1

I just completed a website crawl on my site to vet SEO, which should return 119 pages. The results are around 490, with errors being returned showing hundreds of 404 pages with strange URLs. These follow a roughly similar pattern, like so:

Where the actual live page looks like this (please note that this is a dummy URL):

www.example.com/about/careers.html

However, there are many 404 errors being returned when crawling the site that look similar or identical to these:

  • www.example.com/about/about/careers.html
  • www.example.com/about/about/about/careers.html
  • www.example.com/about/about/about/about/careers.html

Currently, I cannot find any specific literature on what may be causing this - at first I thought it may be sitemap related, but attempting to re-run a sitemap generator also indicates over 490 pages which is very odd. I cannot see any of these additional pages on the IIS server system where the site is hosted, only the 119 that should be there.

I am totally stumped at this point! What could be causing this?

1

You have a broken link somewhere on your site. My guess is that it is in your navigation template that is used on every page on your site. You probably have a link like <a href="about/careers.html">Careers</a> that points to your career page.

That link will work fine on your home page, but it is a relative link that always adds the "about" directory into the URL, even if the "about" directory is already in the URL. When you get to the career page and click on the "careers" link again, it will go to a 404 page: www.example.com/about/about/careers.html.

The solution to the problem is to use "root relative linking" rather than "document relative linking". To do so, start all your links with a slash (/). So your careers link should be: <a href="/about/careers.html">Careers</a>. When a URL starts with a slash, the it is interpreted as relative to your root domain rather than relative to the current page and directory. Linking to /about/careers.html will work from any page on your site, no matter how deep in the directory structure it is.

| improve this answer | |
  • Many thanks for this feedback! I have gone through every HTML instance on the website and modified the href=" values for these, to include a leading slash. I would imagine the next step is to process a new sitemap. – spl Aug 30 at 13:14
  • The site map has nothing to do with this issue. you could recall your site using some of these tools and ensure the problem is fixed – Stephen Ostermiller Aug 30 at 15:50
  • I see - you mentioned tools but i can't see any links within the message, unfortunately? – spl Aug 31 at 7:51
  • 1
    I believe Stephen means you could use the same tool you used to scan your site to check to make sure there aren't any more broken links. Something like ScreamingFrog has broken link reports built into it. – I Capulet Sep 1 at 0:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.