3

I have exported a list of 404 and 403 pages from Google Search Console. I have compared these URLs against my site crawl. The 404 pages in Google Search Console are not within my site crawl pages list.

Does this mean I don't need to 301 redirect these pages that have been picked up by GSC?

| improve this question | |
3

These pages have likely come from broken external links. Google crawls them, receives a 404 but will occasionally revisit them in case they've been fixed.

If you wanted to preserve traffic/backlink benefits from these links then you will still need to 301 redirect them.

| improve this answer | |
  • Thanks for the reply - Just looked at where some of the links are coming from and they're either from the same domain or from the 'http' version which redirects to the HTTPS. Does it not matter if the pages from GSC are not in the crawl then? – Sean Lloyd-Clarke Oct 28 '16 at 10:32
  • If the redirect is already in place - then Google will eventually merge the indexed URLs. It doesn't sound like you have anything to worry about! – L Martin Oct 28 '16 at 10:38
2

If you click on the URL in the Crawl Errors report you will get a popup containing a "Linked From" tab. In that tab, Google will tell you where it found the link to those pages:

You may find that they are:

  • Broken links on your site
  • Broken links on 3rd party sites
  • Text URLs Googlebot discovered that are not linked
  • Bits of JavaScript that Googlebot thinks might be URLs

Your own site crawl is only likely to find the broken links on your own site. Since Googlebot crawls the whole web and has heuristics for finding what it thinks might be links, it will find more errors than your own crawl of just your site.

I would only fix a 404 error if it effects users as well as Googlebot. Broken links from your site are always worth fixing. Broken links from 3rd party sites are worth fixing if you can tell what they are supposed to point to. Googlebot stupidity of picking things that could be links out of JavaScript is just noise.

Google expects properly functioning sites to have some 404 errors. In fact, Google gets worried if Googlebot never gets a 404 error for any URL, especially those that should be junk. As long as the URLs that are returning a 404 are not supposed to have content, those 404 errors will not hurt your site in any way. It doesn't matter if there are 10 of them or 10 million of them.

There are a few basic methods for fixing crawl errors:

  • Remove the link to the 404 page if it is on your site
  • Fix the link to the 404 page such that it points to the correct thing
  • 301 redirect the 404 URL to the actual page (ex: /page.ht -> /page.html)
  • Put up a new page of content at the error URL
| improve this answer | |
  • Also consider that not all auditing tools can chase down all links such as javascript links and therefore may not always see all the possible link combinations. ScreamingFrog is an example. I have seen this confusion before. Cheers!! – closetnoc Oct 28 '16 at 16:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.