Google bot requesting URLs from old 301 redirectd site and then reporting a bunch of 404 errors

Question

I'm reviewing some errors in Google Webmaster Tools / Search Console.

I'm getting a lot of 404 errors for pages that don't exist such as:

• example.com/page12.php
• example.com/page111.html
• example.com/wpimages/34f126905bc8Pc7454118.xml

My site doesn't use WordPress or a /page URL structure as the above would suggest.

I looked to see where Webmaster Tools says they are linked from. I found that they are being linked from a old site which I now 301 to this current site. Somehow it must be trying /page12.html on the old site and because the old site has a site wide 301 to the new site it is causing the crawl errors. It doesn't actually matter if its doing this as its a "bot only bug" which users would never see, but could that effect our rankings?

Answer 1

Once Googlebot finds content on a URL, it is very tenacious. It will continue to fetch that URL and follow redirects from it forever. When you redirect an old site to your new site you inherit all its URLs. Googlebot will then fetch them on your site.

Those 404 errors won't hurt your current site. Google's John Mueller says so here:

404 errors on invalid URLs do not harm your site’s indexing or ranking in any way. It doesn’t matter if there are 100 or 10 million, they won’t harm your site’s ranking.

It does however make your reports in Webmaster Tools much less useful.

Your options are:

• Ignore the 404 errors -- they shouldn't hurt you
• Remove the 301 redirect from the old site although Googlebot may still try to crawl those URLs on your site because it has done so before.
• Make those pages "410 Gone" status which will make Googlebot crawl them less frequently (although Googlebot will still crawl some) and should lower their priority in the Webmaster Tools error reports
• Find other pages to which to redirect those URLs. If you have content similar to what used to be on those pages, redirect to that content. Then you would also take advantage of any inbound links that the old site had. It won't help to redirect to your home page. Google considers mass redirects to a central page to be "soft 404" and will still report them as errors.