1

Google Webmaster Tools is reporting 404 errors on URLs that are not present in the submitted sitemap anymore. These URLs were present but since then I have resubmitted my sitemap and GWT shows that it was, indeed, crawled again after the change.

Is there anything I can do to stop Google from indexing these URLs without manually excluding them? There are quite a lot of them so I would love to be able to avoid doing it.

Improve this question
6
  • 1
    possible duplicate of How to effectively close a page?
    – Zistoloen
    Commented Jul 14, 2013 at 12:57
  • 1
    Look at my answer on this topic, it answers to your need.
    – Zistoloen
    Commented Jul 14, 2013 at 12:58
  • @Zistoloen, thank you for the answer. It does help a bit, I have been thinking about using 301s instead of 410s (the pages are not gone, their URLs changed). However, I hoped for a possible solution to the problem of Google crawling pages that are not in sitemaps anymore. But I guess that removing something from a sitemap does not stop Google from trying to access it.
    – user6113
    Commented Jul 14, 2013 at 13:04
  • 1
    Google follows links and maybe there are still links to these webpages. In that case, apply a 301 redirects and Google will remove these webpages as 404.
    – Zistoloen
    Commented Jul 14, 2013 at 13:07
  • 1
    Just wait a little bit and Google will remove them.
    – Zistoloen
    Commented Jul 14, 2013 at 13:13

1 Answer 1

Reset to default
0

Once Google crawls a URL, it will continue to crawl that URL forever. It doesn't matter if it is no longer in your sitemap. It doesn't even matter if you never had a page there at all. If Google found a link to it at some point in the past, you may see it in your 404 report on an ongoing basis.

Google's John Mueller said that 404 errors don't hurt your site's indexing and ranking. There is really no reason to worry about 404 errors in this report unless you expected those pages to be found.

If you want to clean up that report you can always redirect the URLs to other similar pages, or return the 410 Gone status.

Improve this answer
1
  • Thanks, this is what I have done. I used 301s in the end, since these were the kind of redirects that fitted the situation best.
    – user6113
    Commented Jul 26, 2013 at 11:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.