5

I have deleted a couple of thousand pages from a website. They do not appear in my sitemap.xml and no other pages on the web are pointing to these anymore.

Google Webmaster Tools (GWT)'s crawl error pages starts to report them as not found. There is an option to mark them as fixed. Should I mark them as fixed? They are not really fixed and won't ever be fixed.

There is also a remove URLs page, but there is no way I will manually remove those pages one by one (REM: those pages are not in a single directory, they were all over the place, so there is no robots.txt directive I could use).

If Google does not find a page, does it automatically remove it from its index, or if we mark it as fixed, does it try to reach it over and over again?

1
  • I'm not totally sure, but I don't think marking it as 'fixed' or not has any bearing on how Google indexes the site -- I think the notice is there just to help you. It will probably disappear after a while, unless the pages are still linked to.
    – Brendon
    Commented Aug 26, 2013 at 13:39

2 Answers 2

5

Whether or not you mark them fixed will have no bearing on your site performance. The idea of that feature is just to allow webmasters to clean up their report when they have fixed errors. Google will eventually get the idea that a page isn't coming back, but it can take a while.

Arguably, if these pages are gone forever they should return a 410 HTTP code. Apparently there's not much difference in how Google treat 410s and 404s, but John Mueller from Google has suggested that 410s may get cleaned up more quickly.

Alternatively, depending on what you've deleted and why, there may be a good user experience argument to 301 redirect to some near equivalent (say, a parent category landing page). That can have SEO advantages, too.

2

Google updates its entire index regularly. When we crawl the web, they automatically find new pages, remove outdated links, and reflect updates to existing pages, keeping the Google index fresh and as up-to-date as possible.

If outdated pages from your site appear in the search results, ensure that the pages return a status of either 404 (not found) or 410 (gone) in the header. These status codes tell Googlebot that the requested URL isn't valid. Some servers are misconfigured to return a status of 200 (Successful) for pages that don't exist, which tells Googlebot that the requested URLs are valid and should be indexed. If a page returns a true 404 error via the http headers, anyone can remove it from the Google index using the webpage removal request tool. Outdated pages that don't return true 404 errors usually fall out of our index naturally when other pages stop linking to them.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.