3

My site is several years old and there are been pages which were removed a long time ago, have no links from the outside and the sitemaps were rebuilt several times after this, but Google Webmaster Tools shows that Google still crawls them and it counts the missing pages as 404 errors.

Also, for testing purposes I installed one more WordPress in a subdirectory and removed it after a couple of days. The sitemap has been rebuilt since then. This test site doesn't have any links from outside, but still, in crawl errors I find pages from there listed.

How can I resolve this problem and tell Google that these pages no longer exist?

Improve this question
4
  • 1
    Try Remove URLs feature...
    – Gowtham
    Jan 14, 2015 at 9:50
  • 1
    Also check the last crawl date on the errors. Some of these may be old errors originally encountered but no longer valid. In which case mark them as fixed and see if they re-appear.
    – joesk
    Jan 14, 2015 at 11:35
  • 1
    NO! Do not mark them as fixed! Google will think you are telling it that these pages should be there and to try again. Do not mark them as fixed! Let them error out. Google will retry for a number of times before it gives up. Be patient. Also, you can never be sure there are no back links. It is possible that Google found some on a scraper site. These sites are notorious for using old databases of links for years. And these sites can exist for as short as one day of a few hours. So this may happen again.
    – closetnoc
    Jan 14, 2015 at 15:52
  • Perfectly fine to mark the urls as fixed if they are errors you know about. Doing do will remove them from display and wont be reshown unless situation reoccurs.
    – user29671
    Jan 15, 2015 at 9:17

2 Answers 2

Reset to default
2

If those pages got indexed while they were live then they would come up as errors. Some things you can do:

1) 301 redirect to the equivalent pages on the main site

2) 301 to closely related pages on the main site

3) Leave as is because its better to have a 404 error then redirect everything to the homepage.

I would also create a custom 404 page and include some navigation links on it that make sense. This will help people get to the right place quickly.

Have you tried "Fetch As Google" options? Sometimes it will make old errors go away if you have already fixed them. Also, keep in mind that sometimes it just takes a while for the errors to go away.

Last tip: when you copied your WP install to a sub directory you should have blocked the crawlers from accessing it in your robots.txt file. For a single page you can also do a noindex/nofollow.

Improve this answer
3
  • 1
    People think that because Google tells you about an error they encountered that it must be fixed. But think about it. If a page does not exist and Google tells you there is an error accessing a non-existing page, then why on earth would anyone think they should fix the error? But yet they do. The Fetch As Google in this case, being a 404, will not do anything. It is best to either offer up a 410 instead or just let nature take it's course. Since a 404 contains the possibility that the page will return, Google will retry an x number before giving up. Also, these errors disappear after xx days.
    – closetnoc
    Jan 14, 2015 at 16:01
  • And removing URL-s doesn't help?
    – Tsiskreli
    Jan 14, 2015 at 20:14
  • I hate seeing 404 messages. While a home page redirect isn't very useful, how about a redirect to a site map?
    – Sherwood Botsford
    Apr 24, 2016 at 22:58
0

You need to 410 those pages, then mark them as fixed in WMT. They'll return soon enough, but with a 410 error. When that happens, think of it as an advisory note. It will drop off the list in a few 3 months.

A 404 is sort of like a 302, both have little practical use and should be avoided. Google handles them as a temporary issue and will continue trying to crawl the url. When you see these in your crawl report, take action to get rid of them.

A 410 is sort of like a 301. Use it when the content -- which may not be the same as a url, because it may have been incorporated into another page -- is permanently gone. When you see a 302 or 404 in your crawl report, you should either 301 that url to the url where that content now exists, or 410 the url so Google knows it's really permanently gone.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.