Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have just started to put custom error pages on my site. But then I started wondering if google will index these error pages. I searched this up and I didn't get anything, even searched this site, so I did my research. Will google index my error pages?

share|improve this question

2 Answers 2

up vote 2 down vote accepted

Pages that return a 404 not found or 410 Gone code are not indexed by search engines.

From Google webmasters center:

This HTTP response code clearly tells both browsers and search engines that the page doesn’t exist. As a result, the content of the page (if any) won’t be crawled or indexed by search engines. We recommend that you always return a 404 (Not found) or a 410 (Gone) response code in response to a request for a non-existing page.

More info can be found here

Now if you want to remove pages from Google there are 2 techniques that you can use.

  1. Use error code 404 or 410
  2. Use noindex meta tag

Both the techniques error codes or noindex tag, are useful but for other purpopses.

  1. If you have pages that exist, but you dont want them to be indexed eg privacy.html, you should add the noindex meta tag

    <meta name="robots" content="noindex">
    

    The next time that Google will crawl these pages, it will remove them from the search results. More info about how Google treats noindex meta tag can be found here

  2. If you have pages that existed in the past and are no longer existing (eg. you deleted them), you should return a 404 or 410 error code and not use the noindex meta tag. No index meta tag, will tell Google that the page exists, but I dont want it to be indexed

    The html that you return along with the error code does not play a role, what it is important is the error code that you return.

    Google will then recheck the page and will see that the code is not 200 but it is 404 or 410, and will drop the page from the indexes.

    Note that Google must have to recrawl a page that returns 404 or 410 several times before dropping the page completely from the index.

    This is done to prevent a webmaster from doing an error and return wrong codes accidentantly, eg database malfunction.

To sum up:

  • On deleted pages or urls that never existed, you should return a 404 or 410 error code and not using the meta noindex tag, and wait until Google drops them from the search results.
  • On existing pages that you do not want them to be indexed, you should use the meta noindex tag
share|improve this answer

The best solution to ensure they don't get indexed is by adding

<meta name="robots" content="noindex">

If Google only encounters the pages as error pages it shouldn't index them, however, potentially if they were accessible through the sitemap or had inbound links, they could be indexed. It's advisable to just add the noindex tag to prevent this.

share|improve this answer
    
I'm personally more curious about whether error pages (with 4xx/5xx HTTP status code) get indexed when there's no noindex flag. –  Jo Liss Aug 27 '13 at 15:31
    
No, as long as you send the 404 - 410, 503 HTTP status code they are not indexed. –  krokola Feb 22 at 17:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.