Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I understand that a resource with X-Robots-Tag: noindex forces some search engines, including Google, not to index the resource further. I also understand that an HTTP 301 response causes search engines to use the redirected URL instead of the original URL to refer to the resource.

But what happens if both X-Robots-Tag: noindex and status code 301 occur on the same response? It's likely that the original URL will no longer be indexed, but will that cause the redirected URL to no longer be indexed too? This possibility is not mentioned in the X-Robots-Tag specification.

share|improve this question

3 Answers 3

It shouldn't have any effect to have a X-Robots-Tag: noindex header on a 301 redirect URL.

  • Search engines don't index 301 redirect URLs anyway. So that tag is only telling them what they already know.
  • It shouldn't cause the target of the redirect not to be indexed. If it did, that could allow an attacker to redirect to a competitor with that header and get the competition de-indexed.
share|improve this answer
    
As a matter of fact Google does for example put redirects into the results if the bot is not permittet to crawl the redirect target. –  schwarzbrot Apr 30 at 13:06
    
@schwarzbrot: That seems like useful info to put into an answer of your own, especially if you have experience or documentation to back it up. –  Stephen Ostermiller Apr 30 at 14:11

A web page can return a 301 header response code which simply tells a bot not to pay attention to anything else on this page but look at the redirect path and pass on traffic through this redirect.

Subsequently, using a noindex tag on a page that you are redirecting is pointless as it won't get seen anyway.

When permanently redirecting (301), what matters is only what response the redirect destination URL is returning.

share|improve this answer
    
Geoff Jackson, read again what I wrote. –  Gollum Oct 27 '13 at 8:08

The X-Robots-Tag: noindex have a means. I 301 redirected a lot of pages and after several months the original URL still appears in index with the content of destination page. Now I added X-Robots-Tag: noindex tag and I wait to see the results.

share|improve this answer
    
Did you see any impact on the index? It's quite an interesting approach, I'd be happy to hear the result ... –  schwarzbrot Apr 30 at 13:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.