2

I have two questions about robots meta tag and X-Robots-Tag:

  • If a page has both X-Robots-Tag HTTP header and robots meta tag, can this cause problems?

    For example:
    On HTTP response headers: X-Robots-Tag: noindex
    And on page source code: <meta content='noindex' name='robots'/>

    Which one is better? X-Robots-Tag HTTP header or robots meta tag?

  • On a frame redirect/masked page, does it count the inner robots tags?

    For example:
    example1.com/masked-url frame redirect to example2.com/original-url

    Does it count the meta tags of example1.com/masked-url or example2.com/original-url?

0

If a page has both X-Robots-Tag HTTP header and robots meta tag, can this cause problems?

This should not cause "problems". Providing you make sure to provide the same values for both. eg. noindex, as in your example.

For search engines that support the X-Robots-Tag HTTP response header, then this should always take priority. Where there is an equivalent HTTP response header for an in-page meta element, the HTTP header always wins.

Which one is better? X-Robots-Tag HTTP header or robots meta tag?

Well, you can't use the robots meta tag on non-HTML resources, such as PDFs and images etc. This is primarily why the X-Robots-Tag header was invented.

But whilst all the main search engines support the X-Robots-Tag header, it probably doesn't have as wide support as the robots meta tag, which has been around a lot longer.

On a frame redirect/masked page, does it count the inner robots tags?

Yes. The document contained inside the frame is an entirely separate request and consequently, it's fetched just like any other single request.

Does it count the meta tags of example1.com/masked-url or example2.com/original-url?

Both. They are two separate requests.

  • Just updated with your last question. – MrWhite Dec 20 '16 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.