Webmasters Stack Exchange is a question and answer site for webmasters. It only takes a minute to sign up.
Sign up to join this community
I am trying to find out something about this tag:
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
If I have a dynamic page like these
example.com/product.aspx?ProductID=123
example.com/product.aspx?ProductID=456
If I dynamically add that meta tag for URL #1, will it affect crawling of URL #2?
Technically they are the same page, product.aspx. But the content is rendered dynamically based on the ProductID.
If you "dynamically add that meta tag" for only the first URL then it will not affect indexing of the second.
Whilst it might be the same file (product.aspx) that processes both requests, they are two different URLs.
If by "dynamically add that meta tag" you mean that the page only adds the meta tag when it detects the parameter ProductID=123 and does not add it when it detects the parameter ProductID=456, then it will not affect indexing and following links in URL #2.
Allowing/Disallowing Crawling is done by adding the URL to your robots.txt file, not by the robots meta tag.
The best way to verify is to check the rendered source code of both URLs. If you DO see the meta tag for both pages, then it will definitely affect URL #2.
