Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We plan to import tens of thousands of products from affiliate product feeds to a brand new site, where each product will have its own page with identical title and description as the original. I worry that too much duplicated content will give search engine penalties. Should these pages be marked with a "noindex" robots meta tag to avoid the penalty issue?

share|improve this question

If you know these pages will be duplicate content then you will definitely want to make sure you tell Google not to count them as "real" pages. I would say a better option is to use canonical URLs as that is a more semantic indicator of what is going on. These pages are duplicates of an original and you're telling Google which page is the original. Plus that way even though the pages won't be listed in the search engines they should be crawled and you can benefit from the internal links structure they have to offer. If you cannot do canonical URLs then definitely block these using a noindex meta tag or HTTP header. Blocking them via robots.txt will also work.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.