We plan to import tens of thousands of products from affiliate product feeds to a brand new site, where each product will have its own page with identical title and description as the original. I worry that too much duplicated content will give search engine penalties. Should these pages be marked with a "noindex" robots meta tag to avoid the penalty issue?
1 Answer
If you know these pages will be duplicate content then you will definitely want to make sure you tell Google not to count them as "real" pages. I would say a better option is to use canonical URLs as that is a more semantic indicator of what is going on. These pages are duplicates of an original and you're telling Google which page is the original. Plus that way even though the pages won't be listed in the search engines they should be crawled and you can benefit from the internal links structure they have to offer. If you cannot do canonical URLs then definitely block these using a noindex meta tag or HTTP header. Blocking them via robots.txt will also work.
answered Mar 3, 2012 at 14:24
