I am building a Product comparison web/application into my website. It will compare products by specs, price etc.

I am wondering if there any dangers of not making a user log in before using the web page. Ie crawlers going crazy and query the database lots of times? Otherwise, I would be letting any guest user add products to the comparison 'basket' and comparing them.

Edit: I don't want to block robots access to the page either, as I think this could be good for SEO purposes.

Is there any dangers of not making users log in to use it?

2 Answers 2


There is a danger in creating too many landing pages. Not many people will search for "3 inch metal blue widgets". They may be searches for:

  • Widgets
  • Blue Widgets
  • Metal Widgets
  • 3 Inch Widgets

Having a page for every combination of product attribute can confuse Googlebot with many similar pages. The pages end up looking thin and have much duplicate content.

Users on your site may wish to drill down to very specific searches, so it is good to have that functionality. Search engines should be limited to viewing one attribute at a time (or maybe two attributes for popular combinations only.)


write comparisons and let index only products with defined search volume. Otherwise you risk to create pages with very low user metrics.

If you let users add products to compare, set the products to noindex till you checked its search volume.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.