My website was hit with a penalty recently, losing approximately 35% of traffic overnight. I have been reading up on Panda and Penguin (had never heard of them before). One of the recommendations many people offer is to cut down on the number of "low-quality" pages. Now, I run a website with lots of user-submitted content, and each user submitted page also has pages like "comment", "report abuse", "tell friend", etc, associated with it. The content of all those pages is 95% identical, just a few words changing, but they all have unique URLs. I suspect that these are the pages that are giving me the penalty, since they are actually 80% of all the pages on the site, because each item of legitimate content links to several of these "auxiliary" pages.

Should I block Googlebot from visiting these pages?

  • 3
    There's no real content on them so they don't need to be searchable. I'd say block them.
    – John Conde
    Feb 24, 2017 at 19:24

1 Answer 1


These are pages that do not need to be indexed and give no real value to the site.

The quickest way to solve this is to block them in the robots.txt file.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.