Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm building a site based on tags for navigation (not a tree structure).

My idea was to create a unique URL representing a unique combination of tags.

The problem is that the sum of every combination of tags will produce over a million pages.

Is this bad?

These pages contain text and links pointing to the real content pages of the site, so not content at all, just dynamically generated content.

share|improve this question
Does every combination of tags produce a page with results? – w3dk Jan 10 '12 at 12:19
yes, it does... and every page contains listings of links – user12370 Jan 10 '12 at 12:51
up vote 3 down vote accepted

Yes, that wouldn't be great for SEO because many of the pages will be almost identical and there are almost infinite combinations. It is akin to a generic search page, which Google discourages the indexing of.

There is nothing wrong with the pages existing, if they are useful to users. You can simply block search engines indexing those pages - probably the best solution would be to add a meta tag to those pages:

<meta name="robots" content="noindex,follow">

This will prevent pages being indexed, but allow search engines to visit and spider the rest of your site. Using a robots.txt file is another option if the above isn't possible.

Having said the above, individual tag pages (as on Pro Webmasters, Stack Overflow, etc) would be pretty useful as they are much more finite and split your content into clear categories. So if possible, do not block those pages, only the tag combos.

share|improve this answer
thank you, perfect! will be rel="nofollow" sufficient instead of robots.txt? thanks! – user12370 Jan 10 '12 at 13:41
@wes Not really because it doesn't stop Google indexing pages if they are linked from elsewhere. However, you can use a meta tag on the pages you wish to block: <meta name="robots" content="noindex"> – DisgruntledGoat Jan 10 '12 at 13:46
ok thank you! I will use the meta tag :) – user12370 Jan 10 '12 at 13:59
@wes I would have said rel="nofollow" is something you don't want to do as you still want the links followed. – w3dk Jan 10 '12 at 14:33
Don't block indexing using robots.txt, it will only block access. Use "noindex" for this! If someone else links to your page and it gets indexed, google cannot access it (because of the robots) to see your "noindex"!. – Nanne Apr 16 '12 at 13:34

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.