I think I read somewhere that it was a good practice to have certain "totally dynamic and irrelevant" parameters excluded from the nice URL rewriting techniques. Like here on the StackOverflow galaxy, when you login the URL is /users/login?returnurl=parameter.

The reason would also have been that it kind of "prevents" search engines to follow those links, and could be a good idea for "transition pages" that change often and should not be indexed as is, but still followed, like /news?page=5 where this page would be an index of the latest news, and by nature changes over time as old news are pushed down.

Is there any sense to that? Would you agree, and if so maybe point to some resources? If not, what do you think of it, and what should be done with those pages that keep changing?

Thanks for any thought!

5 Answers 5


I'd say stop reading wherever you read that drivel. Dynamic pages are a natural part of the web, very common, and you should definitely want them to be indexed by the search engines. The search engine crawlers regularly crawl pages looking for changed content and, if they detect any, update their indices. That's good for you because it means new content that you have will be available for others to find which is what you want.

And search engines can crawl virtually any URL regardless of the structure of that URL. So if that site is implying query strings are not search engine friendly they are greatly mistaken.


Google and Bing both crawl URLs with parameters in them. Using parameters as a way to confuse Google out of following the links will only cause you SEO issues.

If you want full information on it, check out Google Search Engine Optimization Guide.. It will educate you on what Google does and doesn't do and what they suggest you do.

Finally, if you don't want specific pages to be crawled or given PR by Google or Bing just put the files/directories into a robots.txt file like @Lese suggested or put a rel="nofollow" on your links. Those are the only ways to 100% guarantee you will not give PR to the pages you don't want to have PR.


There is really only one rule: If the page content doesn't change when the parameter changes, then tell Google to ignore it.

Your returnurl example is a good one - I assume the page doesn't change at all, so tell Googlebot to ignore that parameters in Webmaster Tools. (Actually Google is pretty good at detecting these anyway.)

But for page - definitely do not ignore this! You need the search engines to parse all your pages to find the links to the content.

  • Thx! I did not think of the link between parameter and Webmaster Tools. Now it all makes sense.
    – damusnet
    Oct 15, 2010 at 9:21

Not using clean URLs does not exclude a page from being indexed. To do that, you need to use robots.txt.

And I don't know what you mean by "should not be indexed as is, but followed", but a page that changes often (and has lots of backlinks and can be easily gotten to by Google's crawlers) will automatically be crawled more often by Google, keeping the indexed content fresh. You can also adjust the crawl rate in the Google Webmasters tools.

  • It's just that it often happens to me that I see a result on google, lets say from an index of a forum, and then I can't find what I am looking for, either coz the page changed, or is packed with links, whereas what I wish I had gotten was the actual "topic" or page behind the link. So google's index is not "always" up to date and there ain't nothing you can "really" do about it...
    – damusnet
    Oct 15, 2010 at 9:24
  • @Damien: That's why it's important to have lots of backlinks and a high PageRank. If google crawls your site every 2 months, then your longest "stale" time will be 2 months. But if they crawl your site every 2 days, then it will only very rarely be out of date. And forums are a worst case scenario since 99% of forum posts aren't worth indexing, so naturally Google isn't going to crawl the average forum on a daily basis. But even if you exclude the forum index from Google's index, that won't increase the chances of your threads being indexed. Oct 15, 2010 at 13:49

There are some statements attached to this question that claim "search engines can crawl virtually any URL regardless of the structure of that URL".

Taken from the Google guidelines:

If you decide to use dynamic pages (i.e., the URL contains a "?" character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them few.

Here is the official link for further review: Google Web Master Guidelines

I strongly suggest you take this advice into consideration.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.