For example, my site has a list of products, with a filter box which allows you to narrow by price, features, etc. My question is, can google crawl this form, see the url results (which is a bunch of query variables in a long url), and then index that URL?

1 Answer 1


They can follow this type of filters but not if they are forms. As for AJAX, they are mostly not followed but it is possible though rarely by accident!

However if those are links, it can generate an excessive and nearly infinite number of URLs which may confuse crawlers and prevent important pages from being indexed because crawlers would see more URLs to crawl.

Google Webmaster Tools lets you specify how each parameter they encountered is used. This is a good place to see which ones they already crawled. Look under Configuration -> URL Parameters.

The option of interest to you is: Yes: Changes, reorders or narrows page content which you can specify for each of your filter parameters. Then you can tell Google to decide what to do, to index very variation, only one or none. For example, you can let them index only one sort order.

  • Thanks alot. Looking at the webmaster parameters is actually what got me interested in this, since it showed some params in urls that i didnt explicitly link to, so I'm trying to figure out how it was indexed
    – user21100
    Commented May 29, 2013 at 19:39
  • @user21100 - Indeed and even after 8 years of building websites and 2 years working for Google, I still have no idea how so many URLs not linked from anywhere appear so often!
    – Itai
    Commented May 30, 2013 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.