I have a tiny two page site: the parameters page and results page. The results page can either work with URL parameters or hidden POST ones. The output is a giant table of data that contains rankings of US cities by some parameters that were defined in the first page.

I am not sure how crawlers could use this page without choosing the parameters (or which ones it would choose), as it is not a standalone page. Should I just block the results page in robots.txt? Will it affect the ranking of my site if I do not block this page (right now the URL to it is very long and contains many parameters)?


1 Answer 1


There is no need to block the page in its entirety, what you can do is apply a change to your site so that if no parameters are passed then the site returns a 404 not found error (important this has to be a http code and not just printed on the screen) and allow Google to spider your results page.

Why do this? Well blocking any spider access to the results page will mean that if someone links to the results page with specific parameters from another site and Google tries to follow it will miss a large number of keywords and page content which could improve your Google ranking. By returning a HTTP 404 error when no parameters are passed Google won't index the URL however as I recall Google treats pages with different query strings as different pages and so will still index the results pages when query string filters are applied and linked to. As for the POST parameters, well ideally all the parameters would be passed as a GET allowing someone to link to it easily (if you wish this) however you wouldn't be receiving links from external sites trying to pass POST parameters so makes no real difference for the Google side of things.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.