Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I have content behind a POST form (a select box, in my case) will Googlebot, or other Search Engine Spiders be able to crawl the content behind that form?

For example, if I were to have 4 translations of one page, and the language selection were a POST form to the same URL, which then redirected to the language-specific URL, would Googlebot be able to crawl the "other" languages?

share|improve this question
up vote 7 down vote accepted

The Googlebot can submit forms, but it generally doesn't unless it can detect a reason to do so.

So from the links, if your translations were AJAX'd and built properly, Googlebot may very well submit the form to see what the results would be. However, this behavior (especially on POST), is not guaranteed and you should probably use GET to make things more accessible to the robots.

share|improve this answer
@DisgruntledGoat Are you sure? Do you have a source? The linked article states that "For text boxes, our computers automatically choose words from the site that has the form". The article was written in 2008 though, so maybe things have changed? – s427 Aug 14 '13 at 10:50
@s427 Yes you are correct, I have deleted my comment. For GET forms (such as a search box) Googlebot may generate text and submit it. There are some caveats listed there - they do not submit POST forms, nor any forms with password fields. – DisgruntledGoat Aug 14 '13 at 11:41

Bing absolutely does. Before I told it not to it would continually submit both a contact form and a survey form. I knew it was Bingbot because I included $_SERVER['HTTP_USER_AGENT'] in the $_POST information. No other spider ever submitted those forms.

share|improve this answer
How did you tell Bing to stop submitting? Thanks! – Thomas Landauer Feb 25 at 11:39

They can and do. I've got a simple email collection form on the front of a site I'm about ready to launch. I just received a submission from that form from IP, which is a GoogleBot IP.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.