Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a web page, say http://domain/purchase and in this page, I have a web form. User, on submitting this form (which has validation, both client-side and server side and won't be validated until fields are filled appropriately), would be redirected to another page, where (s)he can choose other things, and specify other settings and then purchase our product. Say the second page is http://domain/options.

So, user comes to our site and visits http://domain/purchase, fills the form, submits it, and then would be redirected to the second page, http://doamin/options?parameter1=value1&parameter2=value2, which contains parameters from the first page. This is very common in passing parameters between web pages (or technically, between URLs).

Now I was reviewing my website, and saw that Google had indexed some of my redirected web pages and URLs, like:

  1. http://domain/options?parameter1=value1&parameter2=value2
  2. http://domain/options?parameter1=value3&parameter2=value4
  3. http://domain/options?parameter1=value5&parameter2=value6
  4. http://domain/options?parameter1=value7&parameter2=value8
  5. http://domain/options?parameter1=value9&parameter2=value10

This means that Google Bot has visited our http://domain/purchase page, and has filled our form, and has submitted it, and was being redirected to the other URL, with corresponding parameters. This is the only way that makes sense to me. Does Google really fills forms?

PS: All parameters are meaningful, meaning that they are not filled arbitrarily. For example, the phone parameter in indexed pages has correct phone numbers. How is it possible?

share|improve this question
add comment

2 Answers

up vote 6 down vote accepted

Google has been able to crawl HTML forms since 2008.

Having chosen the values for each input, we generate and then try to crawl URLs that correspond to a possible query a user may have made. If we ascertain that the web page resulting from our query is valid, interesting, and includes content not in our index, we may include it in our index much as we would include any other web page.

On the other hand, since it's unlikely that GoogleBot knows the correct parameters to put into those fields, there may be another explanation. Is it possible that your users are linking to the redirected pages?

share|improve this answer
    
This Google Webmaster video also covers both points: youtube.com/watch?v=xS0oHYZafTQ –  DisgruntledGoat Nov 30 '11 at 12:14
    
It says that they only crawl GET forms. My form method's is POST. Thus, not applicable. Yeah, why not, URLs are always link-able. –  Saeed Neamati Nov 30 '11 at 13:30
4  
If you're seeing ?parameter1=value9&parameter2=value10 in the URLs, they're GETting them; POST doesn't embed the form variables in the URL. –  Wooble Nov 30 '11 at 14:28
add comment

Google is getting more aggressive when it comes to submitting forms.

share|improve this answer
    
The key line is: "We’ve started experiments to rewrite POST requests to GET" ;) –  Zhaph - Ben Duguid Dec 2 '11 at 10:43
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.