11

I have a web page, say http://domain/purchase and in this page, I have a web form. User, on submitting this form (which has validation, both client-side and server side and won't be validated until fields are filled appropriately), would be redirected to another page, where (s)he can choose other things, and specify other settings and then purchase our product. Say the second page is http://domain/options.

So, user comes to our site and visits http://domain/purchase, fills the form, submits it, and then would be redirected to the second page, http://doamin/options?parameter1=value1&parameter2=value2, which contains parameters from the first page. This is very common in passing parameters between web pages (or technically, between URLs).

Now I was reviewing my website, and saw that Google had indexed some of my redirected web pages and URLs, like:

  1. http://domain/options?parameter1=value1&parameter2=value2
  2. http://domain/options?parameter1=value3&parameter2=value4
  3. http://domain/options?parameter1=value5&parameter2=value6
  4. http://domain/options?parameter1=value7&parameter2=value8
  5. http://domain/options?parameter1=value9&parameter2=value10

This means that Google Bot has visited our http://domain/purchase page, and has filled our form, and has submitted it, and was being redirected to the other URL, with corresponding parameters. This is the only way that makes sense to me. Does Google really fills forms?

PS: All parameters are meaningful, meaning that they are not filled arbitrarily. For example, the phone parameter in indexed pages has correct phone numbers. How is it possible?

2 Answers 2

8

Google has been able to crawl HTML forms since 2008.

Having chosen the values for each input, we generate and then try to crawl URLs that correspond to a possible query a user may have made. If we ascertain that the web page resulting from our query is valid, interesting, and includes content not in our index, we may include it in our index much as we would include any other web page.

On the other hand, since it's unlikely that GoogleBot knows the correct parameters to put into those fields, there may be another explanation. Is it possible that your users are linking to the redirected pages?

4
  • This Google Webmaster video also covers both points: youtube.com/watch?v=xS0oHYZafTQ Nov 30, 2011 at 12:14
  • It says that they only crawl GET forms. My form method's is POST. Thus, not applicable. Yeah, why not, URLs are always link-able. Nov 30, 2011 at 13:30
  • 5
    If you're seeing ?parameter1=value9&parameter2=value10 in the URLs, they're GETting them; POST doesn't embed the form variables in the URL.
    – Wooble
    Nov 30, 2011 at 14:28
  • @Woobie: In this case the URL parameters would seem to be as a result of a server-side redirection, after the form has been POST'd.
    – MrWhite
    Jun 6, 2014 at 11:13
4

Google is getting more aggressive when it comes to submitting forms.

1
  • The key line is: "We’ve started experiments to rewrite POST requests to GET" ;) Dec 2, 2011 at 10:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.