We are currently trying to see if our site is being indexed successfully by google. Mind you we got millions of pages.
A trick some people use is the google site: command. I always assumed that google.com and google.nl would be the same. Data being fetched by the same GoogleBot. And that it was more of a latency thing where google.nl would be closer to home than google.com. But this is clearly not the case, since google.nl and google.com give back different results.
http://www.google.nl/search?sourceid=chrome&ie=UTF-8&q=site:amazon.com returns 37.400.000 results and http://www.google.com/#sclient=psy&hl=en&site=&source=hp&q=site:amazon.com returns 487,000,000 results
Does anybody know how this works?
Another related question is what other tricks people use to see if google.com successfully indexes all data.
It seems the best way is to do some analytics on the apache logs searching for the GoogleBot.