2

Does Googlebot, when crawling pages which display maps using the Javascript API, cause a billable API usage on each page request?

1

Yes. At the end of August, a 20-fold increase in the Googlebot crawl rate pushed my site into the paying band for the Maps API. Three days of being monstered cost me €174.

Fish in a barrel?

| improve this answer | |
0

An investigation into the Google Robots.txt file...

https://www.google.com/robots.txt
Allow: /maps?*output=classic*
Allow: /maps?*file=
Allow: /maps/api/js?
Allow: /maps/d/

Has Google's own crawler pushed up the maps views on your website? It looks like it might have.

| improve this answer | |
  • Can you elaborate on how this answers the question? Your answer as written is not very clear to me. – John Conde Sep 4 '18 at 17:10
  • Sure. It appears that Google's Robots.txt file (the file that tells Google what to crawl and what not to crawl on its own domain) may allow the crawling of the /maps. When you add Google Maps to your website, you're basically embedding code from their domain, so crawl rates are controlled by them. – Rob Sep 6 '18 at 5:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.