2

Does Googlebot, when crawling pages which display maps using the Javascript API, cause a billable API usage on each page request?

2 Answers 2

1

Yes. At the end of August, a 20-fold increase in the Googlebot crawl rate pushed my site into the paying band for the Maps API. Three days of being monstered cost me €174.

Fish in a barrel?

0

An investigation into the Google Robots.txt file...

https://www.google.com/robots.txt
Allow: /maps?*output=classic*
Allow: /maps?*file=
Allow: /maps/api/js?
Allow: /maps/d/

Has Google's own crawler pushed up the maps views on your website? It looks like it might have.

2
  • Can you elaborate on how this answers the question? Your answer as written is not very clear to me.
    – John Conde
    Sep 4, 2018 at 17:10
  • Sure. It appears that Google's Robots.txt file (the file that tells Google what to crawl and what not to crawl on its own domain) may allow the crawling of the /maps. When you add Google Maps to your website, you're basically embedding code from their domain, so crawl rates are controlled by them.
    – Rob
    Sep 6, 2018 at 5:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.