We recently encountered an issue where we added a JavaScript variable (e.g. var GLOBAL_VAR = true;) to an HTML page (e.g. /Search/Index) and updated the separate JS file (e.g. /Scripts/search/index.js) to access and use that JS variable. After publishing these changes live to the web, we began getting JS errors generated by Googlebot that are sent to us by our window.onerror function. The errors are like:
Uncaught ReferenceError: GLOBAL_VAR is not defined
Line #: 1
When I visit the /Search/Index page on our site, I clearly see the new JS variable is there and defined.
It's almost as if when Googlebot is crawling our site, it isn't detecting that the HTML page changed. So that means it is using old, cached HTML that doesn't include the setting of the GLOBAL_VAR variable, which causes the JS error.
We published the HTML/JS changes on 2/6, and we are still getting JS errors as of 2/10. I would have thought Googlebot would recognize the HTML changes by now. We've never experienced this problem in the past.
Why would Googlebot not update their cache if the HTML of a page changes? Most importantly, how can we get Googlebot to detect the HTML changes and update their cache, so we stop getting these JS errors?
2019 UPDATE - One thing I'll add here is that we use Microsoft Azure Cloud Services that have a production and a staging environment. We publish to staging, swap staging to production, test the new production, then delete staging. The scenario described above appears to only occur when swapping/deleting environments, so it must have something to do with that. It's like Googlebot is crawling our site when we swap staging/production, and it starts getting some data from staging and some from production, which distorts their cached resources from our site. It is still happening and still filling up our error logs.
window.onerrorto send each JS error to our server via Ajax.