My question is about javascript websites.

Suppose GoogleBot has crawled example.com/index.html?id=1

Does it request all the javascript files again when crawling example.com/index.html?id=2?

index.html?id=1 and index.html?id=2 have identical javascript files, but their generated contents are different.

The javascript files are included like so

<script src="data.js" type="text/javascript"></script>

Here is the code for cache control in the .htaccess.

ExpiresActive On ExpiresDefault "access plus 1 year"

  • I do not believe that Google caches resource files such as JS at all. Or not longer than it takes to render the page and processes it. – closetnoc May 24 '18 at 16:44
  • @closetnoc I struggle with the idea that Google doesn't cache resource files "at all". If Google was in the process of crawling 100 or so pages then I would think it a complete waste if it didn't at least cache the common resources between these 100 or so pages. But at the same time I certainly can't see Google honouring a cache time of "one year" (or even one week) as this could potentially/inadvertantly kill a site if the webmaster/developer has made a simple caching error. In an ideal world I would perhaps expect the SE to honour the caching directives; but alas the world is pretty far – DocRoot May 24 '18 at 22:26
  • ... from ideal. – DocRoot May 24 '18 at 22:26
  • @DocRoot I remember reading a lenthy paper outlining the difficulty of caching JS when so much of it is the same resources. At that time, the thought was that caching JS was a waste and yielded no advantage. I can see caching long enough to the work that is needed of course. After that, what would be the point? The cache is supposed to be helpful, but not to replace the experience of going to the sites page. We did have a question a while ago where the OP was updating his JS regularly and feared breaking any cached page. I do not remember the consensus on that question if there was one. – closetnoc May 24 '18 at 22:41
  • 1
    I will be examining this by tracking GoogleBot activity in such a website, where a resource file is shared with a lot of pages. If the there will be a request for data.js per request for index.html by GoogleBot, then it may not cache resource files at all. Thank you, everybody, for your comments. – Siamak May 25 '18 at 2:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.