Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

What headers should I be sending to outright stop all requests to the server after the content has been cached?

We have a very high latency server (Sigh, VMWare) so even sending a HEAD request to the server takes +40ms.

Currently these are the headers being sent/received;

First request

Client sends;

GET http://dugong:8080/Rvi24mYJkxFRGNzq73PPvgWGh1j/IMG_2071.jpg HTTP/1.1
Host: dugong:8080
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:9.0) Gecko/20100101 Firefox/9.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Pragma: no-cache, no-cache, no-cache
Cache-Control: no-cache, no-cache, no-cache

Server responds;

HTTP/1.1 200 OK
Server: nginx/1.0.11
Date: Wed, 01 Feb 2012 14:51:51 GMT
Content-Type: text/plain
Vary: Accept-Encoding
Last-Modified: Tue, 31 Jan 2012 10:45:11 GMT
Content-Length: 14
Expires: Thu, 31 Jan 2013 14:51:51 GMT
Cache-Control: max-age=31536000

So it sends a Cache-Control and Expires header set to 365 days in the future. Unfortunately on the second refresh it requests the object again with an If-Modified-Since header.

Second Request

GET http://dugong:8080/Rvi24mYJkxFRGNzq73PPvgWGh1j/IMG_2071.jpg HTTP/1.1
Host: dugong:8080
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:9.0) Gecko/20100101 Firefox/9.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
If-Modified-Since: Tue, 31 Jan 2012 10:45:11 GMT
Cache-Control: max-age=0

Response;

HTTP/1.1 304 Not Modified
Server: nginx/1.0.11
Date: Wed, 01 Feb 2012 14:58:00 GMT
Vary: Accept-Encoding
Expires: Thu, 31 Jan 2013 14:58:00 GMT
Cache-Control: max-age=31536000

Unfortunately due to silly outdated proxy software we can't use Keep-Alive, or put any other servers/proxys in front of the application. We also can't improve the performance of the server and reduce the network latency. I've been trying to figure out what headers we can send to get rid of the 301 requests. I've tried using ETags but that makes no difference, it still sends a If-modified-since header. I've also tried removing the Last-Modified header but that just causes a standard GET request with no caching (Checked the logs, server still receiving requests).

Clients are a mix of Firefox (mostly), IE 7, 8 and (some) 9, Chrome and Safari but this behavior seems to be appearing in all browsers tested.

TL;DR;

Terrible network, what headers should I send to tell clients to never ever ever send If-modified-since requests to the server to validate their cache, and keep content cached until the Expires header is met?

I'm probably missing something obvious but everything I try seems to yield the same results.

We have an NGINX server sat in front of our application server so I can add/remove any headers as I please. Our proxy doesn't support Keep-Alive and theirs no way to improve the attrocious network performance. Due to terrible software design the web app loads +100 resources on each page load (Yeah, enterprise software sucks) with a latency of ~40-50ms per object.

share|improve this question
    
Hmm, that's odd. Sending the Expires and max age headers should prevent further requests of the image. Edit: by the way, what's the deal with sending a JPG as text/plain? –  DisgruntledGoat Feb 1 '12 at 15:46
    
@DisgruntledGoat Ahh, you've made the assumption a .jpg file is actually an image rather than a text document. Welcome to my world =) (It's actually a text file that contains 'Hello World' for my testing, the software just renames all files sequentially IMG_xxxx.jpg regardless of type. Cool huh?) –  sam Feb 1 '12 at 15:58

1 Answer 1

up vote 12 down vote accepted

You can't really control what headers user agents decide to send to you. If the file in question is in the browser's cache and it decides it need to check for a new version then it will. According to this article, these are the situations browsers will request using If-Modified-Since:

  • The cached entry has no expiration date and the content is being accessed for the first time in a browser session
  • The cached entry has an expiration date but it has expired
  • The user has requested a page update by clicking the Refresh button or pressing F5

So if you are reloading the page to test your caching, it won't work since the browser will re-request the images. Try clicking a link then another link back to the first page. If your users are regularly reloading pages then you may need to rethink your site/app structure to prevent that.

One thing that may help is adding "public" to the cache control header, i.e. Cache-Control: public, max-age=31536000. I also learned recently that an expiry date of more than one year is invalid. Since your expiry date is exactly one year, perhaps lowering that by a few days or weeks would ensure the file stays in browser caches and is not discarded.

share|improve this answer
    
Interesting, I'll lower the expires to 60 days and add the public flag, and see what happens. This seemed to be happening on link-clicks rather than F5s (according to Firebug and the server logs) –  sam Feb 1 '12 at 17:52
    
Technically, the HTTP/1.1 spec only says that "servers SHOULD NOT send Expires dates more than one year in the future" (presumably because that's a ridiculously long time before expiration) and that "approximately one year" in the future is the appropriate expiration time to send for content that is never expected to expire. –  Ilmari Karonen Feb 1 '12 at 19:07
1  
After a bit of playing around, I've concluded the 365d expiry is not affecting our clients however I've dropped it down to be safe, it seems the Cache-Control: public,... was the key to this specific situation. –  sam Feb 2 '12 at 8:42
    
Do you mean the "public" header fixed the unnecessary round-trips ? I tried it, but with no success... –  phtrivier Jan 16 '13 at 10:47
2  
In case it's not clear from my answer, reloading the page in your browser will request the files again. Just click links to open pages and the browser uses its cache. –  DisgruntledGoat Oct 18 '13 at 12:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.