Eliminating 'cookie' from 'vary:' without breaking website functionality

Question

I read that some middle-man servers won't cache websites that have cookie set in the vary: header. For example, If my website issues a vary:accept-encoding,cookie as opposed to vary:accept-encoding then caching will unlikely happen. This URL has further documentation:

https://groups.google.com/forum/#!topic/page-speed-service-discuss/82-_dQfTFhc

I'm using a home-brew version of PHP sessions in an attempt to reduce Time to First Byte (TTFB). and currently the sessions are stored as tracking cookies which makes some pages on my site not work correctly if cookie was removed from the vary: header.

Another way I could save user configuration instead of cookies would be to check the client's IP address and load data from a local file as if it was a cookie. But before I think of implementing this, would it be a good idea, and would middle-man (I think they're called intermediate) caches always provide the correct page?

If I'm confusing, I'll explain by example.

Say I have a website where a user can change the background color of the screen with a click of one of two buttons and when other pages on the site are navigated, the last background color used will be shown until the user changes the color again.

I can easily do this by storing the user's preference as a cookie and issuing vary:cookie on all responses, and the correct color will always load, even through middle-man caches.

Now the question is, could I store user's preference as a file on the server and attach the remote IP address to it and not use vary:cookie? or will the middle-man cache disrupt passing the correct IP address and/or provide incorrect data due to its own caches?

The point is I want to somehow eliminate cookie from the vary: header while still allowing a user to browse a website with their preferences always saved.

Anyone got any idea how I can accomplish this?

** Just found out... **

According to https://serverfault.com/questions/195654/how-to-cache-websites-using-varnish-php-and-cookies , Varnish cache will not cache any output that includes cookie in the vary header. I'm curious if its legal across all browsers and systems to use my own header as a cookie header and if it would solve problems.

Answer 1

Basically, you want to get to a point where your HTML page is the same for all users, regardless of their configuration. This is only really possible if you can apply all of the customisation with javascript.

To continue your example, let's say you allow users to customise the background colour of the page, and this choice is stored in a cookie. If you apply this colour change using javascript (which reads the cookie and then makes the necessary CSS change client side), your HTML page remains the same for all users, and you don't have any issues with this page being cached.

Using IP address would be a bad idea. You can't rely on each user having a different IP address, and you can't rely a user always having the same IP address. But regardless of these factors, caching the configuration by IP as you described wouldn't solve your problem.

I'm assuming by "middle-man cache" you're referring to HTTP proxies like Squid - caches between you and your users that you have no control over. (This is slightly different to something like Varnish, which you mentioned, which you'd install on your server and would have full control over.)

I went to a campus university which had around 10,000 students. All external web traffic went through a web cache in order in order to minimise unnecessary outside traffic. So this is an example of the type of cache I think your question refers to.

Let's say you've setup your site to cache configuration by IP address and removed the vary cookie header. A student at my old uni visits your site and sets the background colour to pink. Your server stores this configuration along with the users IP address (which would be the uni web cache IP), and sends back the modified HTML page with the pink colour applied. The uni web cache caches this pink page since its headers indicate it can do so. 5 minutes later student B visits your site. The uni cache would serve student A's pink page to student B, and crucially this request would never reach your server at all. Even if this second request did reach your servers, student B would have the same IP as student A, so your server would just send back the same pink page.

Now let's say you'd implemented this using javascript instead. Student A visits your site, sets the page to pink. Using JS, your page sets a cookie and applies the pink colour (client side). The uni web cache caches the HTML page. Student B visits your site, the uni cache serves the cached page, which would be the standard one. Student B would see the normal, non-pink page, since they wouldn't have the colour change cookie.

Of course, it's not always possible (or desirable) to do all customisation using javascript. Whether or not this will work for you will depend on your specific use case (which I assume is more complicated than setting a background colour).

To give another example, I worked on a site a few years ago which used Varnish. This was a high traffic site, so we wanted it to be as cache friendly as possible, but users who were logged in needed some extra functionality which was very specific to them. Logged in users represented a very small proportion of the site traffic (<1%). I wrote a custom PHP session handler which stored sessions in the database, but would only create them if they contained data. I configured Varnish to not serve cached pages if the request included the session cookie. This meant Varnish could handle the vast majority of traffic, but as soon as a user logged in, all their requests would go straight through to PHP.