Cache an external site that may have a mobile version

Question

In continuation of the answer for the "Nginx Reverse Proxy for Tumblr" question, what is the best way to cache a web-site like Tumblr with nginx, which returns distinct results depending on the User-Agent?

From the one hand, we wouldn't want to keep a distinct cache copy for every possible user-agent string -- that would be insane.

From the other, the information of what parts of the request they use to determine which version to serve, is proprietary, so it's not like we even know what we're dealing with.

What the best way to do the caching with the minimum chance of distinct user-agents polluting and disturbing the cache?

Answer 1

You are correct that proxying less effective when the document has lots of variations for various clients. In this case it sounds like there might only be two versions, which makes the problem a little easier. The ideal solution would proxy based on user agent. It would serve the mobile version to mobile clients and the desktop version to other clients.

For example there are regular expressions that could be used in nginx rewrite rules to do the proxying. Such rules are available here.

Here is how you might implement it in nginx:

if ($http_user_agent ~* "/Mobile|Android|BlackBerry/") {
    proxy_pass    http://m.example.com$request_uri;
}

proxy_pass    http://www.example.com$request_uri;