What hosting solutions help with traffic spikes?

Question

A web application is hypothetically featured on Digg or Reddit, and the server load immediately spikes up by a couple of orders of magnitude. What kind of hosting solution will allow the website to survive through a large flux like this? (and not cost a fortune)

Hosting through AWS seems to be in fashion these days. However, many complain about the cost. Will a VPS hosting solution also be able to scale-up in a timely fashion, while costing less? Alternatives? Thanks.

Answer 1

Optimise your application before optimising your server infrastructure
It's tempting to throw more servers at traffic spikes or overpay for resources that you may never use "just in case". A better solution is to optimise your application to withstand spikes before they arrive. Specifically:

Cache dynamic code
Your application should serve static html files instead of dynamically generated ones using a caching system. WordPress offers plugins for this (e.g. W3 Total Cache) but it's not too tricky to roll your own solution if your application is home grown. (Search for "[programming language] static cache".)

Offload static assets
Serve images, CSS, and JavaScript files from somewhere else. Amazon's Cloudfront and S3 packages are popular options, but I favour Cloudflare, which offers automatic caching of static assets and a list of other CPU and bandwidth-saving benefits for $20/month.

Use Google's Page Speed Online tool
Optimise your site further with Google's Page Speed Online tool. It will present suggestions for things you can do to improve page speed (and reduce load) without touching your server infrastructure.

When you've done all that
Only then is it worth looking at hardware and environment optimisation. Some options worth considering:

1. VPS solutions that provide 'instant scaling', such as VPS.net.
2. Application hosting solutions such as Duostack, PHP Fog, and Orchestra.io, which optimise and load balance the infrastructure for you.
3. Optimised environments such as Nginx with PHP-FPM, or Ruby Enterprise Edition.
4. If you're just hosting a blog, consider a hosted platform like WordPress.com or tumblr. These tend to withstand traffic surges better than default home-grown setups (and they're cheaper).

If all fails...
Consider creating a static page to outline your service and collect email addresses. Put this up in place of the application itself should your site go down during a traffic spike. I use Wufoo or MailChimp for this.

If you host the emergency 'whoops! we got dugg!' form on a separate (cheap) shared hosting account, you can use DNS failover to automatically redirect traffic to the static form if your main application goes down. That way, if Digg sends you 100,000 visitors and your app chokes, at least you'll have built up your mailing list and piqued interest for a proportion of those visitors.

Answer 2

This really depends heavily on how your frontend servers distribute load. If it's not designed to have extra capacity added to it at the flip of a switch (or maybe better yet, automatically when a sustained spike is detected), then planning for this sort of thing is tough.

If you design your load balancing in such a way that you can flip on extra servers in a matter of minutes (e.g. via AWS), then you'll only pay for the extra capacity while you're using it. You'll have to do some pricing research with some kind of scenario to see if the pricing works out.

HOWEVER if you have a sudden traffic spike due to popularity, there are worse things in the world than going down for a brief period, unless you're running life support systems or something.

If your app/startup/project is especially cost sensitive, then I'd suggest building what you need, and keeping it at that. If it explodes in popularity and you have to spend a couple of days adding capacity, you'd call that a good problem to have.

Furthermore, planning too far ahead for capacity you don't yet need will cause you to spend money now that you could spend on more critical resources.

As just one example, the first StackExchange Site (StackOverflow) ran on a single server until maybe last year, I believe. Jeff Atwood talks about it in his blog, and they were getting millions of pageviews/day before they added additional infrastructure to scale out for the StackExchange network.