A web application is hypothetically featured on Digg or Reddit, and the server load immediately spikes up by a couple of orders of magnitude. What kind of hosting solution will allow the website to survive through a large flux like this? (and not cost a fortune)
Hosting through AWS seems to be in fashion these days. However, many complain about the cost. Will a VPS hosting solution also be able to scale-up in a timely fashion, while costing less? Alternatives? Thanks.
Optimise your application before optimising your server infrastructure
It's tempting to throw more servers at traffic spikes or overpay for resources that you may never use "just in case". A better solution is to optimise your application to withstand spikes before they arrive. Specifically:
Cache dynamic code
Your application should serve static html files instead of dynamically generated ones using a caching system. WordPress offers plugins for this (e.g. W3 Total Cache) but it's not too tricky to roll your own solution if your application is home grown. (Search for "[programming language] static cache".)
Offload static assets
Serve images, CSS, and JavaScript files from somewhere else. Amazon's Cloudfront and S3 packages are popular options, but I favour Cloudflare, which offers automatic caching of static assets and a list of other CPU and bandwidth-saving benefits for $20/month.
Use Google's Page Speed Online tool
Optimise your site further with Google's Page Speed Online tool. It will present suggestions for things you can do to improve page speed (and reduce load) without touching your server infrastructure.
When you've done all that
Only then is it worth looking at hardware and environment optimisation. Some options worth considering:
If all fails...
Consider creating a static page to outline your service and collect email addresses. Put this up in place of the application itself should your site go down during a traffic spike. I use Wufoo or MailChimp for this.
If you host the emergency 'whoops! we got dugg!' form on a separate (cheap) shared hosting account, you can use DNS failover to automatically redirect traffic to the static form if your main application goes down. That way, if Digg sends you 100,000 visitors and your app chokes, at least you'll have built up your mailing list and piqued interest for a proportion of those visitors.
This really depends heavily on how your frontend servers distribute load. If it's not designed to have extra capacity added to it at the flip of a switch (or maybe better yet, automatically when a sustained spike is detected), then planning for this sort of thing is tough.
If you design your load balancing in such a way that you can flip on extra servers in a matter of minutes (e.g. via AWS), then you'll only pay for the extra capacity while you're using it. You'll have to do some pricing research with some kind of scenario to see if the pricing works out.
HOWEVER if you have a sudden traffic spike due to popularity, there are worse things in the world than going down for a brief period, unless you're running life support systems or something.
If your app/startup/project is especially cost sensitive, then I'd suggest building what you need, and keeping it at that. If it explodes in popularity and you have to spend a couple of days adding capacity, you'd call that a good problem to have.
Furthermore, planning too far ahead for capacity you don't yet need will cause you to spend money now that you could spend on more critical resources.
As just one example, the first StackExchange Site (StackOverflow) ran on a single server until maybe last year, I believe. Jeff Atwood talks about it in his blog, and they were getting millions of pageviews/day before they added additional infrastructure to scale out for the StackExchange network.
During the high traffic period your server should be able to handle all requests made by visitors to your website. But there are some limits in concurrent connections handled by the server. So it's best to serve the page requests as fast as possible.
Here are some suggestions to consider in these situations,
Application level improvements:
a) Combine all JS files together in a single combined JS file, and all CSS files in a single combined CSS file.
b) Minify JS, and CSS files, so the file size will be reduced and it will be downloaded faster.
c) Use CSS Sprites - When you combine most or all of your images into a sprite, you turn multiple images requests into just one. Then you just use the background-image CSS property to display the section of the image you need.
d) Delay image download with lazy-loading, this will be helpful to reduce the http requests.
a)Exclude decorative elements like images or Flash wherever possible; use text instead of images in the site navigation and chrome, and put most of the content in HTML.
b)Use static HTML pages rather than dynamic ones; the latter place more load on your servers. You can also cache the static output of dynamic pages to reduce server load.
Server level improvements:
When timeouts are lower the connection will be released soon, so the server will be able to handle more connections.
Use third party services like CloudFlare for static data caching, and to protect your website from malicious users and attacks like DDOS.
Upgrade your server hardware - Upgrade physical and Virtual memories, increase I/O and Entry processes limits, if required. Your hosting provider will be able to help you better.
Cache dynamic code - Use APC to cache PHP opcode.
Load Balancing - Distribute load across multiple load balancing servers.
When all required actions are taken, now it's time to check if the website is ready for a huge traffic spike.
There are some third party services like loadimpact.com who provide load testing with simulated traffic. The analysis will help you to understand how much load your website can handle and what can be improved.
Also, during the traffic spike period, avoid high CPU usage operations like website backup cronjobs etc.
