47

I read that Stack Overflow uses 10 or more servers for serving the Stack Overflow site. Different servers have different functions such as reverse proxy, database server, or HTTP server.

I have seen a powerful standalone single server with these specifications:

  • 2 x Xeon E5-2630v2 @2.60 GHz, total 12 core, 24 threads; 30 MB
  • 64 GB ECC Reg. up to 768 GB DDR3 at 1600 MHz
  • 4 x 120 GB Intel 520/530 Series (80k random IOPS, ~550 MB/s)
  • HP iLo4 Advanced with dedicated Ethernet management port.

Why not use a single server with higher specifications like 768 GB RAM, 20 TB + HDD, 4+ x Xeon? What are the benefits of using many servers or the drawbacks of using a single high-specification server?

Improve this question
7
  • 4
    SE not only has 10+ servers, they have a duplicate setup in another datacenter for failover. And, the server has not been invented yet that could handle all the traffic of Facebook or Google.
    – Michael Hampton
    Nov 21, 2014 at 1:17
  • 8
    What happens when you need to reboot that super server?
    – Liath
    Nov 21, 2014 at 11:04
  • Redundancy... :)
    – William
    Nov 21, 2014 at 16:40
  • 1
    Parallelism...
    – Lightness Races in Orbit
    Nov 23, 2014 at 14:40
  • 1
    @SSpoke: you are not limited to one connection per port. All that matters is that the combination of (src address, src port, dst address, dst port) is unique.
    – David
    Nov 24, 2014 at 6:17

6 Answers 6

Reset to default
60

A single powerful server can only be upgraded so far. Once you have the most powerful server available, your site cannot grow more without splitting it between servers or making it more efficient.

There is also the cost factor. A single server that is super powerful may cost ten times as much as two servers that are half as powerful. You want to be able to buy your hardware at the price point that is cheapest and not be locked into a higher price point because it's the only thing that will work.

Uptime and reliability also come into play. With two or more servers, one can fail or be taken off-line for maintenance and the site can stay up. You can't do that with a single server.

Most large websites use load balancers and multiple servers. I used to work for TripAdvisor. They published a great article about the TripAdvisor architecture and how they make it highly scalable with multiple servers.

It is possible to run an advanced service on a single server. One example that I know of is Mailinator. The author published an article about the architecture of Mailinator. He focuses on making his code more efficient rather than buying new servers. This ends up being a limitation that dictates how his service works. It keeps mail only a few hours before the single machine deletes it to make room for more.

Upgrading a single server is known as scaling vertically. Adding more servers is known as scaling horizontally. For more information on this topic, here are some articles that compare the two:

Improve this answer
7
  • 9
    If you have multiple servers (more than a few), and some cpu dies, you have the other servers to keep everything running. If you have 1 server, and that breaks done, you're done.
    – Martijn
    Nov 20, 2014 at 13:51
  • 3
    Another point that people forget is that is not necessarily a good thing to run a server at max capacity or near it. We gauged our servers at a global telecom (that shall remain nameless) at about half capacity maximum as a general rule (no real logic behind it- just watching metrics). You start running into trouble with the compute queue, IO sub-systems, memory addressing and swapping, and so on at some point regardless of hardware capacity simply because the balance between sub-systems can run into conflicts depending upon the OS of course. There are some robust systems that allow more.
    – closetnoc
    Nov 20, 2014 at 16:47
  • @closetnoc I think the best way to describe it is that you are trying to avoid bottlenecks. A properly balanced system could theoretically run at 100% capacity with no ill side-effects, but anything that the system has to wait on (CPU time, I/O, bus transfer, etc...) will cause performance issues. By running your systems at half max capacity, you have found a good spot where you don't run into such bottlenecks.
    – Thebluefish
    Nov 20, 2014 at 18:07
  • 1
    @Thebluefish Yes and no. I am an old system internals guy. Most systems have bottlenecks within the OS and internal hardware that cannot be made up for with faster raids, memory, CPU's etc. As well, there are limits within the OS. Windows was pretty good because it was based upon VMS, but still had limits that could not be tuned like VMS. Linux is obviously better. Some servers are designed with little hardware limitations such as HP which is what we used. But even then, it is never a good idea to run a compute queue at %100 capacity because of increases in interrupts and CPU swaps.
    – closetnoc
    Nov 20, 2014 at 18:38
  • 3
    One other advantage to scaling horizontally: there's only so much electricity, bandwidth, cooling, etc. that you can have directed to a single server. Netflix could have a box with infinite processing power and memory, but it would do them no good without a fat enough pipe to get their traffic out.
    – Chris Hayes
    Nov 21, 2014 at 4:45
31

From Rear Admiral Grace Hopper:

On the building of bigger computers: "In pioneer days they used oxen for heavy pulling, and when one ox couldn't budge a log, they didn't try to grow a larger ox. We shouldn't be trying for bigger computers, but for more systems of computers."

source

Improve this answer
10
  • 1
    I met Grace Hopper a few times in my early career and spent some time with her. She was really something! One cool cat! We all loved her. She was so kind and generous with her time and graces (pun intended). Kudos for quoting her! One up-vote for the way-back. Thanks!
    – closetnoc
    Nov 20, 2014 at 16:36
  • 6
    While this is a relevant quote, this doesn't answer the question. One person's unsubstantiated opinion shouldn't be valuable here.
    – TankorSmash
    Nov 20, 2014 at 21:08
  • 7
    @NoahSpurrier Because it doesn't actually answer any part of the question? It's just one quote that makes an unsubstantiated analogy and doesn't explain why we should shoot for more servers.
    – Chris Hayes
    Nov 21, 2014 at 4:43
  • 2
    I would say that it is a useful answer, but should not be accepted as THE answer as it does not detail the specific reasons. It does, however, clearly state the over arching reason for the principal of load splitting.
    – Ian T. Small
    Nov 21, 2014 at 19:10
  • 1
    @Bobson I'm not at all arguing she's an important player, I'm just saying I'd like to see an answer with some content, instead of a sentence or two that just sounds nice.
    – TankorSmash
    Nov 21, 2014 at 19:51
12

Stephen explains the major consideration to make when deciding on a systems architecture: the tradeoff in vertical and horizontal scaling. I'll add a few other considerations:

  • Separation of concerns: you mention multiple radically different systems: reverse proxies, DB, content servers, etc. From a maintenance and security standpoint it is clearly advantageous to keep these responsibilities spread out over different systems so they can run a different OS (version) if necessary, can be updated separately and don't impact other services when compromised.
  • Content delivery: this is the ultimate goal of a web server and it lends itself well to a distributd model. Systems can be duplicated and spread out geographically so that the latency of long-distance connections is minimised. It also allows for redundancy. Large websites use load balancers (yet another set of servers!) to allow for automatic failover to keep the service up at all times.

There is actually an entire server class that takes vertical scaling to another level: mainframes. They have a variety of advantages (speed, reliability) and disadvantages (cost) but on the whole they are usually used when tremendous volumes of data have to handled via Input-Output processing in what we call transaction processing (think credit card purchases, banking, election and census data). Banks for instance serve sites from vertically scaled web servers while the back-end would end up processing transactions via the mainframe.

Interestingly companies like Paypal and Visa have been moving away from mainframe towards clustered systems of thousands of horizontally scaled systems. In the rapidly evolving digital world even mainframes are hitting the horizontal scaling ceiling:

“With all the availability and performance requirements, we couldn’t keep processing payments on mainframes,

Source: Adam Banks, in ComputerWorldUK

Improve this answer
8

  • Size limit. We like to pretend that a single box with multiple processors, memory chips and disks is uniform. This isn't entirely true, but it's true enough if your numbers don't get too big. There are technical limits on heat, energy, proximity etc. which means there'll always be a practical limit on how big a single server can be.

  • Scalability - there's a vast difference between a single server system, using shared memory for IPC and a multi server system that uses networking or clustering. However the difference between two servers and 200 is considerably smaller - if you've built a system that scales, you can scale it MUCH larger before there's a problem... and if you have, then there's actually no need for a huge single server in the first place.

  • Resilience - One server is a place that one admin might 'oops'. Or there's a physical problem that means service to that whole piece of tin is interrupted. (Datacentre water leak, someone crashing into a rack and knocking it over, that kind of thing). Multiple servers can be distributed within a datacentre, or better yet distributed geographically. And if you're already distributing your app, scaling on 'medium' sized machines is almost always cheaper than the same amount of CPU/memory/IO on a smaller number of bigger machines.

  • Updates - If I patch a server, this may make a service unstable, require a reboot, or otherwise demand some downtime. If I have 4 servers running the same thing, I can take one out of service for a while to do this. And leave it out of service if the patching/update cycle goes wrong.

Improve this answer
7

Let's take the problem at small scale. A tiny office with one server running mail, ActiveDirectory, file share, and the web site for the company.

Hackers hit it and you have to reboot because IIS is messed up. Or Exchange needs an update and a reboot. Or Active Directory got corrupted.

Any of these isolated "one service is down" problems affects the entire server, so anything sharing on that server is going to impact them by virtue of having to reboot or anything else.

Once a real IT guy shows up and sees that server, he's going to recommend splitting them up into separate servers (and having a backup domain controller server).

It's the old adage of "don't put all your eggs in one basket"

Now that philosophy is applied to web servers. If I have just a single web server and I publish my web app (the new MyFaceLink.com) and it gets really popular, I've got new troubles. I can't take the site down to do maintenance while users are on it. And if it crashes or I get too many users, I'm hosed. Even the worlds biggest single server will get overwhelmed by 1 billion FB converts coming over.

So, load balancing comes into play, for the same "eggs in basket" reason. Spread the site across 3 servers, and if one goes down, the remaining 2 handle the capacity. If I need to do patches, I just do one at a time, and nobody notices.

At the simplest, it isn't about price of the mega-server or whether it can truly handle the load (though it can be). It's about single point of failure. Once business is busy enough, and happening 24x7 instead of for 5 users working 8-5, downtime is not acceptable. Scheduled outages are harder to schedule. So, you spread the load.

Improve this answer
1
2

If one tries to have one machine do the work of two, some parts of the machine will need to be larger but run at the same speed, some can stay the same size but will need to run faster, and some will need to be larger and faster. The extent to which it makes sense to combine the roles of smaller machines into a larger one, or split the roles of larger machines into smaller ones, depends in large part upon what sort of scaling would apply to the most expensive parts of the machines. If the workloads of too many machines are combined into one huge colossus, then costs will be dominated by things which would need to get larger and faster to handle increased workloads. Even if the costs of such things were linear with respect to speed and size, doubling the workload would more than double the cost of a machine to process it. The fact that speed increases beyond a certain point result a (much) greater-than-linear cost increase magnify the effect.

There isn't really a fixed point where practicality compels subdivision of labor; depending upon the type of work to be done, a machine that combines the workloads of two might get by with less than twice as much memory, or running at less than twice the speed. On the other hand, the more tasks a machine is given to do, the greater the extent to which memory and speed requirements start scaling linearly with workload. The further one goes beyond that, the greater the increase in relative cost for each doubling of workload.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.