Let's take the problem at small scale. A tiny office with one server running mail, ActiveDirectory, file share, and the web site for the company.
Hackers hit it and you have to reboot because IIS is messed up. Or Exchange needs an update and a reboot. Or Active Directory got corrupted.
Any of these isolated "one service is down" problems affects the entire server, so anything sharing on that server is going to impact them by virtue of having to reboot or anything else.
Once a real IT guy shows up and sees that server, he's going to recommend splitting them up into separate servers (and having a backup domain controller server).
It's the old adage of "don't put all your eggs in one basket"
Now that philosophy is applied to web servers. If I have just a single web server and I publish my web app (the new MyFaceLink.com) and it gets really popular, I've got new troubles. I can't take the site down to do maintenance while users are on it. And if it crashes or I get too many users, I'm hosed. Even the worlds biggest single server will get overwhelmed by 1 billion FB converts coming over.
So, load balancing comes into play, for the same "eggs in basket" reason. Spread the site across 3 servers, and if one goes down, the remaining 2 handle the capacity. If I need to do patches, I just do one at a time, and nobody notices.
At the simplest, it isn't about price of the mega-server or whether it can truly handle the load (though it can be). It's about single point of failure. Once business is busy enough, and happening 24x7 instead of for 5 users working 8-5, downtime is not acceptable. Scheduled outages are harder to schedule. So, you spread the load.