Load balancing and passing sockets; was: Re: Google and Python

[Bryan Olson]

Hendrik van Rooyen wrote:
> "Nick Craig-Wood" wrote: [about passing sockets between processes]
>> It is trivial to pass a socket to a new thread or a forked child - you
>> don't need this mechanism for that.  It doesn't work on different
>> machines though - it has to be on the same machine.

> 
> How does a very large volume site work then? - there must be some
> way of sharing the load without bottlenecking it through one machine?

Several ways. The Domain Name System can provide multiple IP
addresses for the same name. IP addresses often often lead
to HTTP "reverse proxies" that shoot back cached replies to
common simple requests, and forward the harder ones to the
file/application servers, with intelligent load balancing.

The services are surprisingly basic, and some excellent
software is free:

   http://en.wikipedia.org/wiki/Round_robin_DNS
   http://en.wikipedia.org/wiki/Reverse_proxy
   http://en.wikipedia.org/wiki/Squid_proxy


Web apps tend to scale just great, except when they need
data that is both shared and modifiable.


-- 
--Bryan