1 Million users.. I can't Scale!!

[simonwittber at gmail.com]

yoda wrote:
> I'm considering moving to stackless python so that I can make use of
> continuations so that I can open a massive number of connections to the
> gateway and pump the messages out to each user simultaneously.(I'm
> thinking of 1 connection per user).

This won't help if your gateway works synchronously. You need to
determine what your gateway can do. If it works asynchronously,
determine the max bandwidth it can handle, then determine how many
messages you can fit into 4 seconds of that bandwidth. That should
provide you with a number of connections you can safely open and still
recieve acceptable response times.

> My questions therefore are:
> 1)Should I switch to stackless python or should I carry out experiments
> with mutlithreading the application?

You will build a more scalable solution if you create a multi process
system. This will let you deploy across multiple servers, rather than
CPU's. Multithreading and Multiprocessing will only help you if your
application is IO bound.

If your application is CPU bound, multiprocessing and multithreading
will likely hurt your performance. You will have to build a parallel
processing application which will work across different machines. This
is easier than it sounds, as Python has a great selection of IPC
mechanisms to choose from.

> 2)What architectural suggestions can you give me?

Multithreading will introduce extra complexity and overhead. I've
always ended up regretting any use of multithreading which I have
tried. Avoid it if you can.

> 3)Has anyone encountered such a situation before? How did you deal with
> it?

Profile each section or stage of the operation. Find the bottlenecks,
and reduce it whichever way you can. Check out ping times. Use gigabit
or better. Remove as many switches and other hops between machines
which talk to each other.

Cache content, reuse it if you can. Pregenerate content, and stick it
in a cache. Cache cache cache! :-)

> 4)Lastly, and probably most controversial: Is python the right language
> for this? I really don't want to switch to Lisp, Icon or Erlang as yet.

Absolutely. Python will let you easily implement higher level
algorithms to cope with larger problems.

Sw.