1 Million users.. I can't Scale!!

[Tim Daneliuk]

yoda wrote:

> Hi guys,
> My situation is as follows:
> 
> 1)I've developed a service that generates content for a mobile service.
> 2)The content is sent through an SMS gateway (currently we only send
> text messages).
> 3)I've got a million users (and climbing).
> 4)The users need to get the data a minimum of 5 seconds after it's
> generated. (not considering any bottlenecks external to my code).
> 5)Generating the content takes 1 second.
> 

We need more information on just where the bottleneck might be.
There are any number of places that things could be getting
choked up and you need to get a profile of where things are
falling down before trying to fix it.

However, that said:

A possible culprit is session setup/teardown - I'm assuming connection
to the SMS gateway is connection-oriented/reliable, not datagram-based.
I suggest this because this is quite often the culprit in connection-
oriented performance problems.

If this is the case, you need to preestablish sessions and pool them for
reuse somehow so that each and every message transmission does not incur
the overhead of message setup and teardown to the gateway. It is a good
idea to make that session pooling logic adaptive. Have it start with a
minimum number of preestablished sessions to the gateway and then
monitor the message 'highwater' mark. As the system becomes starved for
sessions, alocate more to the pool. As system utililization declines,
remove spare sessions from the pool until the count falls back to the
initial minimum

Write the pooling manager to be able to configure both the initial
session count as well as the interval for adjusting that count up and
down (i.e. Over what interval you will 'integrate' the function that
figures out just how many sessions the pool needs). Too short an interval
and the system will throw itself into feedback hystersis trying to
figure out just many sessions you need. Too long an interval, and the
system will exhbit poor response to changing load.

P.S. My firm does consultancy for these kinds of problems. We're always
      looking for a great new customer.

Always-Developing-New-Business-ly Yours,

----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/