1 Million users.. I can't Scale!!

[Paul Rubin]

"yoda" <nochiel at gmail.com> writes:
> Currently, the content is generated and a number of SMS per user are
> generated. I'll have to measure this more accurately but a cursory
> glance indicated that we're generting approximately 1000 sms per
> second. (I'm sure this can't be right.. the parser\generator should be
> faster than that:)

Don't be sure.  Try some profiling, and maybe Psyco, C extensions,
etc.  Python is many things but it's not terribly fast at
interpreting.  It sounds like you have more bottlenecks than just the
python app though, starting with the SMS gateway issue that you just
mentioned.  If you get more gateway bandwidth and your application can
split across multiple boxes, then the simple short-term fix is buy
more boxes.  Longer term you need to re-examine the sw architecture
and possibly make changes.