[melbourne-pug] multi threaded/process web apps

[William ML Leslie]

On 11 July 2011 12:58, James Alford <mydnite1 at gmail.com> wrote:
> Ok, that was quick and rightfully flamed for not being specific.
>
> This is the scenario I have solved in the past on the ipad but with
> php it is a litte complex.
>
> Scenario:
> 1. Incoming web service request
> 2. Data is processed
> 3. Outgoing web request sent.
> 4. Web service request response sent
>
> With step 3, I would fired an event or spawn another thread to do the
> outgoing request.  With php it is possible but I have to start a
> separate process as a php cli process.  With the ipad I'm easily able
> to fire an event that handles this in a separate thread.

This list is more or less your twisted callback chain.  The nature of
"Data is processed" determines exactly how you make it scale with
twisted - for example, if it is a bunch of queries you want to run,
you could use the adbapi on top of your existing database driver.  If
you want to interact with a number of internal services, maybe AMP is
the way to go.  Otherwise, you could start a bunch of subprocesses.

Starting a separate thread per request scales (in performance) poorly,
and that is even more true on cpython, so it should be avoided where
possible.  Starting separate processes or interacting asynchronously
with the web server tends to be faster besides resulting in code that
is easier to comprehend.

-- 
William Leslie