python threads

[Michael Hudson]

curt at journyx.com (curt finch) writes:

> We ship a web application completely written in python.
> 
> The application server daemon that sits between apache/IIS and the 
> dbserver would like to take advantage of multi-chip architectures
> and have many threads.  Each thread would be owned by a logged
> in user with it's own db connection and share cached db data with
> other threads.
> 
> My understanding is that python's current threads implementation
> does not really allow for this in the sense that only one thread
> can be running at a time, regardless of the number of chips on
> the machine, and additionally that the threads cannot really
> interrupt each other and still maintain correct state.
> 
> Is that all correct?

Yes.  You can of course run multiple Python processes which will
utilise multiple processors, but that being possible is not a given.
Rearchitecturing your application to make this possible may be easier
than making Python free threaded[1], though.

> If it is correct, it would help the scalability with regards to
> thruput of our app to get this fixed and we'd be willing to help
> with both the coding, the testing, and the cost of getting this
> accomplished.

Note that this is not just a matter of a "fix"; there are some truly
subtle issues here, and also the desire to not harm performance in the
single-threaded case too much.  Googling for "python free threading"
should yield a few thousand interesting reads (and indeed the first
hit raises some of those subtle issues I was talking about).

> Who would be the right people to talk to about this?

Like Oleg said, python-dev, or the PythonLabs team at Zope Corp.

Cheers,
M.

[1] "free threading" is the name this issue traditionally goes by in
    the Python community.  Don't really know why.