Embedding Python, threading and scalability

[Donn Cave]

Quoth Jeff Epler <jepler at unpythonic.net>:
| On Thu, Jul 10, 2003 at 11:14:47AM +0800, Simon Wittber (Maptek) wrote:
|> Seriously though, this is an issue, which is a major hurdle Python *has*
|> to cross if it is ever going to be seriously considered for use on large
|> projects, on large SMP hardware.
|
| Has anybody proposed "how to get there from here" for this problem
| (useful multithreading of non-blocking pure Python code)?  I'm not
| bright enough to see how, that's for sure.  Especially if you are
| talking about an incremental approach, not the mythical "Python 3000".
| What I mean is that you have to work this magic *and* somehow let
| existing modules written in C work, with at worst a recompile.  (as
| someone who doesn't *need* threads, but works on a project with piles of
| Python modules written in C, that's my bias anyway)

Ha - I knew I'd find an answer for this if I searched for "free threading" -
and among other hits, I found this very thread!  So in Andrew Dalke's words:

> Ahh, the Global Interpreter Lock (GIL). 
>
> Years ago, Greg Stein had a version of Python 1.4 running with no GIL. 
>
> http://www.python.org/ftp/python/contrib-09-Dec-1999/System/threading.README
>
> Search for "free threading" to get more hits on this topic. 

At any rate, it sure isn't because Guido can't scare up an SMP machine.

| I've written Python programs that use threads in a way that was expedient
| to get a user interface running without long seizures, while I've never
| written a thread in tcl or perl.  OTOH if I'd had to treat everything
| as explicit message passing (a la tcl threading), I'd have just found
| another way (like "after idle" and "update" in tcl)

Hm, not sure what you're alluding to here.  I actually kind of like an
I/O thread dispatch model myself, but I don't think Python cares either
way nor would it if the GIL were abolished.

| Jython will give you Java's thread model today, for Python code, won't
| it?  Back when I benchmarked it, Jython code and Python code ran at
| fairly similar speeds (in pybench), if only you had a machine that could
| keep the Jython interpreter from swapping...

Don't know, but I noticed in another thread among the hits that Ype Kingma
asserts that Jython does indeed support fine grained multithreading.

	Donn Cave, donn at drizzle.com