[Python-Dev] Making python C-API thread safe (try 2)

[Jeff Epler]

On Wed, Sep 17, 2003 at 06:22:19AM +0300, Harri Pesonen wrote:
> But wouldn't it be better if Python had real multitasking? Comments like 
> the above mean that you accept Python as it is, fine. But usually people 
> want to make things better when they see that something can be improved. 
> If my roof is leaking, I have two choices: fix it or accept it. If I am 
> able to fix it, then I'll probably do it.
> 
> The old code base really is the problem here. If Python threads really 
> run at the same time in the future, how many current applications stop 
> working because they depend on the fact that only one thread runs at any 
> given time, and do not acquire and release locks as needed? On the other 
> hand, my suggestion probably means that we couldn't have a threading 
> module compatible with thread or threading anyhow (we could have 
> freethreading, with specific functions for inter-thread communication).

I don't think "the roof is leaking".  In the current situation, users
can write threads in the cases where they make sense (and easily share
any data without jumping through hoops) and even derive a performance
benefit when the threads' work is done without the GIL held.  Or, they
can use multiple processes with explicit data sharing, and derive a
performance benefit even when each process does its work with the GIL
held.

In your situation, the first paradigm becomes impossible, and the second
paradigm has been modeled with a "shared nothing" approach which adds
complexity to Python to achieve with threads what the operating system
already offers using processes!

Keep in mind that probably more than of 99.9% of machines out there
have only one CPU anyway, so all you re-claim is the GIL overhead if you
remove it.  Yay, a 2%* performance increase for a huge loss in flexibility.
And processes still win, because you can probably easily hack out the
thread/threading modules, turn the GIL operations into no-ops, and use
fork() to get that 2%* performance increase.

Let me know when you have benchmark figures to the contrary.

Jeff
* Number picked out of the air