Parallelization on muli-CPU hardware?

[Jeff Shannon]

Daniel Dittmar wrote:

> Andreas Kostyrka wrote:
>
>> So, IMHO, there are basically the following design decisions:
>> GIL: large granularity
>> MSL: (many small locks) would slow down the overall execution of Python
>>      programs.
>> MSLu: (many small locks, unsafe) inacceptable because it would change
>>       Python experience ;)
>
>
> You forgot GarbageCollection instead of reference counting.


That's an orthogonal issue.  GC/Refcounting may depend on the 
data-locking scheme being thread-safe, but even if one used a GC scheme 
that was completely thread-safe regardless of locking, you'd still need 
data-locking to safely use objects in your own code.  The fact that I 
can be completely confident that the memory space pointed to by a, b, 
and c won't be deallocated behind my back says nothing about whether 
another worker thread may change b in between when I test it and when I 
use it.  This means that in a free-threaded environment (without GIL), 
given

    a = b + c
    d = b + c
    assert (a == d)

I cannot guarantee that the assertion is true, unless I explicitly lock 
all the variables involved -- four locking operations plus four 
unlocking operations.  Locking the local namespace won't work, because 
any of those could be a global reference or a cross-module reference or 
whatever.  One could, I suppose, explicitly acquire/release a 
GIL-granularity lock (process-global "critical section", in essence), 
but that leaves the possibility of mistakenly unprotected variable 
causing a segfault -- again, a violation of one of Python's design 
constraints.  And I don't think that it's practical to have the 
interpreter infer where it'd be safe to release the GIL.

Jeff Shannon
Technician/Programmer
Credit International