Multiple scripts versus single multi-threaded script

[Dave Angel]

On 3/10/2013 12:50, Chris Angelico wrote:

> On Fri, Oct 4, 2013 at 2:41 AM, Roy Smith <roy at panix.com> wrote:
>> The downside to threads is that all of of this sharing makes them much
>> more complicated to use properly.  You have to be aware of how all the
>> threads are interacting, and mediate access to shared resources.  If you
>> do that wrong, you get memory corruption, deadlocks, and all sorts of
>> (extremely) difficult to debug problems.  A lot of the really hairy
>> problems (i.e. things like one thread continuing to use memory which
>> another thread has freed) are solved by using a high-level language like
>> Python which handles all the memory allocation for you, but you can
>> still get deadlocks and data corruption.
>
> With CPython, you don't have any headaches like that; you have one
> very simple protection, a Global Interpreter Lock (GIL), which
> guarantees that no two threads will execute Python code
> simultaneously. No corruption, no deadlocks, no hairy problems.
>
> ChrisA

The GIL takes care of the gut-level interpreter issues like reference
counts for shared objects.  But it does not avoid deadlock or hairy
problems.  I'll just show one, trivial, problem, but many others exist.

If two threads process the same global variable as follows,
    myglobal = myglobal + 1

Then you have no guarantee that the value will really get incremented
twice.  Presumably there's a mutex/critsection function in the threading
module that can make this safe, but once you use it in two different
places, you raise the possibility of deadlock.

On the other hand, if you're careful to have the thread use only data
that is unique to that thread, then it would seem to be safe.  However,
you still have the same risk if you call some library that wasn't
written to be thread safe.  I'll assume that print() and suchlike are
safe, but some third party library could well use the equivalent of a
global variable in an unsafe way.



-- 
DaveA