threading support in python

[Paul Rubin]

Steve Holden <steve at holdenweb.com> writes:
> You write as though the GIL was invented to get in the programmer's
> way, which is quite wrong. It's there to avoid deep problems with
> thread interaction. Languages that haven't bitten that bullet can bite
> you in quite nasty ways when you write threaded applications.

And yet, Java programmers manage to write threaded applications all
day long without getting bitten (once they're used to the issues),
despite usually being less skilled than Python programmers ;-).

> Contrary to your apparent opinion, the GIL has nothing to do with
> reference-counting.

I think it does, i.e. one of the GIL's motivations was to protect the
management of reference counts in CPython, which otherwise wasn't
thread-safe.  The obvious implementation of Py_INCREF has a race
condition, for example.  The GIL documentation at

   http://docs.python.org/api/threads.html

describes this in its very first paragraph.

> > Will the madness never end?
> 
> This reveals an opinion of the development team that's altogether too
> low. I believe the GIL was introduced for good reasons.

The GIL was an acceptable tradeoff when it was first created in the
previous century.  First of all, it gave a way to add threads to the
existing, non-threadsafe CPython implementation without having to
rework the old code too much.  Second, Python was at that time
considered a "scripting language" and there was less concern about
writing complex apps in it, especially multiprocessing apps.  Third,
multiprocessor computers were themselves exotic, so people who wanted
to program them probably had exotic problems that they were willing to
jump through hoops to solve.

These days, even semi-entry-level consumer laptop computers have dual
core CPU's, and quad Opteron boxes (8-way multiprocessing using X2
processors) are quite affordable for midrange servers or engineering
workstations, and there's endless desire to write fancy server apps
completely in Python.  There is no point paying for all that
multiprocessor hardware if your programming language won't let you use
it.  So, Python must punt the GIL if it doesn't want to keep
presenting undue obstacles to writing serious apps on modern hardware.