Will Python 3.0 remove the global interpreter lock (GIL)

[Paul Boddie]

On 20 Sep, 00:59, TheFlyingDutchman <zzbba... at aol.com> wrote:
>
> Paul it's a pleasure to see that you are not entirely against
> complaints.

Well, it seems to me that I'm usually the one making them. ;-)

> The very fastest Intel processor of the last 1990's that I found came
> out in October 1999 and had a speed around 783Mhz. Current fastest
> processors are something like 3.74 Ghz, with larger caches.

True, although you're paying silly money for a 3.8 GHz CPU with a
reasonable cache. However, as always, you can get something not too
far off for a reasonable sum. When I bought my CPU two or so years
ago, there was a substantial premium for as little as 200 MHz over the
3.0 GHz CPU I went for, and likewise a 3.4 GHz CPU seems to be had for
a reasonable price these days in comparison to the unit with an extra
400 MHz.

Returning to the subject under discussion, though, one big difference
between then and now is the availability of dual core CPUs, and these
seem to be fairly competitive on price with single cores, although the
frequencies of each core are lower and you have to decide whether you
believe the AMD marketing numbers: is a dual 2.2 GHz core CPU "4200+"
or not, for example? One can argue whether it's better to have two
cores, especially for certain kinds of applications (and CPython,
naturally), but if I were compiling lots of stuff, the ability to do a
"make -j2" and have a decent speed-up would almost certainly push me
in the direction of multicore units, especially if the CPU consumed
less power. And if anyone thinks all this parallelism is just
hypothetical, they should take a look at distcc to see a fairly clear
roadmap for certain kinds of workloads.

> Memory is also faster and larger. It appears that someone running a non-GIL
> implementation of CPython today would have significantly faster
> performance than a GIL CPython implementation of the late 1990's.
> Correct me if I am wrong, but it seems that saying non-GIL CPython is
> too slow, while once valid, has become invalid due to the increase in
> computing power that has taken place.

Although others have picked over these arguments, I can see what
you're getting at: even if we take a fair proportion of the increase
in computing power since the late 1990s, rather than 100% of it,
CPython without the GIL would still faster and have more potential for
further speed increases in more parallel architectures, rather than
running as fast as possible on a "sequential" architecture where not
even obscene amounts of money will buy you significantly better
performance. But I don't think it's so interesting to consider this
situation as merely a case of removing the GIL and using lots of
threads.

Let us return to the case I referenced above: even across networks,
where the communications cost is significantly higher than that of
physical memory, distributed compilation can provide a good
performance curve. Now I'm not arguing that every computational task
can be distributed in such a way, but we can see that some
applications of parallelisation are mature, even mainstream. There are
also less extreme cases: various network services can be scaled up
relatively effectively by employing multiple processes, as is the UNIX
way; some kinds of computation can be done in separate processes and
the results collected later on - we do this with relational databases
all the time. So, we already know that monolithic multithreaded
processes are not the only answer. (Java put an emphasis on extensive
multithreading and sandboxing because of the requirements of running
different people's code side-by-side on embedded platforms with
relatively few operating system conveniences, as well as on Microsoft
Windows, of course.)

If the programmer cost in removing the GIL and maintaining a GIL-free
CPython ecosystem is too great, then perhaps it is less expensive to
employ other, already well-understood mechanisms instead. Of course,
there's no "global programmer lock", so everyone interested in doing
something about removing the GIL, writing their own Python
implementation, or whatever they see to be the solution can freely do
so without waiting for someone else to get round to it. Like those
more readily parallelisable applications mentioned above, more stuff
can get done provided that everyone doesn't decide to join the same
project. A lesson from the real world, indeed.

Paul