Will Python 3.0 remove the global interpreter lock (GIL)

[Chris Mellon]

On 9/19/07, TheFlyingDutchman <zzbbaadd at aol.com> wrote:
> On Sep 19, 5:08 pm, "Terry Reedy" <tjre... at udel.edu> wrote:
> > "Terry Reedy" <tjre... at udel.edu> wrote in message
>
> This is a little confusing because google groups does not show your
> original post (not uncommon for them to lose a post in a thread - but
> somehow still reflect the fact that it exists in the total-posts
> number that they display) that you are replying to.
>
>
> >
> > This assumes that comparing versions of 1.5 is still relevant.  As far as I
> > know, his patch has not been maintained to apply against current Python.
> > This tells me that no one to date really wants to dump the GIL at the cost
> > of half Python's speed.  Of course not.  The point of dumping the GIL is to
> > use multiprocessors to get more speed!  So with two cores and extra
> > overhead, Stein-patched 1.5 would not even break even.
> >
> > Quad (and more) cores are a different matter.  Hence, I think, the
> > resurgence of interest.
>
> I am confused about the benefits/disadvantages of the "GIL removal".
> Is it correct that the GIL is preventing CPython from having threads?
>

No. Python has threads, and they're wrappers around true OS level
system threads. What the GIL does is prevent *Python* code in those
threads from running concurrently.

> Is it correct that the only issue with the GIL is the prevention of
> being able to do multi-threading?
>

This sentence doesn't parse in a way that makes sense.

> If you only planned on writing single-threaded applications would GIL-
> removal have no benefit?
>

Yes.

> Can threading have a performance benefit on a single-core machine
> versus  running multiple processes?
>

A simple question with a complicated answer. With the qualifier "can",
I have to say yes to be honest although you will only see absolute
performance increases on a single core from special purposed APIs that
call into C code anyway - and the GIL doesn't effect those so GIL
removal won't have an effect on the scalability of those operations.

Pure CPU bound threads (all pure Python code) will not increase
performance on a single core (there's CPU level concurrency that can,
but not OS level threads). You can improve *perceived* performance
this way (latency at the expense of throughput), but not raw
performance.

Very, very few operations are CPU bound these days, and even fewer of
the ones were Python is involved. The largest benefits to the desktop
user of multiple cores are the increase in cross-process performance
(multitasking), not single applications.

Servers vary more widely. However, in general, there's not a huge
benefit to faster threading when you can use multiple processes
instead. Python is not especially fast in terms of pure CPU time, so
if you're CPU bound anyway moving your CPU bound code into C (or
something else) is likely to reap far more benefits - and sidestepping
the GIL in the process.

In short, I think any problem that would be directly addressed by
removing the GIL is better addressed by other solutions.

> > So now this question for you:  "CPython 2.5 runs too slow in 2007: true or
> > false?"
>
> I guess I gotta go with Steven D'Aprano - both true and false
> depending on your situation.
>
> > If you answer false, then there is no need for GIL removal.
>
> OK, I see that.
>
> > If you answer true, then cutting its speed for 90+% of people is bad.
>
> OK, seems reasonable, assuming that multi-threading cannot be
> implemented without a performance hit on single-threaded applications.
> Is that a computer science maxim - giving an interpreted language
> multi-threading will always negatively impact the performance of
> single-threaded applications?
>

It's not a maxim, per se - it's possible to have lockless concurrency,
although when you do this it's more like the shared nothing process
approach - but in general, yes. The cost of threading is the cost of
the locking needed to ensure safety, and the amount of locking is
proportional to the amount of shared state. Most of the common uses of
threading in the real world do not improve absolute performance and
won't no matter how many cores you use.

> >
> > | Most people are not currently bothered by the GIL and would not want its
> > | speed halved.
> >
> > And another question: why should such people spend time they do not have to
> > make Python worse for themselves?
> >
> Saying they don't have time to make a change, any change, is always
> valid in my book. I cannot argue against that. Ditto for them saying
> they don't want to make a change with no explanation. But it seems if
> they make statements about why a change is not good, then it is fair
> to make a counter-argument. I do agree with the theme of Steven
> D'Aprano's comments in that it should be a cordial counter-argument
> and not a demand.
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>