Are the critiques in "All the things I hate about Python" valid?

[Chris Angelico]

On Sat, Feb 17, 2018 at 5:25 PM, boB Stepp <robertvstepp at gmail.com> wrote:
> I've just reread everyone's replies and one point you mentioned about
> the GIL caught my eye ...
>
> On Fri, Feb 16, 2018 at 11:16 PM, Chris Angelico <rosuav at gmail.com> wrote:
>
>> Asynchronicity and concurrency are hard. Getting your head around a
>> program that is simultaneously doing two things is inherently tricky.
>> Anything that simplifies this (such as providing atomic and
>> thread-safe operations on high-level data structures) will make this
>> sort of work FAR easier. That's what Python gives you. CPython's
>> current implementation (with the GIL) is reasonably easy to
>> conceptualize, and requires very few other locks to make everything
>> work sanely; all attempts to remove the GIL from CPython have involved
>> other tradeoffs, usually resulting in significant performance
>> penalties on single-threaded code.
>
> I am curious as to what efforts have been attempted to remove the GIL
> and what tradeoffs resulted and why?  Is there a single article
> somewhere that collates this information?  I fear if I try to Google
> this I will get a lot of scattered pieces that I will have to wade
> through to get to what I want to know, where you (or someone else)
> might be able to point me to a good link.  Or kindly summarize
> yourself the relevant information.
>
> Thanks!

No, there isn't a single article, at least not that I know of. A good
word to search for is "gilectomy", which brought me to this talk by
Larry Hastings:

https://www.youtube.com/watch?v=pLqv11ScGsQ

Broadly speaking, what happens is that removing a large-scale lock
(the GIL) requires using a whole lot of small-scale locks. That gives
finer granularity, but it also adds a whole lot of overhead; the CPU
features required for implementing those locks are not fast. With the
GIL, you claim it, and you can do what you like. Without the GIL, you
have to claim a lock on each object you manipulate, or something along
those lines. (Different attempts have gone for different forms of
granularity, so I can't generalize too much here.) That means claiming
and relinquishing a lot more locks, which in turn means a lot more
CPU-level "lock" primitives. That's a lot of overhead.

One of the best ways to multi-thread CPU-intensive work is to push a
lot of the work into an extension library. Take a function like this:

def frobnosticate(stuff):
    magic magic magic
    magic magic more magic
    return result

As long as the returned object is a newly-created one and the
parameter is not mutated in any way, you can do all the work in the
middle without holding the GIL. That can't be done in pure Python, but
in a C function, it certainly can. You still have the coarse-grained
locking of the GIL, you still have all the protection, but you can now
have two threads frobnosticating different stuff at the same time.

ChrisA