ESR "Waning of Python" post

[Peter J. Holzer]

On 2018-10-12 14:07:56 -0500, Tim Daneliuk wrote:
> On 10/11/2018 12:15 AM, Gregory Ewing wrote:
> > But it's not like that at all. As far as I know, all the
> > attempts that have been made so far to remove the GIL have
> > led to performance that was less than satisfactory. It's a
> > hard problem that we haven't found a good solution to yet.
> 
> Do you happen to have a reference that explains what the issues are
> for GIL removal?

I'm certainly not an expert on CPython internals, but what I've gathered
from talks and discussions on the topic is that the CPython interpreter
accesses shared state a lot (the reference count fields are an obvious
example, but there are others), so to remove the GIL you would have to
replace it with a myriad of small locks which are taken and released all
the time - this adds a lot of overhead compared to a single lock which
is basically always taken and just released before blocking syscalls. 

(If you ask your favourite search engine for "gilectomy", you'll
probably find lots of info)

It might be better to start from scratch: Design a new VM suitable for
Python which can run mostly without locks. But this is of course a lot
of work with no guarantee of success. (JPython and IronPython did
something similar, although they didn't design new VMs, but reused VMs
designed for other languages).

        hp

-- 
   _  | Peter J. Holzer    | we build much bigger, better disasters now
|_|_) |                    | because we have much more sophisticated
| |   | hjp at hjp.at         | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20181013/7bf55625/attachment.sig>