[Python-ideas] solving multi-core Python

[Nick Coghlan]

On 24 June 2015 at 15:33, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Sun, Jun 21, 2015 at 7:06 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Nick Coghlan schrieb am 21.06.2015 um 03:28:
>>> * there may be restrictions on some extension modules that limit them
>>> to "main interpreter only" (e.g. if the extension module itself isn't
>>> thread-safe, then it will need to remain fully protected by the GIL)
>>
>> Just an idea, but C extensions could opt-in to this. Calling into them has
>> to go through some kind of callable type, usually PyCFunction. We could
>> protect all calls to extension types and C functions with a global runtime
>> lock (per process, not per interpreter) and Extensions could set a flag on
>> their functions and methods (or get it inherited from their extension types
>> etc.) that says "I don't need the lock". That allows for a very
>> fine-grained transition.
>
> Exactly.  PEP 489 helps facilitate opting in as well, right?

Yep, as PEP 489 requires subinterpreter compatibility as a
precondition for using multi-phase initialisation :)

Cheers,
Nick.

P.S. Technically, what it actually requires is support for "multiple
instances of the module existing in the same process at the same
time", as it really recreates the module if you remove it from
sys.modules and import it again, unlike single phase initialisation.
But that's a mouthful, so "must support subinterpreters" is an easier
shorthand.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia