[Import-SIG] PEP 489: Multi-phase extension module initialization; version 5

[Eric Snow]

On Wed, May 20, 2015 at 4:16 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 21 May 2015 at 00:56, Eric Snow <ericsnowcurrently at gmail.com> wrote:
>> On Wed, May 20, 2015 at 4:55 AM, Petr Viktorin <encukou at gmail.com> wrote:
>>> The point is that exec_module doesn't a priori depend on the module
>>> being in sys.modules, which I think is a good thing.
>>
>> Well, there's an explicit specification about how sys.modules is used
>> during loading.  For post-exec sys.modules lookup specifically,
>> https://docs.python.org/3.5//reference/import.html#id2.  The note in
>> the language reference says that it is an implementation detail.
>> However, keep in mind that this PEP is a CPython-specific proposal.
>>
>> That said, I'm only -0 on not matching the sys.modules lookup behavior
>> of module loading.  It could be okay if we were to document the
>> behavior clearly.  My concern is with having different semantics even
>> if it only relates to a remote corner case.  It may be a corner case
>> that someone will rely on.
>
> We *will* match the semantics for the *overall* loading process. What
> Petr is saying is that *while* executing the "execution slots",
> they'll all receive the object returned by Py_mod_create (or the
> automatically created module if that slot is not defined), rather than
> any replacement injected into sys.modules.
>
> There's no Python level parallel for that "multiple execution slots"
> behaviour, so it makes sense to define the semantics based on
> simplicity of implementaiton and the fact we want to encourage the use
> of Py_mod_create for extension modules over sys.modules injection.

I was thinking along those same lines.  I'm okay with that rationale.
The PEP should be updated to clarify this point and its rationale.

>
>>> No other implementation of exec_module checks sys.modules in the middle
>>> of its operation. So I think the semantics are consistent.
>>
>> I was thinking of each exec slot as a parallel to Loader.exec_module.
>> Thus I was expecting the same sys.modules lookup behavior that you get
>> during module loading.  That's why I would expect the module to get
>> updated to sys.modules[spec.name] after each exec slot runs.
>
> I changed my mind when Petr posted the clarification that this is
> really just a matter of iterating over the defined slots in the
> loader's exec_module method, and calling any of them that are defined
> as execution slots (for the time, just Py_mod_exec).
>
> The entirety of a Python module runs in the same module namespace,
> regardless of what is done with sys.modules, so having all execution
> slots called with the same object is the extension module equivalent.

Sounds good.  Thanks for clarifying.

-eric