[Import-SIG] Running C extension modules using -m switch

[Petr Viktorin]

On 05/19/2017 12:24 PM, Nick Coghlan wrote:
> On 18 May 2017 at 22:50,  <gmarcel.plch at gmail.com> wrote:
>> Greetings,
>>
>> This has been already sent to python-ideas, but since I got no
>> response, so I'm re-sending it to this SIG. I would welcome any
>> comments.
> 
...
>>
>> This new method calls into the _imp module, which executes the module
>> as a script.
>> I can see two ways of doing this. Both expect that the module uses PEP
>> 489 multi-phase initialization.
> 
> The main reason I didn't immediately reply is that I had a vague
> recollection of thinking this could be done *without* a new method on
> loaders, but I needed to refresh my memory of our plans in that
> regard.
> 
> I've now done that, and I'm pretty sure the unwritten plan was to
> change runpy to do something like the following:
> 
>      spec = importlib.find_spec(modname)
>      created = spec.loader.create_module()
>      if created is not None:
>          raise RuntimeError("Cannot use customised module instance as __main__")
>      spec.loader.exec_module(main_mod)
> 
> That's oversimplified quite a bit, but it gives the general idea.

The problem here is that for extension modules, 
`spec.loader.create_module()` returns None. It can't: the PyModuleDef is 
attached to the returned module, and that's where the Py_mod_exec 
function is stored. This is unlike with source modules, where the code 
is always looked up by module name.

So I see these ways to make things work:
- Make spec.loader.create_module() return None if Py_mod_create is 
missing, and either store the PyModuleDef on the loader (which doesn't 
really fit in with how importlib works), or re-load it from the .so 
every time (which seems wasteful and hacky).
- Make exec_module take two modules – the module in whose namespace to 
run, and the module whose code should run. Or make it take a module and 
a spec of a different module. This would be an API change, affecting all 
third-party loaders, so it's out.
- Add a new loader method taking two modules (or module and spec) as above
- Add a new loader method to explicitly run as main