[Import-SIG] PEP 451 (ModuleSpec) round 3

[Nick Coghlan]

On 31 Aug 2013 01:31, "Brett Cannon" <brett at python.org> wrote:
>
>
>
>
> On Fri, Aug 30, 2013 at 11:12 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> On 31 August 2013 00:57, Brett Cannon <brett at python.org> wrote:
>> >> So perhaps a better name might be "prepare_module" (by analogy to PEP
>> >> 3115), and have it accept a "reloading" parameter, which is an
>> >> existing module to be reused.
>> >
>> >
>> > Is this to replace create_module() or exec_module()?
>>
>> It replaces create_module.
>>
>> >> The signature would be something like:
>> >>
>> >>     def prepare_module(reloading=None):
>> >>         """Create a module object for execution. Returning None will
>> >> created a default module.
>>
>> Oops, stuffed up the signature. First arg should be the module spec:
>>
>>     def prepare_module(spec, reloading=None):
>>         ...
>>
>> > I can't follow that sentence. =) What does returning None represent?
>>
>> Returning None indicates that the *loader* defines a module creation
>> API, but the particular module being loaded doesn't take advantage of
>> it.
>
>
> IOW returning None means "I don't have anything special to say here, so
do what you want"?
>
>>
>>
>> It's a feature I need for the new extension module loader API, where
>> the creation hook allows the extension module to build a completely
>> custom object (perhaps with additional state). You can request an
>> ordinary module just by not defining the creation hook, and only
>> defining the execution hook (which accepts an already created module).
>
>
> OK, so this is purely for special-cases and not meant to always return
something, just return something when needed.
>
>>
>>
>> By switching to a *preparation* hook, rather than creation, I think we
>> can make this play more nicely with reloading. In the reloading case,
>> the preparation hook would be responsible for checking that the
>> existing object was a suitable execution target.
>
>
> Ah, OK. It's more of a pre-condition check in that case, otherwise it's a
chance to say "use this rather than whatever you default to".
>
>>
>>
>> >>         If *reloading* is set, specifies an existing sys.modules entry
>> >> that is being reloaded.
>> >
>> > As in the key into sys.modules?
>>
>> No, as in the object itself. Technically it doesn't *have* to be in
>> sys.modules, and the loader really shouldn't care if it is or not.
>
>
> That's what I figured.
>
>>
>>
>> >> Must return None or that
>> >>         specific object if reloading is supported.
>> >
>> >
>> > What's "that" supposed to represent?
>>
>> s/that specific object/the passed in object/
>>
>> >> Returning a
>> >> different module object or explicitly raising ImportError
>> >>         indicates that reloading is not supported. (Or perhaps define
>> >> a "ReloadError" subclass of ImportError?)
>> >>         """
>> >
>> >
>> > I'm really not following what this method is supposed to do. Is it
simply
>> > mucking with sys.modules? Is it creating a module to use? If it's the
latter
>> > then how does return None do anything? Are you saying returning None
means
>> > "I didn't do anything special, do what you want"?
>>
>> It replace create_module with something that can also serve as the
>> pre-check for the reloading case.
>
>
> In ModuleSpec.load():
>
> module = self.loader.prepare_module(self)
> if module is None:
>   module = types.ModuleType(self.name)
>
> And in reload():
>
> module = self.loader.prepare_module(self, module_being_reloaded)
>
> That way some custom object can be used, and in the reload case
ImportError can just propagate up if it turns out the module can't be
reloaded.

Yep, that's exactly what I had in mind, although reload would also have an
extra check to ensure the module returned was the same as the one passed in
(that way, in-place reloading support for custom loaders that define
create_module would always be opt-in rather than opt-out).

Talking to Stefan about making this work on the extension module API side
has confirmed my belief that this is the way to go, since it also deals
nicely with placing custom objects in sys.modules.

The one downside is that it means preconditions will be checked twice in
the reload case (once in prepare, once in exec), but I can live with that
for the likely reliability gains in the reloading API.

If it works as well as I hope, I may finally be comfortable with proposing
"imp.reload_fresh" for 3.5 :)

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/import-sig/attachments/20130901/c19399bd/attachment.html>