[Python-Dev] PEP: Post import hooks

[Phillip J. Eby]

At 10:10 PM 1/11/2008 +0100, Christian Heimes wrote:
>Phillip J. Eby wrote:
> > *sigh*.  We seem to be getting further and further off course,
> > here.  The more different you make the semantics of the system, the
> > harder it will be to verify that it's doing the right thing without
> > having real field experience with the new approach.
>
>*sigh*, too. :/
>
>This discussion has neither helping me nor you. Could you please write
>an unit test or two to show me exactly what my implementation is doing
>wrong and how you think the callbacks should be called. I know a simple
>test won't proof the correctness of the implementation but a failing
>test would at least show the incorrectness.

when_imported('a.b')(func_ab1)
when_imported('a.b')(func_ab2)

@when_imported('a')
def func_a1(module_a):
     when_imported('a.b')(func_ab3)
     notify_module('a.b')   # <- this is here to foil trivial implementations

when_imported('a')(func_a2)
notify_module('a.b')

This should produce the calling sequence:

func_a1, func_a2, func_ab1, func_ab2, func_ab3.



>I'm still not sure which way is the correct way in your opinion and I
>hate guessing what you are trying to explain to me.

The invariants to ensure are:

1. notification is only done once for a given module, ever, even if 
the notification function is called more than once, even if it's 
called during notifications for that module

2. notifications for a child module/package may not begin until the 
notifications for the parent package have begun

3. two registrations for the same module must always be invoked in 
the same order as they were registered, even if some of the 
registrations are done during notification.

In order to implement these invariants, you will have to have a way 
to know whether notifications have been begun for a given module.  In 
peak.util.imports, the module objects effectively keep track of this, 
although they don't have a specific flag.  For the Python 
implementation, you could add a __notified__ field to module objects, 
and implement the notify function thus:

     def notify(name):
         try:
             module = sys.modules[name]
         except KeyError:
             raise ImportError("Module %s has not been imported" % (name,))
         if module.__notified__:
             return
         if '.' in name:
             notify(name[:name.rfind('.')])
         try:
             module.__notified__ = True
             for callback in post_import_hooks[name]:
                callback(module)
         finally:
             post_import_hooks[name] = None

Of course, __notified__ would actually be a structure slot, rather 
than an attribute, so as to avoid any attribute lookup issues with 
module subtypes (e.g. lazy modules).

The register function would simply append a hook to the entry in 
post_import_hooks if it's not None, or call the hook otherwise.

With this implementation, I could make a version of peak.util.imports 
that did its own lazy modules, but used the base system for all the hooks.