[Python-Dev] PEP: Post import hooks

[Phillip J. Eby]

At 02:28 AM 1/16/2008 +0100, Christian Heimes wrote:
>Phillip J. Eby wrote:
> > At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote:
> >> My code queues up new hooks while a sequence of hooks is processed. It
> >> makes sure that hooks for a parent aren't called in the middle of a
> >> child's hook chain.
> >
> > Notice that that's not necessary with the notification algorithm I gave,
> > since the list in post_import_hooks suffices as a queue.  So, just as in
> > peak.util.imports, the registration code doesn't need to know whether
> > callbacks are being run; it only needs to know whether they're *finished*.
>
>Are you sure your proposed algorithm and output match for the test case?
>I'm confident I got it right in C but I'm getting a different output.

I guess it's not right then.  ;-)  Though I shouldn't make fun, since 
it turns out that my code sketch was not a correct translation of 
peak.util.imports.  (See below.)


>Without the extra imp.notify_module_loaded('a.b') in func_a1(mod)::
>
>    ['func_a1', 'func_a2', 'func_ab1', 'func_ab2', 'func_ab3']
>
>
>With the extra imp.notify_module_loaded('a.b') in func_a1(mod)::
>
>    ['func_a1', 'func_ab1', 'func_ab2', 'func_ab3', 'func_a2']

Right - that's why I put it in there, to foil trivial implementations 
that don't really satisfy the invariant.


>I can't see how your implementation results in the first output when
>func_a1() calls the notification method.

Hm, you're right, my implementation sketch waits too long to set the 
__notified__ flag.  It should have read:

      def notify(name):
          try:
              module = sys.modules[name]
          except KeyError:
              raise ImportError("Module %s has not been imported" % (name,))
          if module.__notified__:
              return
          try:
              module.__notified__ = True
              if '.' in name:
                  notify(name[:name.rfind('.')])
              for callback in post_import_hooks[name]:
                 callback(module)
          finally:
              post_import_hooks[name] = None

That is, module.__notified__ has to be set *before* the recursive 
notification call.  This effectively happens in peak.util.imports 
now, except that __notified__ isn't an explicit attribute, just a 
side effect of other module state changes.


>I'm aware of the implications and my code already uses the lock. The
>PyImport_NotifyLoaded() method excepts to be called with the importer
>lock acquired. So I'm locking the importer lock in
>imp_notify_module_loaded(). The PyImport_RegisterPostImportHook() method
>does the locking before it accesses sys.modules and sys.post_import_hooks.

Great!