Clearing a session and reload() problem (with repro error)

[Rafe]

On Sep 10, 2:28 pm, "Gabriel Genellina" <gagsl-... at yahoo.com.ar>
wrote:
> En Wed, 10 Sep 2008 00:56:43 -0300,Rafe<rafesa... at gmail.com> escribió:
>
>
>
> > On Sep 9, 11:03 pm, "Gabriel Genellina" <gagsl-... at yahoo.com.ar>
> > wrote:
> >> En Mon, 08 Sep 2008 05:37:24 -0300,Rafe<rafesa... at gmail.com> escribió:
> >> ...
> >> This dependency between modules, applied to all modules in your project,
> >> defines a "dependency graph". In some cases, one can define a partial
> >> ordering of its nodes, such that no module depends on any other module
> >> *after* it (it may depend only on modules *before* it). Look for
> >> "topological sort".
>
> >> Doing that in the generic case is not easy. If you *know* your
> >> dependencies, reload the modules in the right order by hand.
> >> ...
> > I was hoping there would be a way to just wipe out the module cache
> > and let it get rebuilt by executing my code (since I'm not using
> > reload as part of my program, but rather, to test it in an environment
> > where I cannot restart the Python session).
>
> Ok, I think the following sequence *might* work:
>
> - replace the __import__ and reload builtins with a custom callable  
> object. This way you can hook into any import attempt. The object should  
> keep a list of already reloaded modules. When a module is imported: if it  
> is already in the list, just delegate to the original __import__; if it is  
> not in the list, locate the module in sys.modules and reload it.
>
> - iterate over sys.modules and reload the desired modules, as you did in  
> your previous attempt.
>
> - restore the original __import__ function.
>
> This way, you effectively transform any import statement into a recursive  
> reload (for the first time); subsequent imports of the same module behave  
> as usual. This may work for you, or perhaps not, or it may screw all your  
> running environment up, or even cause the next global thermonuclear war...  
> (I hope not!)
>
> Note: some modules don't work well with reload(). A common case: global  
> mutable values, like a list of objects which starts empty:
>         my_list = []
> To make it more "reload friendly", use this:
>         try: my_list
>         except NameError: my_list = []
> (this way the list will keep its previous values).
>
> The example below shows how to hook into the import mechanism - it just  
> prints the module being imported. Implementing the functionality outlined  
> above is left as an exercise to the reader :)
>
> py> class ImportHook(object):
> ...     _orig_import = None
> ... #
> ...     def __call__(self, name, globals={}, locals={}, fromlist=[],  
> level=-1):
> ...         if fromlist:
> ...             print "-> from %s import %s" % (name, ','.join(fromlist))
> ...         else:
> ...             print "-> import %s" % name
> ...         return self._orig_import(name, globals, locals, fromlist,  
> level)
> ... #
> ...     def hook(self):
> ...         import __builtin__
> ...         self._orig_import = __builtin__.__import__
> ...         __builtin__.__import__ = self
> ... #
> ...     def unhook(self):
> ...         assert self._orig_import is not None, "unhook called with no  
> previous hook"
> ...         import __builtin__
> ...         __builtin__.__import__ = self._orig_import
> ...         del self._orig_import
> ... #
> ...     # support the "with" statement
> ...     def __enter__(self):
> ...         self.hook()
> ...         return self
> ... #
> ...     def __exit__(self, type, value, tb):
> ...         self.unhook()
> ...
> py>
> py> ih = ImportHook()
> py> ih.hook()
> py> import htmllib
> -> import htmllib
> -> import sgmllib
> -> import markupbase
> -> import re
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import re
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> import sre_parse
> -> from formatter import AS_IS
> -> import sys
> -> from htmlentitydefs import entitydefs
> py> reload(htmllib)
> -> import sgmllib
> -> from formatter import AS_IS
> -> from htmlentitydefs import entitydefs
> <module 'htmllib' from 'C:\apps\Python25\lib\htmllib.pyc'>
> py> ih.unhook()
> -> import __builtin__
> py> import htmllib
> py>
>
> > I have been keeping a diagram of my module inheritance to make sure it
> > is as clean as possible, so I could just right a list of reloads as
> > you suggest. However, one of the sub-packages is designed to allow
> > users to add more modules. Because these get dynamically imported, a
> > guess I could add an argument to the reload function to allow a user
> > to give the 'add-on' module they are working on... so much work just
> > to get a clean environment...
>
> > Separate of my program, I was really hoping to write a generic reload
> > tool for anyone developing in the same application as I am. I just
> > don't see a way to trace import dependencies in systems which include
> > dynamic imports. Any ideas?
>
> You may adapt the example above.
> Good luck!
>
> --
> Gabriel Genellina

Thanks again Gabriel! I'll see if I can take another swing at this
when I catch up to my milestones ;). I'll post anything useful.

- Rafe