Clearing a session and reload() problem (with repro error)

[Gabriel Genellina]

En Wed, 10 Sep 2008 00:56:43 -0300, Rafe <rafesacks at gmail.com> escribió:
> On Sep 9, 11:03 pm, "Gabriel Genellina" <gagsl-... at yahoo.com.ar>
> wrote:
>> En Mon, 08 Sep 2008 05:37:24 -0300,Rafe<rafesa... at gmail.com> escribió:
>> ...
>> This dependency between modules, applied to all modules in your project,
>> defines a "dependency graph". In some cases, one can define a partial
>> ordering of its nodes, such that no module depends on any other module
>> *after* it (it may depend only on modules *before* it). Look for
>> "topological sort".
>>
>> Doing that in the generic case is not easy. If you *know* your
>> dependencies, reload the modules in the right order by hand.
>> ...

> I was hoping there would be a way to just wipe out the module cache
> and let it get rebuilt by executing my code (since I'm not using
> reload as part of my program, but rather, to test it in an environment
> where I cannot restart the Python session).

Ok, I think the following sequence *might* work:

- replace the __import__ and reload builtins with a custom callable  
object. This way you can hook into any import attempt. The object should  
keep a list of already reloaded modules. When a module is imported: if it  
is already in the list, just delegate to the original __import__; if it is  
not in the list, locate the module in sys.modules and reload it.

- iterate over sys.modules and reload the desired modules, as you did in  
your previous attempt.

- restore the original __import__ function.

This way, you effectively transform any import statement into a recursive  
reload (for the first time); subsequent imports of the same module behave  
as usual. This may work for you, or perhaps not, or it may screw all your  
running environment up, or even cause the next global thermonuclear war...  
(I hope not!)

Note: some modules don't work well with reload(). A common case: global  
mutable values, like a list of objects which starts empty:
	my_list = []
To make it more "reload friendly", use this:
	try: my_list
	except NameError: my_list = []
(this way the list will keep its previous values).

The example below shows how to hook into the import mechanism - it just  
prints the module being imported. Implementing the functionality outlined  
above is left as an exercise to the reader :)

py> class ImportHook(object):
...     _orig_import = None
... #
...     def __call__(self, name, globals={}, locals={}, fromlist=[],  
level=-1):
...         if fromlist:
...             print "-> from %s import %s" % (name, ','.join(fromlist))
...         else:
...             print "-> import %s" % name
...         return self._orig_import(name, globals, locals, fromlist,  
level)
... #
...     def hook(self):
...         import __builtin__
...         self._orig_import = __builtin__.__import__
...         __builtin__.__import__ = self
... #
...     def unhook(self):
...         assert self._orig_import is not None, "unhook called with no  
previous hook"
...         import __builtin__
...         __builtin__.__import__ = self._orig_import
...         del self._orig_import
... #
...     # support the "with" statement
...     def __enter__(self):
...         self.hook()
...         return self
... #
...     def __exit__(self, type, value, tb):
...         self.unhook()
...
py>
py> ih = ImportHook()
py> ih.hook()
py> import htmllib
-> import htmllib
-> import sgmllib
-> import markupbase
-> import re
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import re
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> import sre_parse
-> from formatter import AS_IS
-> import sys
-> from htmlentitydefs import entitydefs
py> reload(htmllib)
-> import sgmllib
-> from formatter import AS_IS
-> from htmlentitydefs import entitydefs
<module 'htmllib' from 'C:\apps\Python25\lib\htmllib.pyc'>
py> ih.unhook()
-> import __builtin__
py> import htmllib
py>

> I have been keeping a diagram of my module inheritance to make sure it
> is as clean as possible, so I could just right a list of reloads as
> you suggest. However, one of the sub-packages is designed to allow
> users to add more modules. Because these get dynamically imported, a
> guess I could add an argument to the reload function to allow a user
> to give the 'add-on' module they are working on... so much work just
> to get a clean environment...
>
> Separate of my program, I was really hoping to write a generic reload
> tool for anyone developing in the same application as I am. I just
> don't see a way to trace import dependencies in systems which include
> dynamic imports. Any ideas?

You may adapt the example above.
Good luck!

-- 
Gabriel Genellina