Problem with Dynamically unloading a module

[Steven D'Aprano]

On Wed, 23 Dec 2009 01:41:27 -0800, lordofcode wrote:

> I need to dynamically load a module and unload it and load another
> module.

Why bother unloading it? Unless you're programming for an embedded 
device, it's highly unlikely that you're so strapped for memory that a 
module will make any real difference.

Just stop using the module and start using another. (See below for 
example.)


> For example I have many files(All files in Python are modules right?)
> like mobile_1.py ,mobile_2.py, mobile_3.py  etc.. in my project folder
> which contains classes and methods with same name but different
> functionality.(am afraid I cannot change this structure as these files
> are generated randomly by the user)
>
> So initially when my program starts I have to load a default module. I
> do this as follows:
> ##############################
>>>MODULE_name = "mobile_1"
>>>exec "from "+MODULE_name+" import *"
> ##############################
> And use the methods defined in "mobile_1.py" file
> 
> Now as the application continues , I may have to use the methods defined
> in "mobile_2.py" or "mobile_3.py" etc instead of the previously loaded
> module,


Sounds like an absolute horrible design. I recommend that you re-think 
the design and do something less fragile and difficult.

The obvious way would be something like this:

# Import the first module and use it.
user_module = __import__("mobile_1")
x = user_module.function(123)
y = user_module.SomeClass(123)

# Now change modules.
user_module = __import__("mobile_2")
x = user_module.function(456)
y = user_module.SomeClass(456)


Note that:

(1) The function __import__ loads the module given a name which is only 
known at runtime instead of compile time.

(2) Don't use the form "from module import name" form. That injects the 
objects into the current namespace, which generally leads to more trouble 
than benefit.

(3) After replacing the module, you have to get rid of the old classes 
that refer to the old module, and create a new objects. Classes don't 
magically change their behaviour just because the name of the class 
points to something different:


>>> class C:
...     def method(self):
...             return "Something!"
...
>>> instance = C()
>>> instance.method()
'Something!'
>>> class C:  # replace the old class
...     def method(self):
...             return "Something different"
...
>>> instance.method()
'Something!'


I'm betting that once you understand that behaviour, you will understand 
why your approach is failing.



> which I incorrectly try to do as below: 
> ####################
>>>MODULE_name = "mobile_2"
>>>exec "from "+MODULE_name+" import *"
> #####################
> The above import does not have any impact and the methods called from my
> application still pertain to mobile_1.py as its still in the current
> namespace(?).

The most likely problem is that you have misunderstood how Python's 
object model works.

Suppose you do this:

from module_1 import MyClass
x = MyClass(1, 2, 3)  # create a class instance
x.method()

The object called "x" is now an instance of MyClass from module_1. So far 
so good. Suppose that you do this:

from module_2 import MyClass
x.method()

I'm guessing that you expect that this will call the method from MyClass 
in module_2. But that's not what happens: the object x is still the same 
object it was before, and it still calls the method from module_1. You 
have to throw away that object and create a new one:

from module_2 import MyClass
x = MyClass(1, 2, 3)
x.method()

(This, BTW, is exactly the same situation as the same code above with 
class C and instance.method, except an import is added to the mix.)


Now, it's not *impossible* to play tricks like this:

from module_2 import MyClass
# replace the reference to the old MyClass with the new MyClass:
x.__class__ = MyClass
x.method()

which *may* work, for some classes. But better to avoid this unless you 
can control the classes yourself, which you can't.


Or find a better design that doesn't rely on dynamically replacing 
modules.


> I tried below code with del(), reload() etc but could not figure it out.
> ###Code to unload a dll####

This has nothing to do with DLLs. Python modules are Python modules, not 
DLLs, and they don't work the same way.


>>>del sys.modules[MODULE_name]    #==> does not delete the reference in
>>>namespace

Deleting the module from the cache does nothing except slow Python down 
the next time you import the module.


> 1)How do I unload a module dynamically and completely remove the
> references in the module so a new module with same name references can
> be loaded?

You don't have to. You just have to understand how Python actually works, 
and work *with* it instead of *against* it.


> 2)Are there any alternative way to do the above requirement? Currently I
> am working around by restarting the whole initial setup for each new
> module which is unnecessary waste.

What makes you think it's unnecessary?


> Can I avoid this "reset"?

I don't know, because I don't fully understand what "restarting the whole 
initial setup" actually means.




-- 
Steven