Strange problem when using imp.load_module

[Arnaud Delobelle]

pythoncurious at gmail.com writes:

> Hi,

Hi, I have a guess at explaining the behaviour you describe - see below.

> I'm having problem when I'm trying to import modules using the
> imp.load_module function.
> At the end of this post there's some code I wrote to illustrate the
> problem.
> The code istself doesn't make much sense, but what I'm trying to do in
> reality is allow people to customize an application by writing a small
> python module. If I find a file in a known location with a known name,
> I'll import it and use some data or function in it.
> I ran into problems when writing unit tests for it.
>
> What happens when run my code is this:
>
> .E
> ======================================================================
> ERROR: test_2 (__main__.Test)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "test_import.py", line 30, in test_2
>     getattr(imported, 'y')
> AttributeError: 'module' object has no attribute 'y'
> ----------------------------------------------------------------------
> Ran 2 tests in 0.015s
>
>
> So it seems that test_2 fails. It does pretty much the same thing as
> test_1, but with different data.
> Changing test_2 to check for 'x' instead of 'y' makes it pass, but
> that's not the result I should get.
> So it seems that I still have the module that I imported in test_1.
>
> So, I'm thinking that I might be dealing with some sort of gc issue.
> I add the "time.sleep(1)", (commented in the code) and the tests pass.
> 'y' is found in test_2.
> Replacing the sleep with a gc.collect() makes the test fail again, so
> garbage collection doesn't seem to help.
>
> I've tried it on python 2.6.1 on windows, 2.5.2 in cygwin and 2.6.1 on
> solaris with the same results.
> Could anyone explain what I am missing?
>
> Thanks
> /Mattias
>
> Here's the code:
>
> import unittest
> import tempfile, os, imp, time, gc
>
> module_file_name=os.path.join(tempfile.gettempdir(), 'my_module.py')
>
> def write_module(data):
>     f = open(module_file_name, 'w')
>     f.write(data)
>     f.close()
>
> def import_from_file(path):
>     imported_module = imp.load_source(module_file_name, path)
>     return imported_module
>
> class Test(unittest.TestCase):
>     def tearDown(self):
>         os.unlink(module_file_name)
>
>     def test_1(self):
>         module_data='''x=1'''
>         write_module(module_data)
>         imported=import_from_file(module_file_name)

This will compile the module and create a 'my_module.pyc' file.

>         getattr(imported, 'x')
>
>     def test_2(self):
>         # time.sleep(1)
>         module_data='''y=2'''
>         write_module(module_data)
>         imported=import_from_file(module_file_name)

My guess is that without the sleep(1), the imp.load_source function will
use the compiled file 'my_module.pyc' created in test_1 instead of
compiling the new 'my_module.py' file.

This would be because the new 'my_module.py' file is created so soon
after the last compilation of the module that they have the same
timestamp, thus fooling imp.load_source into thinking that
'my_module.py' was not modified since its last compilation.

>         getattr(imported, 'y')
>
> if __name__ == "__main__":
>     unittest.main()

HTH

-- 
Arnaud