Testing automatically on import

[Jeremy Bowers]

On Mon, 30 Aug 2004 09:05:12 +0200, Alex Martelli wrote:
>> I'm not *quite* familiar with the import hook stuff to knock this off
>> right away. I tried prototyping it but got stuck on how to determine if a
>> Python file already has an up-to-date .pyc file. (Although I guess there
>> may be external dependencies... well in the interest of keeping it simple,
>> deal with that later.)
> 
> Hmmm, I could be wrong, but I think you have to check timestamps and
> 'magic' signatures yourself.  Pity, because imp.load_module must also do
> that, but I don't think it makes this logic available separately.
> Still, I believe Demo/imputil/importers.py has all the code you need.

I've knocked up a prototype which can be downloaded at
http://www.jerf.org/importer.py . It unzips into a directory name
"importer", and contains a module "testOnImport".

To see it in action, make that your current directory, start up Python,
and type:

>>> import testOnImport
>>> testOnImport.runOnUserCode.append(testOnImport.UnitTestPattern('%s/tests/%sTest.py'))
>>> import t1

You should then see (between the lines):

----------
This is the validate code in t1.py. If you raise an TestOnImportError here, it will stop the import.
./tests/t1Test.py
F
======================================================================
FAIL: testSchmoo (t1Test.Test1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./tests/t1Test.py", line 5, in testSchmoo
    self.assert_(0)
  File "/usr/lib/python2.3/unittest.py", line 278, in failUnless
    if not expr: raise self.failureException, msg
AssertionError

----------------------------------------------------------------------
Ran 1 test in 0.005s

FAILED (failures=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.3/imputil.py", line 103, in _import_hook
    top_module = self._import_top_module(parts[0])
  File "/usr/lib/python2.3/imputil.py", line 189, in _import_top_module
    module = item.import_top(name)
  File "/usr/lib/python2.3/imputil.py", line 216, in import_top
    return self._import_one(None, name, name)
  File "/usr/lib/python2.3/imputil.py", line 267, in _import_one
    result = self.get_code(parent, modname, fqname)
  File "testOnImport.py", line 104, in get_code
    func(module)
  File "testOnImport.py", line 183, in <lambda>
    permissive)
  File "testOnImport.py", line 169, in UnitTestFromModule
    RunUnitTests(unitTestFile)
  File "testOnImport.py", line 142, in RunUnitTests
    raise TestOnImportError("Unit tests failed.")
testOnImport.TestOnImportError: Unit tests failed.
-----------

The first line is from a __validate__ function in the t1 module. The
second line you typed to start the process told the import hook to check
for my style of test naming, and it looked for and found tests/t1Test.py.
(You should be able to change that string to "%s/tests/test_%s.py" for
your style.)

This is poorly tested as it is late for me (come to think of it I haven't
tested the "permissive" flag), and I know there are bugs, but I don't know
how to fix some of them. (In particular, I don't know what to do with
"import pychecker.checker", which fails.) Please review brutally.

If we can work out a few more bugs, and the issue of whether we want to
prevent/delete .pyc files if the tests fail, I'll take the next step of
trying to integrate this into a real project. (Of course others are
welcome to as well, just expressing my willingness.)

More info is in the testOnImport.py file, including my known bug/issue
list.

Thanks for the idea, this could turn out useful.