[Distutils] buildout: bin/test --coverage

[Marius Gedminas]

On Tue, Feb 12, 2008 at 02:28:21PM +0100, Thomas Lotze wrote:
> I've noticed a bug which is most probably the fault of
> zc.recipe.testrunner, and as I'm not sure where to report bugs against
> this buildout recipe, I'll drop a note here.
>
> The problem occurs when checking test coverage using the test runner's
> --coverage option. I've run bin/test --coverage on a suite of doc tests,
> and it gave me some 70% of coverage for a particular code file. After
> adding a test for one of the uncovered pieces of code, I got less than 70%
> coverage for the same file, everything else being equal.

It's not a problem with the recipe, it's a problem with zope.testing,
or, more fundamentally, with Python.  The sys.settrace hook used to
implement coverage tracing is non-recursive.

zope.testing tests its own ability to do coverage tracing, and since the
test is run in the same Python process as the test runner, the side
effect of the test's calling sys.settrace is that the runner is no
longer able to trace coverage.

There are other ways to reset the sys.settrace hook.  My favourite is

    class SomeClass(object):
        def __iter__(self):
            ...
        def __len__(self):
            return len(list(self))

list(obj) cals obj.__len__ as an optimization.  This results in an
infinite recursive loop, but then list(obj) traps that, hides it, and
does the brute-force list conversion.  As a side effect of sys.settrace,
the stack overflow RuntimeError happens inside the tracing hook and
Python removes it for misbehaving.  Tracing stops from that point on.

Python's doctest also had side effects that used to disable tracing.

Marius Gedminas
-- 
You can't have megalomania.  *I* have megalomania.
		-- Joe Bednorz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.python.org/pipermail/distutils-sig/attachments/20080212/098daa4a/attachment.pgp