coverage.py: "Statement coverage is the weakest measure of code coverage"

[Kay Schluehr]

On Oct 29, 4:15 am, Ben Finney <bignose+hates-s... at benfinney.id.au>
wrote:
> Kay Schluehr <kay.schlu... at gmx.net> writes:
> > I used to write once a coverage tool ( maybe I can factor this out
> > of my tool suite some time )
>
> That'd be wonderful. I'd like to see comparisons between different
> test-coverage tools, just as we have the different but comparable
> 'pyflakes' and 'pylint' code inspection tools.
>
> > Given it's nature it might act transformative. So a statement:
>
> > if a and b:
> >     BLOCK
>
> > can be transformed into
>
> > if a:
> >     if b:
> >         BLOCK
>
> I don't see that this actually helps in the cases described in the
> original post. The lack of coverage checking isn't "are both sides of
> an 'and' or 'or' expression evaluated", since that's the job of the
> language runtime, and is outside the scope of our unit test.
>
> what needs to be tested is "do the tests execute both the 'true' and
> 'false' branches of this 'if' statement", or "do the tests exercise
> the 'no iterations' case for this loop", et cetera. That is, whether
> all the functional branches are exercised by tests, not whether the
> language is parsed correctly.

You are right. I re-read my coverage tool documentation and found also
the correct expansion for the statement

if a and b:
   BLOCK

which is:

if a:
    if b:
        BLOCK
    else:
        BLOCK
else:
    BLOCK

This will cover all relevant traces. The general idea still holds.

Note I would like to see some kind of requirement specification ( a
PEP style document ) of different coverage purposes and also a test
harness. I'm all for advancing Python and improve the code base not
just accidentally. Something in the way of an MVC framework was nice
in addition which implements UI functions independently s.t. the basic
coverage functionality can be factored out into components and
improved separately. I do not think it's a good idea to have 10
coverage tools that handle presentation differently.