[py-dev] pytest-cov

[holger krekel]

Hi Meme,

On Tue, Aug 03, 2010 at 21:57 +1000, meme dough wrote:
> On 3 August 2010 18:14, holger krekel <holger at merlinux.eu> wrote:
> > On Mon, Aug 02, 2010 at 16:05 +0200, holger krekel wrote:
> >> Hum, how does the "out-of-the-box" work exactly?
> >> It won't always be enabled just because pytest-cov is installed, will it?
> >> (if so, i'd rather suggest to do an indirection and only perform coverage
> >> if a certain environment varialbe like PYTEST_COV_AUTOSUB=1 or so it st).
> >
> > FYI i think pytest-cov begins to work better than pytest-figleaf
> > in some cases! :)
> >
> > However, i consider it crucial that installing pytest-cov has no side effects
> > by default.  With the above suggestion we could make py.test set the environment
> > variable when a coverage-run is selected and otherwise avoid any side effects,
> > particularly when py.test is not even run.  IOW, i'd be very happy with a
> > pytest-cov-1.1 streamlining this issue so it's safe to point people to it :)
> 
> When pytest-cov installed (well really cov-core) an init_cov_core.pth
> will be created. During python site initialisation this will cause
> coverage to start only if appropriate.  From the doc string:
> 
> Activate coverage at python startup if appropriate.
> 
> The python site initialisation will ensure that anything we import
> will be removed and not visible at the end of python startup.  However
> we minimise all work by putting these init actions in this separate
> module and only importing what is needed when needed.
> 
> For normal python startup when coverage should not be activated we
> only import os, look for one env var and get out.
> 
> For python startup when an ancestor process has set the env indicating
> that code coverage is being collected we activate coverage based on
> info passed via env vars.
 
If i understand correctly you are describing the situation as is. 
Sorry, i didn't look at the content of cov_core_init.py which would
have revealed the mechanics that you describe. 

> So I completely agree with you.
> 
> One issue is creating the pth file during installation.  It works for
> me with what I have to test with.  If it can't find site-packages /
> dist-packages then it outputs a message describing what file to create
> and content to go in it.
> 
> Another issue is uninstalling.  I added a big note indicating that the
> pth file must be uninstalled manually.

I guess somehow "pip" should be better able to uninstall.  But ok,
cov_core_init.py is hopefully not causing trouble so it's not so bad. 
 
> >> * --cov=path ... the help string should say what path, file-system or
> >>   python import-path?
> 
> I was tying to be consistent with other pytest options and path looked
> the best as used by others.  Under hood it is currently morf (modules
> or files coverage speak) but will change to source option which is a
> root of a source tree so I think path is the good like following.
> Suggestions?
> 
> --junitxml=path
> --ignore=path
> --genscript=path

these are all file system paths.   

> >> * when i ran "py.test --cov=py._code" it worked fine, i.e. only showed
> >>   results for the py._code subpackage.  When i mistype the path,
> >>   e.g. "py.test --cov=py._code2" it will give me *all* results which
> >>   is wrong: i'd expect "no coverage data for import path "py._code2" found
> >>   or something like this.
> 
> If all --cov options don't give something valid then everything
> collected is dumped (and everything was collect unless you use omit
> option in config file and omit is changing a lot with new source /
> include / omit).

This remains then the only issue i guess: rather give information 
that the specified "--cov=XYZ" restriction was used and turn up
*no* results (even if coverage defaults disagree).  Seeing everything 
is almost never what wants to see if "--cov=MORF" was specified. 
Do you agree and see a way to implement this within pytest-cov? 

Another minor note: when using --cov-report=html no information
is printed at all.  Like the junitxml or figleaf plugin i suggest
to print the location of the generated html directory or directly
the index.html file. 

best & thanks for your nice work!
holger
 
> This is where source option comes in.  With source option specify the
> roots of trees then limited to those roots since measurement itself is
> not done outside of those trees.
> 
> :)
> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev
> 

--