[Python-Dev] We now have C code coverage!

[Terry Reedy]

On 6/22/2018 8:43 PM, Terry Reedy wrote:
> On 6/22/2018 6:21 PM, Brett Cannon wrote:
>> Thanks to a PR from Ammar Askar we now run Python under lcov as part 
>> of the code coverage build. And thanks to codecov.io 
>> <http://codecov.io> automatically merging code coverage reports we get 
>> a complete report of our coverage (the first results of which can now 
>> be seen at https://codecov.io/gh/python/cpython).
>>
>> And funny enough the coverage average changed less than 1%. :)
> 
> Questions:
> 
> 1. Is it possible, given that we are not paying for those reports, to 
> customize the .coveragerc exclude_lines definitions?  Without such, the idlelib 
> measures are biased downward.
> 
> 2. What do the colors of test files mean?  Every line of nearly all the 
> idlelib test files are executed, but over half are red.
> 
> The Learn More page does not say anything about either.

I discovered the answer to 2. by shift-clicking on a text_x file to see 
their coverage report for the file.  The colors actually do reflect the 
test lines executed.  codecov.io excludes gui tests*, so the reported 
coverage for tkinter, idlelib, and turtle is deceptive and bogus, and 
under-reports the total cpython coverage by a percent or two.  It would 
be better to exclude these modules.

* I assume that codecov.io uses linux servers.  I have read that there 
are programs that simulate X-Windows so that gui code will execute 
without actual terminals.

-- 
Terry Jan Reedy