[python-committers] Codecov and PR

[INADA Naoki]

> While I use code coverage to improve automated unittesting, I am opposed to
> turning a usable but limited and sometime faulty tool into a blind robotic
> master that blocks improvements.  The prospect of this being done has
> discouraged me from learning the new system.  (More on 'faulty tool' later.)
>
> The temptation to write artificial tests to satisfy an artificial goal is
> real.  Doing so can eat valuable time better used for something else.  For
> instance:
>
>     def meth(self, arg):
>         mod.inst.meth(arg, True, ob=self, kw='cut')
>
> Mocking mod.class.meth, calling meth, and checking that the mock is called
> will satisfy the robot, but does not contribute much to the goal of
> providing a language that people can use to solve problems.
>
> Victor, can you explain 'tested indirectly' and perhaps give an example?
>

This is one examples I merged "untested line of code".
https://github.com/python/cpython/pull/162/files#diff-0ad86c44e7866421ecaa5ad2c0edb0e2R552

+            file_object = builtins.open(f, 'wb')
+            try:
+                self.initfp(file_object)
+            except:
+                file_object.close()
+                raise

`self.initfp()` is very unlikely raise exceptions.  But MemoryError,
KeyboardInterrupt or
other rare exceptions may be happen.
Test didn't cover this except clause.  But I merged it because:

* this code is simple enough.
* I can write test for it with mocking `self.initfp()` to raise
exception.  But such test code
  have significant maintenance cost.  I don't think this except clause
is important enough
  to maintain such code.

If I remove the except clause, all lines are tested, but there is
(very unlikely)
leaking unclosed file.  Which are "broken"?

Coverage is very nice information to find code which should be tested,
but not tested.
But I don't think "all code should be tested".
It may make hard to improve Python.

So I agree with Victor and Terry.

Regards,