Unit tests and coverage

[Roy Smith]

In article <mailman.4694.1388291001.18130.python-list at python.org>,
 "Joseph L. Casale" <jcasale at activenetwerx.com> wrote:

> I have a script that accepts cmdline arguments and receives input via stdin.
> I have a unit test for it that uses Popen to setup an environment, pass the 
> args and provide the stdin.
> 
> Problem is obviously this does nothing for providing coverage. Given the 
> above specifics, anyone know of a way to work around this?

What do you mean by "providing coverage"?  If you mean, "automatically 
generate every possible combination of inputs", that's not, in general, 
possible.

There are lots of different meanings people give to the word "coverage" 
when talking about testing code.  One common definition is, "provide 
sufficient sets of inputs to cause every branch in the code to be taken 
at least once".  That's a pretty weak definition (since it doesn't talk 
about combinations of branches), but even that can be amazingly hard to 
satisfy.  For example, imagine the following code:

try:
   big_hairy_operation()
except MemoryError:
   print "Oh noes, out of memory!"

what combination of inputs do you have to provide to cause the except 
clause to be executed?  Or, to touch on an example I mentioned in a 
recent thread:

t0 = time.time()
while True:
   t1 = time.time()
   if t1 < t0:
        print "Distortion detected in the time-space continuum"
    t0 = t1

So, back to my original question; what do you mean by "providing 
coverage"?