Sphinx Doctest: test the code without comparing the output.

[Luca Cerone]

> It won't be very good documenation any more but nothing stops you
> 
> from examining the result in the next doctest and making yourself
> 
> happy about it.
> 
> 
> 
>   >>> x = input("indeterminate:")
> 
>   >>> result = "'{}'".format(x))
> 
>   >>> result.startswith("'") and result.endswith("'")
> 
>   True
> 

Hi Neil, thanks for the hint, but this won't work.

The problem is that the function displays some output informing you of what steps are being performed (some of which are displayed by a 3rd party function that I don't control).

This output "interferes" with the output that should be checked by doctest.

For example, you can check that the following doctest would fail:

.. doctest:: example_fake

   >>> def myfun(x,verbose):
   ...    print "random output"
   ...    return x
   >>> myfun(10)
   10

When you run make doctest the test fails with this message:

File "tutorial.rst", line 11, in example_fake
Failed example:
    myfun(10)
Expected:
    10
Got:
    random output
    10

In this case (imagine that "random output" is really random, therefore I can not easily filter it, if not ignoring several lines. This would be quite easy if ellipsis and line continuation wouldn't have the same sequence of characters, but unfortunately this is not the case.

The method you proposed still is not applicable, because I have no way to use startswith() and endswith()...

The following code could do what I want if I could ignore the output...

   >>> def myfun(x,verbose):
   ...    print "random output"
   ...    return x
   >>> result = myfun(10) #should ignore the output here!
   >>> print result
   10

fails with this message:

File "tutorial.rst", line 11, in example_fake
Failed example:
    result = myfun(10)
Expected nothing
Got:
    random output

(line 11 contains: >>> result = myfun(10))

A SKIP directive is not feasible either:

.. doctest:: example_fake

   >>> def myfun(x):
   ...    print "random output"
   ...    return x
   >>> result = myfun(10) # doctest: +SKIP 
   >>> result
   10

fails with this error message:
File "tutorial.rst", line 12, in example_fake
Failed example:
    result
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/doctest.py", line 1289, in __run
        compileflags, 1) in test.globs
      File "<doctest example_fake[2]>", line 1, in <module>
        result
    NameError: name 'result' is not defined

As you can see is not that I want something too weird, is just that sometimes you can't control what the function display and ignoring the output is a reasonable way to implement a doctest.

Hope these examples helped to understand better what my problem is.

Thanks all of you guys for the hints, suggestions and best practices :)