doctest random output?

[Chris Angelico]

On Tue, Aug 29, 2017 at 5:39 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Tue, 29 Aug 2017 12:25:45 +1000, Chris Angelico wrote:
>
>> For a lot of functions, this completely destroys the value of
>> doctesting.
>
>
> "The" value? Doc tests have two values: documentation (as examples of
> use) and as tests. Disabling the test aspect leaves the value as
> documentation untouched, and arguably is the least-worst result. You can
> always write a unit test suite to perform more detailed, complicated
> tests. Doc tests are rarely exhaustive, so you need unit tests as well.

You can have a docstring that isn't crafted to be runnable tests. The
point about doc tests is that they're executable documentation - the
point of them is to be tests, not just docs. You can always write your
unit tests separately, and let your docstrings merely be
documentation, and then none of this matters.

> For example, imagine a function that returns a randomly selected prime
> number. The larger the prime, the less likely it is to be selected, but
> there's no upper limit. So you write:
>
>    >>> num = my_thing()
>    >>> isinstance(num, int) and 2 <= num
>    True
>
>
> Not very informative as documentation, and a lousy test too.

Yes, but you could have some sort of primality test on it.

>>> is_prime(my_thing())
True

Even if all you have is a "probably prime" test, that would still make
for better documentation AND better testing than no test at all.

> Wait... are you saying that importing test_mymodule monkey-patches the
> current library? And doesn't un-patch it afterwards? That's horrible.
>
> Or are you saying that test_module has its own version of roll(), and so
> you're using *that* version instead of the one in the library?
>
> That's horrible too.

My original plan was to have *a function in* that module that does the
monkey-patching, but I seem to have not actually typed that part in...
mea culpa. I agree that merely importing your test helpers shouldn't
do the changes! Even with "import test_mymodule;
test_mymodule.stable()" it's still just one extra line (better than
the full version).

Not un-patching it afterwards? Yes. Since part of its purpose is to
seed the RNG with a fixed value, it's not really possible or practical
to "undo" that, and so I wouldn't worry too much about an "afterwards"
- after testing, you exit the interpreter, if you want to get back to
normality.

>> It can have its own
>> implementations of randint and whatever else you use. That way, at least
>> there's only one line that does the messing around. I still don't like
>> it though - so quite honestly, I'm most likely to go the route of "don't
>> actually use doctests".
>
> Are you saying don't use doctests for *this* problem, or don't use them
> *at all*?

For this and any other problem where doctesting is impractical.
Because let's face it, laziness is a big thing. If it's too much
hassle to make a docstring executable, I'm just not going to make it
executable. Which has the unfortunate downside of allowing the
docstrings to get out of sync with the code, but that's the cost.

ChrisA