doctest random output?

[Chris Angelico]

On Tue, Aug 29, 2017 at 11:53 AM, Steve D'Aprano
<steve+python at pearwood.info> wrote:
> (1) Disable doctesting for that example, and treat it as just documentation:
>
> def my_thing():
>     """blah blah blah
>
>     >>> my_thing()  #doctest:+SKIP
>     4
>
>     """

For a lot of functions, this completely destroys the value of doctesting.

> (2) Monkey-patch the random module for testing. This is probably the worst idea
> ever, but it's an idea :-)
>
> That makes for a fragile test and poor documentation.

This highlights the inherent weakness of doctests. For proper unit
testing, I would definitely recommend this. Maybe a hybrid of 1 and 2
could be organized... hmm.

> (3) Write your functions to take an optional source of randomness, and then in
> your doctests set them:
>
> def my_thing(randint=None):
>     """blah blah blah
>
>     >>> my_thing(randint=lambda a,b: 4)
>     4
>
>     """
>     if randint is None:
>         from random import randint
>     ...

Unless that would be useful for other reasons, not something I like
doing. Having code in your core that exists solely (or even primarily)
to make testing easier seems like doing things backwards.

> (4) Write your doctests to test the most general properties of the returned
> results:
>
>
> def my_thing(randint=None):
>     """blah blah blah
>
>     >>> num = my_thing()
>     >>> isinstance(num, int) and 0 <= my_thing() <= 6
>     True
>
>     """

This is what I'd probably do, tbh.

None of the options really appeal though. Personally, I'd probably
either go with #4, or maybe something like this:

def roll(sequence):
    """Roll a set of dice

    >>> from test_mymodule import * # ensure stable RNG
    >>> roll("d12 + 2d6 + 3")
    You roll d12: 8
    You roll 2d6: 1, 6, totalling 7.
    You add a bonus of 3
    For d12 + 2d6 + 3, you total: 18
    """

and bury all the monkey-patching into test_mymodule. It can have its
own implementations of randint and whatever else you use. That way, at
least there's only one line that does the messing around. I still
don't like it though - so quite honestly, I'm most likely to go the
route of "don't actually use doctests".

ChrisA