How best to initialize in unit tests?

[Steve D'Aprano]

On Thu, 5 Oct 2017 12:07 am, Skip Montanaro wrote:

> Suppose you want to test a package (in the general sense of the word,
> not necessarily a Python package).

I'm... not sure I understand. Given that we're discussing Python, in a Python
group, what are senses are relevant? 


> You probably have specific unit 
> tests, maybe some doctests scattered around in doc strings. Further,
> suppose that package requires you call an initialize function of some
> sort. Where does that go? 

Ew. Sounds like a badly designed package. Importing the package should
initialise it. We write:

import math
x = math.sin(1)

not:

import math
math.initialise()  # I actually do want to use the module I just imported
x = math.sin(1)  # fails mysteriously if we forget to initialise first


If you see a code unit (a package, a module, a class, or even a function) that
requires a manual call to something called "initialise" or equivalent, you
probably have a poorly designed code unit.

(I suppose there may be -- hopefully rare -- cases where that's unavoidable,
or less bad than the alternative.)

But given that you have to test the code you have, not the code you wish you
had... 

Where are the doc tests?

(1) If they are in the module you are testing itself, you will probably need
to call initialise() at the start of each doc string before running any
tests. That's because doctest makes a shallow copy of the module's globals
before running the tests, so any global state initialised in one doc string
will be lost when the next is called.

https://docs.python.org/2/library/doctest.html#what-s-the-execution-context

(2) If the doc tests are in a separate file, the entire file will be run in a
single context. You probably already start the file by importing the module
you are testing:

    >>> import Module

so just add the initalise call after that.

In unit tests, you import the module, so just call initialise immediately:

    import Module
    Module.initialise()

and then you don't have to worry about it in the test cases themselves.


> I know about setUp and setUpClass methods in 
> unittest.TestCase. Is order of execution of doctests deterministic
> across modules (say, when tests are run through nosetests)?

I can't speak for nose, but doctest runs the tests in undocumented, and
therefore arbitrary, order.


> In my case, I didn't know what order things would be called, so I
> added a call to initialize() at the start of every doctest and added a
> setUpClass class method to all my TestCase subclasses. Just in case.
> It worked okay because my initialize function can be called multiple
> times. What about situations where it can only be called once? 

If the initialise function isn't idempotent, there may be subtle problems with
doctest if making a shallow copy of the module globals isn't enough to reset
all the initialised state.


> Do you 
> have to define some sort of super_initialize() function for your tests
> which guarantees that your initialize function is called precisely
> once?

No.


> This all seems rather messy. 

Indeed. And now you know why importing a module/package should be sufficient
to initialise it.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.