[Tutor] problems with doctest: apparent interferance between tests (LONG)

[Kent Johnson]

Brian van den Broek wrote:
> I've not fully grokked the doctest code (which I delved into after Lee
> Harr suggested I do so), but I would have thought that each doctest
> had its own copy of the Wall_clock class from copying globals. But
> here, I surely have more work to do myself :-)

doctest makes a shallow copy of the globals (see 
http://docs.python.org/lib/doctest-execution-context.html)
So all the tests have references to the same Wall_clock class.

Copying the globals protects each test from depending on objects created or destroyed by other 
tests, but it doesn't isolate tests from changes to mutable objects in the global namespace.

> This suggests to me quite strongly that order in which the tests are run 
> is determined at one or more points by dictionary access. (I suspect 
> that one or both of the module objects to be examined and the docstrings 
> in module objects are at some point stored in a dictionary.[*]) Since 
> dictionary access is random, any fetching of a list of items to feed to 
> the DocTest class could have the order of testing of two items altered 
> by the presence of other items.

doctest finds classes, functions and methods by inspecting the __dict__ attributes of modules and 
classes; see lines 901 and 932 of doctest.py (in Python2.4). So this sounds like a good theory to 
explain at least why adding a new method changes the order of the tests.

Kent