unittest and threading

[Ross Boylan]

On Tue, 2012-01-24 at 13:54 -0800, Ross Boylan wrote:
> Is it safe to use unittest with threads?
> 
> In particular, if a unit test fails in some thread other than the one
> that launched the test, will that information be captured properly?
> 
> A search of the net shows a suggestion that all failures must be
> reported in the main thread, but I couldn't find anything definitive.
> 
> If it matters, I'm using CPython 2.7.
> 
> Thanks.  If you're using email, I'd appreciate a cc.
> Ross Boylan
> 
Steven D'Aprano wrote

> I think you need to explain what you mean here in a little more detail.
> 
> If you mean, "I have a library that uses threads internally, and I want 
> to test it with unittest", then the answer is almost certainly yes it is 
> safe.
> 
> If you mean, "I want to write unit tests which use threads as part of the 
> test", then the answer again remains almost certainly yes it is safe.
Thanks for your responses (only partially excerpted above).

The code I want to test uses threads, but that is not entirely internal
from the standpoint of the unit test framework.  The unit test will be
executing in one thread, but some of the assertions may occur in other
threads.  The question is whether that will work, in particular whether
assertion failures will be properly captured and logged by the test
framework.

Concretely, a test may exercise some code that triggers a callback; the
callback might come in a different thread, and the code that is
triggered might make various assertions.

There are two issues: whether assertions and their failures that happen
in other threads will be correctly received by the test framework, and
whether the framework is robust against several assertions being raised
"simultaneously" in different threads.  The latter seems a bit much to
hope for.

I assume that, at a minimum, the my test code will need to use locks or
other coordination mechanisms so the test doesn't end before all code
under test executes.

Finally, I'll mention two senses of threads in tests that my question
does not concern, although they are also interesting.

I am not concerned with testing the performance of my code, in the sense
of asserting  that an operation must complete before x seconds or after
y seconds.  Some potential implementations of such tests might use
threads even if the code under test was single-threaded.

The question also does not concern running lots of unit tests in
parallel.

Ross