[New-bugs-announce] [issue7197] test_multiprocessing crashes under Windows when run in verbose mode

[Antoine Pitrou]

New submission from Antoine Pitrou <pitrou at free.fr>:

When run under Windows in verbose mode ("python -m test.regrtest -v
test_multiprocessing"), most tests in test_multiprocessing fail with a
recursion limit error.

The explanation is that most tests are written in the following manner:

class _TestArray(BaseTestCase):
    [...]
    def test_array(self, raw=False):
        [...]
        p = self.Process(target=self.f, args=(arr,))

Running a Process under Windows involved pickling it to send it to the
child process. This also pickles the `target` function, which here is a
method of an unittest instance. This consequently pickles the unittest
instance, which has a reference to the unittest runner, which has a
reference to a unittest.runner._WritelnDecorator instance.

The infinite recursion occurs when unpickling the _WritelnDecorator
instance, because the `stream` attribute is not restored when calling
__new__, and the __getattr__ method then recurses when trying to return
`getattr(self.stream,attr)`.

I see two possible resolutions:
- make unittest.runner._WritelnDecorator properly (un)pickleable 
- change all tests in test_multiprocessing to avoid pickling instances
of unittest.TestCase

The former is simpler and probably more future-proof than the latter.

(NB: in non-verbose mode, test.support uses a custom test runner which
doesn't involve the _WritelnDecorator)


Appendix: here is a traceback example (noticed on the newgil branch but
it really happens on stock trunk and py3k):

test_notify (test.test_multiprocessing.WithProcessesTestCondition) ...
Traceback
 (most recent call last):
  File "<string>", line 1, in <module>
  File "Z:\py3k\newgil\lib\multiprocessing\forking.py", line 339, in main
    self = load(from_parent)
  File "Z:\py3k\newgil\lib\pickle.py", line 1365, in load
    encoding=encoding, errors=errors).load()
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
    return getattr(self.stream,attr)
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
    return getattr(self.stream,attr)
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
    return getattr(self.stream,attr)
[... snipped ...]
RuntimeError: maximum recursion depth exceeded while calling a Python
object

----------
components: Library (Lib), Tests
messages: 94406
nosy: jnoller, michael.foord, pitrou
priority: high
severity: normal
status: open
title: test_multiprocessing crashes under Windows when run in verbose mode
type: behavior
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7197>
_______________________________________