[issue33884] [multiprocessing] Multiprocessing in spawn mode doesn't work when the target is a method in a unittest.TestCase subclass, when run either with unittest or with pytest

[Yoni Rozenshein]

New submission from Yoni Rozenshein <yoni.vl at gmail.com>:

multiprocessing will attempt to pickle things using ForkingPickler when starting a new process in spawn mode (in Windows this is the only mode, in Linux this is a non-default but settable mode).

When run within the context of a unit test, if it has to pickle a TestCase subclass, it encounters objects that can't be pickled. The attached file shows a minimum working example (uncomment the two commented lines under __main__ to run with pytest).

When run with unittest.main(), it raises:

TypeError: cannot serialize '_io.TextIOWrapper' object

When run with pytest.main(), it raises:

AttributeError: Can't pickle local object 'ArgumentParser.__init__.<locals>.identity'

Note that changing the _child_process in my example to a classmethod/staticmethod or moving it to a top-level function outside the class works around this issue (both with unittest and with pytest).

----------
components: Library (Lib)
files: mp_pickle_issues.py
messages: 319811
nosy: Yoni Rozenshein
priority: normal
severity: normal
status: open
title: [multiprocessing] Multiprocessing in spawn mode doesn't work when the target is a method in a unittest.TestCase subclass, when run either with unittest or with pytest
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47645/mp_pickle_issues.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33884>
_______________________________________