[New-bugs-announce] [issue18999] Allow multiple calls to multiprocessing.set_start_method

[Lars Buitinck]

New submission from Lars Buitinck:

The new multiprocessing based on forkserver (issue8713) looks great, but it has two problems. The first:

"set_start_method() should not be used more than once in the program."

The documentation does not explain what the effect of calling it twice would be. Judging from the documentation, it should be possible to do

start_method = get_start_method()
if start_method is None:
    set_start_method('forkserver')

but this code shows the second problem: it always succeeds with the (undocumented!) side-effect of setting the start method in get_start_method, to the system default, if it hasn't been set already.

I was just going to forge a patch for joblib (http://pythonhosted.org/joblib/) to set the start method to forkserver at import time. But in the current state of affairs, it would be impossible for the user to safely override the start method before importing joblib, because joblib can't figure out if it's been set already without setting it.

The enclosed patch solves the problem by making the new functions more robust:

* get_start_method no longer sets anything, but returns None if the start method has not been set already;
* set_start_method raises a RuntimeError (for want of a better exception type) when resetting the start method is attempted.

Unfortunately, I had to hack up the tests a bit, because they were violating the set_start_method contract. There is a test for the new set_start_method behavior, though, and all {fork,forkserver,spawn} tests pass on Linux.

----------
components: Library (Lib)
files: mp_getset_start_method.patch
keywords: patch
messages: 197444
nosy: larsmans
priority: normal
severity: normal
status: open
title: Allow multiple calls to multiprocessing.set_start_method
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file31721/mp_getset_start_method.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18999>
_______________________________________