multiprocessing module backport from 3 to 2.7 - spawn feature

[Sturla Molden]

Skip Montanaro <skip.montanaro at gmail.com> wrote:

> Can you explain what you see as the difference between "spawn" and "fork"
> in this context? Are you using Windows perhaps? I don't know anything
> obviously different between the two terms on Unix systems.

spawn is fork + exec.

Only a handful of POSIX functions are required to be "fork safe", i.e.
callable on each side of a fork without an exec. 

An example of an API which is not safe to use on both sides of a fork is
Apple's GCD. The default builds of NumPy and SciPy depend on it on OSX
because it is used in Accelerate Framework. You can thus get problems if
you use numpy.dot in a process started with multiprocessing. What will
happen is that the call to numpy.dot never returns, given that you called
any BLAS or LAPACK function at least once before the instance of
multiprocessing.Process was started. This is not a bug in NumPy or in
Accelerate Framework, it is a bug in multiprocessing because it assumes
that BLAS is fork safe. The correct way of doing this is to start processes
with spawn (fork + exec), which multiprocessing does on Python 3.4. 

Sturla