[issue18999] Robustness issues in multiprocessing.{get, set}_start_method

[Richard Oudkerk]

Richard Oudkerk added the comment:

With your patch, I think if you call get_start_method() without later calling set_start_method() then the helper process(es) will never be started.

With the current code, popen.Popen() automatically starts the helper processes if they have not already been started.

Also, set_start_method() can have the side-effect of starting helper process(es).  I do not really approve of new processes being started as a side-effect of importing a library.  But it is reasonable for a library to want a specific start method unless the user demands otherwise.

I will have to think this over.


BTW, the reason for discouraging using set_start_method() more than once is because some shared resources are created differently depending on what the current start method is.

For instance using the fork method semaphores are created and then immediately unlinked.  But with the other start methods we must not unlink the semaphore until we are finished with it (while being paranoid about cleanup).

Maybe it would be better to have separate contexts for each start method.  That way joblib could use the forkserver context without interfering with the rest of the user's program.

    from multiprocessing import forkserver_context as mp
    l = mp.Lock()
    p = mp.Process(...)
    with mp.Pool() as pool:
        ...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18999>
_______________________________________