[issue19946] Handle a non-importable __main__ in multiprocessing

[Nick Coghlan]

Nick Coghlan added the comment:

OK, fixed test case attached. Turns out the ipython workaround test was completely wrong and never even loaded multiprocessing, and hence always passed, even with the workaround disabled. So I fixed that test case, and used the same approach for the zipfile, directory and package tests. I also fixed the submodule test to check that explicit relative imports work properly from __mp_main__ in the child processes.

With this updated test cast, the v2 patch handles everything correctly, but there are 4 failures on Linux without the patch. Specifically:

- test_basic_script_no_suffix fails for the spawn and forkserver start methods (the child processes fail to find a spec for __mp_main__)
- test_module_in_package fails for the spawn and forkserver start methods (the explicit relative import from __mp_main__ fails because the import system isn't initialised correctly in the child processes)

The former case is the one Olivier reported in this issue. It's a new case for 3.4, since the spawn start method was previously only available on Windows, where scripts always have an extension.

The latter edge case is the one my "XXX (ncoghlan): The following code makes several bogus assumptions regarding the relationship between __file__ and a module's real name." comment was about.

I believe we could actually adjust earlier versions to handle things as well as this new PEP 451 based approach (by using a combination of __package__ and __file__ rather than __spec__), but that's much harder for me to work on in those versions where the "spawn" start method is only available on Windows :)

----------
Added file: http://bugs.python.org/file33175/test_multiprocessing_main_handling.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19946>
_______________________________________