Multiprocessing vs. concurrent.futures, Linux vs. Windows

[Terry Reedy]

On 5/4/2020 9:05 PM, John Ladasky wrote:
> On Monday, May 4, 2020 at 4:09:53 PM UTC-7, Terry Reedy wrote:
[snip]

> Hi Terry,
> 
> Thanks for your reply.  I have been hacking at this for a few hours.  I have learned two things:
> 
> 1. Windows hangs unless you explicitly pass any references you want to use in the subprocess through args.  That would include the Pipe connection.
>   Using multiprocessing in Linux requires the reference names to be global, however the use of args is not required.  Finally, Linux does not appear to cause any problems if args are specified.
> 2. Even if you fix problem 1, the parent process must be distinguished by creating the subprocess inside an "if __name__ == '__main__'" block.  Again, Linux just pushes on through, but Windows will hang if you don't do this.
> 
> The example code you posted shows exactly these two changes.  They are OS-specific, and my multiprocessing code has been (up to now) only required to run on Linux.  These recommendations can be found in the official Python docs, e.g.:
> 
> https://docs.python.org/3.7/library/multiprocessing.html#programming-guidelines

I was aware of point 2 (use __name__ clause) because there was a time 
when some of the examples in the doc did not do that, and they failed 
when run on Windows.  So there was a patch to fix the examples and add 
the explicit advice.

I must have read point 1 (pass resources in args) but forgot.  But 
knowing that the examples are, I believe, now tested on Windows, I 
looked for the one closest to your code and then saw the difference.

Good luck with moving forward.