Multiprocessing vs. concurrent.futures, Linux vs. Windows

[John Ladasky]

Several years ago I built an application using multiprocessing.  It only needed to work in Linux.  I got it working fine.  At the time, concurrent.futures did not exist.

My current project is an application which includes a PyQt5 GUI, and a live video feed with some real-time image processing.  Running all this in one process resulted in unacceptable performance, so I'm trying to move all of the heavy video work into its own process.  I only need to define one child process.  I don't need a Pool.  The child Process can run indefinitely, and it will communicate multiple messages to the main process using a Pipe.

I built a working example in Linux, but it hangs in Windows.  I built a minimum example.  My problem has nothing to do with PyQt.  In Windows, my example hangs when I try to create the child Process.  Code:


import os
from multiprocessing import Pipe, Process

def child_app():
    inlet.send("Child process id = {}".format(os.getpid()))

if __name__ == "__main__":
    outlet, inlet = Pipe()
    print("Parent process id =", os.getpid())
    child_process = Process(target=child_app)  # Windows hangs here
    child_process.start()
    print(outlet.recv())
    child_process.join()
    child_process.close()
    print("Program complete.")


I'm working in Python 3.7.6 on Windows 10, and 3.7.5 on Ubuntu Linux 19.10.

I am reading through the multiprocessing documentation, and I'm guessing that I've run into a problem with spawning vs. forking a new Process.  The former is the Windows default, and the latter is the Posix default.

Before I dig too deeply, I am wondering whether this cross-platform problem is something that concurrent.futures might handle automatically.  The concurrent API looks foreign to me.  But if it is meant to replace multiprocessing, I suppose this would be a good time for me to learn it.

Thanks for any advice and suggestions!