Python concurrent.futures.ProcessPoolExecutor

[Dan Stromberg]

On Wed, Dec 16, 2020 at 9:23 AM Israel Brewster <ijbrewster at alaska.edu>
wrote:

> > On Dec 16, 2020, at 7:04 AM, Rob Rosengard <robert.rosengard at gmail.com>
> wrote:
> >
> > Warning:  I am new to this group
> > Warning:  I am not an expert at Python, I've written a few small
> programs, and spend 20 hours of online classes, and maybe a book or two.
> > Warning:  I am new to trying to use
> concurrent.futures.ProcessPoolExecutor
> > - Prior to writing this question I updated to Python 3.9 and PyCharm
> 2020.3.  And confirmed the problem still exists.
> > - Running on Windows 10 Professional
> > - I've been trying to run a simple piece of code to exactly match what I
> have seen done in various training videos.  By I am getting a different and
> > unexpected set of results.  I.e. the instructor got different results
> than I did on my computer.  My code is very simple:
> >
> > import concurrent.futures
> > import time
> >
> >
> > start = time.perf_counter()
> >
> >
> > def task(myarg):
> >    print(f'Sleeping one second...{myarg}')
> >    time.sleep(1)
> >    return 'Done sleeping...'
> >
> >
> > if __name__ == '__main__':
> >    with concurrent.futures.ProcessPoolExecutor() as executor:
> >        future1 = executor.submit(task, 1)
> >        future2 = executor.submit(task, 2)
> > finish = time.perf_counter()
> > print(f'Finished in {round(finish-start,2)} seconds')
> >
> > And the output is:
> > Finished in 0.0 seconds
> > Finished in 0.0 seconds
> > Sleeping one second...1
> > Sleeping one second...2
> > Finished in 1.14 seconds
> >
> > Process finished with exit code 0
>


> Assuming the code above is indented exactly as you run it, you have an
> indentation error. That is, the finish and print() are not indented to be
> part of the if __name__… call. As such, they run on import. When you launch
> a new process, it imports the module, which then runs those lines, since
> they are not guarded by the if statement.
>
> Indent those last two lines to be under the if (they don’t need to be
> indented to be under the with, just the if), and it should work as intended.
>

Windows has a problem forking, so the indentation is more persnickety
there. On Linux and I believe on Mac, you don't have to be so careful with
multiprocessing and concurrent.futures.