Python concurrent.futures.ProcessPoolExecutor

[Israel Brewster]

> On Dec 16, 2020, at 7:04 AM, Rob Rosengard <robert.rosengard at gmail.com> wrote:
> 
> Warning:  I am new to this group
> Warning:  I am not an expert at Python, I've written a few small programs, and spend 20 hours of online classes, and maybe a book or two. 
> Warning:  I am new to trying to use concurrent.futures.ProcessPoolExecutor
> - Prior to writing this question I updated to Python 3.9 and PyCharm 2020.3.  And confirmed the problem still exists. 
> - Running on Windows 10 Professional
> - I've been trying to run a simple piece of code to exactly match what I have seen done in various training videos.  By I am getting a different and 
> unexpected set of results.  I.e. the instructor got different results than I did on my computer.  My code is very simple:
> 
> import concurrent.futures
> import time
> 
> 
> start = time.perf_counter()
> 
> 
> def task(myarg):
>    print(f'Sleeping one second...{myarg}')
>    time.sleep(1)
>    return 'Done sleeping...'
> 
> 
> if __name__ == '__main__':
>    with concurrent.futures.ProcessPoolExecutor() as executor:
>        future1 = executor.submit(task, 1)
>        future2 = executor.submit(task, 2)
> finish = time.perf_counter()
> print(f'Finished in {round(finish-start,2)} seconds')
> 
> And the output is: 
> Finished in 0.0 seconds
> Finished in 0.0 seconds
> Sleeping one second...1
> Sleeping one second...2
> Finished in 1.14 seconds
> 
> Process finished with exit code 0
> 
> --- 
> QUESTIONS and CONCERNS that I have...
> It seems that both calls to task not only runs that function, but then keeps executing the rest of the main line code.  I only expected it to run the function and then immediately quit/disappear.   That is, I expect the output to look like (i.e. not having the three lines of "Finished in x.x seconds", rather, just one line like that):
> Sleeping one second...1
> Sleeping one second...2
> Finished in 1.14 seconds
> 
> Goal:  I need the executor tasks to only run that one function, and then completely go away and stop.  Not keep executing more code that doesn't belong to the task function. 
> 
> I've tried many iterations of this issue, and placed PRINT statements all over to try to track what is going on.  And I used if/else statements in the main code, which caused even more problems.  I.e. both the IF and the ELSE was executed each time through the code. Which completely blows my mind. 
> 
> Any thoughts on this?  Thanks for your time and help!  

Assuming the code above is indented exactly as you run it, you have an indentation error. That is, the finish and print() are not indented to be part of the if __name__… call. As such, they run on import. When you launch a new process, it imports the module, which then runs those lines, since they are not guarded by the if statement.

Indent those last two lines to be under the if (they don’t need to be indented to be under the with, just the if), and it should work as intended.

---
Israel Brewster
Software Engineer
Alaska Volcano Observatory 
Geophysical Institute - UAF 
2156 Koyukuk Drive 
Fairbanks AK 99775-7320
Work: 907-474-5172
cell:  907-328-9145

> 
> R
> -- 
> https://mail.python.org/mailman/listinfo/python-list