Python concurrent.futures.ProcessPoolExecutor

[MRAB]

On 2020-12-16 16:04, Rob Rosengard wrote:
> Warning:  I am new to this group
> Warning:  I am not an expert at Python, I've written a few small programs, and spend 20 hours of online classes, and maybe a book or two.
> Warning:  I am new to trying to use concurrent.futures.ProcessPoolExecutor
> - Prior to writing this question I updated to Python 3.9 and PyCharm 2020.3.  And confirmed the problem still exists.
> - Running on Windows 10 Professional
> - I've been trying to run a simple piece of code to exactly match what I have seen done in various training videos.  By I am getting a different and
> unexpected set of results.  I.e. the instructor got different results than I did on my computer.  My code is very simple:
> 
> import concurrent.futures
> import time
> 
> 
> start = time.perf_counter()
> 
> 
> def task(myarg):
>      print(f'Sleeping one second...{myarg}')
>      time.sleep(1)
>      return 'Done sleeping...'
> 
> 
> if __name__ == '__main__':
>      with concurrent.futures.ProcessPoolExecutor() as executor:
>          future1 = executor.submit(task, 1)
>          future2 = executor.submit(task, 2)
> finish = time.perf_counter()
> print(f'Finished in {round(finish-start,2)} seconds')
> 
> And the output is:
> Finished in 0.0 seconds
> Finished in 0.0 seconds
> Sleeping one second...1
> Sleeping one second...2
> Finished in 1.14 seconds
> 
> Process finished with exit code 0
> 
> ---
> QUESTIONS and CONCERNS that I have...
> It seems that both calls to task not only runs that function, but then keeps executing the rest of the main line code.  I only expected it to run the function and then immediately quit/disappear.   That is, I expect the output to look like (i.e. not having the three lines of "Finished in x.x seconds", rather, just one line like that):
> Sleeping one second...1
> Sleeping one second...2
> Finished in 1.14 seconds
> 
> Goal:  I need the executor tasks to only run that one function, and then completely go away and stop.  Not keep executing more code that doesn't belong to the task function.
> 
> I've tried many iterations of this issue, and placed PRINT statements all over to try to track what is going on.  And I used if/else statements in the main code, which caused even more problems.  I.e. both the IF and the ELSE was executed each time through the code. Which completely blows my mind.
> 
> Any thoughts on this?  Thanks for your time and help!
> 
It imports the module to run the function, so any top-level code _will_ 
be run.

Here's a modified version:

--8<----------------------------8<--

import concurrent.futures
import time

print('Main code, __name__ is {!a}'.format(__name__))

start = time.perf_counter()


def task(myarg):
     print('In task, __name__ is {!a}'.format(__name__))
     print(f'Sleeping one second...{myarg}')
     time.sleep(1)
     return 'Done sleeping...'


if __name__ == '__main__':
     with concurrent.futures.ProcessPoolExecutor() as executor:
         future1 = executor.submit(task, 1)
         future2 = executor.submit(task, 2)
finish = time.perf_counter()
print(f'Finished in {round(finish-start,2)} seconds')

--8<----------------------------8<--

It prints:

--8<----------------------------8<--

Main code, __name__ is '__main__'
Main code, __name__ is '__mp_main__'
Finished in 0.0 seconds
Main code, __name__ is '__mp_main__'
Finished in 0.0 seconds
In task, __name__ is '__mp_main__'
Sleeping one second...1
In task, __name__ is '__mp_main__'
Sleeping one second...2
Finished in 1.8 seconds

--8<----------------------------8<--

Any top-level code that you don't want it to run when it re-imports the 
module should be protected by the __name__ test.