Nesting concurrent.futures.ThreadPoolExecutor

[Andrew McLean]

I have a program where I am currently using a 
concurrent.futures.ThreadPoolExecutor to run multiple tasks 
concurrently. These tasks are typically I/O bound, involving access to 
local databases and remote REST APIs. However, these tasks could 
themselves be split into subtasks, which would also benefit from 
concurrency.

What I am hoping is that it is safe to use a 
concurrent.futures.ThreadPoolExecutor within the tasks. I have coded up 
a toy example, which seems to work. However, I'd like some confidence 
that this is intentional. Concurrency is notoriously tricky.

I very much hope this is safe, because otherwise it would not be safe to 
use a ThreadPoolExecutor to execute arbitrary code, in case it also used 
concurrent.futures to exploit concurrency.

Here is the toy example:

> |importconcurrent.futures definner(i,j):returni,j,i**j 
> defouter(i):withconcurrent.futures.ThreadPoolExecutor(max_workers=5)asexecutor:futures 
> ={executor.submit(inner,i,j):j forj inrange(5)}results =[]forfuture 
> inconcurrent.futures.as_completed(futures):results.append(future.result())returnresults 
> defmain():withconcurrent.futures.ThreadPoolExecutor(max_workers=5)asexecutor:futures 
> ={executor.submit(outer,i):i fori inrange(10)}results =[]forfuture 
> inconcurrent.futures.as_completed(futures):results.extend(future.result())print(results)if__name__ 
> =="__main__":main()|
I have previously posted this on Stack Overflow, but didn't get any 
replies. Apologies if you are seeing this twice.

https://stackoverflow.com/questions/44989473/nesting-concurrent-futures-threadpoolexecutor