[Async-sig] async executors

[Christoph Groth]

Łukasz Langa wrote:

>> As far as I can see the original 
>> concurrent.futures.ProcessPoolExecutor has no advantage when 
>> used in asyncio-based programs except when some coroutine 
>> blocks for long enough for the call queue to become empty. 
>> (But hey, async programming is about non-blocking coroutines!)
>
> That’s incorrect. The ProcessPoolExecutor is a way for you to 
> run CPU-intensive tasks without affecting the responsiveness of 
> the main event loop in your program. As long as you keep your 
> arguments small (pickling/unpickling big data structures is 
> pretty detrimental to performance here) and the duration of the 
> task is non-trivial, the child processes enable you to use more 
> CPU cores, and since they have their own GILs, this doesn't 
> affect your loop at all, unlike using a thread pool.

I was comparing the (possible) advantages of the traditional 
concurrent.futures.ProcessPoolExecutor with the new 
aexecutor.ProcessPoolExecutor.  Both use processes for the 
workers, so the GIL is not a problem for either.

Christoph
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/async-sig/attachments/20160613/7cbf8c11/attachment.sig>