[Python-Dev] [PEP 3148] futures - execute computations asynchronously

[Phillip J. Eby]

At 01:19 AM 3/6/2010, Jeffrey Yasskin wrote:
>On Fri, Mar 5, 2010 at 10:11 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> > I'm somewhat concerned that, as described, the proposed API ... 
> [creates] yet another alternative (and
> > mutually incompatible) event loop system in the stdlib ...
>
>Futures are a blocking construct; they don't involve an event loop.

And where they block is in a loop, waiting for events (completed 
promises) coming back from other threads or processes.

The Motivation section of the PEP also stresses avoiding reinvention 
of such loops, and points to the complication of using more than one 
at a time as a justification for the mechanism.  It seems relevant to 
at least address why wrapping multiprocessing and multithreading is 
appropriate, but *not* dealing with any other form of sync/async 
boundary, *or* composition of futures.

On which subject, I might add, the PEP is silent on whether executors 
are reentrant to the called code.  That is, can I call a piece of 
code that uses futures, using the futures API?  How will the called 
code know what executor to use?  Must I pass it one explicitly?  Will 
that work across threads and processes, without explicit support from the API?

IOW, as far as I can tell from the PEP, it doesn't look like you can 
compose futures without *global* knowledge of the application...  and 
in and of itself, this seems to negate the PEP's own motivation to 
prevent duplication of parallel execution handling!

That is, if I use code from module A and module B that both want to 
invoke tasks asynchronously, and I want to invoke A and B 
asynchronously, what happens?  Based on the design of the API, it 
appears there is nothing you can do except refactor A and B to take 
an executor in a parameter, instead of creating their own.

It seems therefore to me that either the proposal does not define its 
scope/motivation very well, or it is not well-equipped to address the 
problem it's setting out to solve.  If it's meant to be something 
less ambitious -- more like a recipe or example -- it should properly 
motivate that scope.  If it's intended to be a robust tool for 
composing different pieces of code, OTOH, it should absolutely 
address the issue of writing composable code...  since, that seems to 
be what it says the purpose of the API is.  (I.e., composing code to 
use a common waiting loop.)

And, existing Python async APIs (such as Twisted's Deferreds) 
actually *address* this issue of composition; the PEP does 
not.  Hence my comments about not looking at existing implementations 
for API and implementation guidance.  (With respect to what the API 
needs, and how it needs to do it, not necessarily directly copying 
actual APIs or implementations.  Certainly some of the Deferred API 
naming has a rather, um, "twisted" vocabulary.)