[Python-ideas] [Python-Dev] Python needs a standard asynchronous return object

[Guido van Rossum]

On Sun, Sep 12, 2010 at 9:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 12 Sep 2010 08:49:56 -0700
> Guido van Rossum <guido at python.org> wrote:
>>
>> Sure, but the tricky thing is to make it pluggable so that PEP 3148
>> and Twisted and other frameworks can use it all together, and a single
>> call will accept a mixture of Futures.
>
> Having a common abstraction (Future or Deferred) allows for
> scheduling-agnostic libraries which consume and/or produce these
> abstractions (*). I'm not sure it is desireable to mix scheduling models
> in a single process (let alone a single thread), though.

IIRC even Twisted supports putting stuff in a thread if you really
need it. And have you looked at Go's Goroutines? They are a hybrid --
they don't map 1:1 to OS threads, but they aren't pure coroutines
either, so that if a goroutine blocks on I/O the others will still
make progress.

> (*) Of course, the abstraction is somehow leaky since being called from
> different threads, depending on the scheduling model, could have adverse
> consequences

Yeah, this is always a problem with pure async frameworks -- if one
callback or coroutine blocks by mistake, the whole world is blocked.
(So Goroutines attempt to fix this; I have no idea how successful they
are.)

>> ISTM that the main difference is that add_done_callback() isn't meant
>> for callbacks that return a value. So then the exceptions that might
>> be raised are kind of "out of band".
>
> It implies that it's mostly useful for simple callbacks (which would
> e.g. print out a success report, or set an Event to wake up another
> thread). The Twisted model allows the major part of processing to occur
> in the callbacks themselves, in which case proper error handling and
> propagation is mandatory.

A generator-based coroutines approach can do this too (just put the
work between the yields in the generator) and has all the proper
exception-propagation stuff built in since PEP 342 (PEP 380 will just
make it easier).

And a Futures-based approach can do it too -- it's not described in
PEP 3148, but you can easily design an API for wrappable Futures.

-- 
--Guido van Rossum (python.org/~guido)