[Python-Dev] PEP 492: async/await in Python; version 4

[Paul Moore]

On 6 May 2015 at 09:20, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> That doesn't cover any of the higher level abstractions like tasks or
>> futures (at least not by those names or with those interfaces).
>
> Because a minimal event loop doesn't *need* those.

It doesn't *need* them, but as abstractions they allow easier building
of reusable higher-level libraries. You can write an event loop with
nothing but coroutines, but to build reusable libraries on top of it,
you need some common interfaces.

> In my little scheduler, a "task" is nothing more than
> a yield-frommable object sitting on a queue of things
> to be run. There is no need to wrap it in another
> object.
>
> And there's really no need for the concept of a
> "future" at all, except maybe at the boundary
> between generator-based async code and other things
> that are based on callbacks. Even then, a "future"
> is really just "an object that can be passed to
> yield-from". There is no need for a concrete
> Future class, it's just a protocol.

Agreed, you don't need a Future class, all you need is to agree what
reusable code is allowed to do with the core objects you are passing
around - that's how duck typing works. The objects *I* can see are
futures (in a PEP 492 world, "awaitables" which may or may not be
equivalent in terms of the operations you'd want to focus on) and the
event loop itself.

In your example, the event loop is implicit (as it's a singleton, you
use global functions rather than methods on the loop object) but
that's a minor detail.

>> And I
>> don't see where the PEP 492 additions would fit in (OK, "replace yield
>> from with await" is part of it, but I don't see the rest).
>
> That's really all there is to it. The rest is
> concerned with catching certain kinds of mistakes,
> and providing convenient syntax for some patterns
> of using 'await'.

So, "things you can wait on" have one operation - "wait for a result".
That's OK. You can create such things as coroutines, which is also
fine. You may want to create such things explicitly (equivalent to
generators vs __iter__) - maybe that's where __aiter__ comes in in PEP
492 and the Future class in asyncio. Again, all fine.

You also need operations like "schedule a thing to run", which is the
event loop "interface". Your sample has the following basic event loop
methods that I can see: run, schedule, unschedule, and
expire_timeslice (that last one may be an implementation detail, but
the other 3 seem pretty basic). PEP 492 has nothing to say on the
event loop side of things (something that became clear to me during
this discussion).

>> There's a lot of asyncio
>> that doesn't seem to me to be IO-related. Specifically the future and
>> task abstractions. I view those as relevant to "coroutine programming
>> in Python" because they are referenced in any discussion of coroutines
>> (you yield from a future, for example).
>
> Only because they've been elevated to prominence
> by asyncio and its documentation, which I regard
> as unfortunate.
>
> When Guido was designing asyncio, I tried very
> hard to dissuade him from giving futures such a
> central place in the model. I saw them as an
> unnecessary concept that would only clutter up
> people's thinking. Seeing all the confusion now,
> I'm more convinced than ever that I was right. :-(

Futures seem to me to be (modulo a few details) what "awaitables" are
in PEP 492. I can't see how you can meaningfully talk about event
loops in a Python context without having *some* term for "things you
wait for". Maybe Future wasn't a good name, and maybe the parallel
with concurrent.futures.Future wasn't helpful (I think both things
were fine, but you may not) but we had to have *some* way of talking
about them, and of constructing standalone awaitables. PEP 492 has
new, and hopefully better, ways, but I think that awaitables *have* to
be central to any model where you wait for things...

By the way, it feels to me like I'm now arguing in favour of PEP 492
with a reasonable understanding of what it "means". Assuming what I
said above isn't complete rubbish, thanks to everyone who's helped me
get to this point of understanding through this thread! (And if I
haven't understood, that's my fault, and still thanks to everyone for
their efforts :-))

Paul