[Python-ideas] Expressiveness of coroutines versus Deferred callbacks (or possibly promises, futures)

[Greg Ewing]

Glyph wrote:

> The real problem with generator coroutines is that if you make them the 
> primitive, you have an abstraction inversion if you want to have 
> callbacks

Has anyone suggested making generator coroutines "the primitive",
whatever that means?

Guido seems to have made it clear that he wants the interface
to the event loop layer to be based on plain callbacks. To plug
in a generator coroutine, you install a callback that wakes up
the coroutine. So using generators with the event loop will be
entirely optional.

> I haven't measured in a few 
> years, but as nice as generators can be for structuring complex event 
> flows, that abstraction comes with a non-trivial performance cost. 
> ...  Every return value being replaced with 
> a callback trampoline is bad, but replacing it instead with a generator 
> being advanced, an exception being raised /and /a callback trampoline is 
> worse.

This is where we expect yield-from to help a *lot*, by removing
almost all of that overhead. A return to the trampoline is only
needed when a task wants to yield the CPU, instead of every time
it makes a function call to a subgenerator.

Returns are still a bit more expensive due to the StopIterations,
but raising and catching an exception in C code is still fairly
efficient compared to doing it in Python. (Although not quite as
super-efficient as it was in Python 2.x, unfortunately, due to
tracebacks being attached to exceptions, so that we can't
instantiate exceptions lazily any more.)

-- 
Greg