[Python-Dev] async/await in Python; v2

[Victor Stinner]

Hi,

Guido van Rossum <guido <at> python.org> writes:
> I'm slowly warming up to Greg's notion that you can't call a coroutine (or
whatever it's called) without a special keyword.

A huge part of the asyncio module is based on "yield from fut" where fut is
a Future object.

How do you write this using the PEP 3152? Do you need to call an artifical
method like "cocall fut.return_self()" where the return_self() method simply
returns fut?


When I discovered Python for the first time, I fall into the trap of trying
to call a function without parenthesis: "hello_world" instead of
"hello_world()". I was very surprised that the language didn't protect me
against such obvious bug. But later, I used this feature in almost all my
applications: passing a callback is just a must have feature, and Python
syntax for this is great! (just pass the function without parenthesis)

Would it be possible that creating coroutine object by calling a coroutine
function is a feature, and not a bug? I mean that it may be used in some
cases. I worked a lot of asyncio and I saw a lot of hacks to solve some
corner case issues, to be able to propose a nice API at the end.

@asyncio.coroutine currently calls a function and *then* check if it should
yields from it or not:

    res = func(*args, **kw)
    if isinstance(res, futures.Future) or inspect.isgenerator(res):
        res = yield from res

With the PEP 3152, it's no more possible to write such code. I fear that we
miss cases where it would be needed.

maybeDeferred() is an important function in Twisted. As expected, users ask
for a similar function in asyncio:

http://stackoverflow.com/questions/20730248/maybedeferred-analog-with-asyncio

Currently, it's possible to implement it using yield from.


> OTOH I'm still struggling with what you have to do to wrap a coroutine in
a Task, the way its done in asyncio by the Task() constructor, the
loop.create_task() method, and the async() function (perhaps to be renamed
to ensure_task() to make way for the async keyword).

Logging a warning when a coroutine object is not "consumed" ("yielded
from"?) is only one of the asyncio.CoroWrapper features. It's now also used
to remember where the coroutine object was created: it's very useful to
rebuild the chain of function calls/tasks/coroutines to show where the bug
comes from.

(I still have a project to enhance debugging to create a full stack where a
task was created. Currently, the stack stops at the last "Task._step()", but
it's technically possible to go further (I have a PoC somewhere). I already
introduced BaseEventLoop._current_handle as a first step.)

Oh, and CoroWrapper also provides a better representation. But we might
enhance repr(coroutine_object) directly in Python.

Yury proposed to store the source (filename, line number) of the most recent
frame where a coroutine object was created. But a single frame is not enough
(usually, the interesting frame is at least the 3rd frame, not the most
recent one). Storing more frames would kill performances in debug mode
(and/or create reference cycles if we keep frame objects, not only
filename+line number).

For all these reasons, I'm in favor of keeping the ability of wrapping
coroutine objects. It has a negligible impact in release mode and you can do
whatever you want in debug mode which is very convenient.

Victor