[Python-Dev] async/await in Python; v2

[Andrew Svetlov]

On Thu, Apr 23, 2015 at 3:35 AM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Apr 22, 2015 at 5:12 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
> wrote:
>>
>> Guido van Rossum wrote:
>>>
>>> On Wed, Apr 22, > OTOH I'm still struggling with what you have to do to
>>> wrap a coroutine in a Task, the way its done in asyncio by the Task()
>>> constructor, the loop.create_task() method, and the async() function
>>
>>
>> That's easy. You can always use costart() to adapt a cofunction
>> for use with something expecting a generator-based coroutine,
>> e.g.
>>
>> codef my_task_func(arg):
>>   ...
>>
>> my_task = Task(costart(my_task_func, arg))
>>
>> If you're willing to make changes, Task() et al could be made to
>> recognise cofunctions and apply costart() where needed.
>
>
> Hm, that feels backwards incompatible (since currently I can write
> Task(my_task_func(arg)) and also a step backwards in elegance (having to
> pass the args separately).
>
> OTOH the benefit is that it's much harder to accidentally forget to wait for
> a coroutine. And maybe the backward compatibility issue is not really a
> problem because you have to opt in by using codef or async def.
>
> So I'm still torn. :-)
>


> Somebody would need to take a mature asyncio app and see how often this is
> used (i.e. how many place would require adding costart() as in the above
> example).
>

I have not found fresh patch for 3152 to play with, but at least
aiohttp [1] library very often creates new tasks by `async(coro(...))`
call. The same for aiozmq, aioredis, sockjs (aiohttp-based library for
sock.js), aiokafka etc.

My applications created for my job also has a `async(...)` calls or
direct `Task(f(arg))` creations -- the numbers are between 3 and 10
usage lines per application. Not a big deal to fix them all but it's
backward incompatibility.

In opposite, I've finished experimental branch [2] of aiomysql library
(asyncio driver for MySQL database) with support for `async for` and
`async with`.

The main problem with public released version is impossibility to
handle transactions (requires async context manager) and iteration
with async fetching data from cursor (required for server-side cursors
for example).

Now both problems are solved with keeping full backward compatibility.
The library can be used with Python 3.3+ but obviously no new features
are available for old Pythons.

I use asyncio coroutines, not async functions, e.g.:

class Cursor:

    # ...

    @asyncio.coroutine
    def __aiter__(self):
        return self

    @asyncio.coroutine
    def __anext__(self):
        ret = yield from self.fetchone()
        if ret is not None:
            return ret
        else:
            raise StopAsyncIteration

The whole aiomysql code is correct from Python 3.3+ perspective. For
testing new features I use new syntax of in separate test files, test
runner will skip test modules with syntax errors on old Python but run
those modules on python from PEP 492 branch.

Usage example (table 'tbl' is pre-filled, DB engine is connected to server):

    async def go(engine):
        async with engine.connect() as conn:
            async with (await conn.begin()) as tr:
                await conn.execute("DELETE FROM tbl WHERE (id % 2) = 0")

                async for row in conn.execute("SELECT * FROM tbl"):
                    print(row['id'], row['name'])


[1] https://github.com/KeepSafe/aiohttp

[2] https://github.com/aio-libs/aiomysql/tree/await

> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>



-- 
Thanks,
Andrew Svetlov