[Python-ideas] How the heck does async/await work in Python 3.5

[Ian Kelly]

On Wed, Feb 24, 2016 at 8:23 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Tem Pl <rtempl31 at gmail.com>:
>
>> Here are some concurrency benchmarks for python vs other languages.
>>
>> https://github.com/atemerev/skynet/pull/53
>>
>> Is there something wrong with this implementation?
>
> It's a "fork bomb".

Isn't that the point of the benchmark?

But yeah, I tried playing with this and here are my results. It's
clearly pushing all those coroutines down into the event loop that
causes the problem. If you remove the asyncio.as_completed calls and
just await the descendant coroutines in order, then it's a lot faster
-- but this may be technically cheating because then the coroutines
are effectively just running recursively without involving the event
loop, so there's no exercise of actual concurrency.

# Straight-up recursion: 400 ms
def func(level=0, index=0):
    if level < LEVELS:
        sons = [func(level=level+1, index=index*SONS + x) for x in range(SONS)]
        sum_ = 0
        for son in sons:
            sum_ += son
        return sum_
    else:
        return index

import timeit
print(timeit.repeat('func()', 'from __main__ import func', number=10))
# [4.016848850995302, 4.1330014310078695, 4.149791953997919]

# Concurrent coroutines with event loop: 30 s
import asyncio
async def coroutine(level=0, index=0):
    if level < LEVELS:
        sons = [coroutine(level=level+1, index=index*SONS + x) for x
in range(SONS)]
        sum_ = 0
        for f in asyncio.as_completed(sons):
            got = await f
            sum_ += got
        return sum_
    else:
        return index

print(timeit.repeat('loop.run_until_complete(coroutine())', 'from
__main__ import coroutine; import asyncio; loop =
asyncio.get_event_loop()', number=1))
# [29.884846250002738, 30.26590966898948, 30.716448744002264]

# Recursion with coroutines: 600 ms
async def coro2(level=0, index=0):
    if level < LEVELS:
        sons = [coro2(level=level+1, index=index*SONS + x) for x in range(SONS)]
        sum_ = 0
        for son in sons:
            got = await son
            sum_ += got
        return sum_
    else:
        return index

print(timeit.repeat('loop.run_until_complete(coro2())', 'from __main__
import coro2; import asyncio; loop = asyncio.get_event_loop()',
number=1))
# [0.6264017040084582, 0.590631059021689, 0.5807875629980117]

# Recursive coroutines, no event loop: 600 ms
print(timeit.repeat('list(coro2().__await__())', 'from __main__ import
coro2', number=10))
# [6.028770218021236, 6.002665672975127, 5.987304503010819]