Asynchronous programming

[Paul Rudin]

Steven D'Aprano <steve+python at pearwood.info> writes:

> Thanks to everyone who has answered, I think I'm slowly starting to get it
> now. Let's see if we can come up with a toy example that doesn't involve
> low-level socket programming :-)
>
> Let me simulate a slow function call:
>
>
> import random, time
>
> def work(id):
>     print("starting with id", id)
>     workload = random.randint(5, 15)
>     for i in range(workload):
>         time.sleep(0.2)  # pretend to do some real work
>         print("processing id", id)  # let the user see some progress
>     print("done with id", id)
>     return 10 + id
>
>
> pending = [1, 2, 3, 4]
>
> for i, n in enumerate(pending):
>     pending[i] = work(n)
>
>
> How do I write work() so that it cooperatively multi-tasks with other ...
> threads? processes? what the hell do we call these things? What does this
> example become in the asynchronous world?
>

They're not separate processes or threads, just think tasks that suspend
and restart cooperatively. You need some kind of event loop to
orchestrate running the tasks. Asyncio provides one.

You can do your example like this:

import asyncio, random

async def work(id):
    print("starting with id", id)
    workload = random.randint(5, 15)
    for i in range(workload):
        await asyncio.sleep(0.2)  # pretend to do some real work
        print("processing id", id)  # let the user see some progress
    print("done with id", id)
    return 10 + id

loop = asyncio.get_event_loop()

pending = [1, 2, 3, 4]

jobs = [asyncio.ensure_future(work(n)) for n in pending]
     
loop.run_until_complete(asyncio.gather(*jobs))

loop.close()


> In this case, all the work is pure computation, so I don't expect to save
> any time by doing this, because the work is still all being done in the
> same process/thread, not in parallel. It may even take a little bit longer,
> due to the overhead of switching from one to another.
>

Yeah - you're not gaining anything here. All (?) the interesting use
cases involve waiting for something (e.g. disk access, network
communication, separate process) before you can proceed with your
computation.