Question about asyncio and blocking operations

[Frank Millman]

"Ian Kelly"  wrote in message 
news:CALwzidnGOgpx+CpMVBA8vpEFuq4-BwMVS0gZ3ShB0oWZi0Bw+Q at mail.gmail.com...
>
> On Sat, Jan 23, 2016 at 7:38 AM, Frank Millman <frank at chagford.com> wrote:
> > Here is the difficulty. The recommended way to handle a blocking 
> > operation
> > is to run it as task in a different thread, using run_in_executor(). 
> > This
> > method is a coroutine. An implication of this is that any method that 
> > calls
> > it must also be a coroutine, so I end up with a chain of coroutines
> > stretching all the way back to the initial event that triggered it.
>
> This seems to be a common misapprehension about asyncio programming.
> While coroutines are the focus of the library, they're based on
> futures, and so by working at a slightly lower level you can also
> handle them as such. So  while this would be the typical way to use
> run_in_executor:
>
> async def my_coroutine(stuff):
>     value = await get_event_loop().run_in_executor(None,
> blocking_function, stuff)
>     result = await do_something_else_with(value)
>     return result
>
> This is also a perfectly valid way to use it:
>
> def normal_function(stuff):
>     loop = get_event_loop()
>     coro = loop.run_in_executor(None, blocking_function, stuff)
>     task = loop.create_task(coro)
>     task.add_done_callback(do_something_else)
>     return task

I am struggling to get my head around this.

1. In the second function, AFAICT coro is already a future. Why is it 
necessary to turn it into a task? In fact when I tried that in my testing, I 
got an assertion error -

File: "C:\Python35\lib\asyncio\base_events.py", line 211, in create_task
    task = tasks.Task(coro, loop=self)
File: "C:\Python35\lib\asyncio\tasks.py", line 70, in __init__
    assert coroutines.iscoroutine(coro), repr(coro)
AssertionError: <Future pending ... >

2. In the first function, calling 'run_in_executor' unblocks the main loop 
so that it can continue with other tasks, but the function itself is 
suspended until the blocking function returns. In the second function, I 
cannot see how the function gets suspended. It looks as if the blocking 
function will run in the background, and the main function will continue.

I would like to experiment with this further, but I would need to see the 
broader context - IOW see the 'caller' of normal_function(), and see what it 
does with the return value.

I feel I am getting closer to an 'aha' moment, but I am not there yet, so 
all info is appreciated.

Frank