asyncio: What is the difference between tasks, futures, and coroutines?

[Zachary Ware]

On Tue, May 5, 2015 at 10:22 AM, Paul  Moore <p.f.moore at gmail.com> wrote:
> I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.

I've been using (and, in part at least, understanding :)) asyncio for
work for the past few months, so I can at least give you my
impressions based on use.

> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".

That's basically how I understand it.

> But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(

My understanding is that Futures are somewhat like 'non-executable
coroutines', if that makes any sense whatsoever.  Futures are used as
something you can pass around when you need to start execution of the
"job" in one method and finish it in another.  For instance, talking
to a remote network entity, and you want to just do "yield from
server.get(something)".  Your 'get' method can make the request,
create a Future, stick it somewhere that the method monitoring
incoming traffic can find it (along with some identifying metadata),
and then do 'return (yield from future)' to wait for the Future to be
fulfilled (by the listener) and return its result.  The listener then
matches up incoming requests with Futures, and calls
Future.set_result() or Future.set_exception() to fulfill the Future.
Once one of those methods has been called on the Future (and control
has been passed back to the scheduler), your server.get method
unblocks from its '(yield from future)' call, and either raises the
exception or returns the result that was set on the Future.

> A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?
>
> Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?

You kind of mixed things up in this paragraph (you said "Tasks somehow
wrap up coroutines", then "wrap a task in a coroutine"; the first is
more correct, I believe).  As I understand it, Tasks are
specializations of Futures that take care of the
set_result/set_exception based on the execution of the coroutine.  I'm
not clear on how that is all implemented (Are all coroutines wrapped
in Tasks?  Just those wrapped in asyncio.async()? No idea.), but I use
it something like this:

   # save a reference to this task so it can be cancelled elsewhere if need be
   self.current_task = asyncio.async(some_coroutine())
   # now the coroutine is scheduled, but we still have control
   # we can do anything else we need to here, including scheduling
other coroutines
   return (yield from self.current_task)

> I concede that I've not read the rest of the asyncio documentation in much detail yet, and I'm skipping everything to do with IO (I want to understand the "async" bit for now, not so much the "IO" side). But I don't really want to dive into the details while I am this hazy on the basic concepts.
>
> Can anyone clarify for me?

I hope I've done some good on that front, but I'm still a bit hazy
myself.  I found it worked best to just take examples or otherwise
working code, and poke and prod at it until it breaks and you can
figure out how you broke it.  Just try to avoid mixing asyncio and
threads for as long as you can :).

-- 
Zach