[Python-Dev] Minimal async event loop and async utilities (Was: PEP 492: async/await in Python; version 4)

[Guido van Rossum]

On Wed, May 6, 2015 at 1:27 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 6 May 2015 at 07:46, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> > Another problem with the "core" idea is that
> > you can't start with an event loop that "just does
> > scheduling" and then add on other features such
> > as I/O *from the outside*. There has to be some
> > point at which everything comes together, which
> > means choosing something like select() or
> > poll() or I/O completion queues, and build that
> > into the heart of your event loop. At that point
> > it's no longer something with a simple core.
>
> Looking at asyncio.queues, the only features it needs are:
>
> 1. asyncio.events.get_event_loop()
> 2. asyncio.futures.Future - creating a standalone Future
> 3. asyncio.locks.Event
> 4. @coroutine
>
> locks.Event in turn only needs the other 3 items. And you can ignore
> get_event_loop() as it's only used to get the default loop, you can
> pass in your own.
>
> And asyncio.futures only uses get_event_loop (and _format_callback)
> from asyncio.events.
>
> Futures require the loop to support:
> 1. call_soon
> 2. call_exception_handler
> 3. get_debug
>
> So, to some extent (how far is something I'd need to code up a loop to
> confirm) you can build the Futures and synchronisation mechanisms with
> an event loop that supports only this "minimal interface".
>
> Essentially, that's my goal - to allow people who want to write (say)
> a Windows GUI event loop, or a Windows event loop based of
> WaitForXXXObject, or a Tkinter loop, or whatever, to *not* have to
> write their own implementation of synchronisation or future objects.
>
> That may mean lifting the asyncio code and putting it into a separate
> library, to make the separation between "asyncio-dependent" and
> "general async" clearer. Or if asyncio's provisional status doesn't
> last long enough to do that, we may end up with an asyncio
> implementation and a separate (possibly 3rd party) "general"
> implementation.
>

This is actually a great idea, and I encourage you to go forward with it.
The biggest piece missing from your inventory is probably Task, which is
needed to wrap a Future around a coroutine.

I expect you'll also want to build cancellation into your "base async
framework"; and the primitives to wait for multiple awaitables. The next
step would be some mechanism to implement call_later()/call_at() (but this
needs to be pluggable since for a "real" event loop it needs to be
implemented by the basic I/O selector).

If you can get this working it would be great to include this in the stdlib
as a separate "asynclib" library. The original asyncio library would then
be a specific implementation (using a subclass of asynclib.EventLoop) that
adds I/O, subprocesses, and integrates with the selectors module (or with
IOCP, on Windows).

I don't see any particular hurry to get this in before 3.5; the refactoring
of asyncio can be done later, in a backward compatible way. It would be a
good way to test the architecture of asyncio!

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150506/f856582c/attachment.html>