Standard Asynchronous Python

[Dustin J. Mitchell]

After seeing David Mertz's talk at PyCon 2012, "Coroutines, event
loops, and the history of Python generators" [1], I got thinking again
about Python's expressive power for asynchronous programming.
Generators, particularly with the addition of 'yield from' and
'return' in PEP 380 [2], allow us to write code that is executed "bit
by bit" but still reads naturally.  There are a number of frameworks
that take advantage of this ability, but each is a little different --
enough so that there's negligible code re-use between these
frameworks.  I think that's a shame.

I proposed a design PEP a while back [3] with the intent of defining a
standard way of writing asynchronous code, with the goal of allowing
code re-use and bringing users of the frameworks closer together.
Ideally, we could have libraries to implement network protocols,
database wrappers, subprocess execution, and so on, that would work in
any of the available asynchronous frameworks.

My proposal met with near-silence, and I didn't pursue it.  Instead, I
did what any self-respecting hacker would do - I wrote up a framework,
uthreads [4], that implemented my idea.  This was initially a simple
trampoline scheduler, but I eventually refactored it to run atop
Twisted, since that's what I use.  To my knowledge, it's never been
used.

I'm considering re-drafting the PEP with the following changes:

 * De-emphasize the thread emulation aspects, and focus on
code-portability issues:
   * callbacks vs. "blocking" calls (e.g., when accepting incoming
connections on a socket, how is my code invoked?)
   * consistent access to primitives, regardless of framework (e.g.,
where's the function I call to branch execution?)
   * nested asynchronous methods
 * Account for PEP 380 (by making the StopIteration workarounds match
PEP 380, and explicitly deprecating them after Python 3.3)
 * Look forward to a world with software transactional memory [5] by
matching that API where appropriate

As I get to work on the PEP, I'd like to hear any initial reactions to the idea.

Dustin

[1] https://us.pycon.org/2012/schedule/presentation/104/
[2] http://www.python.org/dev/peps/pep-0380
[3] http://code.google.com/p/uthreads/source/browse/trunk/microthreading-pep.txt
[4] http://code.google.com/p/uthreads/
[5] https://bitbucket.org/pypy/pypy/raw/stm-thread/pypy/doc/stm.rst