[Python-ideas] asyncore: included batteries don't fit

[Guido van Rossum]

On Tue, Oct 9, 2012 at 2:11 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Ben Darnell wrote:
>
>> StackContext doesn't quite give you better tracebacks, although I
>> think it could be adapted to do that.  ExceptionStackContext is
>> essentially a try/except block that follows you around across
>> asynchronous operations - on entry it sets a thread-local state, and
>> all the tornado asynchronous functions know to save this state when
>> they are passed a callback, and restore it when they execute it.

> This is something that generator-based coroutines using
> yield-from ought to handle a lot more cleanly. You should
> be able to just use an ordinary try-except block in your
> generator code and have it do the right thing.

Indeed, in NDB this works great. However tracebacks don't work so
great: If you don't catch the exception right away, it takes work to
make the tracebacks look right when you catch it a few generator calls
down on the (conceptual) stack. I fixed this to some extent in NDB, by
passing the traceback explicitly along when setting an exception on a
Future; before I did this, tracebacks looked awful. But there are
still StackContextquite a few situations in NDB where an uncaught
exception prints a baffling traceback, showing lots of frames from the
event loop and other async machinery but not the user code that was
actually waiting for anything. I have to study Tornado's to see if
there are ideas there for improving this.

> I hope that the new async core will be designed so that
> generator-based coroutines can be plugged into it directly
> and efficiently, without the need for a lot of decorators,
> callbacks, Futures, etc. in between.

That has been my hope too. But so far when thinking about this
recently I have found the goal elusive -- somehow it seems there *has*
to be a distinction between an operation you just *yield* (this would
be waiting for a specific low-level I/O operation) and something you
use with yield-from, which returns a value through StopIteration. I
keep getting a headache when I think about this, so there must be a
Monad in there somewhere... :-( Perhaps you can clear things up by
showing some detailed (but still simple enough) example code to handle
e.g. a simple web client?

-- 
--Guido van Rossum (python.org/~guido)