[Python-ideas] async: feedback on EventLoop API

[Jasper St. Pierre]

I guess this is a good place as any to bring this up, but we really need to
address issues with error handling and things like par().

par() has one way to handle errors: if one task (using it as a general term
to encompass futures and coroutines) fails, all tasks fail.

This is nowhere near acceptable. As a simple example, par(grab_page("
http://google.com"), grab_page("http://yahoo.com")) should not fail if one
of the two sites returns a 500; the results of another may still be useful
to us.

I can think of an approach that doesn't require passing more arguments to
par(), but may be absurdly silly: the results generated by par() are not
directly results returned by the task, but instead an intermediate wrapper
value that allows us to hoist the error handling into the caller.

    for intermediate in par(*tasks):
        try:
            result = intermediate.result()
        except ValueError as e:
            print("bad")
        else:
            print("good")

But this makes the trade-off that you can't immediately cancel all the
other tasks when one task fails.

The only truly way to be notified when a task has finished, either with
success, with error, is a callback, which I think we should flesh out
entirely in our Futures model.

And, of course, we should make sure that we can handle the four situations
mentioned in [0] , even if we don't solve them with callbacks.

[0] https://gist.github.com/3889970


On Tue, Dec 18, 2012 at 11:28 PM, Guido van Rossum <guido at python.org> wrote:

> The point of PEP 3156 is not to make using callbacks easy. It is to
> make callbacks mostly disappear in favor of coroutines, but keeping
> them around in order to provide interoperability with callback-based
> frameworks such as Twisted or Tornado.
>
> Your handlers appear to be an attempt at reinventing Twisted's
> Deferred. But Deferred already exists, and it works perfectly fine
> with the current callback-based event loop spec in the PEP. It's not
> clear how your handlers will enable a coroutine to wait for the result
> (or exception) however.
>
> --Guido
>
> On Tue, Dec 18, 2012 at 7:44 PM, Shane Green <shane at umbrellacode.com>
> wrote:
> > Ignoring the API overlap with Futures/Promises for a moment, let me throw
> > out this straw man approach to the event loop that seems to my naive eye
> > like it pull together a lot of these ideas…
> >
> > Rather than passing in your callbacks, factories, etc., asynchronous APIs
> > return a lightweight object you register your callback with.  Unlike
> > promises, deferrers, etc., this is a one-time thing: only one callback
> can
> > register with it.  However, it can be chained.  The registered callback
> is
> > invoked with the output of the operation when it completes.
> >
> > Timer.wait(20).then(callme, *args, **kw)
> > # I could do
> > Timer.wait(20).then(callme, *args, **kw).then(piped_from_callme)
> >
> > #I could not do
> > handler = Timer.wait(20)
> > handler.then(callme)
> > handler.then(callme2) # this would throw an exception.
> >
> > # I/O example…
> > sock.accept().then(handle_connection) # invokes handle_connection(conn,
> > addr)
> > # Read some data
> > conn.read(1024).then(handle_incoming) # handle_incoming invoked with up
> to
> > 1024 bytes, read asynchronously.
> > # Write some data
> > conn.write("data").then(handle_written) # handle_written invoked with up
> > number 5, giving number of bytes written async.
> > # Connect HTTP channel and add it to HTTP dispatcher.
> > channel.connect((hostname,80)).then(dispatcher.add_channel)
> >
> >
> > # Listen to FD's for I/O events
> > descriptors.select(r, w, e).then(handle) # handle(readable, writables,
> oobs)
> >
> > It seems like only supporting a single callback per returned handle lets
> us
> > circumvent a lot of the weight associated with normal
> > promise/future/deferred pattern type implementations, but the chaining
> could
> > come in handy as it may cover some of the use-cases being considered when
> > multiple events per fd came up, plus chaining is pretty powerful,
> especially
> > when it comes at little cost.  The API would be much more extensive than
> > "then()", of course, with things like "every", etc. we'd have to pull
> > examples from everything already discussed.  Just wanted to throw out
> there
> > to get beat up about ;-)
> >
> >
> >
> > Shane Green
> > www.umbrellacode.com
> > 805-452-9666 | shane at umbrellacode.com
> >
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > http://mail.python.org/mailman/listinfo/python-ideas
> >
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
  Jasper
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121218/05d6834e/attachment.html>