[Python-ideas] Async API

[Yury Selivanov]

On 2012-10-26, at 12:57 PM, Itamar Turner-Trauring <itamar at futurefoundries.com> wrote:
> On Fri, Oct 26, 2012 at 12:36 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Oct 26, 2012 at 8:52 AM, Laurens Van Houtven <_ at lvh.cc> wrote:
> > err, I suppose the missing bit there is that you'll probably want to:
> >
> > reactor.callLater(timeout, d.cancel)
> >
> > As opposed to calling d.cancel() directly. (That snippet was in
> > bpython-urwid with the reactor running in the background, but I doubt it'd
> > work well anywhere else outside of manholes :))
> 
> So I think that Yuri's original problem statement, transformed to
> Twisted+Deferred, might still apply, depending on how you implement
> it. Yuri essentially did this:
> 
> def foobar():  # a task
>     try:
>         yield <blocking action>
>     finally:
>         # must clean up regardless of whether action succeeded or failed:
>         yield <blocking cleanup>
> 
> He then calls this with a timeout, with the semantics that if the
> generator is blocked in a yield when the timeout arrives, that yield
> raises a Timeout exception (and at no other time is Timeout raised).
> The problem with this is that if the action succeeds within the
> timeout, but barely, there's a chance that the cleanup of a
> *successful* action receives the Timeout exception. Apparently this
> bit Yuri. I'm not sure how you'd model that using just Deferreds, but
> using inlineCallbacks it seems the same thing might happen. Using
> Deferreds, I assume there's a common pattern to implement this that
> doesn't have this problem. Of course, using coroutines, there is too
> -- spawn the cleanup as an independent task.
> 
> If you call cancel() on a Deferred that already has a result, nothing happens. So you don't get a TimeoutError if the operation has succeeded (or failed some other way). This would also be true when using inlineCallbacks, so there's no issue.
> 
> In general I'm not clear why this is a problem: in a single-threaded program only one thing happens at a time. Your code for triggering a timeout always has the option to check if the operation has succeeded, without worrying about race conditions.

Let me ask you a question that may help me and others to understand
how inlineCallbacks works.

If you write the following:

   def func():
       try:
           yield one_thing()
           yield and_another()
       finally:
           yield and_finally()

Then each of those yields will create a separate Deferred object, that
'inlineCallbacks' transparently dispatches via generator send/throw,
right?

And if you 'yield func()' the same will happen--'inlineCallbacks' will
return a Deferred, that will have a result of 'func' execution?

Thanks,
Yury