[Python-ideas] PEP 3156 feedback: wait_one vs par vs concurrent.futures.wait

[Guido van Rossum]

Yes, I like always returning a future.

On Saturday, December 22, 2012, Nick Coghlan wrote:

> On Sun, Dec 23, 2012 at 1:54 AM, Guido van Rossum <guido at python.org<javascript:;>>
> wrote:
> > On Sat, Dec 22, 2012 at 12:04 AM, Nick Coghlan <ncoghlan at gmail.com<javascript:;>>
> wrote:
> >> I deliberately chose to return coroutines. My rationale is to be able
> >> to handle the case where multiple operations become ready without
> >> having to make multiple trips around the event loop by having the
> >> iterator switch between two modes: when the complete set is empty, it
> >> yields a coroutine that calls wait and then returns the first complete
> >> future, while when there are already complete futures available, it
> >> yields a coroutine that just returns one of them immediately. It's
> >> really the same rationale as that for having @coroutine not
> >> automatically wrap things in Task - if we can avoid the event loop in
> >> cases that don't actually need to wait for an event, that's a good
> >> thing.
> >
> > I think I see it now. The first item yielded is the simplest thing
> > that can be used with yield-from, i.e. a coroutine. Then if multiple
> > futures are ready at once, you return an item of the same type, i.e. a
> > coroutine. This is essentially wrapping a Future in a coroutine! If we
> > could live with the items being alternatingly coroutines and Futures,
> > we could just return the Future in this case. BTW, yield from <future>
> > need not go to the scheduler if the Future is already done -- the
> > Future,__iter__ method should be:
> >
> >     def __iter__(self):
> >         if not self.done():
> >             yield self  # This tells Task to wait for completion.
> >         return self.result()  # May raise too.
> >
> > (I forgot this previously.)
>
> And I'd missed it completely :)
>
> In that case, yeah, yielding any already completed Futures directly
> from as_completed() should work. The "no completed operations" case
> will still need a coroutine, though, as it needs to update the
> "complete" and "incomplete" sets inside the iterator. Since we know
> we're certain to hit the scheduler in that case, we may as well wrap
> it directly in a task so we're always returning some kind of future.
> The impl might end up looking something like:
>
>     def as_completed(fs):
>         incomplete = fs
>         while incomplete:
>             # Phase 1 of the loop, we yield a Task that waits for
> operations
>             @coroutine
>             def _wait_for_some():
>                 nonlocal complete, incomplete
>                 complete, incomplete = yield from tulip.wait(fs,
> return_when=FIRST_COMPLETED)
>                 return complete.pop().result()
>             yield Task(_wait_for_some())
>             # Phase 2 of the loop, we pass back the already complete
> operations
>             while complete:
>                 yield complete.pop()
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com <javascript:;>   |   Brisbane,
> Australia
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121222/423f2052/attachment.html>