[Python-ideas] Add closing and iteration to threading.Queue

[Nathaniel Smith]

On Sun, Oct 21, 2018, 16:48 MRAB <python at mrabarnett.plus.com> wrote:

> On 2018-10-21 22:30, Antoine Pitrou wrote:
> > On Sun, 21 Oct 2018 19:58:05 +0200
> > Vladimir Filipović <hemflit at gmail.com>
> > wrote:
> >>
> >> To anticipate a couple more possible questions:
> >>
> >> - What would this proposal do about multiple producers/consumers
> >> needing to jointly decide _when_ to close the queue?
> >>
> >> Explicitly nothing.
> >>
> >> The queue's state is either closed or not, and it doesn't care who
> >> closed it. It needs to interact correctly with multiple consumers and
> >> multiple producers, but once any one piece of code closes it, the
> >> correct interaction is acting like a closed queue for everybody.
> >
> > Ah.  This is the one statement that makes me favorable to this idea.
> > When there is a single consumer, it's easy enough to send a sentinel.
> > But when there are multiple consumers, suddenly you must send exactly
> > the right number of sentinels (which means you also have to careful
> > keep track of their number, which isn't always easy).  There's some
> > delicate code doing exactly that in concurrent.futures.
> >
> You don't need more than one sentinel. When a consumer sees the
> sentinel, it just needs to put it back for the other consumers.
>

I'm not sure if this is an issue the way Queue is used in practice, but in
general you have to be careful with this kind of circular flow because if
your queue communicates backpressure (which it should) then circular flows
can deadlock.

-n

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20181021/842064a2/attachment.html>