extracting a heapq in a for loop - there must be more elegant solution

[Cameron Simpson]

On 04Dec2013 09:44, Helmut Jarausch <jarausch at igpm.rwth-aachen.de> wrote:
> On Wed, 04 Dec 2013 08:13:03 +1100, Cameron Simpson wrote:
> > I iterate over Queues so often that I have a personal class called
> > a QueueIterator which is a wrapper for a Queue or PriorityQueue
> > which is iterable, and an IterablePriorityQueue factory function.
> > Example:
> > 
> >   from cs.queues import IterablePriorityQueue
> > 
> >   IPQ = IterablePriorityQueue()
> >   for item in [1,2,3,7,6,5,9,4]:
> >     IPQ.put(item)
> > 
> >   for item in IPQ:
> >     ... do stuff with item ...
> 
> Many thanks!
> I think you QueueIterator would be a nice addition to Python's library.

I'm not convinced. I'm still sorting out the edge cases, and it's
my own code and use cases!

The basic idea is simple enough:

  class QueueIterator:
    def __init__(self, Q, sentinel=None):
      self.q = Q
      self.sentinel = sentinel
    def __next__(self):
      item = self.q.get()
      if item is self.sentinel:
        # queue sentinel again for other users
        self.q.put(item)
        raise StopIteration
      return item
    def put(self, item):
      if item is self.sentinel:
        raise ValueError('.put of sentinel object')
      return self.q.put(item)
    def close(self):
      self.q.put(self.sentinel)

  QI = QueueIterator(Queue())

Note that it does not work well for PriorityQueues unless you can
ensure the sentinel sorted later than any other item.

But you really need to be sure nobody does a .get() in the internal
queue, or the close protocol doesn't work. This is why the above
does not expose a .get() method.

You want to prevent .put() on the queue after close().
Of course, I have use cases where I _want_ to allow .put() after
close:-(

You may want to issue a warning if more than one call to
.close happens.

And so on.

And I have a some weird cases where the close situation has trouble,
which probably points to .close() not being a great mapping to
my termination scenarios.

It is easy to code for the simple data-in, data-out case though.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Computer manufacturers have been failing to deliver working systems on
time and to budget since Babbage.       - Jack Schofield, in The Guardian