Question regarding Queue object

[bockman at virgilio.it]

On 27 Apr, 12:27, Terry <terry.yin... at gmail.com> wrote:
> Hello!
>
> I'm trying to implement a message queue among threads using Queue. The
> message queue has two operations:
> PutMsg(id, msg) #  this is simple, just combine the id and msg as one
> and put it into the Queue.
> WaitMsg(ids, msg) # this is the hard part
>
> WaitMsg will get only msg with certain ids, but this is not possible
> in Queue object, because Queue provides no method to peek into the
> message queue and fetch only matched item.
>
> Now I'm using an ugly solution, fetch all the messages and put the not
> used ones back to the queue. But I want a better performance. Is there
> any alternative out there?
>
> This is my current solution:
>
>     def _get_with_ids(self,wait, timeout, ids):
>         to = timeout
>         msg = None
>         saved = []
>         while True:
>             start = time.clock()
>             msg =self.q.get(wait, to)
>             if msg and msg['id'] in ids:
>                 break;
>             # not the expecting message, save it.
>             saved.append(msg)
>             to = to - (time.clock()-start)
>             if to <= 0:
>                 break
>         # put the saved messages back to the queue
>         for m in saved:
>             self.q.put(m, True)
>         return msg
>
> br, Terry

Wy put them back in the queue?
You could have a defaultdict with the id as key and a list of
unprocessed messages with that id as items.
Your _get_by_ids function could first look into the unprocessed
messages for items with that ids and then
look into the queue, putting any unprocessed item in the dictionary,
for later processing.
This should improve the performances, with a little complication of
the method code (but way simpler
that implementing your own priority-based queue).

Ciao
-----
FB