Howto wait for multiple queues (Queue.py)?

[Andreas Ames]

Alex Martelli <aleax at aleax.it> writes:

> ...this doesn't follow -- just make all requests to a thread onto
> a single Queue (tag the requests with 'kind' or whatever as needed)
> and live happily.  A Queue instance *IS* implicitly, automatically
> and intrinsically "synchronized" -- several threads can be reading
> and/or writing to the same Queue instance and you're guaranteed that
> they'll never interfere with each other.  That's just about the
> whole POINT of the Queue class.

Sorry, this confuses me a bit.  If you understood my mail such that I
want to synchronize the access of several threads to one queue, my
mail must have been totally ambiguous.

What I really want to achieve is that several threads can consume
items from several (possibly identical) _sets_ of queues.  As I read
the docs and looked at the source in Queue.py I don't doubt that this
queue implementation is threadsave.

What I don't understand is the reasoning, *why* every thread should
only have exactly one input queue.  This has several disadvantages
from my point of view.  Several threads can't share the same input
queue and I can't distinguish between several input channels within
one thread.  For example in my case the threads should have input
channels for commands and for events (such as command responses or
unsolicited events).  Furthermore there can be different functional
units which write commands and/or events to the same thread.  It is
clear to me that this can be achieved by giving the different
commands/events different types/kinds and use one input queue for all
of them.  But I have at least two reservetions against this:

- The single input queue could be a bottleneck.  If you choose to
  avoid the bottleneck by providing several threads which understand
  the same commands, the producers have to decide about the
  scheduling, i.e. which thread gets which command.

- It seems more intuitive to me to have different input channels for
  different purposes.

This is comparable to the 'select()' function which is widely used.

If you say, there is no good implementation for waiting on several
queues in python, then ok, in this case I admit that using a single
input queue per thread might be the way to go (I'm also not satisfied
by the three alternatives I have enumerated in the original post; I
think I will try alternative 3 for now).


cheers,

andreas