An improvement to Queue

[Michael Abbott]

"Alex Martelli" <aleax at aleax.it> wrote in
news:a3blop$a29$1 at serv1.iunet.it: 

> "Michael Abbott" <michael at rcp.co.uk> wrote in message
> news:Xns91A793581DD79michaelrcpcouk at 194.238.50.13...
> 
>> the queue has just changed state from empty to non-empty.  This allows
>> the generating process to know whether a "wake-up" signal needs to be
>> passed to the client thread, and is useful for non-blocking use of the
>> queue. 
> 
> In this case, for symmetry, you might also want .get to also return a
> boolean about whether the Queue transitioned from full to non-full.

Probably harmless, and might be useful in some other application.
 
> I'm not clear on why the 'client' (getting?) thread would be waiting
> on a wakeup signal rather than on the queue itself.  Example please?

Difficult to give a concise example, but the framework I have in mind is 
this:

I have a worker thread which sits in a tight loop doing the following:

    	1. Waiting on an event
    	2. Doing some work

This can be implemented using the existing Queue as simply (this isn't 
actually how I do it, but the intent is the same):

    	while 1:
    	    (action_queue.get())()

This thread in general will only block on the action_queue.get() call 

Other threads are busily running around in the background preparing work 
for the worker thread to work on.  Here's a typical background activity:

    	on interesting_event:
    	    if my_queue.put(some_work):    	    	# This is the key line
    	        action_queue.put(do_the_work)

    	def do_the_work():
    	    process_work(my_queue.get_all())

The idea here is that until the worker thread is ready I don't want to call 
process_work, indeed process_work is supposed to happen on the worker 
thread, but when it is ready it should process everything that's in hand.


Let's be more explicit.

An interesting event might be a socket read ready event: I read what I can 
from the socket, and queue the results for processing on the worker thread.  
I don't particularly want to queue a separate do_the_work event for each 
incoming block, particularly as the worker thread might be really quite 
busy at times (in fact, at the moment I'm running 100% cpu for minutes at a 
stretch; Python has its challenges).

Of course without my hack I can still work around by testing whether 
my_queue.qsize()==1, assuming only one thread is generating work...