An improvement to Queue

[Aahz Maruch]

In article <Xns91A85CDCAE3FBmichaelrcpcouk at 194.238.50.13>,
Michael Abbott  <michael at rcp.co.uk> wrote:
>"Tim Peters" <tim.one at home.com> wrote in
>news:mailman.1012515655.8263.python-list at python.org: 
>> 
>> It *sounds* like you're using Queue to manipulate items at too low a
>> level of granularity.  If you created a class to encapsulate the notion
>> of "one round of work", then the consumer could pull off "one complete
>> round of work" per Queue.get(), and you wouldn't need other synch
>> gimmicks (or other complications) to communicate where "one round of
>> work's" boundaries lie. 
>
>The problem is that when the consumer gets around to performing its round 
>of work, rather than just performing the "jobs" in sequence and in 
>isolation, in fact the the consumer is in a position to merge jobs together 
>into a single action.
>
>Think of a cursor addressable terminal the wrong end of a serial line (this 
>is close to my application): sending characters down the serial line is the 
>main limiting resource, and the "consumer" in this case is in the business 
>of formatting up an incremental update and sending it on.
>
>However, changes to the display are coming in all the time, too fast to 
>display, on a separate thread.  I therefore want to accumulate these 
>changes and when there is room on the serial line send the entire batch.  
>The consumer should therefore be spending most of its time waiting for the 
>serial line to become free, and then formatting up a batch of updates.
>
>Arguably this accumulation should be done in the generating thread, but I 
>don't really wish to do it that way in this particular case.  In 
>particular, the model I propose makes synchronisation almost trivial, which 
>is I think quite an important consideration!

Well, you're adding complexity to the Python threading model, which is
already difficult enough for many people to understand.  Your getall()
idea *is* a good one, no argument there.  You should either do the
batching on the consumer side (if it's busy, don't do a get(); when it's
free, do a getall() and format) or create an intermediary pipeline to do
just the collapsing.
-- 
                      --- Aahz  <*>  (Copyright 2002 by aahz at pobox.com)

Hugs and backrubs -- I break Rule 6                 http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista   

"The more you drive, the less intelligent you are."  --_Repo Man_