Queue qsize = unreliable?

Tim Peters tim.peters at gmail.com
Fri Aug 6 11:53:11 EDT 2004


[James R. Saker Jr.]
>> I see per pydoc that Queue.Queue()'s .qsize is allegedly unreliable:
>>
>>     |  qsize(self)
>>     |      Return the approximate size of the queue (not reliable!).
>>
>> Any thoughts on why this is unreliable (and more curiously, why it would
>> be put in there as an unreliable function?) Rather than roll my own
>> threaded fifo class, it would seem prudent to use Python's built-in
>> Queue but the warning signs on a rather necessary function seem curious.

[Mike C. Fletcher]
> I would imagine that there is the potential for an item to be added or
> consumed during the running of the qsize method, or between the
> completion of the running of the qsize method and the op-codes which
> deal with the result.
> ...
> Tim, of course, would know better, but that's always been my interpretation.

That's exactly right.  The Queue empty() and full() methods are
similarly "unreliable".  They return the exact truth at the time
they're computed by Queue, but by the time their caller gets the
result they may bear no relationship to then-current reality.

James, why do you think they're necessary functions?  If, e.g., you're
worried about the queue getting "too big", pass a maximum size to the
constructor, and then a producer thread trying to add to a full queue
will block until a consumer thread takes an entry off the queue. 
Indeed, mediating mismatched processing rates is a prime use case for
bounded queues.

There's no use case I know of where knowing "the exact current size"
is necessary, except inside the implementation of Queue (where it can
know it, but at the expense of locking out all other threads for as
long as it takes to compute the result).



More information about the Python-list mailing list