Queue qsize = unreliable?

[Tim Peters]

[Jeff Shannon]
>> No you don't.  You simply execute a non-blocking get(), and be
>> prepared to catch the Queue.Empty exception.  Similarly, if you want
>> a non-blocking producer, then you execute a non-blocking put() and
>> catch the Queue.Full exception.

[Ames Andreas]
> I've actually read the docs some time ago, but I've also read the
> source.  I think the problem is here that "non-blocking" isn't very
> well defined.

That's true, it isn't.

> In the sense that the consumer won't wait until something gets
> available from an empty Queue you are absolutely right with calling
> Queue.get(False).  But if you look at the source you will find that
> the first thing that's done in Queue.get() is aquiring a mutex/cv
> lock.  That's potentially quite blocking as far as I am concerned.

You must be looking at 2.4.  The mutex is never held for longer than
it takes to push or pop a single entry, so there is no pattern of
calls that can force a non-blocking call to wait for an arbitrarily
long time (assuming OS thread scheduling doesn't starve Python). 
That's one meaning of non-blocking.  Before 2.4 it had a different
meaning, one that never waited on a mutex for any reason in the
non-blocking cases.  But then a non-blocking get() could raise Empty
not only if the queue *was* empty, but also if it simply couldn't get
immediate ownership of a mutex in order to *tell* whether it was
empty.  That proved impossible to explain to users, who kept
complaining about it (as if it actually mattered ...).

> With CPython's assertions about atomicity of certain list operations
> it's possible to implement a lock free queue that is even usable for
> the single producer or single consumer case.  That's not to say
> anything against the standard libraries Queue implementation because
> it presumably does the right thing for the general case.

Yes, the Queue class makes no assumptions about how the queue is
implemented concretely.  Its base implementation (in 2.4) uses a
deque, but it's intended that users be able to subclass Queue and use
any concrete implementation.  In order to support that, the base Queue
guarantees that calls to implementation methods are serialized.