Queue qsize = unreliable?

[Ames Andreas (MPA/DF)]

Hi,

Jeff Shannon wrote:

> No you don't.  You simply execute a non-blocking get(), and be
> prepared to catch the Queue.Empty exception.  Similarly, if you want
> a non-blocking producer, then you execute a non-blocking put() and
> catch the Queue.Full exception.

I've actually read the docs some time ago, but I've also read the
source.  I think the problem is here that "non-blocking" isn't very
well defined.

In the sense that the consumer won't wait until something gets
available from an empty Queue you are absolutely right with calling
Queue.get(False).  But if you look at the source you will find that
the first thing that's done in Queue.get() is aquiring a mutex/cv
lock.  That's potentially quite blocking as far as I am concerned.

With CPython's assertions about atomicity of certain list operations
it's possible to implement a lock free queue that is even usable for
the single producer or single consumer case.  That's not to say
anything against the standard libraries Queue implementation because
it presumably does the right thing for the general case.  OTOH, a
single thread within a select-loop (single producer/consumer) feeding
a bunch of workers shouldn't be too exotic either.

> That's really the only reliable way to tell whether the queue is
> empty or full, anyhow...

I don't understand why this is more reliable than what I described in
my scenario.  If, for example, you have a single consumer then:

if !q.empty():
   reliableCode()
else
   unreliableCode()

should be absolutely okay.


cheers,

andreas