thread/queue bug

[Peter Otten]

phil wrote:

> And sorry I got ticked, frustrating week
> 
>  >And I could help more, being fairly experienced with
>  >threading issues and race conditions and such, but
>  >as I tried to indicate in the first place, you've
>  >provided next to no useful (IMHO) information to
>  >let anyone help you more than this
> 
> This is about 5% of the code.
> Uses no locks.

A lock is acquired in Queue.__init__().

> I am mystified, I have written probably 100,000 lines
> of Python and never seen a thread just lock up and quit
> running.  It happens on a Queue() statement so my suspicion
> is a bug.  ??
> 
> I have kludged around it by putting all the thread/queue stuff
> in the main program and import the stuff I don't want to
> distribute.  But mysteries haunt your dreams, sooo...

What I believe to be a minimal example:

<freeze.py>
import Queue
import threading
import time

q = Queue.Queue(4)

def proc():
    while True:
        q.get(1)
        Queue.Queue()
        print "YADDA"

threading.Thread(target=proc).start()

while True:
    print "yadda"
    q.put(None)
    time.sleep(1)
</freeze.py>

<freezemain.py>
import freeze
</freezemain.py>

Invoking freezemain.py produces

yadda
yadda
yadda
yadda
yadda
yadda

i. e. consistently q.maxsize + 2. One item about to be put, one already
taken before Queue.Queue(). Deferring execution of the module-level code
until after the import

<nofreeze.py>
import Queue
import threading
import time

q = Queue.Queue(4)

def proc():
    while True:
        q.get(1)
        Queue.Queue()
        print "YADDA"

def start():
    threading.Thread(target=proc).start()

    while True:
        print "yadda"
        q.put(None)
        time.sleep(1)
</nofreeze.py>

<nofreezemain.py>
import nofreeze
nofreeze.start()
</nofreezemain.py>

and invoking nofreezemain.py produces
yadda
YADDA
yadda
YADDA
yadda
YADDA
yadda
YADDA

apparently ad infinitum. Conclusion? None so far, though you might welcome
the confirmation that this is not a "once in a blue moon" accident as Bengt
Richter surmised.

Import _is_ a sensitive phase...

As an experiment I moved

try:
    import thread
except ImportError:
    import dummy_thread as thread

from the Queue.Queue.__init__() method to the module body -- and now
freezemain.py seems to work, too. So that would be an easy remedy, but sure
there is a reason why that import statement is in such an unusual place?

Peter