A little threading problem

[Jeremy Jones]

Alban Hertroys wrote:

> Hello all,
>
> I need your wisdom again. I'm working on a multi-threaded application 
> that handles multiple data sources in small batches each time. The 
> idea is that there are 3 threads that run simultaneously, each read a 
> fixed number of records, and then they wait for eachother. After that 
> the main thread does some processing, and the threads are allowed to 
> continue reading data.
>
> I summarized this part of the application in the attached python 
> script, which locks up rather early, for reasons that I don't 
> understand (I don't have a computer science education), and I'm pretty 
> sure the problem is related to what I'm trying to fix in my 
> application. Can anybody explain what's happening (Or maybe even show 
> me a better way of doing this)?
>
> Regards,
>
> Alban Hertroys,
> MAG Productions.
>
>------------------------------------------------------------------------
>
>import sys
>import threading
>
>class AThread(threading.Thread):
>	def __init__(self, name, mainCond, allowedCond):
>		self.counter	= 0
>		self.name		= name
>		self.mainCond	= mainCond
>		self.condAllowed = allowedCond
>		self.waitUntilRunning = threading.Condition()
>
>		threading.Thread.__init__(self, None, None, name, [])
>
>	def start(self):
>		threading.Thread.start(self)
>
>		# Let the main thread wait until this thread is ready to accept Notify
>		# events.
>		self.waitUntilRunning.acquire()
>		self.waitUntilRunning.wait()
>		self.waitUntilRunning.release()
>
>	def run(self):
>		threading.Thread.run(self)
>
>		# Print numbers 1 - 25
>		while self.counter < 25:
>			self.condAllowed.acquire()
>
>			# Tell the main thread that we're ready to receive Notifies
>			self.waitUntilRunning.acquire()
>			self.waitUntilRunning.notify()
>			print "Running"
>			self.waitUntilRunning.release()
>
>			# Wait for a Notify from the main thread
>			print "Wait"
>			self.condAllowed.wait()
>			self.condAllowed.release()
>
>			self.counter += 1
>
>			print "Thread %s: counter = %d" % (self.name, self.counter)
>
>
>			# Tell the main thread that a thread has reached the end of the loop
>			self.mainCond.acquire()
>			self.mainCond.notify()
>			self.mainCond.release()
>
>class Main(object):
>	def __init__(self):
>		self.condWait = threading.Condition()
>		self.condAllowed = threading.Condition()
>
>		self.threads = [
>			AThread('A', self.condWait, self.condAllowed),
>			AThread('B', self.condWait, self.condAllowed),
>			AThread('C', self.condWait, self.condAllowed),
>		]
>
>		# Start the threads
>		for thread in self.threads:
>			thread.start()
>
>		while True:
>			# Allow the threads to run another iteration
>			self.condAllowed.acquire()
>			print "Notify"
>			self.condAllowed.notifyAll()
>			self.condAllowed.release()
>
>			# Wait until all threads reached the end of their loop
>			for thread in self.threads:
>				self.condWait.acquire()
>				self.condWait.wait()
>				self.condWait.release()
>
>
>main = Main()
>
>  
>
You've got a deadlock.  I modified your script to add a print "T-%s" % 
self.name before an acquire and after a release in the threads you spun 
off (not in the main thread).  Here is the output:

[jmjones at qatestrunner threading]$ python tt.py
T-A: acquiring condAllowed
T-A: acquiring waitUntilRunning
T-A: Running
T-A: released waitUntilRunning
T-A: Wait
T-B: acquiring condAllowed
T-B: acquiring waitUntilRunning
T-B: Running
T-B: released waitUntilRunning
T-B: Wait
T-C: acquiring condAllowed
T-C: acquiring waitUntilRunning
T-C: Running
T-C: released waitUntilRunning
T-C: Wait
Notify
T-A: released condAllowed
T-A: counter = 1
T-A: acquiring mainCond
T-A: released mainCond
T-A: acquiring condAllowed
T-A: acquiring waitUntilRunning
T-A: Running
T-A: released waitUntilRunning
T-A: Wait
T-C: released condAllowed
T-C: counter = 1
T-C: acquiring mainCond
T-C: released mainCond
T-C: acquiring condAllowed
T-C: acquiring waitUntilRunning
T-C: Running
T-C: released waitUntilRunning
T-C: Wait
T-B: released condAllowed
T-B: counter = 1
T-B: acquiring mainCond
T-B: released mainCond
T-B: acquiring condAllowed
Notify                                         <---------Here is your 
problem
T-A: released condAllowed
T-A: counter = 2
T-A: acquiring mainCond
T-A: released mainCond
T-A: acquiring condAllowed
T-A: acquiring waitUntilRunning
T-A: Running
T-A: released waitUntilRunning
T-A: Wait
T-B: acquiring waitUntilRunning
T-B: Running
T-B: released waitUntilRunning
T-B: Wait
T-C: released condAllowed
T-C: counter = 2
T-C: acquiring mainCond
T-C: released mainCond
T-C: acquiring condAllowed
T-C: acquiring waitUntilRunning
T-C: Running
T-C: released waitUntilRunning
T-C: Wait

Notify is called before thread B (in this case) hits the 
condAllowed.wait() piece of code.  So, it sits at that wait() for 
forever (because it doesn't get notified, because the notification 
already happened), waiting to be notified from the main thread, and the 
main thread is waiting on thread B (again, in this case) to call 
mainCond.notify().  This approach is a deadlock just wanting to happen 
(not waiting, because it already did happen).  What is it exactly that 
you are trying to accomplish?  I'm sure there is a better approach.

Jeremy Jones

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20041201/afbc8aed/attachment.html>