os walk() and threads problems (os.walk are thread safe?)

Marcus Alves Grando marcus at sbh.eng.br
Tue Nov 13 09:06:10 EST 2007


Hello list,

I have a strange problem with os.walk and threads in python script. I 
have one script that create some threads and consume Queue. For every 
value in Queue this script run os.walk() and printing root dir. But if i 
increase number of threads the result are inconsistent compared with one 
thread.

For example, run this code plus sort with one thread and after run again 
with ten threads and see diff(1).

--code--
#!/usr/local/bin/python

import os, time, glob
import Queue
import threading

EXIT=False
POOL=Queue.Queue(0)
NRO_THREADS=1
#NRO_THREADS=10

class Worker(threading.Thread):
	def run(self):
		global POOL, EXIT
		while True:
			try:
				mydir=POOL.get(timeout=1)
				if mydir == None:
					continue

				for root, dirs, files in os.walk(mydir):
					print root

			except Queue.Empty:
				if EXIT:
					break
				else:
					continue
			except KeyboardInterrupt:
				break
			except Exception:
				raise

for x in xrange(NRO_THREADS):
	Worker().start()
try:
	for i in glob.glob('/usr/ports/*'):
		POOL.put(i)

	while not POOL.empty():
		time.sleep(1)
	EXIT = True

	while (threading.activeCount() > 1):
		time.sleep(1)
except KeyboardInterrupt:
	EXIT=True
--code--

If someone can help with this i appreciate.

Regards

-- 
Marcus Alves Grando
marcus(at)sbh.eng.br | Personal
mnag(at)FreeBSD.org  | FreeBSD.org



More information about the Python-list mailing list