GIL on SMP (need quick answer for the boss :) - thanks!

max khesin max at NcOviSsPiAoMntech.com
Fri Feb 13 11:01:58 EST 2004


I am working on a windoze service (deamon) that needs to spawn up to 
NumCPU heavyweight processes on SMP and wait for results, which have to 
  be logged. So I have to do something like (total pseudocode, thread 
pool not implemented, but demonstrates the problem):

Logger log


def ThreadFoo(job):
	log.write(os.open(job).read())

while(1):
	jobs = GetJobs()
	threads = []
	for job in jobs:
		threads.append(SpawnThread(ThreadFoo, job))
	waitFor(threads)


I have 3 questions:

1) (this is the killer) - is GIL going to get me into trouble? (i guess 
not, since all the python threads can run on the same CPU, not 
preventing the processes from being scheduled to different CPUs by the OS)
2) any problems having multiple threads write to the log (the standard 
Python logger). In general, is there an easy way to determine if a 
particular Python lib is thread-safe, at least for the standard libs?
3) Has anyone already implemented a thread pool lib that I can use (GPL 
not ok for this :()

thanks,

max



More information about the Python-list mailing list