SMP, GIL and Threads

[Steve Holden]

catsup wrote:
> Hi,
> 
> I have an app written under version Python 2.3.5.  The problem I'm
> having is that it hangs on one of its threads.  The thread that hangs
> does updates to a standard dictionary shared with another thread that
> only reads this dictionary.  This app works beautifully on a single
> processor boxes in my testing environment, but this problem quickly
> occurs when the software runs on a dual cpu SMP blade server with
> hyperthreading turned off, Windows 2003 server.
> 
I don't see why you should get problems on SMP hardware, since the 
threads are all part of the same process and should therefore (I'd have 
thought) be tagged with the same processor affinity. Hence the GIL 
should manage contention successfully.

> I do not bother using application level locks because I figure the GIL
> is going to do the job for me, and because with one thread doing
> updates and the other only reading, things remain consistent logically
> for the app.  The app will not have a problem if the dictionary changes
> just before it does a read.
> 
I  believe dictionary access is an atomic operation wrt thread switches 
anyway, so I'm (again) not sure why a thread is hanging.

> I have searched this group on this subject and seen one warning against
> sharing objects between threads.  I don't recall every writing a
> threaded app that didn't share data between threads in some way.  I've
> also seen a recomendation in this list against using threads at all
> with Python.  I'm hoping that is an extreme view and not general wisdom
> here.  Python has never failed me when analysis indicated that it would
> be the correct tool for the job.
> 
Threads most often use Queue.Queue to communicate, precisely because its 
operations are guaranteed thread-safe. Could we see some code, or do we 
have to guess what the problem might be ;-) (I appreciate the code may 
be too large or proprietary, but in that case could you build a test to 
demonstrate the problem?).

I wouldn't worry about the "don't use threads" camp. Although there is a 
lot to be said for non-blocking asynchronous operations managed by a 
single thread this is by no means the only way to do things.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/