multithreading

[Cliff Wells]

On 24 May 2002 18:03:15 -0400
François Pinard wrote:

> [Aahz]
> 
> > My response is that instead of trying to take advantage of the few
> > atomic Python constructs, instead code defensively and always use
> > thread-safe mechanisms for passing information.  Because Python has a
> > powerful and simple Queue, this is straightforward to accomplish.
> 
> But abusing Queues for very simple things, a bit everywhere, might yield
> code bloat, and impinge readability.  This is a bit like if someone was
> inviting everyone to abuse fixed point integer arithmetic all over as a
> way to program defensively against floating point arithmetic.
> 
> The key point is proper documentation.  Even saying that a behaviour is
> undefined is good documentation, as it teaches what should be avoided.
> 
> One could be paranoid and setup queues and server to serve `os.listdir()',
> say, in fear that two threads could not simultaneously use that library
> function.  We can go overboard doing such things, maybe without any kind
> of real necessity.  Best is to know how thing works, that is, what is
> guaranteed and can be relied upon, and what is not guaranteed, and need
> synchronisation mechanisms.  Mere testing is out of question, as a working
> program is no proof of a correct usage.  Abusing of a few synchronisation
> primitives is no good either: I hope being able to know Python well enough
> to feel it is on my side, and not have to program defensively as if it
> was a lost cause trying to understand how Python is meant to be used.

In your original post, you mentioned having problems determining which Python
operations are atomic.  IMHO, this is not a good approach to thread-safety. 
While using Queues exclusively would be overkill, a good rule of thumb is to
avoid analyzing what the interpreter does internally to determine thread-safety
and simply use locks whenever accessing shared data.  If you use a Queue, this
is done for you, which is what makes Queues so convenient.  However, trying to
determine if a particular operation is atomic is overkill in the other
direction.  Simply use a Lock (or RLock, as Aahz will undoubtedly suggest) to
protect shared data while a thread is accessing it.  Obviously, if the data is
static, no lock is needed.  However, if there is the possiblity that another
thread could alter said data, then use a lock.  This will save you a ton of
headaches later.  While it can be argued that this approach introduces overhead
(and possible deadlock issues if you aren't careful), I would counter that it
also saves development time, which is why you're using Python in the first
place.

Regards,

-- 
Cliff Wells, Software Engineer
Logiplex Corporation (www.logiplex.net)
(503) 978-6726 x308  (800) 735-0555 x308