[Python-ideas] Simpler thread synchronization using "Sticky Condition"

[Richard Whitehead]

Nathaniel,

You’re quite right, I can’t work out the race condition myself now... Is it possible that an AutoResetEvent is just an Event with a two-line wait() overload???

Richard


From: Nathaniel Smith 
Sent: Tuesday, March 26, 2019 5:10 PM
To: Richard Whitehead 
Cc: Python-Ideas 
Subject: Re: [Python-ideas] Simpler thread synchronization using "Sticky Condition"

On Tue, Mar 26, 2019, 09:50 Richard Whitehead <richard.whitehead at ieee.org> wrote:

  Nathaniel,

  Thanks very much for taking the time to comment.

  Clearing the event after waiting for it will introduce a race condition: if 
  the sender has gone around its loop again and set the semaphore after we 
  have woken but before we've cleared it.

Sounds fine to me. Why is that a problem? Can you write down an example of how two threads could be interleaved to produce incorrect results?

  As you said, this stuff is tricky! 
  The only safe way is to make the wait-and-clear atomic, which can be done 
  with a lock; and this comes essentially back to what I'm proposing.

  I realise this is not a fundamental new primitive - if it was, I wouldn't be 
  able to build it in pure Python - but I've found it extremely useful in our 
  generic threading and processing library.

  You're right about what you say regarding queues; I didn't want to go into 
  the full details of the multi-threading and multi-processing situation at 
  hand, but I will say that we have a pipeline of tasks that can run as either 
  threads or processes, and we want to make it easy to construct this 
  pipeline, "wiring" it as necessary; combining command queues with data 
  queues just gets a real mess.


But you're effectively implementing a multi-producer single-consumer Queue anyway, so without any details it's hard to guess why using a Queue would be messier. I know you don't want to get into too many details, but if the whole motivation for your proposal is based on some details then it's usually a good idea to explain them :-).

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20190326/e85135eb/attachment.html>