Thread scheduling

[Jack Orenstein]

Peter Hansen wrote:
 > Jack Orenstein wrote:
 >
 >> I am using Python 2.2.2 on RH9, and just starting to work with Python
 >> threads.
 >
 >
 > Is this also the first time you've worked with threads in general,
 > or do you have much experience with them in other situations?

Yes, I've used threading in Java.

 > You've got two shared global variables, "done" and "counter".
 > Each of these is modified in a manner that is not thread-safe.
 > I don't know if "counter" is causing trouble, but it seems
 > likely that "done" is.

I understand that. As I said in my posting, "The counter is
incremented without locking -- I expect to see a final count of less
than THREADS * COUNT." This is a test case, and I threw out more and
more code, including synchronization around counter and done, until it
got as simple as possible and still showed the problem.

 > Basically, the statement "done += 1" is equivalent to the
 > statement "done = done + 1" which, in Python or most other
 > languages is not thread-safe.  The "done + 1" part is
 > evaluated separately from the assignment, so it's possible
 > that two threads will be executing the "done + 1" part
 > at the same time and that the following assignment of
 > one thread will be overwritten immediately by the assignment
 > in the next thread, but with a value that is now one less
 > than what you really wanted.

Understood. I was counting on this being unlikely for my test
case. I realize this isn't something to rely on in real software.


 > If you really want to increment globals from the thread, you
 > should look into locks.  Using the "threading" module (as is
 > generally recommended, instead of using "thread"), you would
 > use threading.Lock().

As my note said, I did start with the threading module. And variables
updated by different threads were protected by threading.Condition
variables. As I analyzed my test cases, and threading.py, I started
suspecting thread scheduling.  I then wrote the test case in my email,
which does not rely on the threading module at all. The point of the
test is not to maintain counter -- it's to show that sometimes even
after one thread completes, the other thread never is scheduled
again. This seems wrong. Try running the code, and let me see if you
see this behavior.

If you'd like, replace this:

     counter += 1

by this:

     time.sleep(0.000001 * id)

You should see the same problem. So that removes counter from the
picture. And the two increments of done (one by each thread) are still
almost certainly not going to coincide and cause a problem. Also, if
you look at the output from the code on a hang, you will see that
'thread X: leaving' only prints once. This has nothing to do with what
happens with the done variable.

Jack