Python threading and sharing variables

[Chris Angelico]

On Wed, Jul 5, 2017 at 6:26 PM, Thomas Nyberg <tomuxiong at gmx.com> wrote:
> I think the general rule would be that no it's not safe to skip the
> locks. It's true that with cpython, your method shouldn't run into
> problems, but that's just a quirk of how you're using it. I look at from
> another perspective, if it's true that no locks actually are necessary,
> then why are you using the shared variables in the first place. In this
> case, the information needs to be send from the worker thread to the
> main thread, but you don't need for any other threads to see it. This
> only really requires a single "channel" (e.g. a queue) and not for the
> variable to further exposed.

What would the lock surround? The purpose of a lock is to make an
atomic unit out of something that otherwise wouldn't be, but a single
store operation is already atomic. So you could do something like:

with lock():
    self.cnt = self.cnt + 1

and then multiple threads could safely increment the counter, which
they otherwise couldn't (note that "self.cnt += 1" might be safe, but
probably wouldn't, so I use the longhand); but even with that form of
increment, it's only necessary if there are multiple writers. A
pub-sub model (one writer, any number of readers) only needs locks if
it's possible for the write itself to be half done, which can't happen
with a single operation.

I'm basing my information primarily on CPython here, where context
switches are protected by the language interpreter and the GIL. But
other Python implementations have similar guarantees. If there's any
Python implementation in which a simple assignment can cause problems,
I'd call that a bug to be fixed - especially since, at the CPU level,
you can generally rely on a pointer-sized memory store being atomic.

But I do agree with the recommendation of the queue. That does make
things clearer.

ChrisA