Python threading and sharing variables

[eryk sun]

On Wed, Jul 5, 2017 at 4:04 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, Jul 6, 2017 at 12:39 AM, eryk sun <eryksun at gmail.com> wrote:
>>> This doesn't show a potential concurrency problem. Calculating a hash
>>> on "cnt" is independent of other threads; the actual work of
>>> __setitem__ isn't visible in this view. There certainly are places
>>> where a context switch could cause problems (eg resizing the dict),
>>> but they're the dict's problem, not your Python program's - because
>>> there's no way you could acquire a lock without working with these
>>> same issues.
>>
>> The work in the above special methods is arbitrary bytecode that could
>> do anything, and there's nothing to prevent a context switch here. The
>> GIL provides no protection here. In principle it could be a problem,
>> but in practice it is not.
>
> But what could it do? Most likely, it's going to end up mutating a
> dict (the core type), so unless the __setitem__ is itself maintaining
> complex state that needs a lock, all you've done is move the problem
> around, and the same solutions work.

That was my point. A namespace mapping could override __setitem__ and
__getitem__ to implement a name as something like a computed property
that's based on multiple values. Then if __setitem__ gets interrupted
in the middle of updating this set of values, another thread that gets
the computed 'property' will see a bad state. The GIL doesn't help. It
would need locking to make accessing the 'property' work as an atomic
operation, just like the case with regular properties. Again, I have
never seen anything like this in practice.