Thread-safe way to add a key to a dict only if it isn't already there?

[Steven D'Aprano]

On Sun, 08 Jul 2018 16:37:11 +0100, MRAB wrote:

> On 2018-07-08 14:38, Steven D'Aprano wrote:
>> On Sun, 08 Jul 2018 14:11:58 +0300, Marko Rauhamaa wrote:
>> 
> [snip]
>>> More importantly, this loop may never finish:
>>> 
>>>     # Initially
>>>     quit = False
>>> 
>>>     # Thread 1
>>>     global quit
>>>     while not quit:
>>>         time.sleep(1)
>>> 
>>>     # Thread 2
>>>     global quit
>>>     quit = True
>> 
>> Assuming that thread 2 actually runs *at some point*, I don't see how
>> that can't terminate. Neither thread sets quit to False, so provided
>> thread 2 runs at all, it has to terminate.
>> 
> [snip]
> 
> The compiler could look at the code for thread 1 and see that 'quit' is
> never assigned to, meaning that it could be "optimised" to:
> 
>      global quit
>      if not quit:
>          while True:
>              time.sleep(1)


I'm glad you put "optimized" in scare quotes there, because any optimizer 
that did that to code that runs in a thread is buggy. The re-write has 
changed the semantics of the code.

Of course a compiler "could" do anything. If it re-wrote the code to 
instead perform:

    while True:
        print(math.sin(random.random()+1)

instead, we'd have no trouble recognising that the compiler has changed 
the semantics of our code to something we didn't write, and in just about 
every language apart from C, we would rightly call it a compiler bug.

If I write "Do X", and the compiler instead executes "Do Y", that's a 
compiler bug. The compiler has one job: to take my source code and 
convert it to something which can be executed, and if it cannot do that 
faithfully, it is buggy.

C is the special case: C programmers routinely blame *themselves* when 
the compiler changes "Do X" to "Do Y", because the standard says it is 
permitted to do anything it likes in the event of undefined behaviour. 
Talk about victims coming to rationalise their own abuse and identify 
with their abuser.

W.A. Wulf might have been talking about C compilers when he wrote:

    "More computing sins are committed in the name of efficiency
    (without necessarily achieving it) than for any other single
    reason — including blind stupidity."


> In C you'd declare 'quit' as 'volatile' to tell the compiler that it
> could change unexpectedly, so don't make that assumption.

You shouldn't need to tell the compiler to assume that the variable could 
change. In multi-threaded code, there's no justification for assuming the 
variable can't change unless you've done a whole-program analysis and can 
prove that *no* thread ever writes to that variable.

Even in single-threaded code, I think that optimizations which change 
execution order are dubious. There's far too much opportunity for "it's 
not a bug, because the specification says we can deny it is a bug" 
faults. It's simply *bad engineering practice*.

When engineers design a bridge or a road or a building, the builders have 
to faithfully follow the engineer's design, unless the design itself 
explicitly allows them to make substitutions.

"The blueprints say these supporting pillars have to be filled with steel-
reinforced concrete. How about we just fill them with empty cans instead, 
what could possibly go wrong?"

http://shanghaiist.com/2016/02/08/taiwan_earthquake_building_collapse/


(Disclaimer: there seems to be some controversy over whether the cans 
were actually in supporting columns or not. But the point still stands.)


-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson