Thread safety issue (I think) with defaultdict

[Rhodri James]

On 03/11/17 14:50, Chris Angelico wrote:
> On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James <rhodri at kynesim.co.uk> wrote:
>> On 02/11/17 20:24, Chris Angelico wrote:
>>>
>>> Thank you. I've had this argument with many people, smart people (like
>>> Steven), people who haven't grokked that all concurrency has costs -
>>> that threads aren't magically more dangerous than other options.
>>
>>
>> I'm with Steven.  To be fair, the danger with threads is that most people
>> don't understand thread-safety, and in particular don't understand either
>> that they have a responsibility to ensure that shared data access is done
>> properly or what the cost of that is.  I've seen far too much thread-based
>> code over the years that would have been markedly less buggy and not much
>> slower if it had been written sequentially.
> 
> Yes, but what you're seeing is that *concurrent* code is more
> complicated than *sequential* code. Would the code in question have
> been less buggy if it had used multiprocessing instead of
> multithreading? What if it used explicit yield points?

My experience with situations where I can do a reasonable comparison is 
limited, but the answer appears to be "Yes".
  Multiprocessing
> brings with it a whole lot of extra complications around moving data
> around.

People generally understand how to move data around, and the mistakes 
are usually pretty obvious when they happen.  People may not understand 
how to move data around efficiently, but that's a separate argument.

  Multithreading brings with it a whole lot of extra
> complications around NOT moving data around.

I think this involves more subtle bugs that are harder to spot.  People 
seem to find it harder to reason about atomicity and realising that 
widely separated pieces of code may interact unexpectedly.

  Yield points bring with
> them the risk of locking another thread out unexpectedly (particularly
> since certain system calls aren't async-friendly on certain OSes).

I've got to admit I find coroutines straightforward, but I did cut my 
teeth on a cooperative OS.  It certainly makes the atomicity issues 
easier to deal with.

  All
> three models have their pitfalls.

Assuredly.  I just think threads are soggier and hard to light^W^W^W^W^W 
prone to subtler and more mysterious-looking bugs.

-- 
Rhodri James *-* Kynesim Ltd