[Python-Dev] Issue 14417: consequences of new dict runtime error

[Guido van Rossum]

On Thu, Mar 29, 2012 at 1:48 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Thu, 29 Mar 2012 16:31:03 -0400, "R. David Murray" <rdmurray at bitdance.com> wrote:
>> On Thu, 29 Mar 2012 13:09:17 -0700, Guido van Rossum <guido at python.org> wrote:
>> > My original assessment was that this only affects dicts whose keys
>> > have a user-implemented __hash__ or __eq__ implementation, and that
>> > the number of apps that use this *and* assume the threadsafe property
>> > would be pretty small. This is just intuition, I don't have hard
>> > facts. But I do want to stress that not all dict lookups automatically
>> > become thread-unsafe, only those that need to run user code as part of
>> > the key lookup.
>>
>> You are probably correct, but the thing is that one still has to do the
>> code audit to be sure...and then make sure that no one later introduces
>> such an object type as a dict key.
>
> I just did a quick grep on our project.  We are only defining __eq__
> and __hash__ a couple places, but both are objects that could easily get
> used as dict keys (there is a good chance that's *why* those methods are
> defined) accessed by more than one thread.  I haven't done the audit to
> find out :)

Of course, that doesn't mean they're likely to be used as keys in a
dict that is read and written concurrently by multiple threads.

> The libraries we depend on have many more definitions of __eq__ and
> __hash__, and we'd have to check them too.  (Including SQLAlchemy,
> and I wouldn't want that job.)
>
> So our intuition that this is not common may be wrong.

But how often does one share a dictionary between threads with the
understanding that multiple threads can read and write it?

Here's a different puzzle. Has anyone written a demo yet that provokes
this RuntimeError, without cheating? (Cheating would be to mutate the
dict from *inside* the __eq__ or __hash__ method.) If you're serious
about revisiting this, I'd like to see at least one example of a
program that is broken by the change. Otherwise I think the status quo
in the 3.3 repo should prevail -- I don't want to be stymied by
superstition.

-- 
--Guido van Rossum (python.org/~guido)