[Python-Dev] Proposal: defaultdict

[Ian Bicking]

Martin v. Löwis wrote:
> Adam Olsen wrote:
> 
>>Consider these two pieces of code:
>>
>>if key in d:
>>  dosomething(d[key])
>>else:
>>  dosomethingelse()
>>
>>try:
>>  dosomething(d[key])
>>except KeyError:
>>  dosomethingelse()
>>
>>Before they were the same (assuming dosomething() won't raise
>>KeyError).  Now they would behave differently.
> 
> 
> I personally think they should continue to do the same thing,
> i.e. "in" should return True if there is a default; in the
> current proposal, it should invoke the default factory.

As I believe Fredrik implied, this would break the symmetry between "x 
in d" and "x in d.keys()" (unless d.keys() enumerates all possible 
keys), and either .get() would become useless, or it would also act in 
inconsistent ways.  I think these broken expectations are much worse 
than what Adam's talking about.

> But that's beside the point: Where is the real example
> where this difference would matter? (I'm not asking for
> a realistic example, I'm asking for a real one)

Well, here's a kind of an example: WSGI specifies that the environment 
must be a dictionary, and nothing but a dictionary.  I think it would 
have to be updated to say that it must be a dictionary with 
default_factory not set, as default_factory would break the 
predictability that was the reason WSGI specified exactly a dictionary 
(and not a dictionary-like object or subclass).  So there's something 
that becomes brokenish.

I think this is the primary kind of breakage -- dictionaries with 
default_factory set are not acceptable objects when a "plain" dictionary 
is expected.  Of course, it can always be claimed that it's the fault of 
the person who passes in such a dictionary (they could have passed in 
None and it would probably also be broken).  But now all of the sudden I 
have to say "x(a) takes a dictionary argument.  Oh, and don't you dare 
use the default_factory feature!" where before I could just say 
"dictionary".  And KeyError just... disappears.  KeyError is one of 
those errors that you *expect* to happen (maybe the "Error" part is a 
misnomer); having it disappear is a major change.

Also, I believe there's two ways to handle thread safety, both of which 
are broken:

1) d[key] gets the GIL, and thus while default_factory is being called 
the GIL is locked

2) d[key] doesn't get the GIL and so d[key].append(1) may not actually 
lead to 1 being in d[key] if another thread is appending something to 
the same key at the same time, and the key is not yet present in d.

Admittedly I don't understand the ins and outs of the GIL, so the first 
case might not actually need to acquire the GIL.



-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org