[Python-Dev] PEP 463: Exception-catching expressions

[Terry Reedy]

On 3/5/2014 8:15 PM, Steven D'Aprano wrote:
> On Wed, Mar 05, 2014 at 12:57:03PM -0800, Thomas Wouters wrote:
>> On Thu, Feb 27, 2014 at 1:29 PM, Chris Angelico <rosuav at gmail.com> wrote:
>>
>>> +Had this facility existed early in Python's history, there would have been
>>> +no need to create dict.get() and related methods;
>>
>>
>> FWIW, after experimenting and some consideration I've come to the
>> conclusion that this is incorrect. 'd[k] except KeyError: default' is still
>> much broader than dict.get(k):
>
> I don't think your example proves what you think it does. I think it
> demonstrates a bug in the dict.get method. The documentation for get
> states clearly that get will never raise KeyError:
>
>      Return the value for key if key is in the dictionary, else default.
>      If default is not given, it defaults to None, so that this method
>      never raises a KeyError.
>
> http://docs.python.org/3/library/stdtypes.html#dict.get
>
>
> but your example demonstrates that in fact it can raise KeyError
> (albeit under some rather unusual circumstances):

>> Python 3.4.0rc1+ (default:aa2ae744e701+, Feb 24 2014, 01:22:15)
>> [GCC 4.6.3] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> expensive_calculation = hash
>>>>> class C:
>> ...     _hash_cache = {}
>> ...     def __init__(self, value):
>> ...         self.value = value
>> ...         if value not in self._hash_cache:
>> ...             self._hash_cache[value] = expensive_calculation(value)
>> ...     def __hash__(self):
>> ...         return self._hash_cache[self.value]

This is a buggy special method. According to the docs for hash and 
__hash__ and the general convention on exceptions, a __hash__ method 
should return an int or raise TypeError.

>> ...     def __eq__(self, other):
>> ...         return self.value == other
>> ...
>>>>> a, b, c, d = C(1), C(2), C(3), C(4)
>>>>> D = {a: 1, b: 2, c: 3, d: 4}
>>>>> a.value = 5

This breaks the implied C invariant and makes the object 'a' incoherent 
and buggy

>>>>> print("dict.get:", D.get(a, 'default'))
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>    File "<stdin>", line 8, in __hash__
>> KeyError: 5

> According to the documentation, this behaviour is wrong.

One could argue that an error raised in a special method is not raised 
*by* a function that uses the special method. The docs constantly assume 
that special methods are coded correctly.

'''bool([x])
     Convert a value to a Boolean, using the standard truth testing 
procedure. If x is false or omitted, this returns False; otherwise it 
returns True.'''

... unless x.__bool__ raises or returns something other than True/False 
-- in which case bool itself raises.
TypeError: __bool__ should return bool, returned int

> Now, you might argue that the documentation is wrong. I'm sympathetic to
> that argument, but *as documented now*, dict.get is documented as being
> logically equivalent to:
>
> try:
>      return d[key]
> except KeyError:
>      return default

It appears to be actually equivalent to

key_hash = hash(key)
try:
     return d._hashlookup(key_has)
except KeyError:
     return default

> The fact that it actually isn't is an artifact of the specific
> implementation used. If it were a deliberate design choice,

Given that the choice is that bugs in special methods should not pass 
silently, I would presume that it is intentional.

 >  that design is not reflected in the documentation.

The docs generally describe behavior in the absence of coding errors.

-- 
Terry Jan Reedy