__getattribute__'s error is not available in __getattr__

[Jason Maldonis]

@Steve they asked me to move it here because it was more fitting. I hope
that's okay?


After some testing, it looks like I'm okay with how things work if the
problem-object isn't a descriptor (although I do still things it's a bit
odd that an error gets squashed, but it's no big deal).

However, I use @property all the time, so that's likely why I'm running
into this so often.  If you can help explain to me what's going on in that
case, that would be great!  I have two examples below for the sake of
discussion.

Here's an example where the underlying error is completely hidden:

class A(object):
    def _some_complex_code_hidden_from_the_user(self):
        # Run a bunch of complex stuff that raises an attribute error
internally
        # This could go layers deep into different modules
        return self.this_doesnt_exist

    @property
    def x(self):
        return self._some_complex_code_hidden_from_the_user()

    def __getattr__(self, attr):
        raise AttributeError("raised from A.__getattr__ to stop execution")

a = A()
print(a.x)


This results in the following output:

Traceback (most recent call last):
  File "test3.py", line 17, in <module>
    print(a.x)
  File "test3.py", line 14, in __getattr__
    raise AttributeError("raised from A.__getattr__ to stop execution")
AttributeError: raised from A.__getattr__ to stop execution


Here the real reason the code errors (`sys.this_doesnt_exist` throwing an
AttributeError) is not in the traceback's stack.

My current opinion on this is that the error that triggered __getattr__
should be passed to __getattr__. This would allow us to use the "raise
from" syntax.


Here's an example where I manually fix this for a specific class:

class A(object):
    def _some_complex_code_hidden_from_the_user(self):
        # a bunch of complex stuff that raises an attribute error internally
        # This could go layers deep into different modules
        return self.this_doesnt_exist

    @property
    def x(self):
        return self._some_complex_code_hidden_from_the_user()

    def __getattribute__(self, attr):
        try:
            return super().__getattribute__(attr)
        except AttributeError as error:
            return self.__custom_getattr__(attr, error)

    def __custom_getattr__(self, attr, error):
        raise AttributeError("raised from A.__getattr__ to stop execution")
from error

    #def __getattr__(self, attr):
    #    raise AttributeError("raised from A.__getattr__ to stop
execution") from error

a = A()
print(a.x)

This code correctly prints `AttributeError: module 'sys' has no attribute
'this_doesnt_exist'` because I `raise from error`.

Note that __getattr__ can't be defined;  if it is, it will squash the error
message from __custom_getattr__ analogous to what's going on in the first
example.

More importantly, note that __getattribute__ can't just `return
self.__getattr__(attr)` in its except block because __getattribute__ isn't
supposed to directly call __getattr__ (because __getattr__ gets triggered
later in the python attribute lookup order).

Unfortunately, this behavior is not generalizable without overriding
__getattribute__ and defining __custom_getattr__ on every single class in
the inheritance structure if I wanted it work for each class.


The 2nd example is the behavior I desire, and I don't really understand how
the descriptor figures into this problem to change that behavior.  -- I.e.
if you just remove @property from the first example, it returns the full
error stack exactly like we'd expect. That means the @property is changing
the call order (?) in some way that I don't understand.

Thanks!
Jason



On Tue, May 2, 2017 at 7:11 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 05/02/2017 11:16 AM, Jason Maldonis wrote:
>
> Here is the simplest example showing what I mean (there are many more
>> complicating variations this, and unfortunately I'm running into some of
>> them):
>>
>> class A(object):
>>      def __getattr__(self, attr):
>>          raise AttributeError("raised from A.__getattr__ to stop
>> execution")
>>
>> a = A()
>> print(a.x)
>>
>> results in:
>>
>> Traceback (most recent call last):
>>    File "test.py", line 6, in <module>
>>      print(a.x)
>>    File "test.py", line 3, in __getattr__
>>      raise AttributeError("raised from A.__getattr__ to stop execution")
>> AttributeError: raised from A.__getattr__ to stop execution
>>
>> The thing to note here is that the AttributeError on the normal
>> __getattribute__'s lookup isn't in the stack trace.  That error is:
>> Traceback (most recent call last):
>>    File "test2.py", line 35, in <module>
>>      print(a.x)
>> AttributeError: 'A' object has no attribute 'x'
>>
>> -- that last line, "AttributeError: 'A' object has no attribute 'x'" does
>> not appear in the stack trace for my above example (because __getattr__ is
>> implemented).  This is because in python's attribute lookup order,
>> __getattr__ is called if an AttributeError is raised, and that raised
>> AttributeError gets completely discarded.
>>
>> So basically I want access to the intermediate AttributeError that caused
>> __getattr__ to be raised in the first place.
>>
>
> Why?  In most cases* you know which object is missing -- it's the `attr`
> parameter above.
>
> *The exception being an AttributeError raised inside a descriptor.
>
> --
> ~Ethan~
> --
> https://mail.python.org/mailman/listinfo/python-list
>