returning NotImplemented

[Eric Snow]

On Tue, May 31, 2011 at 6:30 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> Eric Snow wrote:
>
>> Guido indicates earlier in the thread that NotImplemented is used so that
>> you know that it came from the function that you directly called, and not
>> from another call inside that function.  Why does it matter if it came
>> directly from the function or not?  And couldn't the NotImplemented still
>> have come from a call inside the operator, rather than directly?
>>
>
> An exception can bubble up from several levels below the original call.
>  Returning NotImplemented (even if returned from another internal function)
> still has to be deliberately done by the original method.
>
> Later in the thread it is also stated that while exception handling
> performance is not too different in C code, it is very much more expensive
> in Python code.
>
> As to why it would matter if whether the return was from the directly
> called method vs some nested method... not sure, but I would hazard a guess
> that by returning NotImplented the interpreter knows everything worked as it
> should have (assuming no bugs, of course); whereas an exception could easily
> _not_ have come from the directly called method.  Imagine if the methods
> raised exceptions to signal normal behavior, and some object was buggy --
> for __eq__ we'd end up with a False result, instead of getting the exception
> propagated. (!)
>
>
Thanks Ethan.  That was insightful.  I hadn't considered the case where we
would want a TypeError or NotImplementedError or whatever to propagate.  So
simply catching any old exception doesn't cut it.  The alternative is to
have something like a CouldNotHandleTypeError that inherits just from
BaseException, that is only used in the NotImplemented situation.  Then the
specificity resolves that problem.

You're right that performance still suffers for python code, but the current
NotImplemented handling stuff is in the C code that handles operators.  I
guess where it would hit is where you raise the Exception in the __eq__
method (or wherever), instead of returning NotImplemented.

So, my understanding is that NotImplemented is sort of a special case of
raising "exceptions" through return values for performance reasons, where
normally we use exceptions.  Here is an example of the equivalence that I am
seeing:

class X:
    def __eq__(self, other):
        return NotImplemented

result = X().__eq__(1)
if result is NotImplemented:
    raise TypeError

  - vs -

class X:
    def __eq__(self, other):
        raise CouldNotHandleTypeError

try:
    result = X().__eq__(1)
except CouldNotHandleTypeError:
    raise TypeError

I'm fine with this.  It's probably one of those practicality beats purity
things.  And at the C level it's practically the same thing.  As I have
thought about this I've realized that it's really isolated to a specific
place, namely operator handling for numeric and comparison operators, so no
big deal.

-eric

~Ethan~
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110531/fd47d96b/attachment-0001.html>