[Python-ideas] divmod(): fallback to __floordiv__ and __mod__?

[Steven D'Aprano]

On Sat, Sep 17, 2016 at 09:01:53AM +0000, Spencer Brown wrote:

> Currently, calling divmod() on a class with __floordiv__ and __mod__ 
> defined, but not __divmod__ raises a TypeError. Is there any reason 
> why it doesn't fallback to (self // x, self % x)? 

Because things get really complex, fast. Does the added complexity pay 
its own way?

I may have some of the details wrong, but I understand the 
implementation of divmod() is something like this:

# pseudo-code for divmod
def divmod(a, b):
    A, B = type(a), type(b)
    if issubclass(B, A):
        # Give priority to the reverse dunder version
        if hasattr(B, '__rdivmod__'):
            result = B.__rdivmod__(b, a)
            if result is not NotImplemented:
                return result
        if hasattr(A, '__divmod__'):
            result = A.__divmod__(a, b)
            if result is not NotImplemented:
                return result
        raise TypeError
    if hasattr(A, '__divmod__'):
        result = A.__divmod__(a, b)
        if result is not NotImplemented:
            return result
    if hasattr(B, '__rdivmod__'):
        result = B.__rdivmod__(b, a)
        if result is not NotImplemented:
            return result
    raise TypeError
    

only it will be in C, so probably three or five times as much code :-)

Now consider adding a fallback to __floordiv__ and __mod__ as suggested. 
That adds a lot more complexity:

- what if the object has __floordiv__, but not __mod__? Do you try 
  the reversed method, __rmod__, instead?

- likewise for NotImplemented?

- what if they're None?


So let's look at the complexity of the fallback:


# reverse dunder version, for when b is a subclass of a
rdiv = getattr(B, '__rfloordiv__, None)
rmod = getattr(B, '__rmod__, None)
if rdiv is not None and rmod is not None:
    x, y = rdiv(b, a), rmod(b, a)
    if x is NotImplemented:
        div = getattr(A, '__floordiv__', None)
        if div is not None:
            x = div(a, b)
            if x is NotImplemented:
                raise TypeError
    if y is NotImplemented:
        mod = getattr(A, '__mod__', None)
        if mod is not None:
            y = mod(a, b)
            if y is NotImplemented:
                raise TypeError
    assert NotImplemented not in (x, y)
else:
    # try the __floordiv__ and __mod__ versions, without falling 
    # back to reversed versions
    ...
return (x, y)


And then more or less the same for the "standard" case where a has 
priority over b (i.e. isn't a subclass). So, my estimate is that this 
will roughly triple the complexity of the divmod() function. Perhaps 
worse.

Oh, and I forgot the case where __divmod__ exists but is None, but I'm 
not going back to add it in because I'm not even sure where that ought 
to be tested.

Now I daresay we can refactor my naive pseudo-code above, but however 
you look at it, there's going to be a big jump in complexity for not a 
lot of benefit. In the case of __ne__ falling back on not __eq__, there 
is at least an utterly clear and obvious user-expectation that the two 
methods are linked, and there are no reversed __req__ and __rne__ 
methods to care about. That's not the case here.

Going the other way (fall back to __divmod__ if __floordiv__ or __mod__ 
is not defined) will probably be not quite as complex, but it will still 
add a fair amount of complexity.

So, *emotionally* I actually do quite like the idea of having divmod 
fall back to // and %, *and* the other way, but *intellectually* when I 
think about the complexity and the tests that will be needed to cover 
all the cases, I have to say "Uh uh, no way!"

It may be that the actual C implementation is simpler than I think, in 
which case maybe this is a reasonable idea, but on the face of it, I 
think that it will be really hard to get right, add a lot of code, and 
provide very little benefit.



-- 
Steve