[Tutor] could somebody please explain...

[Steven D'Aprano]

On Tue, Sep 30, 2014 at 03:54:42PM -0700, Clayton Kirkwood wrote:

> I don't understand the multiplicity of some tools. Namely, why is there a
> 'a+b', operator.add(a,b), operator.__add__(a,b), operator.iadd(a,b),
> operator.__iadd__(a,b) and their related operators?

The + operator is the public interface, but the implementation that 
makes + work are the special methods __add__ and __radd__ .

When you write in your code:

    result = a + b

how does Python know what to do with a and b? In principle, Python could 
hard-code into the language a handful of types that the interpreter 
knows how to add: int, float, str, list, etc. But that is not easily 
extended when a new type is supported, and Python supports "operator 
overloading" where any custom class can define "a + b" to do whatever 
the class developer wants.

So when the Python interpreter executes a + b, when it does is look for 
a special "dunder" (Double UNDERscore) method on a, __add__, or a 
special dunder method __radd__ ("right add") on b, and calls that.

Actually the rules are a bit more complicated than that, which I'm happy 
to explain if you would like, but for simplicity let's ignore __radd__ 
and just say that when Python sees "a + b" what actually gets called is 
a.__add__(b).

So when you create a new class and want it to support the + operator, 
you write a __add__ method:

class Spam:
    def __add__(self, other):
        ...


and now Python knows how to add your Spam instances together.

Sometimes it is useful to treat the + operator as a function, e.g. so 
that you can pass it to another function like reduce. But operators 
aren't values, you can't pass them to functions. This doesn't work:

py> reduce(+, [1, 2, 3, 4])
  File "<stdin>", line 1
    reduce(+, [1, 2, 3, 4])
            ^
SyntaxError: invalid syntax


But you can wrap the operator in a function using lambda:

py> reduce(lambda a, b: a+b, [1, 2, 3, 4])
10


but a more efficient way is to use the pre-made functions in the 
operator module:

py> import operator
py> reduce(operator.add, [1, 2, 3, 4])
10


So for every operator + - * / ** etc. there is a corresponding function 
version in the operator module, add(), sub() etc.


[ Aside: you might not know about reduce(). It takes a function f, and a 
list [a, b, c, d, ...] and calls the function with the first two values:

    result = f(a, b)

then takes that result and repeatedly calls the function again with the 
next value from the list:

    result = f(result, c)
    result = f(result, d)
    ...

until there are no more values left, then returns the final result. 
These days, now that Python has a sum() function, reduce() doesn't get 
used very often. ]

So for each operator that Python knows about, there is the operator 
itself, a function version, and one or two special dunder methods:

  Operator    Function        Dunder methods
  ==========  ==============  =====================
  +           operator.add    __add__  __radd__
  -           operator.sub    __sub__  __rsub__
  *           operator.mul    __mul__  __rmul__
  **          operator.pow    __pow__  __rpow__
  ==          operator.eq     __eq__
  !=          operator.ne     __ne__

etc.

Then there are the special "augmented assignment" operators, so that 
Python can support writing:

  x += 1
  y -= x

etc. Again, the syntax used is a combined operator-assignment += and 
that ends up calling a special dunder method, __iadd__. And again, there 
are special function versions in the operator module.


In summary:

(1) When you want to add two values, use a + b.

(2) When you want a function that adds two values, use operator.add.

(3) When you want to write a class that supports addition, give it 
    the two special dunder methods __add__ and __radd__.

(4) You almost never should call __add__ yourself.



-- 
Steven