[Python-Dev] Python-versus-CPython question for __mul__ dispatch

[Stefan Richthofer]

>>Should I be worried?

You mean should *I* be worried ;)

Stuff like this is highly relevant for JyNI, so thanks very much for clarifying this
subtle behavior. It went onto my todo-list right now to ensure that JyNI will emulate
this behavior as soon as I am done with gc-support. (Hopefully it will be feasible,
but I can only tell in half a year or so since there are currently other priorities.)
Still, this "essay" potentially will save me a lot of time.

So, everybody please feel encouraged to post things like this as they come up. Maybe
there could be kind of a pitfalls-page somewhere in the docs collecting these things.

Best

Stefan


> Gesendet: Freitag, 15. Mai 2015 um 02:45 Uhr
> Von: "Nathaniel Smith" <njs at pobox.com>
> An: "Python Dev" <python-dev at python.org>
> Betreff: [Python-Dev] Python-versus-CPython question for __mul__ dispatch
>
> Hi all,
> 
> While attempting to clean up some of the more squamous aspects of
> numpy's operator dispatch code [1][2], I've encountered a situation
> where the semantics we want and are using are possible according to
> CPython-the-interpreter, but AFAICT ought not to be possible according
> to Python-the-language, i.e., it's not clear to me whether it's
> possible even in principle to implement an object that works the way
> numpy.ndarray does in any other interpreter. Which makes me a bit
> nervous, so I wanted to check if there was any ruling on this.
> 
> Specifically, the quirk we are relying on is this: in CPython, if you do
> 
>   [1, 2] * my_object
> 
> then my_object's __rmul__ gets called *before* list.__mul__,
> *regardless* of the inheritance relationship between list and
> type(my_object). This occurs as a side-effect of the weirdness
> involved in having both tp_as_number->nb_multiply and
> tp_as_sequence->sq_repeat in the C API -- when evaluating "a * b",
> CPython tries a's nb_multiply, then b's nb_multiply, then a's
> sq_repeat, then b's sq_repeat. Since list has an sq_repeat but not an
> nb_multiply, this means that my_object's nb_multiply gets called
> before any list method.
> 
> Here's an example demonstrating how weird this is. list.__mul__ wants
> an integer, and by "integer" it means "any object with an __index__
> method". So here's a class that list is happy to be multiplied by --
> according to the ordinary rules for operator dispatch, in the example
> below Indexable.__mul__ and __rmul__ shouldn't even get a look-in:
> 
> In [3]: class Indexable(object):
>    ...:     def __index__(self):
>    ...:         return 2
>    ...:
> 
> In [4]: [1, 2] * Indexable()
> Out[4]: [1, 2, 1, 2]
> 
> But, if I add an __rmul__ method, then this actually wins:
> 
> In [6]: class IndexableWithMul(object):
>    ...:     def __index__(self):
>    ...:         return 2
>   ...:     def __mul__(self, other):
>    ...:         return "indexable forward mul"
>    ...:     def __rmul__(self, other):
>    ...:         return "indexable reverse mul"
> 
> In [7]: [1, 2] * IndexableWithMul()
> Out[7]: 'indexable reverse mul'
> 
> In [8]: IndexableWithMul() * [1, 2]
> Out[8]: 'indexable forward mul'
> 
> NumPy arrays, of course, correctly define both __index__ method (which
> raises an array on general arrays but coerces to int for arrays that
> contain exactly 1 integer), and also defines an nb_multiply slot which
> accepts lists and performs elementwise multiplication:
> 
> In [9]: [1, 2] * np.array(2)
> Out[9]: array([2, 4])
> 
> And that's all great! Just what we want. But the only reason this is
> possible, AFAICT, is that CPython 'list' is a weird type with
> undocumented behaviour that you can't actually define using pure
> Python code.
> 
> Should I be worried?
> 
> -n
> 
> [1] https://github.com/numpy/numpy/pull/5864
> [2] https://github.com/numpy/numpy/issues/5844
> 
> -- 
> Nathaniel J. Smith -- http://vorpus.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
>