[Python-Dev] sum(...) limitation - temporary elision take 2

[Julian Taylor]

On 04.08.2014 22:22, Jim J. Jewett wrote:
> 
> 
> 
> Sat Aug 2 12:11:54 CEST 2014, Julian Taylor wrote (in
> https://mail.python.org/pipermail/python-dev/2014-August/135623.html ) wrote:
> 
> 
>> Andrea Griffini <agriff at tin.it> wrote:
> 
>>>    However sum([[1,2,3],[4],[],[5,6]], []) concatenates the lists.
>     
>> hm could this be a pure python case that would profit from temporary
>> elision [ https://mail.python.org/pipermail/python-dev/2014-June/134826.html ]?
> 
>> lists could declare the tp_can_elide slot and call list.extend on the
>> temporary during its tp_add slot instead of creating a new temporary.
>> extend/realloc can avoid the copy if there is free memory available
>> after the block.
> 
> Yes, with all the same problems.
> 
> When dealing with a complex object, how can you be sure that __add__
> won't need access to the original values during the entire computation?
> It works with matrix addition, but not with matric multiplication.
> Depending on the details of the implementation, it could even fail for
> a sort of sliding-neighbor addition similar to the original justification.

The c-extension object knows what its add slot does. An object that
cannot elide would simply always return 0 indicating to python to not
call the inplace variant.
E.g. the numpy __matmul__ operator would never tell python that it can
work inplace, but __add__ would (if the arguments allow it).

Though we may have found a way to do it without the direct help of
Python, but it involves reading and storing the current instruction of
the frame object to figure out if it is called directly from the
interpreter.
unfinished patch to numpy, see the can_elide_temp function:
https://github.com/numpy/numpy/pull/4322.diff
Probably not the best way as this is hardly intended Python C-API but
assuming there is no overlooked issue with this approach it could be a
good workaround for known good Python versions.