[Numpy-discussion] Baffling error: ndarray += csc_matrix -> "ValueError: setting an array element with a sequence"

[Pauli Virtanen]

27.09.2013 22:15, Nathaniel Smith kirjoitti:
[clip]
> 3) The issue of how to make an in-place like ndarray += sparse
> continue to work in the brave new __numpy_ufunc__ world.
> 
> For this last issue, I think we disagree. It seems to me that the
> right answer is that csc_matrix.__numpy_ufunc__ needs to step up and
> start supporting out=! If I have a large dense ndarray and I try to +=
> a sparse array to it, this operation should take no temporary memory
> and nnz time. Right now it sounds like it actually copies the large
> dense ndarray, which takes time and space proportional to its size.
> AFAICT the only way to avoid that is for scipy.sparse to implement
> out=. It shouldn't be that hard...?

Sure, scipy.sparse can easily support also the output argument.

But I still maintain that the implementation of __iadd__ in Numpy is
wrong. What it does now is:

	def __iadd__(self, other):
	    return np.add(self, other, out=self)

But since we know this raises a TypeError if the input is of a type that
cannot be dealt with, it should be:

	def __iadd__(self, other):
	    try:
	        return np.add(self, other, out=self)
	    except TypeError:
	        return NotImplemented

Of course, it's written in C so it's a bit more painful to write this.

I think this will have little performance impact, since the check would
be only a NULL check in the inplace op methods + subsequent handling. I
can take a look at some point at this...

-- 
Pauli Virtanen