[Numpy-discussion] Do we want scalar casting to behave as it does at the moment?

[Peter Cock]

On Fri, Jan 4, 2013 at 12:39 AM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> Since I've actually used NumPy arrays with specific low memory
>> types, I thought I should comment about my use case if case it
>> is helpful:
>>
>> I've only used the low precision types like np.uint8 (unsigned) where
>> I needed to limit my memory usage. In this case, the topology of a
>> graph allowing multiple edges held as an integer adjacency matrix, A.
>> I would calculate things like A^n for paths of length n, and also make
>> changes to A directly (e.g. adding edges). So an overflow was always
>> possible, and neither the old behaviour (type preserving but wrapping
>> on overflow giving data corruption) nor the current behaviour (type
>> promotion overriding my deliberate memory management) are nice.
>> My preferences here would be for an exception, so I knew right away.
>>
>> The other use case which comes to mind is dealing with low level
>> libraries and/or file formats, and here automagic type promotion
>> would probably be unwelcome.
>
> Regards,
>
> Peter

Elsewhere on the thread, Nathaniel Smith <njs at pobox.com> wrote:
>
> To be clear: we're only talking here about the case where you have a mix of
> a narrow dtype in an array and a scalar value that cannot be represented in
> that narrow dtype. If both sides are arrays then we continue to upcast as
> normal. So my impression is that this means very little in practical terms,
> because this is a rare and historically poorly supported situation.
>
> But if this is something you're running into in practice then you may have a
> better idea than us about the practical effects. Do you have any examples
> where this has come up that you can share?
>
> -n

Clarification appreciated - on closer inspection for my adjacency
matrix example I would not fall over the issue in
https://github.com/numpy/numpy/issues/2878

>>> import numpy as np
>>> np.__version__
'1.6.1'
>>> A = np.zeros((100,100), np.uint8) # Matrix could be very big
>>> A[3,4] = 255 # Max value, setting up next step in example
>>> A[3,4]
255
>>> A[3,4] += 1 # Silently overflows on NumPy 1.6
>>> A[3,4]
0

To trigger the contentious behaviour I'd have to do something
like this:

>>> A = np.zeros((100,100), np.uint8)
>>> B = A + 256
>>> B
array([[256, 256, 256, ..., 256, 256, 256],
       [256, 256, 256, ..., 256, 256, 256],
       [256, 256, 256, ..., 256, 256, 256],
       ...,
       [256, 256, 256, ..., 256, 256, 256],
       [256, 256, 256, ..., 256, 256, 256],
       [256, 256, 256, ..., 256, 256, 256]], dtype=uint16)

I wasn't doing anything like that in my code though, just
simple matrix multiplication and in situ element modification,
for example A[i,j] += 1 to add an edge.

I still agree that for https://github.com/numpy/numpy/issues/2878
an exception sounds sensible.

Peter