[Numpy-discussion] Add guaranteed no-copy to array creation and reshape?

[Eric Wieser]

I think np.never_copy is really ugly. I’d much rather simply use ‘never’,

I actually think np.never_copy is the better choice of the two. Arguments
for it:

   - It’s consistent with np.newaxis in spelling (modulo the _)
   - If mistyped, it can be caught easily by IDEs.
   - It’s typeable with mypy, unlike constant string literals which
   currently aren’t
   - If code written against new numpy is run on old numpy, it will error
   rather than doing the wrong thing
   - It won’t be possible to miss parts of our existing API that evaluate a
   copy argument in a boolean context
   - It won’t be possible for downstream projects to misinterpret by not
   checking for ‘never’ - an error will be raised instead.

Arguments against it:

   - It’s more characters than "never"
   - The implementation of np.never_copy is a little verbose / ugly

Eric

On Thu, Dec 27, 2018, 1:41 AM Ralf Gommers <ralf.gommers at gmail.com wrote:


>
> On Wed, Dec 26, 2018 at 3:29 PM Sebastian Berg <sebastian at sipsolutions.net>
> wrote:
>
>> Hi all,
>>
>> In https://github.com/numpy/numpy/pull/11897 I am looking into the
>> addition of a `copy=np.never_copy` argument to:
>>   * np.array
>>   * arr.reshape/np.reshape
>>   * arr.astype
>>
>> Which would cause an error to be raised when numpy cannot guarantee
>> that the returned array is a view of the input array.
>> The motivation is to easier avoid accidental copies of large data, or
>> ensure that in-place manipulation will be meaningful.
>>
>> The copy flag API would be:
>>   * `copy=True` forces a copy
>>   * `copy=False` allows numpy to copy if necessary
>>   * `copy=np.never_copy` will error if a copy would be necessary
>>   * (almost) all other input will be deprecated.
>>
>> Unfortunately using `copy="never"` is tricky, because currently
>> `np.array(..., copy="never")` behaves exactly the same as
>> `np.array(..., copy=bool("never"))`. So that the wrong result would be
>> given on old numpy versions and it would be a behaviour change.
>>
>
> I think np.never_copy is really ugly. I'd much rather simply use 'never',
> and clearly document that if users start using this and they critically
> rely on it really being never, then they should ensure that their code is
> only used with numpy >= 1.17.0.
>
> Note also that this would not be a backwards compatibility break, because
> `copy` is now clearly documented as bool, and not bool_like or some such
> thing. So we do not need to worry about the very improbable case that users
> now are using `copy='never'`.
>
> If others think `copy='never'` isn't acceptable now, there are two other
> options:
> 1. add code first to catch `copy='never'` in 1.17.x and raise on it, then
> in a later numpy version introduce it.
> 2. just do nothing. I'd prefer that over `np.never_copy`.
>
> Cheers,
> Ralf
>
>
>> Some things that are a not so nice maybe:
>>  * adding/using `np.never_copy` is not very nice
>>  * Scalars need a copy and so will not be allowed
>>  * For rare array-likes numpy may not be able to guarantee no-copy,
>>    although it could happen (but should not).
>>
>>
>> The history is that a long while ago I considered adding a copy flag to
>> `reshape` so that it is possible to do `copy=np.never_copy` (or
>> similar) to ensure that no copy is made. In these, you may want
>> something like an assertion:
>>
>> ```
>> new_arr = arr.reshape(new_shape)
>> assert np.may_share_memory(arr, new_arr)
>>
>> # Which is sometimes -- but should not be -- written as:
>> arr.shape = new_shape  # unnecessary container modification
>>
>> # Or:
>> view = np.array(arr, order="F")
>> assert np.may_share_memory(arr, new_arr)
>> ```
>>
>> but is more readable and will not cause an intermediate copy on error.
>>
>>
>> So what do you think? Other variants would be to not expose this for
>> `np.array` and probably limit `copy="never"` to the reshape method. Or
>> just to not do it at all. Or to also accept "never" for `reshape`,
>> although I think I would prefer to keep it in sync and wait for a few
>> years to consider that.
>>
>> Best,
>>
>> Sebastian
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181227/9d9a320d/attachment.html>