[Numpy-discussion] ENH: Proposal to add atleast_nd function

Thu Feb 11 21:31:56 EST 2021

both napari and scikit-image use atleast_ a few times. I don’t have many examples of where I used nd because it didn’t exist. But I have the very distinct impression of needing it repeatedly. In some places, I’ve used `np.broadcast_to` to signal the same intention, where `atleast_nd` would have been the more readable solution.

I don’t buy the argument that it’s just a way to mask errors. NumPy broadcasting also has that same potential but I hope no one would seriously consider deprecating it. Indeed, even if we accept that we (library authors) should force users to provide an array of the right dimensionality, that still argues for making it convenient for users to do that!

I don’t feel super strongly about this. But I think atleast_nd is a move in a positive direction and I’d prefer  it to what’s there now:

In [1]: import numpy as np
In [2]: np.atleast_3d(np.ones(4)).shape
Out[2]: (1, 4, 1)

There might be some linear algebraic reason why those axis positions make sense, but I’m not aware of it...

Juan.

> On 12 Feb 2021, at 5:32 am, Eric Wieser <wieser.eric+numpy at gmail.com> wrote:
> 
> I did a quick search of matplotlib, and found a few uses of all three functions:
> 
> * https://github.com/matplotlib/matplotlib/blob/fed55c63a314351cd39a12783f385009782c06e1/lib/matplotlib/_layoutgrid.py#L441-L446 <https://github.com/matplotlib/matplotlib/blob/fed55c63a314351cd39a12783f385009782c06e1/lib/matplotlib/_layoutgrid.py#L441-L446>
>   This one isn't really numpy at all, and is really just a shorthand for normalizing an argument `x=n` to `x=[n, n]`
> * https://github.com/matplotlib/matplotlib/blob/dd249744270f6abe3f540f81b7a77c0cb728ddbb/lib/matplotlib/mlab.py#L888 <https://github.com/matplotlib/matplotlib/blob/dd249744270f6abe3f540f81b7a77c0cb728ddbb/lib/matplotlib/mlab.py#L888>
>    This one is the classic "either multivariate or single-variable data" thing endemic to the SciPy ecosystem.
> * https://github.com/matplotlib/matplotlib/blob/1eef019109b64ee4085732544cb5e310e69451ab/lib/matplotlib/cbook/__init__.py#L1325-L1326 <https://github.com/matplotlib/matplotlib/blob/1eef019109b64ee4085732544cb5e310e69451ab/lib/matplotlib/cbook/__init__.py#L1325-L1326>
>   Matplotlib has their own `_check_1d` function for input sanitization, although github says it's only used to parse the arguments to `plot`, which at this point are fairly established as being flexible.
> * https://github.com/matplotlib/matplotlib/blob/f72adc49092fe0233a8cd21aa0f317918dafb18d/lib/matplotlib/transforms.py#L631 <https://github.com/matplotlib/matplotlib/blob/f72adc49092fe0233a8cd21aa0f317918dafb18d/lib/matplotlib/transforms.py#L631>
>   This just looks like "defensive programming", and if the argument isn't already 3d then something is probably wrong.
> 
> This isn't an exhaustive list, just a handful of different situations the functions were used.
> 
> Eric
> 
> 
> 
> On Thu, 11 Feb 2021 at 18:15, Stephan Hoyer <shoyer at gmail.com <mailto:shoyer at gmail.com>> wrote:
> On Thu, Feb 11, 2021 at 9:42 AM Benjamin Root <ben.v.root at gmail.com <mailto:ben.v.root at gmail.com>> wrote:
> for me, I find that the at_least{1,2,3}d functions are useful for sanitizing inputs. Having an at_leastnd() function can be viewed as a step towards cleaning up the API, not cluttering it (although, deprecations of the existing functions probably should be long given how long they have existed).
> 
> I would love to see examples of this -- perhaps in matplotlib?
> 
> My thinking is that in most cases it's probably a better idea to keep the interface simpler, and raise an error for lower-dimensional arrays. Automatic conversion is convenient (and endemic within the SciPy ecosystem), but is also a common source of bugs.
> 
> On Thu, Feb 11, 2021 at 1:56 AM Stephan Hoyer <shoyer at gmail.com <mailto:shoyer at gmail.com>> wrote:
> On Wed, Feb 10, 2021 at 9:48 PM Juan Nunez-Iglesias <jni at fastmail.com <mailto:jni at fastmail.com>> wrote:
> I totally agree with the namespace clutter concern, but honestly, I would use `atleast_nd` with its `pos` argument (I might rename it to `position`, `axis`, or `axis_position`) any day over `at_least{1,2,3}d`, for which I had no idea where the new axes would end up.
> 
> So, I’m in favour of including it, and optionally deprecating `atleast_{1,2,3}d`.
> 
> 
> I appreciate that `atleast_nd` feels more sensible than `at_least{1,2,3}d`, but I don't think "better" than a pattern we would not recommend is a good enough reason for inclusion in NumPy. It needs to stand on its own.
> 
> What would be the recommended use-cases for this new function?
> Have any libraries building on top of NumPy implemented a version of this?
>  
> Juan.
> 
>> On 11 Feb 2021, at 9:48 am, Sebastian Berg <sebastian at sipsolutions.net <mailto:sebastian at sipsolutions.net>> wrote:
>> 
>> On Wed, 2021-02-10 at 17:31 -0500, Joseph Fox-Rabinovitz wrote:
>>> I've created PR#18386 to add a function called atleast_nd to numpy and
>>> numpy.ma <http://numpy.ma/>. This would generalize the existing atleast_1d, atleast_2d, and
>>> atleast_3d functions.
>>> 
>>> I proposed a similar idea about four and a half years ago:
>>> https://mail.python.org/pipermail/numpy-discussion/2016-July/075722.html <https://mail.python.org/pipermail/numpy-discussion/2016-July/075722.html>,
>>> PR#7804. The reception was ambivalent, but a couple of folks have asked me
>>> about this, so I'm bringing it back.
>>> 
>>> Some pros:
>>> 
>>> - This closes issue #12336
>>> - There are a couple of Stack Overflow questions that would benefit
>>> - Been asked about this a couple of times
>>> - Implementation of three existing atleast_*d functions gets easier
>>> - Looks nicer that the equivalent broadcasting and reshaping
>>> 
>>> Some cons:
>>> 
>>> - Cluttering up the API
>>> - Maintenance burden (but not a big one)
>>> - This is just a utility function, which can be achieved through
>>> broadcasting and reshaping
>>> 
>> 
>> My main concern would be the namespace cluttering. I can't say I use even the `atleast_2d` etc. functions personally, so I would tend to be slightly against the addition. But if others land on the "useful" side here (and it seemed a bit at least on github), I am also not opposed.  It is a clean name that lines up with existing ones, so it doesn't seem like a big "mental load" with respect to namespace cluttering.
>> 
>> Bike shedding the API is probably a good idea in any case.
>> 
>> I have pasted the current PR documentation (as html) below for quick reference. I wonder a bit about the reasoning for having `pos` specify a value rather than just a side?
>> 
>> 
>> 
>> numpy.atleast_nd(ary, ndim, pos=0)
>> View input as array with at least ndim dimensions.
>> New unit dimensions are inserted at the index given by pos if necessary.
>> Parameters
>> ary  array_like
>> The input array. Non-array inputs are converted to arrays. Arrays that already have ndim or more dimensions are preserved.
>> ndim  int
>> The minimum number of dimensions required.
>> pos  int, optional
>> The index to insert the new dimensions. May range from -ary.ndim - 1 to +ary.ndim (inclusive). Non-negative indices indicate locations before the corresponding axis: pos=0 means to insert at the very beginning. Negative indices indicate locations after the corresponding axis: pos=-1 means to insert at the very end. 0 and -1 are always guaranteed to work. Any other number will depend on the dimensions of the existing array. Default is 0.
>> Returns
>> res  ndarray
>> An array with res.ndim >= ndim. A view is returned for array inputs. Dimensions are prepended if pos is 0, so for example, a 1-D array of shape (N,) with ndim=4becomes a view of shape (1, 1, 1, N). Dimensions are appended if pos is -1, so for example a 2-D array of shape (M, N) becomes a view of shape (M, N, 1, 1)when ndim=4.
>> See also
>> atleast_1d <https://18298-908607-gh.circle-artifacts.com/0/doc/build/html/reference/generated/numpy.atleast_1d.html#numpy.atleast_1d>, atleast_2d <https://18298-908607-gh.circle-artifacts.com/0/doc/build/html/reference/generated/numpy.atleast_2d.html#numpy.atleast_2d>, atleast_3d <https://18298-908607-gh.circle-artifacts.com/0/doc/build/html/reference/generated/numpy.atleast_3d.html#numpy.atleast_3d>
>> Notes
>> This function does not follow the convention of the other atleast_*d functions in numpy in that it only accepts a single array argument. To process multiple arrays, use a comprehension or loop around the function call. See examples below.
>> Setting pos=0 is equivalent to how the array would be interpreted by numpy’s broadcasting rules. There is no need to call this function for simple broadcasting. This is also roughly (but not exactly) equivalent to np.array(ary, copy=False, subok=True, ndmin=ndim).
>> It is easy to create functions for specific dimensions similar to the other atleast_*d functions using Python’s functools.partial <https://docs.python.org/dev/library/functools.html#functools.partial> function. An example is shown below.
>> Examples
>> >>> np.atleast_nd(3.0, 4)
>> array([[[[ 3.]]]])
>> >>> x = np.arange(3.0)
>> >>> np.atleast_nd(x, 2).shape
>> (1, 3)
>> >>> x = np.arange(12.0).reshape(4, 3)
>> >>> np.atleast_nd(x, 5).shape
>> (1, 1, 1, 4, 3)
>> >>> np.atleast_nd(x, 5).base is x.base
>> True
>> >>> [np.atleast_nd(x) for x in ((1, 2), [[1, 2]], [[[1, 2]]])]:
>> [array([[1, 2]]), array([[1, 2]]), array([[[1, 2]]])]
>> >>> np.atleast_nd((1, 2), 5, pos=0).shape
>> (1, 1, 1, 1, 2)
>> >>> np.atleast_nd((1, 2), 5, pos=-1).shape
>> (2, 1, 1, 1, 1)
>> >>> from functools import partial
>> >>> atleast_4d = partial(np.atleast_nd, ndim=4)
>> >>> atleast_4d([1, 2, 3])
>> [[[[1, 2, 3]]]]
>> 
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
>> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20210212/2dfdb06f/attachment-0001.html>