From anntzer.lee at gmail.com Fri Apr 2 09:09:14 2021 From: anntzer.lee at gmail.com (Antony Lee) Date: Fri, 2 Apr 2021 15:09:14 +0200 Subject: [Numpy-discussion] ANN: mplcairo 0.4 release Message-ID: Dear all, I am pleased to announce the release of mplcairo 0.4. mplcairo is a Matplotlib backend based on the well-known cairo library, supporting output to both raster (including interactively) and vector formats. In other words, it provides the functionality of Matplotlib's {,qt5,gtk3,wx,tk,macos}{agg,cairo}, pdf, ps, and svg backends. Per Matplotlib's standard API, the backend can be selected by calling matplotlib.use("module://mplcairo.qt") or setting your MPLBACKEND environment variable to `module://mplcairo.qt` for Qt5, and similarly for other toolkits. mplcairo 0.4 adds support for Matplotlib 3.4, support for retrieving underlying cairo contexts, as well as the usual bugfixes over 0.3. Enjoy, Antony Lee -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Fri Apr 2 16:46:33 2021 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Fri, 02 Apr 2021 15:46:33 -0500 Subject: [Numpy-discussion] Type resolver related deprecations and changes Message-ID: Hi all, I have to do some changes to the type resolution, and I started these here: https://github.com/numpy/numpy/pull/18718 There are four changes: * Deprecate `signature="l"` and `signature=("l",)`, these are confusing since the signature should include all inputs and outputs. To only provide the output use `dtype="l"`. * Using `dtype=` for comparisons (e.g. `np.equal`) used to be weird: np.equal(1, 2, dtype=object) -> returns boolean np.equal(None, 2, dtype=object) -> returns object array The first one will now give a FutureWarning. Comparisons that provide a dtype other than object or bool give a DeprecationWarning (or fail). I hope the warning can be preserved when more refactoring happens. * NumPy *almost* always ignores any metadata, byte-order, time unit information from the `dtype` or `signature` arguments to ufuncs. Practically, the dtypes passed actually denote the DType types rather than the specific instance (which could be byte swapped). NumPy will now do this always and give a warning if byte-order or time unit is ignored! * It is THEORETICALLY possible to call `ufunc->type_resolver` in the C API (as opposed to providing it, which is somewhat OK). If someone does that they have to normalize the type tuple now, I don't really see a reason for keeping support, when NumPy will stop calling it itself almost always and anyone using it would probably be in trouble soon. To be clear: I have NOT found a single instance of such code in a code search. Even *providing* it ? which is much more reasonable ? is probably only done by astropy/pyerfa. ** Long example for the "time unit" dropping change ** For the third point, which is in theory the largest impact. Both pandas and astropy do not notice it (I also grepped scipy, its clean). These are the biggest changes: # The following will now warn on most systems (unchanged result): np.add(3, 5, dtype=">i32") # The biggest impact is for timedelta or datetimes: arr = np.arange(10, dtype="m8[s]") # The examples always ignored the time unit "ns" (using the # unit of `arr`. They now issue a warning: np.add(arr, arr, dtype="m8[ns]") np.maximum.reduce(arr, dtype="m8[ns]") # The following issue a warning but previously did return # a "ns" result. np.add(3, 5, dtype="m8[ns]") # Now return generic time units np.maximum(arr, arr, dtype="m8[ns]") # Now returns "s" (from `arr`) I doubt there is a good way to just keep the old behaviour. It is hopelessly inconsistent. (The result even depends on how you pass things, as the paths I deprecate align with `dtype=` but not with the `signature=(None, None, dtype)` equivalent call.) One thought I had just raising a hard error instead of a UserWarning right now. If you say that why not have `dtype=...` always be honored as the correct output dtype, I don't disagree. But it seems to me we probably would have wade through a FutureWarning first, so a warning is the "right direction". (Or just add an `output_dtypes` keyword argument.) Cheers, Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From rajpoothks07 at gmail.com Sat Apr 3 12:51:56 2021 From: rajpoothks07 at gmail.com (Harsh Kumar Singh) Date: Sat, 3 Apr 2021 22:21:56 +0530 Subject: [Numpy-discussion] Beginner at open source Message-ID: Hi all, As I am beginner at open source. I would be obliged if someone could guide me forward. Provided I know basics of numpy and other machine learning libraries. -------------- next part -------------- An HTML attachment was scrubbed... URL: From melissawm at gmail.com Sat Apr 3 14:51:53 2021 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Sat, 3 Apr 2021 15:51:53 -0300 Subject: [Numpy-discussion] Beginner at open source In-Reply-To: References: Message-ID: Hello, Harsh Do you have any specific questions? If not, I would suggest you look at our contributor guide (https://numpy.org/devdocs/dev/index.html) and maybe check open issues in the NumPy github repo to see if anything looks interesting to you. Cheers, Melissa On Sat, Apr 3, 2021 at 1:53 PM Harsh Kumar Singh wrote: > Hi all, > As I am beginner at open source. I would be obliged if someone could guide > me forward. Provided I know basics of numpy and other machine learning > libraries. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdubravski at gmail.com Tue Apr 6 14:26:57 2021 From: mdubravski at gmail.com (Michael Dubravski) Date: Tue, 06 Apr 2021 14:26:57 -0400 Subject: [Numpy-discussion] MAINT: Use of except-pass blocks Message-ID: <599B07BF-430F-4360-9DF4-41BD91E0492E@gmail.com> Hello everyone, There are multiple instances of except-pass blocks within the codebase that to my knowledge are bad practices (Referencing This StackOverflow Article. For example in numpy/ma/core.py there is an except-pass block that catches all exceptions thrown. Another example of this can be found in numpy/core/function_base.py. I was wondering if it would be a good idea to add some print statements for logging the exceptions caught. Also for cases where except-pass blocks are needed, is there an explanation for not logging exceptions? https://github.com/numpy/numpy/blob/914407d51b878bf7bf34dbd8dd72cc2dbc428673/numpy/ma/core.py#L1034-L1041 https://github.com/numpy/numpy/blob/914407d51b878bf7bf34dbd8dd72cc2dbc428673/numpy/core/function_base.py#L461-L472 Thanks, Michael Dubravski -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben.v.root at gmail.com Tue Apr 6 14:57:38 2021 From: ben.v.root at gmail.com (Benjamin Root) Date: Tue, 6 Apr 2021 14:57:38 -0400 Subject: [Numpy-discussion] MAINT: Use of except-pass blocks In-Reply-To: <599B07BF-430F-4360-9DF4-41BD91E0492E@gmail.com> References: <599B07BF-430F-4360-9DF4-41BD91E0492E@gmail.com> Message-ID: In both of those situations, the `pass` aspect makes sense, although they probably should specify a better exception class to catch. The first one, with the copyto() has a comment that explains what is goingon. The second one, dealing with adding to the docstring, is needed because one can run python in the "optimized" mode, which strips out docstrings. On Tue, Apr 6, 2021 at 2:27 PM Michael Dubravski wrote: > Hello everyone, > > > > There are multiple instances of except-pass blocks within the codebase > that to my knowledge are bad practices (Referencing This StackOverflow > Article > . > For example in numpy/ma/core.py there is an except-pass block that > catches all exceptions thrown. Another example of this can be found in > numpy/core/function_base.py. I was wondering if it would be a good idea > to add some print statements for logging the exceptions caught. Also for > cases where except-pass blocks are needed, is there an explanation for not > logging exceptions? > > > > > https://github.com/numpy/numpy/blob/914407d51b878bf7bf34dbd8dd72cc2dbc428673/numpy/ma/core.py#L1034-L1041 > > > > > https://github.com/numpy/numpy/blob/914407d51b878bf7bf34dbd8dd72cc2dbc428673/numpy/core/function_base.py#L461-L472 > > > > Thanks, > > Michael Dubravski > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdubravski at gmail.com Tue Apr 6 15:12:42 2021 From: mdubravski at gmail.com (Michael Dubravski) Date: Tue, 06 Apr 2021 15:12:42 -0400 Subject: [Numpy-discussion] MAINT: Use of except-pass blocks In-Reply-To: References: <599B07BF-430F-4360-9DF4-41BD91E0492E@gmail.com> Message-ID: <433E3F6B-1732-4088-AD94-1021A8D9ED44@gmail.com> Okay thank you for the input. Do you have any recommendations for the type of exception classes that they could be changed to? From: NumPy-Discussion on behalf of Benjamin Root Reply-To: Discussion of Numerical Python Date: Tuesday, April 6, 2021 at 2:58 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] MAINT: Use of except-pass blocks In both of those situations, the `pass` aspect makes sense, although they probably should specify a better exception class to catch. The first one, with the copyto() has a comment that explains what is goingon. The second one, dealing with adding to the docstring, is needed because one can run python in the "optimized" mode, which strips out docstrings. On Tue, Apr 6, 2021 at 2:27 PM Michael Dubravski wrote: Hello everyone, There are multiple instances of except-pass blocks within the codebase that to my knowledge are bad practices (Referencing This StackOverflow Article. For example in numpy/ma/core.py there is an except-pass block that catches all exceptions thrown. Another example of this can be found in numpy/core/function_base.py. I was wondering if it would be a good idea to add some print statements for logging the exceptions caught. Also for cases where except-pass blocks are needed, is there an explanation for not logging exceptions? https://github.com/numpy/numpy/blob/914407d51b878bf7bf34dbd8dd72cc2dbc428673/numpy/ma/core.py#L1034-L1041 https://github.com/numpy/numpy/blob/914407d51b878bf7bf34dbd8dd72cc2dbc428673/numpy/core/function_base.py#L461-L472 Thanks, Michael Dubravski _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieser.eric+numpy at gmail.com Tue Apr 6 15:24:44 2021 From: wieser.eric+numpy at gmail.com (Eric Wieser) Date: Tue, 6 Apr 2021 20:24:44 +0100 Subject: [Numpy-discussion] MAINT: Use of except-pass blocks In-Reply-To: <433E3F6B-1732-4088-AD94-1021A8D9ED44@gmail.com> References: <599B07BF-430F-4360-9DF4-41BD91E0492E@gmail.com> <433E3F6B-1732-4088-AD94-1021A8D9ED44@gmail.com> Message-ID: I think the add_docstring one is best left alone, since if it fails once it will probably fail for every docstring in the system, and the logging would be pure noise. The ma/core one is suspicious - I can't think of any examples where an error would occur, but if you're interested, I'd encourage you to try and come up with a corner-case that enters that `except` block. Eric On Tue, Apr 6, 2021, 20:13 Michael Dubravski wrote: > Okay thank you for the input. Do you have any recommendations for the type > of exception classes that they could be changed to? > > > > *From: *NumPy-Discussion gmail.com at python.org> on behalf of Benjamin Root > *Reply-To: *Discussion of Numerical Python > *Date: *Tuesday, April 6, 2021 at 2:58 PM > *To: *Discussion of Numerical Python > *Subject: *Re: [Numpy-discussion] MAINT: Use of except-pass blocks > > > > In both of those situations, the `pass` aspect makes sense, although they > probably should specify a better exception class to catch. The first one, > with the copyto() has a comment that explains what is goingon. The second > one, dealing with adding to the docstring, is needed because one can run > python in the "optimized" mode, which strips out docstrings. > > > > On Tue, Apr 6, 2021 at 2:27 PM Michael Dubravski > wrote: > > Hello everyone, > > > > There are multiple instances of except-pass blocks within the codebase > that to my knowledge are bad practices (Referencing This StackOverflow > Article > . > For example in numpy/ma/core.py there is an except-pass block that > catches all exceptions thrown. Another example of this can be found in > numpy/core/function_base.py. I was wondering if it would be a good idea > to add some print statements for logging the exceptions caught. Also for > cases where except-pass blocks are needed, is there an explanation for not > logging exceptions? > > > > > https://github.com/numpy/numpy/blob/914407d51b878bf7bf34dbd8dd72cc2dbc428673/numpy/ma/core.py#L1034-L1041 > > > > > https://github.com/numpy/numpy/blob/914407d51b878bf7bf34dbd8dd72cc2dbc428673/numpy/core/function_base.py#L461-L472 > > > > Thanks, > > Michael Dubravski > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ NumPy-Discussion mailing > list NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Tue Apr 6 18:16:49 2021 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 06 Apr 2021 17:16:49 -0500 Subject: [Numpy-discussion] umPy Development Meeting Wednesday - Triage Focus Message-ID: <0b95433b618a7507cfb0030d16f2fbc904622f96.camel@sipsolutions.net> Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, March 10th at 11 am Pacific Time (18:00 UTC). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From sebastian at sipsolutions.net Thu Apr 8 13:40:04 2021 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Thu, 08 Apr 2021 12:40:04 -0500 Subject: [Numpy-discussion] Type resolver related deprecations and changes In-Reply-To: References: Message-ID: On Fri, 2021-04-02 at 15:46 -0500, Sebastian Berg wrote: > Hi all, > > I have to do some changes to the type resolution, and I started these > here: https://github.com/numpy/numpy/pull/18718 > Just to keep everyone updated. We discussed this yesterdays meeting, and settled with going with these changes and going directly to an error (instead of warning). This may mean that some users have to update their code, but in that case the code is likely to do something unexpected already. I expect very few users to be affected, so please let us know if you are. If anyone notices issues with any of these changes I am happy to look whether we can mitigate the issue or avoid/delay a specific change! In general these changes are necessary for refactors with regards to new DTypes NEP 41, 42, and specifically the draft NEP 43. (In large parts this is because NumPy is currently inconsistent.) Cheers, Sebastian > There are four changes: > > * Deprecate `signature="l"` and `signature=("l",)`, these are > confusing > ? since the signature should include all inputs and outputs.? To only > ? provide the output use `dtype="l"`. > > * Using `dtype=` for comparisons (e.g. `np.equal`) used to be weird: > > ????? np.equal(1, 2, dtype=object)? -> returns boolean > ????? np.equal(None, 2, dtype=object)? -> returns object array > > ? The first one will now give a FutureWarning. Comparisons that > provide > ? a dtype other than object or bool give a DeprecationWarning (or > ? fail). > ? I hope the warning can be preserved when more refactoring happens. > > * NumPy *almost* always ignores any metadata, byte-order, time unit > ? information from the `dtype` or `signature` arguments to ufuncs. > ? Practically, the dtypes passed actually denote the DType types > ? rather than the specific instance (which could be byte swapped). > > ? NumPy will now do this always and give a warning if byte-order or > ? time unit is ignored! > > * It is THEORETICALLY possible to call `ufunc->type_resolver` in the > C > ? API (as opposed to providing it, which is somewhat OK). > ? If someone does that they have to normalize the type tuple now, I > ? don't really see a reason for keeping support, when NumPy will stop > ? calling it itself almost always and anyone using it would probably > be > ? in trouble soon. > ? To be clear: I have NOT found a single instance of such code in a > ? code search. Even *providing* it ? which is much more reasonable ? > ? is probably only done by astropy/pyerfa. > > > > ** Long example for the "time unit" dropping change ** > > For the third point, which is in theory the largest impact. Both > pandas > and astropy do not notice it (I also grepped scipy, its clean). > These are the biggest changes: > > ??? # The following will now warn on most systems (unchanged result): > ??? np.add(3, 5, dtype=">i32") > > ??? # The biggest impact is for timedelta or datetimes: > ??? arr = np.arange(10, dtype="m8[s]") > ??? # The examples always ignored the time unit "ns" (using the > ??? # unit of `arr`.? They now issue a warning: > ??? np.add(arr, arr, dtype="m8[ns]") > ??? np.maximum.reduce(arr, dtype="m8[ns]") > > ??? # The following issue a warning but previously did return > ??? # a "ns" result. > ??? np.add(3, 5, dtype="m8[ns]")? # Now return generic time units > ??? np.maximum(arr, arr, dtype="m8[ns]")? # Now returns "s" (from > `arr`) > > > I doubt there is a good way to just keep the old behaviour.? It is > hopelessly inconsistent.? (The result even depends on how you pass > things, as the paths I deprecate align with `dtype=` but not with the > `signature=(None, None, dtype)` equivalent call.) > One thought I had just raising a hard error instead of a UserWarning > right now. > > If you say that why not have `dtype=...` always be honored as the > correct output dtype, I don't disagree.? But it seems to me we > probably > would have wade through a FutureWarning first, so a warning is the > "right direction".? (Or just add an `output_dtypes` keyword > argument.) > > Cheers, > > Sebastian > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From melissawm at gmail.com Sat Apr 10 12:29:55 2021 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Sat, 10 Apr 2021 13:29:55 -0300 Subject: [Numpy-discussion] Documentation Team meeting - Monday April 12 In-Reply-To: References: Message-ID: Hi all! Our next Documentation Team meeting will be on *Monday, April 12* at ***4PM UTC***. All are welcome - you don't need to already be a contributor to join. If you have questions or are curious about what we're doing, we'll be happy to meet you! If you wish to join on Zoom, use this link: https://zoom.us/j/96219574921?pwd=VTRNeGwwOUlrYVNYSENpVVBRRjlkZz09#success Here's the permanent hackmd document with the meeting notes (still being updated in the next few days!): https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg Hope to see you around! ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentation+Team+Meeting&iso=20210412T16&p1=1440&ah=1 *** You can add the NumPy community calendar to your google calendar by clicking this link: https://calendar.google.com/calendar /r?cid=YmVya2VsZXkuZWR1X2lla2dwaWdtMjMyamJobGRzZmIyYzJqODFjQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20 - Melissa -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sun Apr 11 16:23:33 2021 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 11 Apr 2021 22:23:33 +0200 Subject: [Numpy-discussion] files for 3D printing of NumPy logo Message-ID: Hi all, Andrej Krecker, who is a 3D printing engineer who likes NumPy, shared 3D printing files for the NumPy logo. It comes in two flavors, a really 3-D version, and a flatter (~0.5 cm thick) 2-D version. Check them out at https://www.thingiverse.com/thing:4821992. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Sun Apr 11 18:50:34 2021 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 11 Apr 2021 16:50:34 -0600 Subject: [Numpy-discussion] files for 3D printing of NumPy logo In-Reply-To: References: Message-ID: On Sun, Apr 11, 2021 at 2:24 PM Ralf Gommers wrote: > Hi all, > > Andrej Krecker, who is a 3D printing engineer who likes NumPy, shared 3D > printing files for the NumPy logo. It comes in two flavors, a really 3-D > version, and a flatter (~0.5 cm thick) 2-D version. Check them out at > https://www.thingiverse.com/thing:4821992. > > Cheers, > Ralf > Nice. Next up, trivets. Chuck > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jgostick at gmail.com Mon Apr 12 16:13:49 2021 From: jgostick at gmail.com (Jeff Gostick) Date: Mon, 12 Apr 2021 16:13:49 -0400 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? Message-ID: I often find myself padding an array to do some processing on it (i.e. to avoid edge artifacts), then I need to remove the padding. I wish there was either a built in "unpad" function that accepted the same arguments as "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a padding of [20, 4]). This seems like a pretty obvious feature to me so maybe I've just missed something, but I have looked through all the open and closed issues on github and don't see anything related to this. Jeff G -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben.v.root at gmail.com Mon Apr 12 16:27:55 2021 From: ben.v.root at gmail.com (Benjamin Root) Date: Mon, 12 Apr 2021 16:27:55 -0400 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: Isn't that just slicing? Or perhaps you are looking for a way to simplify the calculation of the slice arguments from the original pad arguments? On Mon, Apr 12, 2021 at 4:15 PM Jeff Gostick wrote: > I often find myself padding an array to do some processing on it (i.e. to > avoid edge artifacts), then I need to remove the padding. I wish there > was either a built in "unpad" function that accepted the same arguments as > "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a > padding of [20, 4]). This seems like a pretty obvious feature to me so > maybe I've just missed something, but I have looked through all the open > and closed issues on github and don't see anything related to this. > > > Jeff G > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Mon Apr 12 16:28:38 2021 From: shoyer at gmail.com (Stephan Hoyer) Date: Mon, 12 Apr 2021 13:28:38 -0700 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: The easy way to unpad an array is by indexing with slices, e.g., x[20:-4] to undo a padding of [(20, 4)]. Just be careful about unpadding "zero" elements on the right hand side, because Python interprets an ending slice of zero differently -- you need to write something like x[20:] to undo padding by [(20, 0)]. On Mon, Apr 12, 2021 at 1:15 PM Jeff Gostick wrote: > I often find myself padding an array to do some processing on it (i.e. to > avoid edge artifacts), then I need to remove the padding. I wish there > was either a built in "unpad" function that accepted the same arguments as > "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a > padding of [20, 4]). This seems like a pretty obvious feature to me so > maybe I've just missed something, but I have looked through all the open > and closed issues on github and don't see anything related to this. > > > Jeff G > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jgostick at gmail.com Mon Apr 12 16:38:36 2021 From: jgostick at gmail.com (Jeff Gostick) Date: Mon, 12 Apr 2021 16:38:36 -0400 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: It's definitely just "slicing", but it's a bit inconvenient. I'm thinking more like: arr = np.random.rand(10, 10, 10) W = [[3, 2], [4, 6]] # or W = 4, or W = [4, 5] arr_padded = np.pad(arr, pad_width=W) < Do some stuff to arr_padded > arr = np.unpad(arr_padded, pad_width=W) # Using W just works, no matter how odd the various pad widths were On Mon, Apr 12, 2021 at 4:29 PM Stephan Hoyer wrote: > The easy way to unpad an array is by indexing with slices, e.g., x[20:-4] > to undo a padding of [(20, 4)]. Just be careful about unpadding "zero" > elements on the right hand side, because Python interprets an ending slice > of zero differently -- you need to write something like x[20:] to undo > padding by [(20, 0)]. > > > On Mon, Apr 12, 2021 at 1:15 PM Jeff Gostick wrote: > >> I often find myself padding an array to do some processing on it (i.e. to >> avoid edge artifacts), then I need to remove the padding. I wish there >> was either a built in "unpad" function that accepted the same arguments as >> "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a >> padding of [20, 4]). This seems like a pretty obvious feature to me so >> maybe I've just missed something, but I have looked through all the open >> and closed issues on github and don't see anything related to this. >> >> >> Jeff G >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Mon Apr 12 19:41:24 2021 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 12 Apr 2021 17:41:24 -0600 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: On Mon, Apr 12, 2021 at 2:29 PM Stephan Hoyer wrote: > > The easy way to unpad an array is by indexing with slices, e.g., x[20:-4] to undo a padding of [(20, 4)]. Just be careful about unpadding "zero" elements on the right hand side, because Python interprets an ending slice of zero differently -- you need to write something like x[20:] to undo padding by [(20, 0)]. You can use x[20:x.shape[0] - 4] to avoid this inconsistency. Or construct the slice based on the original unpadded shape (x[20:20+orig_x.shape[0]]). Aaron Meurer > > > On Mon, Apr 12, 2021 at 1:15 PM Jeff Gostick wrote: >> >> I often find myself padding an array to do some processing on it (i.e. to avoid edge artifacts), then I need to remove the padding. I wish there was either a built in "unpad" function that accepted the same arguments as "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a padding of [20, 4]). This seems like a pretty obvious feature to me so maybe I've just missed something, but I have looked through all the open and closed issues on github and don't see anything related to this. >> >> >> Jeff G >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From jgostick at gmail.com Mon Apr 12 20:12:07 2021 From: jgostick at gmail.com (Jeff Gostick) Date: Mon, 12 Apr 2021 20:12:07 -0400 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: I guess I should have clarified that I was inquiring about proposing a 'feature request'. The github site suggested I open a discussion on this list first. There are several ways to effectively unpad an array as has been pointed out, but they all require more than a little bit of thought and care, are dependent on array shape, and honestly error prone. It would be very valuable to me to have such a 'predefined' function, so I was wondering if (a) I was unaware of some function that already does this and (b) if I'm alone in thinking this would be useful. On Mon, Apr 12, 2021 at 7:42 PM Aaron Meurer wrote: > On Mon, Apr 12, 2021 at 2:29 PM Stephan Hoyer wrote: > > > > The easy way to unpad an array is by indexing with slices, e.g., > x[20:-4] to undo a padding of [(20, 4)]. Just be careful about unpadding > "zero" elements on the right hand side, because Python interprets an ending > slice of zero differently -- you need to write something like x[20:] to > undo padding by [(20, 0)]. > > You can use x[20:x.shape[0] - 4] to avoid this inconsistency. Or > construct the slice based on the original unpadded shape > (x[20:20+orig_x.shape[0]]). > > Aaron Meurer > > > > > > > On Mon, Apr 12, 2021 at 1:15 PM Jeff Gostick wrote: > >> > >> I often find myself padding an array to do some processing on it (i.e. > to avoid edge artifacts), then I need to remove the padding. I wish there > was either a built in "unpad" function that accepted the same arguments as > "pad", or that "pad" accepted negative numbers (e.g [-20, -4] would undo a > padding of [20, 4]). This seems like a pretty obvious feature to me so > maybe I've just missed something, but I have looked through all the open > and closed issues on github and don't see anything related to this. > >> > >> > >> Jeff G > >> > >> _______________________________________________ > >> NumPy-Discussion mailing list > >> NumPy-Discussion at python.org > >> https://mail.python.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Mon Apr 12 21:15:23 2021 From: shoyer at gmail.com (Stephan Hoyer) Date: Mon, 12 Apr 2021 18:15:23 -0700 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick wrote: > I guess I should have clarified that I was inquiring about proposing a > 'feature request'. The github site suggested I open a discussion on this > list first. There are several ways to effectively unpad an array as has > been pointed out, but they all require more than a little bit of thought > and care, are dependent on array shape, and honestly error prone. It would > be very valuable to me to have such a 'predefined' function, so I was > wondering if (a) I was unaware of some function that already does this and > (b) if I'm alone in thinking this would be useful. > Indeed, this is a fair question. Given that this is not entirely trivial to write correctly, I think it would be reasonable to add the inverse operation for pad() into NumPy. This is generally better than encouraging users to write their own thing. >From a naming perspective, here are some possibilities: unpad trim crop I think "trim" would be pretty descriptive, probably slightly better than "unpad." -------------- next part -------------- An HTML attachment was scrubbed... URL: From jgostick at gmail.com Mon Apr 12 21:36:31 2021 From: jgostick at gmail.com (Jeff Gostick) Date: Mon, 12 Apr 2021 21:36:31 -0400 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: It is great to hear that this might be useful. I would LOVE to create a PR on this idea and contribute back to numpy...but let's not get ahead of ourselves :-) Regarding the name, I kinda like "unpad" since it relates directly to "pad", analogous to "ravel" and "unravel" for instance. Or maybe "depad". Although, it's possible to use this on any array, not just a previously padded one, so maybe tying it too directly to "pad" is not right, in which case "trim" and "crop" are both perfect. I must admit that I find it odd that these functions are not in numpy already. I just searched the docs and they show up as keyword args for a few functions but are otherwise conspicuously absent. Also, funnily, there is a link to "padding arrays" but it is basically empty: https://numpy.org/doc/stable/reference/routines.padding.html. Alternatively, I don't hate the idea of passing negative pad widths into "pad". I actually tried this at one point to see if there was a hidden functionality there, to no avail. BTW, we just adding a custom "unpad" function to our PoreSpy package for this purpose: https://github.com/PMEAL/porespy/blob/dev/porespy/tools/_unpadfunc.py On Mon, Apr 12, 2021 at 9:15 PM Stephan Hoyer wrote: > On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick wrote: > >> I guess I should have clarified that I was inquiring about proposing a >> 'feature request'. The github site suggested I open a discussion on this >> list first. There are several ways to effectively unpad an array as has >> been pointed out, but they all require more than a little bit of thought >> and care, are dependent on array shape, and honestly error prone. It would >> be very valuable to me to have such a 'predefined' function, so I was >> wondering if (a) I was unaware of some function that already does this and >> (b) if I'm alone in thinking this would be useful. >> > > Indeed, this is a fair question. > > Given that this is not entirely trivial to write correctly, I think it > would be reasonable to add the inverse operation for pad() into NumPy. This > is generally better than encouraging users to write their own thing. > > From a naming perspective, here are some possibilities: > unpad > trim > crop > > I think "trim" would be pretty descriptive, probably slightly better than > "unpad." > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Tue Apr 13 04:26:12 2021 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Tue, 13 Apr 2021 10:26:12 +0200 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: On Tue, Apr 13, 2021 at 3:37 AM Jeff Gostick wrote: > It is great to hear that this might be useful. I would LOVE to create a > PR on this idea and contribute back to numpy...but let's not get ahead of > ourselves :-) > > Regarding the name, I kinda like "unpad" since it relates directly to > "pad", analogous to "ravel" and "unravel" for instance. Or maybe "depad". > Although, it's possible to use this on any array, not just a previously > padded one, so maybe tying it too directly to "pad" is not right, in which > case "trim" and "crop" are both perfect. I must admit that I find it odd > that these functions are not in numpy already. I just searched the docs > and they show up as keyword args for a few functions but are otherwise > conspicuously absent. Also, funnily, there is a link to "padding arrays" > but it is basically empty: > https://numpy.org/doc/stable/reference/routines.padding.html. > > Alternatively, I don't hate the idea of passing negative pad widths into > "pad". I actually tried this at one point to see if there was a hidden > functionality there, to no avail. > > BTW, we just adding a custom "unpad" function to our PoreSpy package for > this purpose: > https://github.com/PMEAL/porespy/blob/dev/porespy/tools/_unpadfunc.py > > > > On Mon, Apr 12, 2021 at 9:15 PM Stephan Hoyer wrote: > >> On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick wrote: >> >>> I guess I should have clarified that I was inquiring about proposing a >>> 'feature request'. The github site suggested I open a discussion on this >>> list first. There are several ways to effectively unpad an array as has >>> been pointed out, but they all require more than a little bit of thought >>> and care, are dependent on array shape, and honestly error prone. It would >>> be very valuable to me to have such a 'predefined' function, so I was >>> wondering if (a) I was unaware of some function that already does this and >>> (b) if I'm alone in thinking this would be useful. >>> >> >> Indeed, this is a fair question. >> >> Given that this is not entirely trivial to write correctly, I think it >> would be reasonable to add the inverse operation for pad() into NumPy. This >> is generally better than encouraging users to write their own thing. >> >> From a naming perspective, here are some possibilities: >> unpad >> trim >> crop >> >> I think "trim" would be pretty descriptive, probably slightly better than >> "unpad." >> > I'm not a fan of `trim`. We already have `clip` which sounds similar. `unpad` looks like the only one that's completely unambiguous. `crop` sounds like an image processing function, and what we don't want is something like Pillow's `crop(left, top, right, bottom)`. Cheers, Ralf _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieser.eric+numpy at gmail.com Tue Apr 13 04:35:53 2021 From: wieser.eric+numpy at gmail.com (Eric Wieser) Date: Tue, 13 Apr 2021 09:35:53 +0100 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: Some other options here that avoid the need for a new function: * Add a `return_view` argument to `pad`, such that for `padded, orig = np.pad(arr, ..., return_view=True)`, `orig == arr` and `orig.base is padded`. This is useful if `padded` is modified in place, but less useful otherwise. It has the advantage of not having to recompute the slices, as pad already has them. * Accept a `slice` object directly in `np.pad`; for `sl = np.s_[2:-20, 4:-40]`, `padded = np.pad(array, sl)`, we have `padded[sl] == array`. The second idea seems promising to me, but perhaps there are corner cases I haven't thought of that it wouldn't help with. Eric On Tue, 13 Apr 2021 at 09:26, Ralf Gommers wrote: > > > On Tue, Apr 13, 2021 at 3:37 AM Jeff Gostick wrote: > >> It is great to hear that this might be useful. I would LOVE to create a >> PR on this idea and contribute back to numpy...but let's not get ahead of >> ourselves :-) >> >> Regarding the name, I kinda like "unpad" since it relates directly to >> "pad", analogous to "ravel" and "unravel" for instance. Or maybe "depad". >> Although, it's possible to use this on any array, not just a previously >> padded one, so maybe tying it too directly to "pad" is not right, in which >> case "trim" and "crop" are both perfect. I must admit that I find it odd >> that these functions are not in numpy already. I just searched the docs >> and they show up as keyword args for a few functions but are otherwise >> conspicuously absent. Also, funnily, there is a link to "padding arrays" >> but it is basically empty: >> https://numpy.org/doc/stable/reference/routines.padding.html. >> >> Alternatively, I don't hate the idea of passing negative pad widths into >> "pad". I actually tried this at one point to see if there was a hidden >> functionality there, to no avail. >> >> BTW, we just adding a custom "unpad" function to our PoreSpy package for >> this purpose: >> https://github.com/PMEAL/porespy/blob/dev/porespy/tools/_unpadfunc.py >> >> >> >> On Mon, Apr 12, 2021 at 9:15 PM Stephan Hoyer wrote: >> >>> On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick wrote: >>> >>>> I guess I should have clarified that I was inquiring about proposing a >>>> 'feature request'. The github site suggested I open a discussion on this >>>> list first. There are several ways to effectively unpad an array as has >>>> been pointed out, but they all require more than a little bit of thought >>>> and care, are dependent on array shape, and honestly error prone. It would >>>> be very valuable to me to have such a 'predefined' function, so I was >>>> wondering if (a) I was unaware of some function that already does this and >>>> (b) if I'm alone in thinking this would be useful. >>>> >>> >>> Indeed, this is a fair question. >>> >>> Given that this is not entirely trivial to write correctly, I think it >>> would be reasonable to add the inverse operation for pad() into NumPy. This >>> is generally better than encouraging users to write their own thing. >>> >>> From a naming perspective, here are some possibilities: >>> unpad >>> trim >>> crop >>> >>> I think "trim" would be pretty descriptive, probably slightly better >>> than "unpad." >>> >> > I'm not a fan of `trim`. We already have `clip` which sounds similar. > > `unpad` looks like the only one that's completely unambiguous. > > `crop` sounds like an image processing function, and what we don't want is > something like Pillow's `crop(left, top, right, bottom)`. > > Cheers, > Ralf > > > _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion at python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Tue Apr 13 04:39:26 2021 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Tue, 13 Apr 2021 11:39:26 +0300 Subject: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be? In-Reply-To: References: Message-ID: Prior art data point: scikit-image.util.crop : https://scikit-image.org/docs/dev/api/skimage.util.html#skimage.util.crop ??, 13 ???. 2021 ?., 11:37 Eric Wieser : > Some other options here that avoid the need for a new function: > > * Add a `return_view` argument to `pad`, such that for `padded, orig = > np.pad(arr, ..., return_view=True)`, `orig == arr` and `orig.base is > padded`. This is useful if `padded` is modified in place, but less useful > otherwise. It has the advantage of not having to recompute the slices, as > pad already has them. > * Accept a `slice` object directly in `np.pad`; for `sl = np.s_[2:-20, > 4:-40]`, `padded = np.pad(array, sl)`, we have `padded[sl] == array`. > > The second idea seems promising to me, but perhaps there are corner cases > I haven't thought of that it wouldn't help with. > > Eric > > On Tue, 13 Apr 2021 at 09:26, Ralf Gommers wrote: > >> >> >> On Tue, Apr 13, 2021 at 3:37 AM Jeff Gostick wrote: >> >>> It is great to hear that this might be useful. I would LOVE to create a >>> PR on this idea and contribute back to numpy...but let's not get ahead of >>> ourselves :-) >>> >>> Regarding the name, I kinda like "unpad" since it relates directly to >>> "pad", analogous to "ravel" and "unravel" for instance. Or maybe "depad". >>> Although, it's possible to use this on any array, not just a previously >>> padded one, so maybe tying it too directly to "pad" is not right, in which >>> case "trim" and "crop" are both perfect. I must admit that I find it odd >>> that these functions are not in numpy already. I just searched the docs >>> and they show up as keyword args for a few functions but are otherwise >>> conspicuously absent. Also, funnily, there is a link to "padding arrays" >>> but it is basically empty: >>> https://numpy.org/doc/stable/reference/routines.padding.html. >>> >>> Alternatively, I don't hate the idea of passing negative pad widths into >>> "pad". I actually tried this at one point to see if there was a hidden >>> functionality there, to no avail. >>> >>> BTW, we just adding a custom "unpad" function to our PoreSpy package for >>> this purpose: >>> https://github.com/PMEAL/porespy/blob/dev/porespy/tools/_unpadfunc.py >>> >>> >>> >>> On Mon, Apr 12, 2021 at 9:15 PM Stephan Hoyer wrote: >>> >>>> On Mon, Apr 12, 2021 at 5:12 PM Jeff Gostick >>>> wrote: >>>> >>>>> I guess I should have clarified that I was inquiring about proposing a >>>>> 'feature request'. The github site suggested I open a discussion on this >>>>> list first. There are several ways to effectively unpad an array as has >>>>> been pointed out, but they all require more than a little bit of thought >>>>> and care, are dependent on array shape, and honestly error prone. It would >>>>> be very valuable to me to have such a 'predefined' function, so I was >>>>> wondering if (a) I was unaware of some function that already does this and >>>>> (b) if I'm alone in thinking this would be useful. >>>>> >>>> >>>> Indeed, this is a fair question. >>>> >>>> Given that this is not entirely trivial to write correctly, I think it >>>> would be reasonable to add the inverse operation for pad() into NumPy. This >>>> is generally better than encouraging users to write their own thing. >>>> >>>> From a naming perspective, here are some possibilities: >>>> unpad >>>> trim >>>> crop >>>> >>>> I think "trim" would be pretty descriptive, probably slightly better >>>> than "unpad." >>>> >>> >> I'm not a fan of `trim`. We already have `clip` which sounds similar. >> >> `unpad` looks like the only one that's completely unambiguous. >> >> `crop` sounds like an image processing function, and what we don't want >> is something like Pillow's `crop(left, top, right, bottom)`. >> >> Cheers, >> Ralf >> >> >> _______________________________________________ >>>> NumPy-Discussion mailing list >>>> NumPy-Discussion at python.org >>>> https://mail.python.org/mailman/listinfo/numpy-discussion >>>> >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion at python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Tue Apr 13 21:28:43 2021 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 13 Apr 2021 20:28:43 -0500 Subject: [Numpy-discussion] NumPy Community Meeting Wednesday Message-ID: <1db87aeb75e80de64c858d387a37810a2563668f.camel@sipsolutions.net> Hi all, There will be a NumPy Community meeting Wednesday April 14th at 20:00 UTC. Everyone is invited and encouraged to join in and edit the work-in-progress meeting topics and notes at: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From blkzol001 at myuct.ac.za Wed Apr 14 11:49:51 2021 From: blkzol001 at myuct.ac.za (zoj613) Date: Wed, 14 Apr 2021 08:49:51 -0700 (MST) Subject: [Numpy-discussion] Improving performance of the `numpy.any` function. Message-ID: <1618415391719-0.post@n7.nabble.com> Hi All, I was using numpy's `any` function earlier and realized that it might not be as performant as I assumed. See the code below: ``` In [1]: import numpy as np In [2]: a = np.zeros(1_000_000) In [3]: a[100] = 1 In [4]: b = np.zeros(2_000_000) In [5]: b[100] = 1 In [6]: %timeit np.any(a) 1.33 ms ? 15.1 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops each) In [7]: %timeit np.any(b) 2.65 ms ? 71.1 ?s per loop (mean ? std. dev. of 7 runs, 100 loops each) In [8] %timeit any(a) 13.4 ?s ? 354 ns per loop (mean ? std. dev. of 7 runs, 100000 loops each) In [9]: %timeit any(b) 13.3 ?s ? 219 ns per loop (mean ? std. dev. of 7 runs, 100000 loops each) ``` It looks like `np.any` does not terminate early as soon as it finds a non-zero element, compared to the built-in any function. So this prompted me to go look at the source code for `PyArray_Any`: https://github.com/numpy/numpy/blob/623bc1fae1d47df24e7f1e29321d0c0ba2771ce0/numpy/core/src/multiarray/calculation.c#L790 . Taking a peek at the `PyArray_GenericReduceFunction` it looks like it just calls the `reduce` function from python on every single element of the array and there is no mechanism in place to short-circuit the call if an element has been found early. It seems worthwhile to have a specific function that allows early stopping of the reduction? So my question is: Is there a reason why maintainers did not implement such a feature? I would also like to know if everyone thinks its worthwhile to implement one? Im guessing there are many unsuspecting users like myself who call the `np.any` function on arrays assuming early-stopping is supported. I think existing code that calls the function would benefit a lot. Let me know what you think. Regards -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From j.wuttke at fz-juelich.de Wed Apr 14 16:36:25 2021 From: j.wuttke at fz-juelich.de (Joachim Wuttke) Date: Wed, 14 Apr 2021 22:36:25 +0200 Subject: [Numpy-discussion] savetxt -> gzip: nondeterministic because of time stamp Message-ID: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> If argument fname of savetxt(fname, X, ...) ends with ".gz" then array X is not only converted to text, but also compressed using gzip. The format gzip [1] has a timestamp. The Python module gzip.py [2] sets the timestamp according to an optional constructor argument "mtime". By default, the current time is used. This makes the file written by savetxt(*.gz, ...) non-deterministic. This is unexpected and confusing in a numerics context. I let different versions of a program generate *.gz files, and ran the "diff" util over pairs of output files to check whether any bit had changed. To my surprise, confusion, and desperation, output always had changed, and kept changing when I ran unchanged versions of my program over and again. So I learned the hard way that the *.gz files contain a timestamp. Regarding the module gzip.py, I submitted a pull request to improve description of the optional argument mtime, and hint at the possible choice mtime = 0 that makes outputs deterministic [3]. Regarding numpy, I'd propose a bolder measure: To let savetxt(fname, X, ...) store exactly the same information in compressed and uncompressed files, always invoke gzip with mtime = 0. I would like to follow up with a pull request, but I am unable to find out how numpy.savetxt is invoking gzip. Joachim [1] https://www.ietf.org/rfc/rfc1952.txt [2] https://docs.python.org/3/library/gzip.html [3] https://github.com/python/cpython/pull/25410 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5338 bytes Desc: S/MIME Cryptographic Signature URL: From deak.andris at gmail.com Wed Apr 14 16:57:33 2021 From: deak.andris at gmail.com (Andras Deak) Date: Wed, 14 Apr 2021 22:57:33 +0200 Subject: [Numpy-discussion] savetxt -> gzip: nondeterministic because of time stamp In-Reply-To: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> References: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> Message-ID: On Wed, Apr 14, 2021 at 10:36 PM Joachim Wuttke wrote: > > If argument fname of savetxt(fname, X, ...) ends with ".gz" then > array X is not only converted to text, but also compressed using gzip. > > The format gzip [1] has a timestamp. The Python module gzip.py [2] > sets the timestamp according to an optional constructor argument > "mtime". By default, the current time is used. > > This makes the file written by savetxt(*.gz, ...) non-deterministic. > This is unexpected and confusing in a numerics context. Related: same for np.savez https://github.com/numpy/numpy/issues/9439 Andr?s > I let different versions of a program generate *.gz files, and ran > the "diff" util over pairs of output files to check whether any bit > had changed. To my surprise, confusion, and desperation, output > always had changed, and kept changing when I ran unchanged versions > of my program over and again. So I learned the hard way that the > *.gz files contain a timestamp. > > Regarding the module gzip.py, I submitted a pull request to improve > description of the optional argument mtime, and hint at the possible > choice mtime = 0 that makes outputs deterministic [3]. > > Regarding numpy, I'd propose a bolder measure: > To let savetxt(fname, X, ...) store exactly the same information in > compressed and uncompressed files, always invoke gzip with mtime = 0. > > I would like to follow up with a pull request, but I am unable to > find out how numpy.savetxt is invoking gzip. > > Joachim > > [1] https://www.ietf.org/rfc/rfc1952.txt > [2] https://docs.python.org/3/library/gzip.html > [3] https://github.com/python/cpython/pull/25410 > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From robert.kern at gmail.com Wed Apr 14 17:15:02 2021 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 14 Apr 2021 17:15:02 -0400 Subject: [Numpy-discussion] savetxt -> gzip: nondeterministic because of time stamp In-Reply-To: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> References: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> Message-ID: On Wed, Apr 14, 2021 at 4:37 PM Joachim Wuttke wrote: > Regarding numpy, I'd propose a bolder measure: > To let savetxt(fname, X, ...) store exactly the same information in > compressed and uncompressed files, always invoke gzip with mtime = 0. > I agree. > I would like to follow up with a pull request, but I am unable to > find out how numpy.savetxt is invoking gzip. > `savetxt` uses the abstractions in this module to invoke gzip when the filename calls for it: https://github.com/numpy/numpy/blob/main/numpy/lib/_datasource.py#L115 -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From andyfaff at gmail.com Wed Apr 14 18:15:25 2021 From: andyfaff at gmail.com (Andrew Nelson) Date: Thu, 15 Apr 2021 08:15:25 +1000 Subject: [Numpy-discussion] savetxt -> gzip: nondeterministic because of time stamp In-Reply-To: References: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> Message-ID: On Thu, 15 Apr 2021 at 07:15, Robert Kern wrote: > On Wed, Apr 14, 2021 at 4:37 PM Joachim Wuttke > wrote: > >> Regarding numpy, I'd propose a bolder measure: >> To let savetxt(fname, X, ...) store exactly the same information in >> compressed and uncompressed files, always invoke gzip with mtime = 0. >> > > I agree. > I might look into making a PR for this. To be clear what would the desired functionality be: 1. Mandatory to have mtime = 0? 2. Default mtime = 0, but `np.save*` has an extra `mtime` kwd that allows to set the mtime? 3. Default mtime = time.time(), but `np.save*` has an extra `mtime` kwd that allows to set the mtime = 0? As Joachim says for testing/git-related purposes it is nice to have bit-wise unchanged files produced (such that the file-hash is unchanged), but I can also see that it might be nice to have a modification time when files contained in a zip file were last changed (e.g. write with numpy, check/inspect with a different module). Of course with the latter you could just look at the overall file-write date, they should be the same. -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek at astro.physik.uni-goettingen.de Wed Apr 14 17:56:22 2021 From: derek at astro.physik.uni-goettingen.de (Derek Homeier) Date: Wed, 14 Apr 2021 23:56:22 +0200 Subject: [Numpy-discussion] savetxt -> gzip: nondeterministic because of time stamp In-Reply-To: References: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> Message-ID: <2F6D00BB-526F-4D33-B0D8-C98236AA296B@astro.physik.uni-goettingen.de> On 14 Apr 2021, at 11:15 pm, Robert Kern wrote: > > On Wed, Apr 14, 2021 at 4:37 PM Joachim Wuttke wrote: > Regarding numpy, I'd propose a bolder measure: > To let savetxt(fname, X, ...) store exactly the same information in > compressed and uncompressed files, always invoke gzip with mtime = 0. > > I agree. > I would caution though that relying on the checksum or similar of the compressed data still does not seem a very robust check of the data itself - the compressed file would still definitely change with any change in compression level, and quite possibly with changes in the linked compression library (perhaps even a simple version update). Shouldn?t you better verify the data buffers themselves? Outside of Python, the lzma utility xzcmp for example allows binary comparison of the content of compressed files independently from the timestamp or even compression method, and it handles gzip and bzip2 files as well. > I would like to follow up with a pull request, but I am unable to > find out how numpy.savetxt is invoking gzip. > > `savetxt` uses the abstractions in this module to invoke gzip when the filename calls for it: > > https://github.com/numpy/numpy/blob/main/numpy/lib/_datasource.py#L115 > One obstacle would be that this is setting gzip.open as _file_opener, which does not have the mtime option; getting this replaced with gzip.GzipFile to work in text mode would require to somehow replicate gzip.open?s io.TextIOWrapper wrapping of GzipFile. Cheers, Derek From robert.kern at gmail.com Wed Apr 14 18:29:46 2021 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 14 Apr 2021 18:29:46 -0400 Subject: [Numpy-discussion] savetxt -> gzip: nondeterministic because of time stamp In-Reply-To: <2F6D00BB-526F-4D33-B0D8-C98236AA296B@astro.physik.uni-goettingen.de> References: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> <2F6D00BB-526F-4D33-B0D8-C98236AA296B@astro.physik.uni-goettingen.de> Message-ID: On Wed, Apr 14, 2021 at 6:20 PM Derek Homeier < derek at astro.physik.uni-goettingen.de> wrote: > On 14 Apr 2021, at 11:15 pm, Robert Kern wrote: > > > > On Wed, Apr 14, 2021 at 4:37 PM Joachim Wuttke > wrote: > > Regarding numpy, I'd propose a bolder measure: > > To let savetxt(fname, X, ...) store exactly the same information in > > compressed and uncompressed files, always invoke gzip with mtime = 0. > > > > I agree. > > > I would caution though that relying on the checksum or similar of the > compressed data still > does not seem a very robust check of the data itself - the compressed file > would still definitely > change with any change in compression level, and quite possibly with > changes in the > linked compression library (perhaps even a simple version update). > Sure, but in practice, that variability happens more rarely. In particular, I'm not going to see that variability when I'm re-running the same workflow with the same software versions inside of a system like DVC[1] that uses file hashes to determine which parts of the workflow need to get rerun. Whereas, I will see timestamp changes every time and thus invalidating the cache when it's irrelevant. [1] https://dvc.org/ > Shouldn?t you better verify the data buffers themselves? Outside of > Python, the lzma utility > xzcmp for example allows binary comparison of the content of compressed > files independently > from the timestamp or even compression method, and it handles gzip and > bzip2 files as well. > We're often integrating with systems that only know how to compute hashes of files and doesn't try to dig into the semantics of the files any deeper. We're not looking to construct the most provably-correct cache, just make some pragmatic choices that will reduce false cache misses with systems that currently exist and enable *better*, if still imperfect, uses of those systems. > I would like to follow up with a pull request, but I am unable to > > find out how numpy.savetxt is invoking gzip. > > > > `savetxt` uses the abstractions in this module to invoke gzip when the > filename calls for it: > > > > https://github.com/numpy/numpy/blob/main/numpy/lib/_datasource.py#L115 > > > One obstacle would be that this is setting gzip.open as _file_opener, > which does not have the > mtime option; getting this replaced with gzip.GzipFile to work in text > mode would require to > somehow replicate gzip.open?s io.TextIOWrapper wrapping of GzipFile. > That should be easily doable. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Wed Apr 14 18:39:25 2021 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 14 Apr 2021 18:39:25 -0400 Subject: [Numpy-discussion] savetxt -> gzip: nondeterministic because of time stamp In-Reply-To: References: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> Message-ID: On Wed, Apr 14, 2021 at 6:16 PM Andrew Nelson wrote: > On Thu, 15 Apr 2021 at 07:15, Robert Kern wrote: > >> On Wed, Apr 14, 2021 at 4:37 PM Joachim Wuttke >> wrote: >> >>> Regarding numpy, I'd propose a bolder measure: >>> To let savetxt(fname, X, ...) store exactly the same information in >>> compressed and uncompressed files, always invoke gzip with mtime = 0. >>> >> >> I agree. >> > > I might look into making a PR for this. To be clear what would the desired > functionality be: > > 1. Mandatory to have mtime = 0? > > 2. Default mtime = 0, but `np.save*` has an extra `mtime` kwd that allows > to set the mtime? > > 3. Default mtime = time.time(), but `np.save*` has an extra `mtime` kwd > that allows to set the mtime = 0? > > > As Joachim says for testing/git-related purposes it is nice to have > bit-wise unchanged files produced (such that the file-hash is unchanged), > but I can also see that it might be nice to have a modification time when > files contained in a zip file were last changed (e.g. write with numpy, > check/inspect with a different module). Of course with the latter you could > just look at the overall file-write date, they should be the same. > I suspect no one's actually looking at the timestamp inside the file (relevant XKCD comic[1] notwithstanding). I'd lean towards boldness here and just set mtime=0. If someone screams, then we can add in the option for them to get the old functionality. But moving closer to reliably reproducible file formats is a goal worth changing the default behavior. [1] https://xkcd.com/1172/ -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek at astro.physik.uni-goettingen.de Wed Apr 14 18:52:35 2021 From: derek at astro.physik.uni-goettingen.de (Derek Homeier) Date: Thu, 15 Apr 2021 00:52:35 +0200 Subject: [Numpy-discussion] savetxt -> gzip: nondeterministic because of time stamp In-Reply-To: References: <3e511d57-865a-5960-34ad-99cc28ffbee1@fz-juelich.de> Message-ID: <274690ED-CC78-4F53-B90E-CF6ADB3C349B@astro.physik.uni-goettingen.de> On 15 Apr 2021, at 12:39 am, Robert Kern wrote: > > On Wed, Apr 14, 2021 at 6:16 PM Andrew Nelson wrote: > On Thu, 15 Apr 2021 at 07:15, Robert Kern wrote: > On Wed, Apr 14, 2021 at 4:37 PM Joachim Wuttke wrote: > Regarding numpy, I'd propose a bolder measure: > To let savetxt(fname, X, ...) store exactly the same information in > compressed and uncompressed files, always invoke gzip with mtime = 0. > > I agree. > > I might look into making a PR for this. To be clear what would the desired functionality be: > > 1. Mandatory to have mtime = 0? > > 2. Default mtime = 0, but `np.save*` has an extra `mtime` kwd that allows to set the mtime? > > 3. Default mtime = time.time(), but `np.save*` has an extra `mtime` kwd that allows to set the mtime = 0? > > > As Joachim says for testing/git-related purposes it is nice to have bit-wise unchanged files produced (such that the file-hash is unchanged), but I can also see that it might be nice to have a modification time when files contained in a zip file were last changed (e.g. write with numpy, check/inspect with a different module). Of course with the latter you could just look at the overall file-write date, they should be the same. > > I suspect no one's actually looking at the timestamp inside the file (relevant XKCD comic[1] notwithstanding). I'd lean towards boldness here and just set mtime=0. If someone screams, then we can add in the option for them to get the old functionality. But moving closer to reliably reproducible file formats is a goal worth changing the default behavior. > I was also a bit worried about that ? a) people actively using the timestamp for information and b) files with mtime=0 decompressing to files showing a creation date in 1970 or something. But all the Unix timestamps at least seem to be determined differently, so maybe the benefits do outweigh the risks. > [1] https://xkcd.com/1172/ :D Derek From dan_patterson at outlook.com Wed Apr 14 21:53:58 2021 From: dan_patterson at outlook.com (dan_patterson) Date: Wed, 14 Apr 2021 18:53:58 -0700 (MST) Subject: [Numpy-discussion] Improving performance of the `numpy.any` function. In-Reply-To: <1618415391719-0.post@n7.nabble.com> References: <1618415391719-0.post@n7.nabble.com> Message-ID: <1618451638138-0.post@n7.nabble.com> a = np.zeros(1_000_000) a[100] = 1 %timeit np.any(a) 814 ?s ? 17.8 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops each) %timeit np.any(a == 1) 488 ?s ? 5.68 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops each) Haven't investigated further since your times are much longer than mine and secondly the equality check for 1 implies that perhaps a short circuit actually exists somewhere -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From jbrockmendel at gmail.com Wed Apr 14 22:34:04 2021 From: jbrockmendel at gmail.com (Brock Mendel) Date: Wed, 14 Apr 2021 19:34:04 -0700 Subject: [Numpy-discussion] Improving performance of the `numpy.any` function. In-Reply-To: <1618451638138-0.post@n7.nabble.com> References: <1618415391719-0.post@n7.nabble.com> <1618451638138-0.post@n7.nabble.com> Message-ID: FWIW in https://github.com/pandas-dev/pandas/issues/32339 I tried short-circuiting (left == right).all() with a naive cython implementation. In the cases that _dont_ short-circuit, it was 2x slower than np.array_equal. On Wed, Apr 14, 2021 at 6:54 PM dan_patterson wrote: > a = np.zeros(1_000_000) > > a[100] = 1 > > %timeit np.any(a) > 814 ?s ? 17.8 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops each) > > %timeit np.any(a == 1) > 488 ?s ? 5.68 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops each) > > Haven't investigated further since your times are much longer than mine and > secondly the equality check for 1 implies that perhaps a short circuit > actually exists somewhere > > > > -- > Sent from: http://numpy-discussion.10968.n7.nabble.com/ > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From czorio4 at gmail.com Thu Apr 15 05:54:16 2021 From: czorio4 at gmail.com (czorio4) Date: Thu, 15 Apr 2021 11:54:16 +0200 Subject: [Numpy-discussion] Add pad_to_shape and pad_to_match for simpler padding if a target size is already known Message-ID: Hello all, I'm only two weeks late with this message about my pull request that adds functions that allow the user to pad to a target shape, instead of adding a certain amount to each axis. For example: x = np.ones((3, 3)) # We want an output shape of (10, 10) padded = np.pad_to_shape(x, (10, 10)) print(padded.shape) prints: (10, 10) Whereas with the current implementation of np.pad you'd need to first figure out the difference in axis sizes. The proposed function would perform this step for you. As this function passes to np.pad internally, I made sure to include the arguments in the signature that np.pad uses for padding modes. Finally, I've added a logical extension of this function; pad_to_match, this takes another array and pads the input array to match. These calls would be equivalent: x = np.ones((3, 3)) y = np.zeros((10, 10)) padded_to_shape = np.pad_to_shape(x, y.shape) padded_to_match = np.pad_to_match(x, y) For additional functionality description, I refer to the pull request. I'm not too familiar with mailing lists, so I hope this is how things work. Kind Regards, Mathijs -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Thu Apr 15 12:42:18 2021 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Thu, 15 Apr 2021 11:42:18 -0500 Subject: [Numpy-discussion] Improving performance of the `numpy.any` function. In-Reply-To: <1618451638138-0.post@n7.nabble.com> References: <1618415391719-0.post@n7.nabble.com> <1618451638138-0.post@n7.nabble.com> Message-ID: On Wed, 2021-04-14 at 18:53 -0700, dan_patterson wrote: > a = np.zeros(1_000_000) > > a[100] = 1 > > %timeit np.any(a) > 814 ?s ? 17.8 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops > each) > > %timeit np.any(a == 1) > 488 ?s ? 5.68 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops > each) > It may be worth checking which ufunc loop is used (and what casts are necessary due to that), I am not sure immediately. Casting (and the fact that it cannot be "short circuited right now), should be at the core of understanding the speeds properly. We might be able to short-circuit more in the future, or add new loops to avoid casting. In the end it is annoying that `np.any(a)` can be slower than `np.any(a != 0)`, but it is an additional complexity/project to think about optimizing `np.any(a == 1)` if you want short circuiting, you cannot do the `a == 1` greedily after all. Cheers, Sebastian > Haven't investigated further since your times are much longer than > mine and > secondly the equality check for 1 implies that perhaps a short > circuit > actually exists somewhere > > > > -- > Sent from: http://numpy-discussion.10968.n7.nabble.com/ > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From blkzol001 at myuct.ac.za Thu Apr 15 17:31:21 2021 From: blkzol001 at myuct.ac.za (zoj613) Date: Thu, 15 Apr 2021 14:31:21 -0700 (MST) Subject: [Numpy-discussion] Improving performance of the `numpy.any` function. In-Reply-To: <1618451638138-0.post@n7.nabble.com> References: <1618415391719-0.post@n7.nabble.com> <1618451638138-0.post@n7.nabble.com> Message-ID: <1618522281201-0.post@n7.nabble.com> Although still much slower than the builtin `any`, this is an interesting and strange alternative way to improve the timing. My speeds are a result of using a _very_ old machine with a low-grade processor so maybe these times are more exaggerated to me than others. -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From alan.isaac at gmail.com Sat Apr 17 13:55:40 2021 From: alan.isaac at gmail.com (Alan G. Isaac) Date: Sat, 17 Apr 2021 13:55:40 -0400 Subject: [Numpy-discussion] two questions about `choose` Message-ID: <2e3e7faf-668b-c2ba-8c66-63347acc6914@gmail.com> 1. Is there a technical reason for `choose` not accept a `dtype` argument? 2. Separately, mypy is unhappy with my 2nd argument to `choose`: Argument 2 to "choose" has incompatible type "Tuple[int, Sequence[float]]"; expected "Union[Union[int, float, complex, str, bytes, generic], Sequence[Union[int, float, complex, str, bytes, generic]], Sequence[Sequence[Any]],_SupportsArray]" However, `choose` is happy to have e.g. `choices=(0,seq)` (and I hope it will remain so!). E.g., a = a = (0,1,1,0,0,0,1,1) #binary array np.choose((0,range(8)) #array([0, 1, 2, 0, 0, 0, 6, 7]) Thanks, Alan Isaac From kevin.k.sheppard at gmail.com Sat Apr 17 16:27:13 2021 From: kevin.k.sheppard at gmail.com (Kevin Sheppard) Date: Sat, 17 Apr 2021 21:27:13 +0100 Subject: [Numpy-discussion] two questions about `choose` In-Reply-To: <2e3e7faf-668b-c2ba-8c66-63347acc6914@gmail.com> References: <2e3e7faf-668b-c2ba-8c66-63347acc6914@gmail.com> Message-ID: 1. I suppose it only uses the (Native int or int64) dtype since each one would need a code path to run quickly. 2. I would describe this a a bug. I think sequences are converted to arrays and in this case the conversion is not returning a 2 element object array but expanding and then concatenation. Kevin On Sat, Apr 17, 2021, 18:56 Alan G. Isaac wrote: > 1. Is there a technical reason for `choose` not accept a `dtype` argument? > > 2. Separately, mypy is unhappy with my 2nd argument to `choose`: > Argument 2 to "choose" has incompatible type "Tuple[int, Sequence[float]]"; > expected "Union[Union[int, float, complex, str, bytes, generic], > Sequence[Union[int, float, complex, str, bytes, generic]], > Sequence[Sequence[Any]],_SupportsArray]" > However, `choose` is happy to have e.g. `choices=(0,seq)` (and I hope it > will remain so!). > > E.g., > a = a = (0,1,1,0,0,0,1,1) #binary array > np.choose((0,range(8)) #array([0, 1, 2, 0, 0, 0, 6, 7]) > > Thanks, Alan Isaac > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From blkzol001 at myuct.ac.za Sun Apr 18 04:01:09 2021 From: blkzol001 at myuct.ac.za (zoj613) Date: Sun, 18 Apr 2021 01:01:09 -0700 (MST) Subject: [Numpy-discussion] Inaccurate documentation of the random c-api Message-ID: <1618732869894-0.post@n7.nabble.com> Hi All, https://numpy.org/devdocs/reference/random/c-api.html has an inaccurate description of the API that goes as follows: "/zig in the name are based on a ziggurat lookup algorithm is used instead of calculating the log, which is significantly faster. The non-ziggurat variants are used in corner cases and for legacy compatibility./". It appears this is no longer the case and instead there are functions that have `inv` in the name to signal the use of the inverse method for sampling. I submited a PR to reflect this and also added the missing function signatures at https://github.com/numpy/numpy/pull/18797/files Regards -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From mukulikapahari at gmail.com Sun Apr 18 08:30:08 2021 From: mukulikapahari at gmail.com (Mukulika Pahari) Date: Sun, 18 Apr 2021 18:00:08 +0530 Subject: [Numpy-discussion] GSoD'21 Ideas Disussion Message-ID: Hello everyone! I am Mukulika Pahari, a Computer Engineering student from India. I wanted to implement the following ideas for Google Summer of Docs 2021 if given the opportunity. I have referred to both- the GSoD proposal and the NEP 44 proposal. Please give me feedback about the ideas! 1. Reorganising contents of the documentation into Reference Guide, How-Tos, Tutorials and Explanations as per structure proposed in NEP 44 . - Auditing existing documentation - Clearing misplaced content for example Explanations in Reference Guide, How-Tos in Explanations etc. - Establishing distinct Reference, How-Tos, Tutorials and Explanations sections with crosslinking where required 1. Reorganising landing page of NumPy docs . - Moving the documentation structure to the left sidebar - Having NumPy Quickstart as the first thing people see when they land on Documentation 1. Writing new must-have tutorials (based on most-searched tutorials on Google). - ?How to write a tutorial? guide - 3 Beginner Tutorials - 3 Intermediate Tutorials - 3 Advanced Tutorials 1. Writing How-Tos based on most-used functions, most asked doubts on StackOverflow, etc. 2. Revamping the User Guide. - Updating out-of-date references and refactoring content to the latest best practices - Adding non-textual images or graphics to enhance the textual explanations - Removing duplication to improve searchability I have proposed this work keeping in mind 30ish weeks of work including a few weeks for becoming familiar with the organisation and ironing out details like exactly which tutorials and how-tos to write. Please let me know if I can aim to achieve more in the timeframe. My experience with NumPy is currently limited to one data analysis project but I would love to learn more about its applications while restructuring and developing its docs! Thank you for your time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.isaac at gmail.com Sun Apr 18 08:39:36 2021 From: alan.isaac at gmail.com (Alan G. Isaac) Date: Sun, 18 Apr 2021 08:39:36 -0400 Subject: [Numpy-discussion] two questions about `choose` In-Reply-To: References: <2e3e7faf-668b-c2ba-8c66-63347acc6914@gmail.com> Message-ID: <33f4c5b6-be64-2dc8-d30a-e94f0d2448e8@gmail.com> I think you are saying that this current behavior of `choose` should be considered a bug. I hope not, because as illustrated, it is useful. How would you propose to efficiently do the same substitutions? On 4/17/2021 4:27 PM, Kevin Sheppard wrote: > 2. I would describe this a a bug. I think sequences are converted to arrays and in this case the conversion is not returning a 2 element object array but > expanding and then concatenation. > On Sat, Apr 17, 2021, 18:56 Alan G. Isaac > wrote: > 2. Separately, mypy is unhappy with my 2nd argument to `choose`: > Argument 2 to "choose" has incompatible type "Tuple[int, Sequence[float]]"; > expected "Union[Union[int, float, complex, str, bytes, generic], Sequence[Union[int, float, complex, str, bytes, generic]], > Sequence[Sequence[Any]],_SupportsArray]" > However, `choose` is happy to have e.g. `choices=(0,seq)` (and I hope it will remain so!). > > E.g., > a = a = (0,1,1,0,0,0,1,1)? #binary array > np.choose(a, (0,range(8))? ? ?#array([0, 1, 2, 0, 0, 0, 6, 7]) From rsoklaski at gmail.com Sun Apr 18 12:10:13 2021 From: rsoklaski at gmail.com (Ryan Soklaski) Date: Sun, 18 Apr 2021 12:10:13 -0400 Subject: [Numpy-discussion] ANN: MyGrad 2.0 - Drop-in autodiff for NumPy Message-ID: All, I am excited to announce the release of MyGrad 2.0. MyGrad's primary goal is to make automatic differentiation accessible and easy to use across the NumPy ecosystem (see [1] for more detailed comments). Source: https://github.com/rsokl/MyGrad Docs: https://mygrad.readthedocs.io/en/latest/ MyGrad's only dependency is NumPy and (as of version 2.0) it makes keen use of NumPy's excellent protocols for overriding functions and ufuncs. Thus you can "drop in" a mygrad-tensor into your pure NumPy code and compute derivatives through it. Ultimately, MyGrad could be extended to bring autodiff to other array-based libraries like CuPy, Sparse, and Dask. For full release notes see [2]. Feedback, critiques, and ideas are welcome! Cheers, Ryan Soklaski [1] MyGrad is not meant to "compete" with the likes of PyTorch and JAX, which are fantastically-fast and powerful autodiff libraries. Rather, its emphasis is on being lightweight and seamless to use in NumPy-centric workflows. [2] https://mygrad.readthedocs.io/en/latest/changes.html#v2-0-0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Sun Apr 18 12:54:54 2021 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 18 Apr 2021 12:54:54 -0400 Subject: [Numpy-discussion] two questions about `choose` In-Reply-To: References: <2e3e7faf-668b-c2ba-8c66-63347acc6914@gmail.com> Message-ID: On Sat, Apr 17, 2021 at 4:28 PM Kevin Sheppard wrote: > 1. I suppose it only uses the (Native int or int64) dtype since each one > would need a code path to run quickly. > > 2. I would describe this a a bug. I think sequences are converted to > arrays and in this case the conversion is not returning a 2 element object > array but expanding and then concatenation. > No, it's broadcasting the two to a common shape, as documented. https://numpy.org/doc/stable/reference/generated/numpy.choose.html > E.g., >> a = a = (0,1,1,0,0,0,1,1) #binary array >> np.choose(a, (0,range(8)) #array([0, 1, 2, 0, 0, 0, 6, 7]) >> > This is equivalent to `np.choose(a, (np.zeros(8, dtype=int), range(8)))` -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.k.sheppard at gmail.com Sun Apr 18 14:08:07 2021 From: kevin.k.sheppard at gmail.com (Kevin Sheppard) Date: Sun, 18 Apr 2021 19:08:07 +0100 Subject: [Numpy-discussion] two questions about `choose` In-Reply-To: References: <2e3e7faf-668b-c2ba-8c66-63347acc6914@gmail.com> Message-ID: Oh. I answered thinking about choice and not choose. Please ignore both parts. On Sun, Apr 18, 2021, 17:56 Robert Kern wrote: > On Sat, Apr 17, 2021 at 4:28 PM Kevin Sheppard > wrote: > >> 1. I suppose it only uses the (Native int or int64) dtype since each one >> would need a code path to run quickly. >> >> 2. I would describe this a a bug. I think sequences are converted to >> arrays and in this case the conversion is not returning a 2 element object >> array but expanding and then concatenation. >> > > No, it's broadcasting the two to a common shape, as documented. > https://numpy.org/doc/stable/reference/generated/numpy.choose.html > > >> E.g., >>> a = a = (0,1,1,0,0,0,1,1) #binary array >>> np.choose(a, (0,range(8)) #array([0, 1, 2, 0, 0, 0, 6, 7]) >>> >> > This is equivalent to `np.choose(a, (np.zeros(8, dtype=int), range(8)))` > > -- > Robert Kern > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Sun Apr 18 19:15:52 2021 From: shoyer at gmail.com (Stephan Hoyer) Date: Sun, 18 Apr 2021 16:15:52 -0700 Subject: [Numpy-discussion] ANN: MyGrad 2.0 - Drop-in autodiff for NumPy In-Reply-To: References: Message-ID: On Sun, Apr 18, 2021 at 9:11 AM Ryan Soklaski wrote: > MyGrad is not meant to "compete" with the likes of PyTorch and JAX, which > are fantastically-fast and powerful autodiff libraries. Rather, its > emphasis is on being lightweight and seamless to use in NumPy-centric > workflows. > Thanks for sharing, this looks like an interesting project! I would also be curious how MyGrad compares to Autograd [1], which is currently probably the most popular package implementing "drop-in autodiff for NumPy." From a quick look, it appears that you take a slightly different approach for the design -- object oriented rather than functional. [1] https://github.com/HIPS/autograd -------------- next part -------------- An HTML attachment was scrubbed... URL: From fiolj at yahoo.com Mon Apr 19 11:22:31 2021 From: fiolj at yahoo.com (fiolj) Date: Mon, 19 Apr 2021 12:22:31 -0300 Subject: [Numpy-discussion] timing of abs(complex) References: Message-ID: Hi all, I recently noted that absolute values of complex arrays are very slow (about a factor of five compared to some straightforward implementations). I would like to understand the origin, but could not trace the code. Please, consider these timings: In [67]: z = np.random.random(10000) + 1j*np.random.random(10000) In [68]: %timeit np.abs(z)**2 215 ?s ? 2.09 ?s per loop (mean ? std. dev. of 7 runs, 1000 loops each) In [69]: %timeit ((np.sqrt((z.real**2 + z.imag**2)))**2) 78.2 ?s ? 193 ns per loop (mean ? std. dev. of 7 runs, 10000 loops each) In [70]: %timeit (z.real**2 + z.imag**2) 40.1 ?s ? 196 ns per loop (mean ? std. dev. of 7 runs, 10000 loops each) In [71]: %timeit (z.conjugate()*z).real 43.7 ?s ? 230 ns per loop (mean ? std. dev. of 7 runs, 10000 loops each) Even considering the square root and/or additional square, does not account for the time spent In [72]: %timeit np.abs(z) 206 ?s ? 970 ns per loop (mean ? std. dev. of 7 runs, 1000 loops each) In [86]: %timeit np.sqrt(z.real**2 + z.imag**2) 70.1 ?s ? 303 ns per loop (mean ? std. dev. of 7 runs, 10000 loops each) In [73]: %timeit np.sqrt((z.conjugate()*z).real) 105 ?s ? 2.23 ?s per loop (mean ? std. dev. of 7 runs, 10000 loops each) I could not follow the code to understand what calculations are taking place for `abs()`. Any ideas? Regards, Juan From bas.vanbeek at hotmail.com Mon Apr 19 12:22:33 2021 From: bas.vanbeek at hotmail.com (bas van beek) Date: Mon, 19 Apr 2021 16:22:33 +0000 Subject: [Numpy-discussion] two questions about `choose` In-Reply-To: References: <2e3e7faf-668b-c2ba-8c66-63347acc6914@gmail.com> Message-ID: After a taking a closer look at the `np.choose` docs I think we should change `choices: npt.ArrayLike` into `choices: Sequence[npt.ArrayLike]`. This would resolve the issue, but it?d also mean that directly passing an array will be prohibited (as far as type checkers are concerned). The docs do mention that the outermost container should be a list or tuple anyway, so I?m not convinced that this new typing restriction would be a huge loss. Regards, Bas From: NumPy-Discussion On Behalf Of Kevin Sheppard Sent: 18 April 2021 20:08 To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] two questions about `choose` Oh. I answered thinking about choice and not choose. Please ignore both parts. On Sun, Apr 18, 2021, 17:56 Robert Kern > wrote: On Sat, Apr 17, 2021 at 4:28 PM Kevin Sheppard > wrote: 1. I suppose it only uses the (Native int or int64) dtype since each one would need a code path to run quickly. 2. I would describe this a a bug. I think sequences are converted to arrays and in this case the conversion is not returning a 2 element object array but expanding and then concatenation. No, it's broadcasting the two to a common shape, as documented. https://numpy.org/doc/stable/reference/generated/numpy.choose.html E.g., a = a = (0,1,1,0,0,0,1,1) #binary array np.choose(a, (0,range(8)) #array([0, 1, 2, 0, 0, 0, 6, 7]) This is equivalent to `np.choose(a, (np.zeros(8, dtype=int), range(8)))` -- Robert Kern _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: From melissawm at gmail.com Mon Apr 19 12:36:53 2021 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Mon, 19 Apr 2021 13:36:53 -0300 Subject: [Numpy-discussion] GSoD'21 Ideas Disussion In-Reply-To: References: Message-ID: Hello, Mukulika! Your project seems very ambitious! My advice would be to focus on one of two of those items and really spend some time on them. For example, items 1 and 5 could come together, or items 3 and 4 (we are working on item 2 and welcome contributions, but that may be a smaller task at this point). It depends on what you would like to focus on: writing new content or revamping existing content. Let me know what you think. Cheers, - Melissa On Sun, Apr 18, 2021 at 9:31 AM Mukulika Pahari wrote: > Hello everyone! > I am Mukulika Pahari, a Computer Engineering student from India. I wanted > to implement the following ideas for Google Summer of Docs 2021 if given > the opportunity. I have referred to both- the GSoD proposal > > and the NEP 44 > proposal. > Please give me feedback about the ideas! > > > 1. > > Reorganising contents of the documentation into Reference Guide, > How-Tos, Tutorials and Explanations as per structure proposed in NEP 44 > . > > > - > > Auditing existing documentation > - > > Clearing misplaced content for example Explanations in Reference > Guide, How-Tos in Explanations etc. > - > > Establishing distinct Reference, How-Tos, Tutorials and Explanations > sections with crosslinking where required > > > 1. > > Reorganising landing page of NumPy docs . > > > - > > Moving the documentation structure to the left sidebar > - > > Having NumPy Quickstart as the first thing people see when they land > on Documentation > > > 1. > > Writing new must-have tutorials (based on most-searched tutorials on > Google). > > > - > > ?How to write a tutorial? guide > - > > 3 Beginner Tutorials > - > > 3 Intermediate Tutorials > - > > 3 Advanced Tutorials > > > 1. > > Writing How-Tos based on most-used functions, most asked doubts on > StackOverflow, etc. > 2. > > Revamping the User Guide. > > > - > > Updating out-of-date references and refactoring content to the latest > best practices > - > > Adding non-textual images or graphics to enhance the textual > explanations > - > > Removing duplication to improve searchability > > > I have proposed this work keeping in mind 30ish weeks of work including > a few weeks for becoming familiar with the organisation and ironing out > details like exactly which tutorials and how-tos to write. Please let me > know if I can aim to achieve more in the timeframe. > > My experience with NumPy is currently limited to one data analysis project > but I would love to learn more about its applications while restructuring > and developing its docs! > > Thank you for your time. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Mon Apr 19 13:16:55 2021 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 19 Apr 2021 13:16:55 -0400 Subject: [Numpy-discussion] two questions about `choose` In-Reply-To: References: <2e3e7faf-668b-c2ba-8c66-63347acc6914@gmail.com> Message-ID: On Mon, Apr 19, 2021 at 12:24 PM bas van beek wrote: > After a taking a closer look at the `np.choose` docs I think we should > change `*choices: npt.ArrayLike` into `choices: Sequence[npt.ArrayLike]`.* > > > > *This would resolve the issue, but it?d also mean that directly passing an > array will be prohibited (as far as type checkers are concerned).* > > *The docs do mention that the outermost container should be a list or > tuple anyway, so I?m not convinced that this new typing restriction would > be a huge loss.* > Why not allow both? -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Tue Apr 20 08:17:59 2021 From: matti.picus at gmail.com (Matti Picus) Date: Tue, 20 Apr 2021 08:17:59 -0400 Subject: [Numpy-discussion] NEP 49: Data allocation strategies Message-ID: <5f06c34e-939a-d6ba-309d-189ef7ffc950@gmail.com> I have submitted NEP 49 to enable user-defined allocation strategies for the ndarray.data homogeneous memory area. The implementation is in PR 17582 https://github.com/numpy/numpy/pull/17582 Here is the text of the NEP: Abstract -------- The ``numpy.ndarray`` requires additional memory allocations to hold ``numpy.ndarray.strides``, ``numpy.ndarray.shape`` and ``numpy.ndarray.data`` attributes. These attributes are specially allocated after creating the python object in ``__new__`` method. The ``strides`` and ``shape`` are stored in a piece of memory allocated internally. This NEP proposes a mechanism to override the memory management strategy used for ``ndarray->data`` with user-provided alternatives. This allocation holds the arrays data and is can be very large. As accessing this data often becomes a performance bottleneck, custom allocation strategies to guarantee data alignment or pinning allocations to specialized memory hardware can enable hardware-specific optimizations. Motivation and Scope -------------------- Users may wish to override the internal data memory routines with ones of their own. Two such use-cases are to ensure data alignment and to pin certain allocations to certain NUMA cores. User who wish to change the NumPy data memory management routines will use :c:func:`PyDataMem_SetHandler`, which uses a :c:type:`PyDataMem_Handler` structure to hold pointers to functions used to manage the data memory. The calls are wrapped by internal routines to call :c:func:`PyTraceMalloc_Track`, :c:func:`PyTraceMalloc_Untrack`, and will use the :c:func:`PyDataMem_EventHookFunc` mechanism? already present in NumPy for auditing purposes. Since a call to ``PyDataMem_SetHandler`` will change the default functions, but that function may be called during the lifetime of an ``ndarray`` object, each ``ndarray`` will carry with it the ``PyDataMem_Handler`` struct used at the time of its instantiation, and these will be used to reallocate or free the data memory of the instance. Internally NumPy may use ``memcpy` or ``memset`` on the data ``ptr``. Usage and Impact ---------------- The new functions can only be accessed via the NumPy C-API. An example is included later in the NEP. The added ``struct`` will increase the size of the ``ndarray`` object. It is one of the major drawbacks of this approach. We can be reasonably sure that the change in size will have a minimal impact on end-user code because NumPy version 1.20 already changed the object size. Backward compatibility ---------------------- The design will not break backward compatibility. Projects that were assigning to the ``ndarray->data`` pointer were already breaking the current memory management strategy (backed by ``npy_alloc_cache``) and should restore ``ndarray->data`` before calling ``Py_DECREF``. As mentioned above, the change in size should not impact end-users. Matti From bobbyocean at gmail.com Tue Apr 20 14:20:59 2021 From: bobbyocean at gmail.com (Robert) Date: Tue, 20 Apr 2021 11:20:59 -0700 (MST) Subject: [Numpy-discussion] Polynomial Improvements Message-ID: <1618942859820-0.post@n7.nabble.com> Hi, I am new to contributing to open source projects and not sure where to begin (maybe emailing this distro?). In any case, the main improvement I would like to work on would be adding multivariate polynomials/differentials to numpy. I would love any insight into the current status and intensions of numpy polynomial implementations. 1) why do we have np.polynomial and np.lib.polynomial? which one is older? is there a desire to see these merged? 2) why does np.polynomial.Polynomial have a domain and window? they appear to have no implementation (except the chebyshev interpolation) as far as I can tell? what is their intended use? why is their no variable representation implemented like in np.polynomial and is not installed to numpy directly, @set_module('numpy')? 3) how many improvements are allowed to be done all at once? is there an expectation of waiting before improving implementations or code? Thinking further down the road, how much is too much to add to numpy? I would like to see multivariate implemented, but after that it would be nice to have a collection of known polynomials like standard, pochhammer, cyclotomic, hermite, lauguerre, legendre, chebyshev, jacobi. After that it would be of interest to extend .diff() to really be a differential operator (implemented as sequences of polynomials) and to allow differential constructions. Then things like this would be possible: x = np.poly1d([1,0]) D = np.diff1d([1,0]) assert Hermite(78) == (2*x - D)(Hermite(77)) Or if series were implemented then we would see things like: assert Hermite(189) == (-1)**189 * Exp(x**2)*(D**189)(Exp(-x**2)) assert Hermite(189) == Exp(-D**2)(x**189)(2*x) 4) at what point does numpy polynomial material belong in scipy material and visa-versa? is numpy considered the backend to scipy or visa-versa? at times it seems like things are double implemented in numpy and scipy, but in general should not implement things in numpy that already exist in scipy? Thanks, Robert -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From oscar.j.benjamin at gmail.com Tue Apr 20 15:57:23 2021 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 20 Apr 2021 20:57:23 +0100 Subject: [Numpy-discussion] Polynomial Improvements In-Reply-To: <1618942859820-0.post@n7.nabble.com> References: <1618942859820-0.post@n7.nabble.com> Message-ID: On Tue, 20 Apr 2021 at 19:21, Robert wrote: > > I am new to contributing to open source projects and not sure where to begin > (maybe emailing this distro?). In any case, the main improvement I would > like to work on would be adding multivariate polynomials/differentials to > numpy. I would love any insight into the current status and intensions of > numpy polynomial implementations. > > 1) why do we have np.polynomial and np.lib.polynomial? which one is older? > is there a desire to see these merged? > 2) why does np.polynomial.Polynomial have a domain and window? they appear > to have no implementation (except the chebyshev interpolation) as far as I > can tell? what is their intended use? why is their no variable > representation implemented like in np.polynomial and is not installed to > numpy directly, @set_module('numpy')? > 3) how many improvements are allowed to be done all at once? is there an > expectation of waiting before improving implementations or code? > > Thinking further down the road, how much is too much to add to numpy? I > would like to see multivariate implemented, but after that it would be nice > to have a collection of known polynomials like standard, pochhammer, > cyclotomic, hermite, lauguerre, legendre, chebyshev, jacobi. After that it > would be of interest to extend .diff() to really be a differential operator > (implemented as sequences of polynomials) and to allow differential > constructions. Then things like this would be possible: > > x = np.poly1d([1,0]) > D = np.diff1d([1,0]) > assert Hermite(78) == (2*x - D)(Hermite(77)) > > Or if series were implemented then we would see things like: > > assert Hermite(189) == (-1)**189 * Exp(x**2)*(D**189)(Exp(-x**2)) > assert Hermite(189) == Exp(-D**2)(x**189)(2*x) > > 4) at what point does numpy polynomial material belong in scipy material and > visa-versa? The distinction between numpy and scipy comes from the way things have worked out over time to some extent. I expect that if numpy was being rewritten today it wouldn't have any polynomial functions. Are you aware of sympy? What you are suggesting seems more appropriate for a symbolic library rather than a numerical one. SymPy already has most of this e.g.: In [47]: x = Symbol('x') In [48]: h78 = hermite(78, x) In [49]: h77 = hermite(77, x) In [50]: h78 == expand(2*x*h77 - h77.diff(x)) Out[50]: True Being a symbolic library sympy can handle Hermite polynomials of symbolic order: In [51]: n = Symbol('n') In [52]: hermite(n, x) Out[52]: hermite(n, x) In [53]: hermite(n, x).diff(x) Out[53]: 2?n?hermite(n - 1, x) There is also support for series etc. Oscar From bobbyocean at gmail.com Tue Apr 20 18:02:31 2021 From: bobbyocean at gmail.com (Robert) Date: Tue, 20 Apr 2021 15:02:31 -0700 (MST) Subject: [Numpy-discussion] Polynomial Improvements In-Reply-To: References: <1618942859820-0.post@n7.nabble.com> Message-ID: <1618956151552-0.post@n7.nabble.com> I had not looked into sympy that closes, thinking it was mostly a symbolic package. However, there appears to be functions that convert back to numpy expressions so that np.ndarray's and such can work. There also appears to be extensive polynomial classes already defined. Thanks for pointing me in the right direction. -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From sebastian at sipsolutions.net Tue Apr 20 18:53:03 2021 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 20 Apr 2021 17:53:03 -0500 Subject: [Numpy-discussion] NumPy Development Meeting Wednesday - Triage Focus Message-ID: Hi all, Our bi-weekly triage-focused NumPy development meeting is Wednesday, Arpil 21st at 11 am Pacific Time (18:00 UTC). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From rsoklaski at gmail.com Tue Apr 20 23:57:06 2021 From: rsoklaski at gmail.com (Ryan Soklaski) Date: Tue, 20 Apr 2021 20:57:06 -0700 (MST) Subject: [Numpy-discussion] ANN: MyGrad 2.0 - Drop-in autodiff for NumPy In-Reply-To: References: Message-ID: <1618977426717-0.post@n7.nabble.com> Hi Stephan, You are correct that MyGrad takes an object-oriented design, rather than a functional one. This enables a more imperative style of workflow [1], which is how many people approach doing data science in notebooks and REPLs. MyGrad feels similar to NumPy and PyTorch in this way. Ultimately, swapping one (or more) `ndarray` with a `Tensor`is all your need to do differentiate your numpy-based code with respect to that variable: from stephans_library import func_using_numpy x = mygrad.tensor(1.) y = mygrad.tensor(2.) z = func_using_numpy(x, y) # coerced into returning a Tensor z.backward() # computes dz/dx and dz/dy x.grad # stores dz/dx y.grad # stores dz/dy Thus with MyGrad, you can truly drop in a tensor into code that is written in vanilla NumPy (assuming said code involves numpy functions currently implemented in MyGrad), no matter what style of code that is ? functional or otherwise. Regarding autograd, I would maybe describe them as "swap out" autodiff for NumPy, rather than "drop-in". Indeed, you need to use the functions supplied by `autograd.numpy` in order to leverage their functionality, in addition to adopting a functional code style. Which means that `stephans_library.func_using_numpy` can't be differentiated either unless it used autograd.numpy. Furthermore, autograd does not really aim to be "NumPy with autodiff" to the same fidelity. For example, and in contrast with MyGrad, it does not support: - in-place operations - specifying dtype, where, or out in ufuncs - common use-cases of einsum like traces and broadcast-reduction [2] And, unfortunately, autograd has some long-standing bugs, including cases where it simply gives you the wrong derivatives for relatively simple functions [3]. In general, it doesn't seem like there is much activity towards dealing with bug reports in the library. That all being said, here are some pros and cons of MyGrad by my own estimate: Some cons of MyGrad: - autograd provides rich support for computing Jacobians and higher-order derivatives. MyGrad doesn't. - Still plenty of NumPy functions that need implementing - Supporting a flexible imperative style along with in-place operations and views comes at a (mitigable) performance cost [4] - Currently maintained just by me in my personal time (hard to match Harvard/Google/Facebook!), which doesn't scale - Nowhere close to the level of adoption of autograd Some pros of MyGrad: - Easy for NumPy users to just pick up and use (NumPy + `Tensor.backward()`) - Big emphasis on correctness and completeness in terms of parity with NumPy - Object-oriented approach has lots of perks - Easy for users to implement their own differentiable functions [5] - Tensor can be wrapped by, say, a xarray-Datarray for backprop through xarray [6] - Tensor could wrap CuPy/Dask/sparse array instead of ndarray to bring autodiff to them - Polished docs, type hints, UX - High-quality test suite (leverages Hypothesis [7] extensively) [1] https://gist.github.com/rsokl/7c2812264ae622bbecc990fad4af3fd2#getting-a-derivative [2] https://gist.github.com/rsokl/7c2812264ae622bbecc990fad4af3fd2#some-derivatives-not-supported-by-autograd [3] https://gist.github.com/rsokl/7c2812264ae622bbecc990fad4af3fd2#some-places-where-autograd-returns-incorrect-derivatives [4] https://mygrad.readthedocs.io/en/latest/performance_tips.html#controlling-memory-guarding-behavior [5] https://mygrad.readthedocs.io/en/latest/operation.html [6] Still lots of sharp edges here: https://gist.github.com/rsokl/7c2812264ae622bbecc990fad4af3fd2#a-crude-prototype-of-using-mygrad-w-xarray [7] https://hypothesis.readthedocs.io/en/latest/ -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From ralf.gommers at gmail.com Wed Apr 21 03:10:22 2021 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Wed, 21 Apr 2021 09:10:22 +0200 Subject: [Numpy-discussion] NEP 49: Data allocation strategies In-Reply-To: <5f06c34e-939a-d6ba-309d-189ef7ffc950@gmail.com> References: <5f06c34e-939a-d6ba-309d-189ef7ffc950@gmail.com> Message-ID: On Tue, Apr 20, 2021 at 2:18 PM Matti Picus wrote: > I have submitted NEP 49 to enable user-defined allocation strategies for > the ndarray.data homogeneous memory area. The implementation is in PR > 17582 https://github.com/numpy/numpy/pull/17582 Here is the text of the > NEP: > Thanks Matti! > > Abstract > -------- > > The ``numpy.ndarray`` requires additional memory allocations > to hold ``numpy.ndarray.strides``, ``numpy.ndarray.shape`` and > ``numpy.ndarray.data`` attributes. These attributes are specially allocated > after creating the python object in ``__new__`` method. The ``strides`` and > ``shape`` are stored in a piece of memory allocated internally. > > This NEP proposes a mechanism to override the memory management strategy > used > for ``ndarray->data`` with user-provided alternatives. This allocation > holds > the arrays data and is can be very large. As accessing this data often > becomes > a performance bottleneck, custom allocation strategies to guarantee data > alignment or pinning allocations to specialized memory hardware can enable > hardware-specific optimizations. > > Motivation and Scope > -------------------- > > Users may wish to override the internal data memory routines with ones > of their > own. Two such use-cases are to ensure data alignment and to pin certain > allocations to certain NUMA cores. > It would be great to expand a bit on these two sentences, and add some links. There's a lot of history here in NumPy development to refer to as well: https://numpy-discussion.scipy.narkive.com/MvmMkJcK/numpy-arrays-data-allocation-and-simd-alignement http://numpy-discussion.10968.n7.nabble.com/Aligned-configurable-memory-allocation-td39712.html http://numpy-discussion.10968.n7.nabble.com/Numpy-s-policy-for-releasing-memory-td1533.html https://github.com/numpy/numpy/issues/5312 https://github.com/numpy/numpy/issues/14177 There must also be a good amount of ideas/discussion elsewhere. https://bugs.python.org/issue18835 discussed an aligned allocator for Python itself, with fairly detailed discussion about whether/how NumPy could benefit. With (I think) the conclusion it shouldn't be in Python, but NumPy/Arrow/others are better off doing their own thing. I'm wondering if improved memory profiling is a use case as well? Fil ( https://github.com/pythonspeed/filprofiler) for example seems to use such a strategy: https://github.com/pythonspeed/filprofiler/blob/master/design/allocator-overrides.md Does it interact with our tracemalloc support ( https://numpy.org/doc/stable/release/1.13.0-notes.html#support-for-tracemalloc-in-python-3-6 )? > User who wish to change the NumPy data memory management routines will use > This is design, not motivation or scope. Try to not refer to specific function names in this section. I suggest moving this content to the "Detailed design" section (or better, a "high level design" at the start of that section). Cheers, Ralf :c:func:`PyDataMem_SetHandler`, which uses a :c:type:`PyDataMem_Handler` > structure to hold pointers to functions used to manage the data memory. The > calls are wrapped by internal routines to call > :c:func:`PyTraceMalloc_Track`, > :c:func:`PyTraceMalloc_Untrack`, and will use the > :c:func:`PyDataMem_EventHookFunc` mechanism already present in NumPy for > auditing purposes. > > Since a call to ``PyDataMem_SetHandler`` will change the default > functions, but > that function may be called during the lifetime of an ``ndarray`` > object, each > ``ndarray`` will carry with it the ``PyDataMem_Handler`` struct used at the > time of its instantiation, and these will be used to reallocate or free the > data memory of the instance. Internally NumPy may use ``memcpy` or > ``memset`` > on the data ``ptr``. > > Usage and Impact > ---------------- > > The new functions can only be accessed via the NumPy C-API. An example is > included later in the NEP. The added ``struct`` will increase the size > of the > ``ndarray`` object. It is one of the major drawbacks of this approach. > We can > be reasonably sure that the change in size will have a minimal impact on > end-user code because NumPy version 1.20 already changed the object size. > > Backward compatibility > ---------------------- > > The design will not break backward compatibility. Projects that were > assigning > to the ``ndarray->data`` pointer were already breaking the current memory > management strategy (backed by ``npy_alloc_cache``) and should restore > ``ndarray->data`` before calling ``Py_DECREF``. As mentioned above, the > change > in size should not impact end-users. > > Matti > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgacha at quansight.com Wed Apr 21 18:44:22 2021 From: sgacha at quansight.com (=?UTF-8?Q?Stephannie_Jim=C3=A9nez_Gacha?=) Date: Wed, 21 Apr 2021 17:44:22 -0500 Subject: [Numpy-discussion] Add smallest_normal and smallest_subnormal attributes to finfo Message-ID: Good afternoon, Given the discussions happened in the Data API consortium when looking into the attributes of `finfo` used in the wild, we found that `tiny` is used regularly but in a good amount of cases not for its intended purpose but rather as "just give me a small number". Following this we are proposing the addition of `smallest_normal` and `smallest_subnormal` attributes. Personally, I think that the `tiny` name is a little bit odd and misleading, so it will be great to leave that as an alias but have a clear name in this class. Right now the PR: https://github.com/numpy/numpy/pull/18536 has all the changes and all the values added were checked against IEEE-754 standard. One of the main concerns is the support of subnormal numbers in certain architectures, where the values can't be calculated accurately. Given the state of the discussion, we don't know if the best alternative is to not add the `smallest_subnormal` attribute and just add the `smallest_number` attribute as an alias to `tiny`. We open this to discussion to see what way we can go in order to get this PR merged. *Stephannie Jimenez Gacha*Software developer *Quansight* | Your Data Experts w: www.quansight.com e: sgacha at quansight.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Wed Apr 21 21:26:11 2021 From: matti.picus at gmail.com (Matti Picus) Date: Wed, 21 Apr 2021 21:26:11 -0400 Subject: [Numpy-discussion] NEP 49: Data allocation strategies In-Reply-To: References: <5f06c34e-939a-d6ba-309d-189ef7ffc950@gmail.com> Message-ID: See my comments interspersed in Ralf's reply. Thanks for the additional context. Matti On 21/4/21 3:10 am, Ralf Gommers wrote: > > > ... > > Motivation and Scope > -------------------- > > Users may wish to override the internal data memory routines with > ones > of their > own. Two such use-cases are to ensure data alignment and to pin > certain > allocations to certain NUMA cores. > > > It would be great to expand a bit on these two sentences, and add some > links. There's a lot of history here in NumPy development to refer to > as well: > > https://numpy-discussion.scipy.narkive.com/MvmMkJcK/numpy-arrays-data-allocation-and-simd-alignement > > http://numpy-discussion.10968.n7.nabble.com/Aligned-configurable-memory-allocation-td39712.html > > http://numpy-discussion.10968.n7.nabble.com/Numpy-s-policy-for-releasing-memory-td1533.html > > https://github.com/numpy/numpy/issues/5312 > > https://github.com/numpy/numpy/issues/14177 > > > There must also be a good amount of ideas/discussion elsewhere. I added more context to this section, trying to focus on the large data allocations in NumPy. > > https://bugs.python.org/issue18835 > discussed an aligned allocator > for Python itself, with fairly detailed discussion about whether/how > NumPy could benefit. With (I think) the conclusion it shouldn't be in > Python, but NumPy/Arrow/others are better off doing their own thing. > > I'm wondering if improved memory profiling is a use case as well? Fil > (https://github.com/pythonspeed/filprofiler > ) for example seems to use > such a strategy: > https://github.com/pythonspeed/filprofiler/blob/master/design/allocator-overrides.md > > Thanks. I added a sentence about this as well. > Does it interact with our tracemalloc support > (https://numpy.org/doc/stable/release/1.13.0-notes.html#support-for-tracemalloc-in-python-3-6 > )? > I added a sentence about this. The new C-API wrapper functions preserve the current status vis-a-vis tracemalloc support. I am not sure that support is complete. The NEP should not change the situation for better or worse. > > User who wish to change the NumPy data memory management routines > will use > > > This is design, not motivation or scope. Try to not refer to specific > function names in this section. I suggest moving this content to the > "Detailed design" section (or better, a "high level design" at the > start of that section). > Done. > Cheers, > Ralf > > From khkiran01 at gmail.com Fri Apr 23 09:29:54 2021 From: khkiran01 at gmail.com (KiranKhanna) Date: Fri, 23 Apr 2021 18:59:54 +0530 Subject: [Numpy-discussion] GSOD Message-ID: <51FC800E-8131-4CF0-A185-E49915E467C2@hxcore.ol> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Resume.pdf Type: application/pdf Size: 54702 bytes Desc: not available URL: From melissawm at gmail.com Fri Apr 23 12:24:54 2021 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Fri, 23 Apr 2021 13:24:54 -0300 Subject: [Numpy-discussion] GSOD In-Reply-To: <51FC800E-8131-4CF0-A185-E49915E467C2@hxcore.ol> References: <51FC800E-8131-4CF0-A185-E49915E467C2@hxcore.ol> Message-ID: Hello, Kiran! If you are interested in participating as part of the Google Season of Docs program, please take a look at our project page: https://github.com/numpy/numpy/wiki/Google-Season-of-Docs-2021:-Submitted-Project As a reminder to you (and others), you must submit a Statement of Interest ( https://developers.google.com/season-of-docs/docs/tech-writer-statement) at some point - the earlier the better, as the deadline for hiring Technical Writers is May 17. Meanwhile, if you have any questions or would like to discuss your ideas, you can send a message to numpy-scipy-gsod at googlegroups.com. Cheers! Melissa On Fri, Apr 23, 2021 at 11:47 AM KiranKhanna wrote: > Respected Sir/ma?am, > > I am Kiran Khanna ,I am interested in contributing in the > NumPy organization .I am second year computer science with AI engineering > student . I am not professional but I am interested in learning more thing > by contributing in the organization . I have attached my resume below . > > > > Thanking you , > > Your faithfully, > > Kiran Khanna > > > > > > Sent from Mail for > Windows 10 > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From melissawm at gmail.com Fri Apr 23 17:55:11 2021 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Fri, 23 Apr 2021 18:55:11 -0300 Subject: [Numpy-discussion] Documentation Team meeting - Monday April 26 In-Reply-To: References: Message-ID: Hi all! Our next Documentation Team meeting will be on *Monday, April 26* at ***4PM UTC***. All are welcome - you don't need to already be a contributor to join. If you have questions or are curious about what we're doing, we'll be happy to meet you! If you wish to join on Zoom, use this link: https://zoom.us/j/96219574921?pwd=VTRNeGwwOUlrYVNYSENpVVBRRjlkZz09#success Here's the permanent hackmd document with the meeting notes (still being updated in the next few days!): https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg Hope to see you around! ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentation+Team+Meeting&iso=20210426T16&p1=1440&ah=1 *** You can add the NumPy community calendar to your google calendar by clicking this link: https://calendar.google.com/calendar /r?cid=YmVya2VsZXkuZWR1X2lla2dwaWdtMjMyamJobGRzZmIyYzJqODFjQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20 - Melissa -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sat Apr 24 11:54:28 2021 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sat, 24 Apr 2021 17:54:28 +0200 Subject: [Numpy-discussion] Add pad_to_shape and pad_to_match for simpler padding if a target size is already known In-Reply-To: References: Message-ID: On Thu, Apr 15, 2021 at 11:54 AM czorio4 wrote: > Hello all, > > I'm only two weeks late with this message about my pull request > that adds functions that > allow the user to pad to a target shape, instead of adding a certain amount > to each axis. > No worries at all, things often go at "whenever I have some time after life stops getting in the way" pace here. > For example: > > x = np.ones((3, 3)) > # We want an output shape of (10, 10) > padded = np.pad_to_shape(x, (10, 10)) > print(padded.shape) > prints: (10, 10) > > Whereas with the current implementation of np.pad you'd need to first > figure out the difference in axis sizes. The proposed function would > perform this step for you. As this function passes to np.pad internally, I > made sure to include the arguments in the signature that np.pad uses for > padding modes. > > Finally, I've added a logical extension of this function; pad_to_match, > this takes another array and pads the input array to match. > Adding this functionality makes sense - if we find a good place outside of the main namespace, as was also remarked on in your PR. Luckily there is plan to create such a space, as submodules under `numpy.lib`. This draft PR needs to be finished for that: https://github.com/numpy/numpy/pull/18447 > These calls would be equivalent: > > x = np.ones((3, 3)) > y = np.zeros((10, 10)) > padded_to_shape = np.pad_to_shape(x, y.shape) > padded_to_match = np.pad_to_match(x, y) > Question, why two functions with slighty awkward (double underscores) names rather than: def pad_to(x, shape=None, like=None): Note also the `like` instead of `match`; this seems more consistent with functions like `ones_like` and the `like=` keyword which also take a shape from another array. > For additional functionality description, I refer to the pull request. > > I'm not too familiar with mailing lists, so I hope this is how things work. > Yep, you got it all right. Mailing lists are starting to get old-fashioned, one of these days we'll move to something more modern. Cheers, Ralf > Kind Regards, > Mathijs > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From czorio4 at gmail.com Sat Apr 24 12:24:32 2021 From: czorio4 at gmail.com (czorio4) Date: Sat, 24 Apr 2021 18:24:32 +0200 Subject: [Numpy-discussion] Add pad_to_shape and pad_to_match for simpler padding if a target size is already known In-Reply-To: References: Message-ID: Hello Ralf, Thanks for the feedback! Instead of the two separate arguments, would it be possible to define it like: def pad_to(x, target, ...) where the `target` can be either a tuple or an ndarray (and/or some other array-like), and the function determines the shape based on what input type was provided. My worry here is that this might introduce some more room for bugs with input types. Alternatively (and probably simpler, too), drop the entire `pad_to_match` function, and instead have the user explicitly provide the other array's shape: padded = np.pad_to(x, y.shape, ...) Kind regards, Mathijs Op za 24 apr. 2021 om 17:54 schreef Ralf Gommers : > > > On Thu, Apr 15, 2021 at 11:54 AM czorio4 wrote: > >> Hello all, >> >> I'm only two weeks late with this message about my pull request >> that adds functions that >> allow the user to pad to a target shape, instead of adding a certain amount >> to each axis. >> > > No worries at all, things often go at "whenever I have some time after > life stops getting in the way" pace here. > > >> For example: >> >> x = np.ones((3, 3)) >> # We want an output shape of (10, 10) >> padded = np.pad_to_shape(x, (10, 10)) >> print(padded.shape) >> prints: (10, 10) >> >> Whereas with the current implementation of np.pad you'd need to first >> figure out the difference in axis sizes. The proposed function would >> perform this step for you. As this function passes to np.pad internally, I >> made sure to include the arguments in the signature that np.pad uses for >> padding modes. >> >> Finally, I've added a logical extension of this function; pad_to_match, >> this takes another array and pads the input array to match. >> > > Adding this functionality makes sense - if we find a good place outside of > the main namespace, as was also remarked on in your PR. Luckily there is > plan to create such a space, as submodules under `numpy.lib`. This draft PR > needs to be finished for that: https://github.com/numpy/numpy/pull/18447 > > >> These calls would be equivalent: >> >> x = np.ones((3, 3)) >> y = np.zeros((10, 10)) >> padded_to_shape = np.pad_to_shape(x, y.shape) >> padded_to_match = np.pad_to_match(x, y) >> > > Question, why two functions with slighty awkward (double underscores) > names rather than: > > def pad_to(x, shape=None, like=None): > > Note also the `like` instead of `match`; this seems more consistent with > functions like `ones_like` and the `like=` keyword which also take a shape > from another array. > > >> For additional functionality description, I refer to the pull request. >> >> I'm not too familiar with mailing lists, so I hope this is how things >> work. >> > > Yep, you got it all right. Mailing lists are starting to get > old-fashioned, one of these days we'll move to something more modern. > > Cheers, > Ralf > > > >> Kind Regards, >> Mathijs >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sat Apr 24 12:26:29 2021 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sat, 24 Apr 2021 18:26:29 +0200 Subject: [Numpy-discussion] Add pad_to_shape and pad_to_match for simpler padding if a target size is already known In-Reply-To: References: Message-ID: On Sat, Apr 24, 2021 at 6:25 PM czorio4 wrote: > Hello Ralf, > > Thanks for the feedback! > > Instead of the two separate arguments, would it be possible to define it > like: > > def pad_to(x, target, ...) > > where the `target` can be either a tuple or an ndarray (and/or some other > array-like), and the function determines the shape based on what input type > was provided. > My worry here is that this might introduce some more room for bugs with > input types. > > Alternatively (and probably simpler, too), drop the entire `pad_to_match` > function, and instead have the user explicitly provide the other array's > shape: > > padded = np.pad_to(x, y.shape, ...) > Yes I agree, that's an even better idea. +1 for avoiding unions when the solution is as simple as using `.shape`. Cheers, Ralf > Kind regards, > Mathijs > > Op za 24 apr. 2021 om 17:54 schreef Ralf Gommers : > >> >> >> On Thu, Apr 15, 2021 at 11:54 AM czorio4 wrote: >> >>> Hello all, >>> >>> I'm only two weeks late with this message about my pull request >>> that adds functions that >>> allow the user to pad to a target shape, instead of adding a certain amount >>> to each axis. >>> >> >> No worries at all, things often go at "whenever I have some time after >> life stops getting in the way" pace here. >> >> >>> For example: >>> >>> x = np.ones((3, 3)) >>> # We want an output shape of (10, 10) >>> padded = np.pad_to_shape(x, (10, 10)) >>> print(padded.shape) >>> prints: (10, 10) >>> >>> Whereas with the current implementation of np.pad you'd need to first >>> figure out the difference in axis sizes. The proposed function would >>> perform this step for you. As this function passes to np.pad internally, I >>> made sure to include the arguments in the signature that np.pad uses for >>> padding modes. >>> >>> Finally, I've added a logical extension of this function; pad_to_match, >>> this takes another array and pads the input array to match. >>> >> >> Adding this functionality makes sense - if we find a good place outside >> of the main namespace, as was also remarked on in your PR. Luckily there is >> plan to create such a space, as submodules under `numpy.lib`. This draft PR >> needs to be finished for that: https://github.com/numpy/numpy/pull/18447 >> >> >>> These calls would be equivalent: >>> >>> x = np.ones((3, 3)) >>> y = np.zeros((10, 10)) >>> padded_to_shape = np.pad_to_shape(x, y.shape) >>> padded_to_match = np.pad_to_match(x, y) >>> >> >> Question, why two functions with slighty awkward (double underscores) >> names rather than: >> >> def pad_to(x, shape=None, like=None): >> >> Note also the `like` instead of `match`; this seems more consistent with >> functions like `ones_like` and the `like=` keyword which also take a shape >> from another array. >> >> >>> For additional functionality description, I refer to the pull request. >>> >>> I'm not too familiar with mailing lists, so I hope this is how things >>> work. >>> >> >> Yep, you got it all right. Mailing lists are starting to get >> old-fashioned, one of these days we'll move to something more modern. >> >> Cheers, >> Ralf >> >> >> >>> Kind Regards, >>> Mathijs >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion at python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Sun Apr 25 22:19:08 2021 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Sun, 25 Apr 2021 20:19:08 -0600 Subject: [Numpy-discussion] ANN: SciPy 1.6.3 Message-ID: Hi all, On behalf of the SciPy development team I'm pleased to announce the release of SciPy 1.6.3, which is a bug fix release. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.6.3 One of a few ways to install this release with pip: pip install scipy==1.6.3 ===================== SciPy 1.6.3 Release Notes ===================== SciPy 1.6.3 is a bug-fix release with no new features compared to 1.6.2. Authors ====== * Peter Bell * Ralf Gommers * Matt Haberland * Peter Mahler Larsen * Tirth Patel * Tyler Reddy * Pamphile ROY + * Xingyu Liu + A total of 8 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.6.3 ------------------------------- * `#13772 `__: Divide by zero in distance.yule * `#13796 `__: CI: prerelease_deps failures * `#13890 `__: TST: spatial rotation failure in (1.6.3) wheels repo (ARM64) Pull requests for 1.6.3 ------------------------------ * `#13755 `__: CI: fix the matplotlib warning emitted during builing docs * `#13773 `__: BUG: Divide by zero in yule dissimilarity of constant vectors * `#13799 `__: CI/MAINT: deprecated np.typeDict * `#13819 `__: substitute np.math.factorial with math.factorial * `#13895 `__: TST: add random seeds in Rotation module Checksums ========= MD5 ~~~ 3b75d493f6c93b662f927d6c2ac60053 scipy-1.6.3-cp37-cp37m-macosx_10_9_x86_64.whl ada6fa32f066dc58033ab47a4fbcd208 scipy-1.6.3-cp37-cp37m-manylinux1_i686.whl 29c6d6edbe9c2ba17dc4edd89ed31abb scipy-1.6.3-cp37-cp37m-manylinux1_x86_64.whl cde2f4824337fda5b2522795fff2135d scipy-1.6.3-cp37-cp37m-manylinux2014_aarch64.whl 27fa4101babcfa7928e4c23e33212e71 scipy-1.6.3-cp37-cp37m-win32.whl b0ef1b6ba4d3b54ad8f17dd10156c3f8 scipy-1.6.3-cp37-cp37m-win_amd64.whl 66b3ea5bb7869af010ae9d6a5015360b scipy-1.6.3-cp38-cp38-macosx_10_9_x86_64.whl 0ef8582f203fdd4afe1ca56fc5b309f5 scipy-1.6.3-cp38-cp38-manylinux1_i686.whl 1194e9d88ef98595619acd657439fae4 scipy-1.6.3-cp38-cp38-manylinux1_x86_64.whl 4b8fe1b85fd8d60bbc787a06c0d545ee scipy-1.6.3-cp38-cp38-manylinux2014_aarch64.whl 1d55aff23261c2fa8d7882bc7220c25d scipy-1.6.3-cp38-cp38-win32.whl e82a80c799f1e5190be0cbd25b554766 scipy-1.6.3-cp38-cp38-win_amd64.whl 8309d581e025539bafaa98ba9c3122a7 scipy-1.6.3-cp39-cp39-macosx_10_9_x86_64.whl 75a9ba6865ff2fefcb5df917a412d99d scipy-1.6.3-cp39-cp39-manylinux1_i686.whl fbfc53ba0ea23f56afb62c2a159e6303 scipy-1.6.3-cp39-cp39-manylinux1_x86_64.whl 4076d3cdb2b16f7d64229ac76c945b6f scipy-1.6.3-cp39-cp39-manylinux2014_aarch64.whl bd553432d6a55a6e139f76951766f31d scipy-1.6.3-cp39-cp39-win32.whl 6b9526aca2adf7b062ac92f184328147 scipy-1.6.3-cp39-cp39-win_amd64.whl 05b4ca400aa1157290abff69016f1cab scipy-1.6.3.tar.gz cef7aa950dfc7f5be79b8c630bfe0cc1 scipy-1.6.3.tar.xz f13ea8ab38fb03c17e38c3cda7877972 scipy-1.6.3.zip SHA256 ~~~~~~ 2a799714bf1f791fb2650d73222b248d18d53fd40d6af2df2c898db048189606 scipy-1.6.3-cp37-cp37m-macosx_10_9_x86_64.whl 9e3302149a369697c6aaea18b430b216e3c88f9a61b62869f6104881e5f9ef85 scipy-1.6.3-cp37-cp37m-manylinux1_i686.whl b79104878003487e2b4639a20b9092b02e1bad07fc4cf924b495cf413748a777 scipy-1.6.3-cp37-cp37m-manylinux1_x86_64.whl 44d452850f77e65e25b1eb1ac01e25770323a782bfe3a1a3e43847ad4266d93d scipy-1.6.3-cp37-cp37m-manylinux2014_aarch64.whl b30280fbc1fd8082ac822994a98632111810311a9ece71a0e48f739df3c555a2 scipy-1.6.3-cp37-cp37m-win32.whl 10dbcc7de03b8d635a1031cb18fd3eaa997969b64fdf78f99f19ac163a825445 scipy-1.6.3-cp37-cp37m-win_amd64.whl 1b21c6e0dc97b1762590b70dee0daddb291271be0580384d39f02c480b78290a scipy-1.6.3-cp38-cp38-macosx_10_9_x86_64.whl 1caade0ede6967cc675e235c41451f9fb89ae34319ddf4740194094ab736b88d scipy-1.6.3-cp38-cp38-manylinux1_i686.whl 19aeac1ad3e57338723f4657ac8520f41714804568f2e30bd547d684d72c392e scipy-1.6.3-cp38-cp38-manylinux1_x86_64.whl ad7269254de06743fb4768f658753de47d8b54e4672c5ebe8612a007a088bd48 scipy-1.6.3-cp38-cp38-manylinux2014_aarch64.whl d647757373985207af3343301d89fe738d5a294435a4f2aafb04c13b4388c896 scipy-1.6.3-cp38-cp38-win32.whl 33d1677d46111cfa1c84b87472a0274dde9ef4a7ef2e1f155f012f5f1e995d8f scipy-1.6.3-cp38-cp38-win_amd64.whl d449d40e830366b4c612692ad19fbebb722b6b847f78a7b701b1e0d6cda3cc13 scipy-1.6.3-cp39-cp39-macosx_10_9_x86_64.whl 23995dfcf269ec3735e5a8c80cfceaf384369a47699df111a6246b83a55da582 scipy-1.6.3-cp39-cp39-manylinux1_i686.whl fdf606341cd798530b05705c87779606fcdfaf768a8129c348ea94441da15b04 scipy-1.6.3-cp39-cp39-manylinux1_x86_64.whl f68eb46b86b2c246af99fcaa6f6e37c7a7a413e1084a794990b877f2ff71f7b6 scipy-1.6.3-cp39-cp39-manylinux2014_aarch64.whl 01b38dec7e9f897d4db04f8de4e20f0f5be3feac98468188a0f47a991b796055 scipy-1.6.3-cp39-cp39-win32.whl 3274ce145b5dc416c49c0cf8b6119f787f0965cd35e22058fe1932c09fe15d77 scipy-1.6.3-cp39-cp39-win_amd64.whl a75b014d3294fce26852a9d04ea27b5671d86736beb34acdfc05859246260707 scipy-1.6.3.tar.gz 3851fdcb1e6877241c3377aa971c85af0d44f90c57f4dd4e54e1b2bbd742635e scipy-1.6.3.tar.xz 0a723da627af61665c7793fdf869ad2606be9cf32e2c0abc0d230f62c4f4914f scipy-1.6.3.zip -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahoviyanikita at gmail.com Mon Apr 26 12:13:13 2021 From: mahoviyanikita at gmail.com (NIKITA MAHOVIYA) Date: Mon, 26 Apr 2021 21:43:13 +0530 Subject: [Numpy-discussion] Applying as a potential technical writer for 2021 Season of Docs Message-ID: Dear Mentor, I am a 3rd-year B.Tech undergraduate pursuing Dual Degree in Electronics and Communication Engineering at National Institute of Technology Hamirpur, Himachal Pradesh, India. I aspire to contribute to the open-source community with my technical writing skills that I have acquired by contributing to various projects. I have hands-on experience in handling WordPress websites and am currently the technical writing head of the leading page of the largest student community I AM AN NITIAN . During my sophomore year, I undertook a remote internship activity at MATLAB Helper as a MATLAB Developer Intern for four and a half months from 27th April 2020 to 9th September 2020. My internship activity included working in the field of MATLAB & promotional content creation. I have contributed to the organization with my blogs on GUIDE, SIR Modelling and webinar on Image Processing using Fuzzy Logic Toolbox. I have also worked on the revision of the online course of Neural Network. During this internship, I have used tools like MATLAB, Slack, Zoom, Trello, Canva, etc. Here I am attaching links to my writing work Blog: Creating a GUI in MATLAB https://matlabhelper.com/blog/matlab/creating-a-gui-in-matlab/ Blog: Image Processing Using Fuzzy Logic Toolbox https://matlabhelper.com/blog/matlab/image-processing-using-fuzzy-logic-toolbox/ Blog: COVID-19 SIR Modelling for Andhra Pradesh, India https://matlabhelper.com/blog/matlab/covid-19-sir-modelling-for-andhra-pradesh-india/ Blog: COVID-19 SIR Modelling for West Bengal, India https://matlabhelper.com/blog/matlab/covid-19-sir-modelling-for-west-bengal-india/ Medium https://mahoviyanikita.medium.com/ Website https://iamannitian.com/ Open Source Contribution: How do object behaviours and attributes differ PR#1326 Defining the correct usage of data types in Java PR#1340 Updated the documentation to reflect the current state of implementation of Adorsys PR#1204 Code for saying hello in different languages PR#5 Updated the README file of scorelab/GSoC PR#116 Updated the README file of the squiral library PR#4 I am interested in working on the project idea of Creating high-level documentation, in the Tutorial or How-to format, covering topics that are missing from the official documentation, organizing the content which is currently scattered through the reference documentation thereby structuring the documentation. I have exposure to documenting projects which involve beginning from the very initial stage to explaining the various code snippets and their uses and reaching the conclusion as can be seen from the blogs which I have developed. I would like to write relevant and easily readable documentation that could help people to explore the full potential of the services. I would be honoured if I get an opportunity to work on the documentation project which will be a pillar for future developers. The documentation will provide an understanding of the entire landscape of solutions. Please find my resume as an attachment with this mail. Waiting in anticipation. Yours sincerely Nikita Mahoviya 3rd Year B.Tech Undergraduate Department of Electronics and Communication Engineering National Institute of Technology Hamirpur (H.P.) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Resume_NikitaMahoviya.pdf Type: application/pdf Size: 140955 bytes Desc: not available URL: From melissawm at gmail.com Tue Apr 27 12:49:02 2021 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Tue, 27 Apr 2021 13:49:02 -0300 Subject: [Numpy-discussion] Applying as a potential technical writer for 2021 Season of Docs In-Reply-To: References: Message-ID: Hello, Nikita Thank you for your interest. It would be nice to discuss a bit more about concrete ideas you have regarding our project for NumPy. Can you elaborate on this? The deadline for hiring a technical writer is May 17, and before that all candidates must submit a Statement of Interest according to the GSoD website. We can talk before that to elaborate a more concrete proposal or discuss different ideas. If you have specific questions you can send them to numpy-scipy-gsod at googlegroups.com Cheers, Melissa On Mon, Apr 26, 2021 at 1:14 PM NIKITA MAHOVIYA wrote: > Dear Mentor, > > I am a 3rd-year B.Tech undergraduate pursuing Dual Degree in Electronics > and Communication Engineering at National Institute of Technology Hamirpur, > Himachal Pradesh, India. > > > > I aspire to contribute to the open-source community with my technical > writing skills that I have acquired by contributing to various projects. I > have hands-on experience in handling WordPress websites and am currently > the technical writing head of the leading page of the largest student > community I AM AN NITIAN . > > During my sophomore year, I undertook a remote internship activity at > MATLAB Helper as a MATLAB Developer Intern for four and a half months > from 27th April 2020 to 9th September 2020. My internship activity included > working in the field of MATLAB & promotional content creation. I have > contributed to the organization with my blogs on GUIDE, SIR Modelling and > webinar on Image Processing using Fuzzy Logic Toolbox. I have also worked > on the revision of the online course of Neural Network. During this > internship, I have used tools like MATLAB, Slack, Zoom, Trello, Canva, > etc. > > Here I am attaching links to my writing work > > Blog: Creating a GUI in MATLAB > > https://matlabhelper.com/blog/matlab/creating-a-gui-in-matlab/ > > Blog: Image Processing Using Fuzzy Logic Toolbox > > > https://matlabhelper.com/blog/matlab/image-processing-using-fuzzy-logic-toolbox/ > > > > Blog: COVID-19 SIR Modelling for Andhra Pradesh, India > > > https://matlabhelper.com/blog/matlab/covid-19-sir-modelling-for-andhra-pradesh-india/ > > > > Blog: COVID-19 SIR Modelling for West Bengal, India > > > https://matlabhelper.com/blog/matlab/covid-19-sir-modelling-for-west-bengal-india/ > > Medium > > https://mahoviyanikita.medium.com/ > > Website > > https://iamannitian.com/ > > Open Source Contribution: > > How do object behaviours and attributes differ PR#1326 > > > Defining the correct usage of data types in Java PR#1340 > > > > Updated the documentation to reflect the current state of implementation > of Adorsys PR#1204 > > > Code for saying hello in different languages PR#5 > > > Updated the README file of scorelab/GSoC PR#116 > > > Updated the README file of the squiral library PR#4 > > > I am interested in working on the project idea of Creating high-level > documentation, in the Tutorial or How-to format, covering topics that are > missing from the official documentation, organizing the content which is > currently scattered through the reference documentation thereby structuring > the documentation. I have exposure to documenting projects which involve > beginning from the very initial stage to explaining the various code > snippets and their uses and reaching the conclusion as can be seen from the > blogs which I have developed. I would like to write relevant and easily > readable documentation that could help people to explore the full potential > of the services. > > I would be honoured if I get an opportunity to work on the documentation > project which will be a pillar for future developers. The documentation > will provide an understanding of the entire landscape of solutions. > > Please find my resume as an attachment with this mail. > > Waiting in anticipation. > > Yours sincerely > Nikita Mahoviya > 3rd Year B.Tech Undergraduate > Department of Electronics and Communication Engineering > National Institute of Technology Hamirpur (H.P.) > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Tue Apr 27 23:14:26 2021 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 27 Apr 2021 22:14:26 -0500 Subject: [Numpy-discussion] NumPy Community Meeting Wednesday Message-ID: Hi all, There will be a NumPy Community meeting Wednesday April 28th at 20:00 UTC. Everyone is invited and encouraged to join in and edit the work-in-progress meeting topics and notes at: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian From mahesh6947foss at gmail.com Tue Apr 27 23:31:37 2021 From: mahesh6947foss at gmail.com (Mahesh S) Date: Wed, 28 Apr 2021 09:01:37 +0530 Subject: [Numpy-discussion] GSoD 2021 - Statement of Interest Message-ID: Hello, I am Mahesh from India. I am interested in doing Google Season of Docs with NumPy on the project HIGH-LEVEL RESTRUCTURING AND END-USER FOCUS - NumPy I have past experience in documentation writing with WordPress and have completed *Google Summer of Code 2018 *with KDE. I have been an open source enthusiast and contributor since 2017.My past experience with coding ,code documentation , understanding code bases ,will help achieve this task without much input from your end. I have about four years experience working with Open-Source and currently working as a *Quality Assurance specialis*t . I have delivered technical talks at international conferences (KDE-Akademy) and was *mentor of Google Code-In* also. All these makes me the best candidate for the project. I am attaching a brief proposal for the work along with this mail. A more in-depth A more in depth proposal with timelines , all planned strategies , document structures will be submitted after more discussion with the mentors and community. I am ready to start the work as soon as possible and will be able work 7 to 9 hours per day including weekends. I strongly wish to be part of the NumPy community even after the project. Hope to hear from you soon -- Thank you Mahesh S Nair -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Enhancing NumPy Documentation- GSoD 2021- Brief Proposal.pdf Type: application/pdf Size: 55770 bytes Desc: not available URL: From melissawm at gmail.com Thu Apr 29 12:23:15 2021 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Thu, 29 Apr 2021 13:23:15 -0300 Subject: [Numpy-discussion] GSoD 2021 - Statement of Interest In-Reply-To: References: Message-ID: Hello, Mahesh Thank you for your proposal. One thing I would say is that some of the things you are suggesting are already there - for example, the How-tos and Explanations section - but I agree they can be improved! Another question I could pose is about duplication of content: can you give examples of duplicated content that you found in the docs? Cheers, Melissa On Wed, Apr 28, 2021 at 12:33 AM Mahesh S wrote: > Hello, > > I am Mahesh from India. I am interested in doing Google Season of Docs > with NumPy on the project HIGH-LEVEL RESTRUCTURING AND END-USER FOCUS - > NumPy > > I have past experience in documentation writing with WordPress and have > completed *Google Summer of Code 2018 *with KDE. I have been an open > source enthusiast and contributor since 2017.My past experience with coding > ,code documentation , understanding code bases ,will help achieve this task > without much input from your end. I have about four years experience > working with Open-Source and currently working as a *Quality > Assurance specialis*t . I have delivered technical talks at international > conferences (KDE-Akademy) and was *mentor of Google Code-In* also. All > these makes me the best candidate for the project. > > I am attaching a brief proposal for the work along with this mail. A more > in-depth A more in depth proposal with timelines , all planned strategies > , document structures will be submitted after more discussion with the > mentors and community. > I am ready to start the work as soon as possible and will be able work 7 > to 9 hours per day including weekends. I strongly wish to be part of the > NumPy community even after the project. > > > Hope to hear from you soon > > -- > Thank you > Mahesh S Nair > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From melissawm at gmail.com Thu Apr 29 15:38:30 2021 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Thu, 29 Apr 2021 16:38:30 -0300 Subject: [Numpy-discussion] NumPy Tutorials Message-ID: Hello, all! Recently, we added a link to NumPy Tutorials to our main page ( https://numpy.org/numpy-tutorials) This set of tutorials has been built with the collaboration of a few contributors, who have put effort and dedication into them. Some of this content was generated as part of last year's Google Season of Docs program, with authors kubedoc and Ryan Cooper. These tutorials are meant to be not only about NumPy, but about using NumPy to solve concrete problems, which means more flexibility in terms of content and dependencies for executing them. They are also executable as Jupyter Notebooks, and contributors can submit their content as notebooks. This hopefully means a lower barrier of entry for some people, and allows us to have interactive materials online (with Binder). We also have a dedicated tutorials section in our main documentation, which should serve a different purpose (for example, focusing on concepts and operations specific to NumPy). The Documentation Team is growing and we would be delighted to have your help. If you have comments, suggestions for improvements or other content, please open an issue or PR in the https://github.com/numpy/numpy-tutorials repo. Special thanks to Ross Barnowski who has also put a lot of work into this new site. Cheers! Melissa -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Fri Apr 30 02:19:02 2021 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Fri, 30 Apr 2021 08:19:02 +0200 Subject: [Numpy-discussion] NumPy Tutorials In-Reply-To: References: Message-ID: Thanks Melissa, Ryan, Olz, Ross, and everyone else who contributed! This site looks really nice, and the content is a big improvement on what we had before as tutorials! Cheers, Ralf On Thu, Apr 29, 2021 at 9:39 PM Melissa Mendon?a wrote: > Hello, all! > > Recently, we added a link to NumPy Tutorials to our main page ( > https://numpy.org/numpy-tutorials) > > This set of tutorials has been built with the collaboration of a few > contributors, who have put effort and dedication into them. Some of this > content was generated as part of last year's Google Season of Docs program, > with authors kubedoc and Ryan Cooper. > > These tutorials are meant to be not only about NumPy, but about using > NumPy to solve concrete problems, which means more flexibility in terms of > content and dependencies for executing them. They are also executable as > Jupyter Notebooks, and contributors can submit their content as notebooks. > This hopefully means a lower barrier of entry for some people, and allows > us to have interactive materials online (with Binder). We also have a > dedicated tutorials section in our main documentation, which should serve a > different purpose (for example, focusing on concepts and operations > specific to NumPy). > > The Documentation Team is growing and we would be delighted to have your > help. If you have comments, suggestions for improvements or other content, > please open an issue or PR in the https://github.com/numpy/numpy-tutorials > repo. > > Special thanks to Ross Barnowski who has also put a lot of work into this > new site. > > Cheers! > > Melissa > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahesh6947foss at gmail.com Fri Apr 30 08:26:47 2021 From: mahesh6947foss at gmail.com (Mahesh S) Date: Fri, 30 Apr 2021 17:56:47 +0530 Subject: [Numpy-discussion] GSoD 2021 - Statement of Interest In-Reply-To: References: Message-ID: Hi there, The current documentation of NumPy is really good but a bit more improvement can be made to it, which is the prime objective of my project. The improvements which I mentioned in my brief proposal are strategies that can be applied to every documentation to make it better. Apart from general improvements most the documentation related issues in the NumPy's GitHub issue tracker will be addressed. . Some needs more technical information and help from the community. Some are due to the lack of visual aids. Most of them will be addressed and improvements will be made such that similar issues will not be generated in future. Rearranging of sections in the User Guide can be done after further discussions Some examples regarding the need of restructuring and duplication are given in the attached document. Apologies in advance if my observations are inaccurate or nitpicks. The given doc is just a very brief one. An in-depth proposal with all the planned changes along with solutions to close as many issues in the tracker will be prepared. I am getting familiar with the community and NumPy. On Thu, Apr 29, 2021 at 9:54 PM Melissa Mendon?a wrote: > Hello, Mahesh > > Thank you for your proposal. One thing I would say is that some of the > things you are suggesting are already there - for example, the How-tos and > Explanations section - but I agree they can be improved! > > Another question I could pose is about duplication of content: can you > give examples of duplicated content that you found in the docs? > > Cheers, > > Melissa > > On Wed, Apr 28, 2021 at 12:33 AM Mahesh S > wrote: > >> Hello, >> >> I am Mahesh from India. I am interested in doing Google Season of Docs >> with NumPy on the project HIGH-LEVEL RESTRUCTURING AND END-USER FOCUS - >> NumPy >> >> I have past experience in documentation writing with WordPress and have >> completed *Google Summer of Code 2018 *with KDE. I have been an open >> source enthusiast and contributor since 2017.My past experience with coding >> ,code documentation , understanding code bases ,will help achieve this task >> without much input from your end. I have about four years experience >> working with Open-Source and currently working as a *Quality >> Assurance specialis*t . I have delivered technical talks at >> international conferences (KDE-Akademy) and was *mentor of Google >> Code-In* also. All these makes me the best candidate for the project. >> >> I am attaching a brief proposal for the work along with this mail. A more >> in-depth A more in depth proposal with timelines , all planned strategies >> , document structures will be submitted after more discussion with the >> mentors and community. >> I am ready to start the work as soon as possible and will be able work 7 >> to 9 hours per day including weekends. I strongly wish to be part of the >> NumPy community even after the project. >> >> >> Hope to hear from you soon >> >> -- >> Thank you >> Mahesh S Nair >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -- Thank you Mahesh S Nair -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Some Examples for improvement.pdf Type: application/pdf Size: 91679 bytes Desc: not available URL: From mikofski at berkeley.edu Fri Apr 30 16:31:02 2021 From: mikofski at berkeley.edu (Dr. Mark Alexander Mikofski PhD) Date: Fri, 30 Apr 2021 13:31:02 -0700 Subject: [Numpy-discussion] Fwd: PhD position in the Netherlands; solar energy/Python In-Reply-To: <481b3153-21e8-4450-8710-53f38b175af7n@googlegroups.com> References: <2076294944.9931.1618395112339.JavaMail.bbuser@fgprd-309451-198615-app002> <481b3153-21e8-4450-8710-53f38b175af7n@googlegroups.com> Message-ID: Hello, sharing a PhD position opening at Utrecht in Netherlands. Read more below... ---------- Forwarded message --------- From: Mark Mikofski Date: Fri, Apr 30, 2021, 12:29 PM Subject: Fwd: PhD position in the Netherlands; solar energy/Python To: Mark Alexander Mikofski PhD positions with Wilfried van Sark at Utrecht Netherlands ---------- Forwarded message --------- From: abedw... at googlemail.com Date: Monday, April 19, 2021 at 9:22:00 AM UTC-7 Subject: PhD position in the Netherlands; solar energy/Python To: pvlib-python Hello all, I did my MSc with Wilfried at Utrecht (the Netherlands). He's a truly wonderful mentor with an amazing and broad grasp of the knowledge. 10/10 would recommend doing a PhD there if you or someone you know would like to develop solar forecasting models. They are looking for pythonistas. All the best, Abed ---------- Forwarded message --------- From: dr. Wilfried van Sark - W.G.J.H.... at uu.nl Date: Wed, Apr 14, 2021 at 4:11 AM Subject: Master Energy Science: Looking for PhD/junior researchers To: Dear all, I am looking for talented and motivated junior researchers that would be interested to work in two projects: 1) ROBUST project: this recently granted project is focused on investigating options for flexibility in local electricity grids by using vehicle-to-grid technology in neighborhoods in Utrecht and Arnhem. We require experience with data analysis and simulation using Python. In this project we collaborate with WeDriveSolar and TUDelft. We have funding for at least 44 months available. 2) PVObs and SolFaSi projects. Both projects deal with photovoltaic performance and forecasting. The PVObs project aims to create a large database of PV systems in the Netherlands in order to assess their performance and from that determine with better accuracy the contribution of PV to the national electricity demand. Parnters in this project are CBS and SunData. The SolFaSi project aims to develop PV forecasting at high time and spatial resolutions using all-sky imagers (aka cloud cameras). We will install three cameras at the UU campus and one at KNMI. This network allows to construct a moving shadow field over the campus. This should be correlated with the power generation of the 4000+ PV panels at campus buildings. We require experience with (big) data analysis and simulation using Python. We have funding for 24 months available. If you are interested, or if you know someone who may be interested, please send me a mail at w.g.j.h.... at uu.nl best regards Wilfried -------------- next part -------------- An HTML attachment was scrubbed... URL: