From millman at berkeley.edu Tue Dec 1 18:53:14 2020 From: millman at berkeley.edu (Jarrod Millman) Date: Tue, 1 Dec 2020 15:53:14 -0800 Subject: [Numpy-discussion] Officially drop Python 3.6 from NumPy 1.20 (was: NumPy 1.20.x branch in two weeks) In-Reply-To: <1e974b9d8d746dac2a79ecb507ea180788e04404.camel@sipsolutions.net> References: <65b3fe9e-6f6a-4c5e-943e-e5747076eb0d@www.fastmail.com> <1e974b9d8d746dac2a79ecb507ea180788e04404.camel@sipsolutions.net> Message-ID: Hi all, It looks like Python 3.6 support has not been dropped. For example, - https://github.com/numpy/numpy/blob/master/setup.py#L30 - https://github.com/numpy/numpy/blob/master/setup.py#L43 - https://github.com/numpy/numpy/blob/maintenance/1.20.x/setup.py#L29 - https://github.com/numpy/numpy/blob/maintenance/1.20.x/setup.py#L43 Other files (e.g., tox.ini) also make it appear that 3.6 has not been dropped. Did you decide to not drop Python 3.6? (Sorry if I missed the discussion.) Best regards, Jarrod PS. I am trying to decide whether NetworkX should drop 3.6 and prefer to follow NumPy's lead rather than NEP 29. On Thu, Nov 5, 2020 at 7:28 AM Sebastian Berg wrote: > > Hi all, > > just to note: We discussed this yesterday briefly and decided to drop > official support for 3.6 in the 1.20 release. We never had ambition to > support 1.20 and there seems advantage in dropping it, if mainly for > clarity and consistency with many other projects. > > If you disagree with this decision, please just bring it up so we can > reconsider. > > Cheers, > > Sebastian > > > PS: We may keep testing on 3.6 for the moment, at least for PyPy for > technical reasons. > > > > On Tue, 2020-11-03 at 11:58 -0800, Brigitta Sipocz wrote: > > Hi, > > > > For what it's worth, python 3.6 is also dropped for astropy 4.2 (RC1 > > to be > > released in the next few days). We haven't yet formally adopted > > NEP29, but > > are very close to it peding some word smithing, and no one from the > > dev > > team was fighting for keeping support for 3.6. or numpy 1.16. > > > > Cheers, > > Brigitta > > > > On Tue, 3 Nov 2020 at 10:53, Thomas Caswell > > wrote: > > > > > I am in favor of dropping py36 for np1.20, I think it would be good > > > to > > > lead by example. > > > > > > Similar to pandas, the next Matplotlib release (3.4 targeted for > > > Dec/Jan) > > > will not support py36. > > > > > > Tom > > > > > > > > > > > > On Tue, Nov 3, 2020 at 9:18 AM Mark Harfouche < > > > mark.harfouche at gmail.com> > > > wrote: > > > > > > > Juan made a pretty good argument for keeping 3.6 support in the > > > > next > > > > scikit-image release, let me try to paraphrase: > > > > > > > > - Since nobody has made the PR to explicitly drop python 3.6 from > > > > the > > > > scikit-image build matrix, we will continue to support it, but if > > > > somebody > > > > were to make the PR, I (Juan) would support it. > > > > > > > > As for supporting PyPy: it already exists in the build matrix > > > > AFAICT. > > > > Breaking PyPy would be a deliberate action, as opposed to an > > > > accidental > > > > byproduct of dropping CPython 3.6. > > > > > > > > On Mon, Nov 2, 2020, 13:50 Sebastian Berg < > > > > sebastian at sipsolutions.net> > > > > wrote: > > > > > > > > > On Mon, 2020-11-02 at 06:49 -0600, Juan Nunez-Iglesias wrote: > > > > > > I like Ralf's email, and most of all I agree that the > > > > > > existing > > > > > > wording is clearer. > > > > > > > > > > > > My view on the NEP is that it does not mandate dropping > > > > > > support, but > > > > > > encourage it. In my projects I would drop it if I had use for > > > > > > Python > > > > > > 3.7+ features. It so happens that we want to use PEP-593 so > > > > > > we were > > > > > > grateful for NEP-29 giving us "permission" to drop 3.6. > > > > > > > > > > > > I would suggest that 3.6 be dropped immediately if there are > > > > > > any open > > > > > > PRs that would benefit from it, or code cleanups that it > > > > > > would > > > > > > enable. The point of the NEP is to short-circuit discussion > > > > > > about > > > > > > whether it's "worth" dropping 3.6. If it's valuable at all, > > > > > > do it. > > > > > > > > > > > > > > > > Probably the only thing that requires 3.7 in NumPy at this time > > > > > is the > > > > > module level `__getattr__`, which is used only for deprecations > > > > > (and to > > > > > make the financial removal slightly more gentle). > > > > > I am not sure if PyPy already has stable support for 3.7 yet? > > > > > Although > > > > > PyPy is maybe not a big priority. > > > > > > > > > > We don't have to support 3.6 and I don't care if we do. Until > > > > > this > > > > > discussion my assumption was we would probably drop it. > > > > > > > > > > But, current master is tested against 3.6, so the main work > > > > > seems > > > > > release related. If Chuck thinks that is no hassle I don't mind > > > > > if > > > > > NumPy is a bit more conservative than NEP 29. > > > > > > > > > > Or is there a danger of setting a precedent where projects are > > > > > wrongly > > > > > expected to keep support just because NumPy still has it, so > > > > > that NumPy > > > > > not being conservative actually helps everyone? > > > > > > > > > > - Sebastian > > > > > > > > > > > > > > > > Thanks all, > > > > > > > > > > > > Juan. > > > > > > > > > > > > On Mon, 2 Nov 2020, at 2:01 AM, Ralf Gommers wrote: > > > > > > > On Mon, Nov 2, 2020 at 7:47 AM Stephan Hoyer < > > > > > > > shoyer at gmail.com> > > > > > > > wrote: > > > > > > > > On Sun, Nov 1, 2020 at 7:47 PM Stefan van der Walt < > > > > > > > > stefanv at berkeley.edu> wrote: > > > > > > > > > On Sun, Nov 1, 2020, at 18:54, Jarrod Millman wrote: > > > > > > > > > > I also misunderstood the purpose of the NEP. I > > > > > > > > > > assumed it > > > > > > > > > > was > > > > > > > > > > intended to encourage projects to drop old versions > > > > > > > > > > of > > > > > > > > > > Python. > > > > > > > > > > > > > > It was. It is. I think the NEP is very clear on that. > > > > > > > Honestly we > > > > > > > should just follow the NEP and drop 3.6 now for both NumPy > > > > > > > and > > > > > > > SciPy, I just am tired of arguing for it - which the NEP > > > > > > > should > > > > > > > have prevented being necessary, and I don't want to do > > > > > > > again right > > > > > > > now, so this will probably be my last email on this thread. > > > > > > > > > > > > > > > > > > > > > > > Other > > > > > > > > > > people have viewed the NEP similarly: > > > > > > > > > > https://github.com/networkx/networkx/issues/4027 > > > > > > > > > > > > > > > > > > Of all the packages, it makes sense for NumPy to behave > > > > > > > > > most > > > > > > > > > conservatively with depreciations. The NEP suggests > > > > > > > > > allowable > > > > > > > > > support periods, but as far as I recall does not > > > > > > > > > enforce > > > > > > > > > minimal support. > > > > > > > > > > > > > > It doesn't *enforce* it, but the recommendation is very > > > > > > > clear. It > > > > > > > would be good to follow it. > > > > > > > > > > > > > > > > Stephan Hoyer had a good recommendation on how we can > > > > > > > > > clarify > > > > > > > > > the NEP to be easier to intuit. Stephan, shall we make > > > > > > > > > an > > > > > > > > > ammendment to the NEP with your idea? > > > > > > > > > > > > > > > > For reference, here was my proposed revision: > > > > > > > > https://github.com/numpy/numpy/pull/14086#issuecomment-649287648 > > > > > > > > Specifically, rather than saying "the latest release of > > > > > > > > NumPy > > > > > > > > supports all versions of Python released in the 42 months > > > > > > > > before > > > > > > > > NumPy's release", it says "NumPy will only require > > > > > > > > versions of > > > > > > > > Python that were released more than 24 months ago". In > > > > > > > > practice, > > > > > > > > this works out to the same thing (at least given Python's > > > > > > > > old 18 > > > > > > > > month release cycle). > > > > > > > > > > > > > > > > This changes the definition of the support window (in a > > > > > > > > way that > > > > > > > > I think is clearer and that works better for infrequent > > > > > > > > releases), but there is still the question of how large > > > > > > > > that > > > > > > > > window should be for NumPy. > > > > > > > > > > > > > > I'm not sure it's clearer, the current NEP has a nice > > > > > > > graphic and > > > > > > > literally says "a project with a major or minor version > > > > > > > release in > > > > > > > November 2020 should support Python 3.7 and newer."). > > > > > > > However happy > > > > > > > to adopt it if it makes others happy - in the end it comes > > > > > > > down to > > > > > > > the same thing: it's recommended to drop Python 3.6 now. > > > > > > > > > > > > > > > My personal opinion is that somewhere in the range of 24- > > > > > > > > 36 > > > > > > > > months would be appropriate. > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > Cheers, > > > > > > > Ralf > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > NumPy-Discussion mailing list > > > > > > > NumPy-Discussion at python.org > > > > > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > NumPy-Discussion mailing list > > > > > > NumPy-Discussion at python.org > > > > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > > > > > _______________________________________________ > > > > > NumPy-Discussion mailing list > > > > > NumPy-Discussion at python.org > > > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > > > > _______________________________________________ > > > > NumPy-Discussion mailing list > > > > NumPy-Discussion at python.org > > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > > > > > -- > > > Thomas Caswell > > > tcaswell at gmail.com > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion at python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Dec 1 19:17:50 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 1 Dec 2020 17:17:50 -0700 Subject: [Numpy-discussion] Officially drop Python 3.6 from NumPy 1.20 (was: NumPy 1.20.x branch in two weeks) In-Reply-To: References: <65b3fe9e-6f6a-4c5e-943e-e5747076eb0d@www.fastmail.com> <1e974b9d8d746dac2a79ecb507ea180788e04404.camel@sipsolutions.net> Message-ID: On Tue, Dec 1, 2020 at 4:54 PM Jarrod Millman wrote: > Hi all, > > It looks like Python 3.6 support has not been dropped. For example, > - https://github.com/numpy/numpy/blob/master/setup.py#L30 > - https://github.com/numpy/numpy/blob/master/setup.py#L43 > - https://github.com/numpy/numpy/blob/maintenance/1.20.x/setup.py#L29 > - https://github.com/numpy/numpy/blob/maintenance/1.20.x/setup.py#L43 > > Other files (e.g., tox.ini) also make it appear that 3.6 has not been > dropped. > > Did you decide to not drop Python 3.6? (Sorry if I missed the discussion.) > > Best regards, > Jarrod > > PS. I am trying to decide whether NetworkX should drop 3.6 and prefer > to follow NumPy's lead rather than NEP 29. > Yes, we did decide to drop 3.6 support, but apparently we need a checklist when dropping support :) It is gone from testing and the wheel builds. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Tue Dec 1 23:58:45 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 01 Dec 2020 22:58:45 -0600 Subject: [Numpy-discussion] NumPy Development Meeting Wednesday - Triage Focus Message-ID: <79ab9fac3098efce52269bd97d28e235583efa87.camel@sipsolutions.net> Hi all, Our bi-weekly triage-focused NumPy development meeting is today (Wednesday, November 18th) at 11 am Pacific Time (19:00 UTC). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From sebastian at sipsolutions.net Wed Dec 2 18:34:46 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Wed, 02 Dec 2020 17:34:46 -0600 Subject: [Numpy-discussion] Rules for argument parsing/forwarding in __array_function__ and __array_ufunc__ Message-ID: Hi all, I am curious about the correct argument for "normalizing" and forwarding arguments in `__array_function__` and `__array_ufunc__`. I have listed the three rules as I understand how the "right" way is below. Mainly this is just to write down the rules that I think we should aim for in case it comes up. I admit, I do have "hidden agendas" where it may come up: * pytorch breaks rule 2 in their copy of `__array_function__`, because it allows to easily optimize away some overheads. * `__array_ufunc__` breaks rule 1 (it allows too much) and I think we should be OK to change that [1]. * A few `__array_function__`s break rule 3. That is completely harmless, but it might be nice to just be clear whether we consider it technically wrong. [2] Rules ----- 1. If an argument is invalid in NumPy it is considered and error. For example: np.log(arr, my_weird_argument=True) is always an error even if the `__array_function__` implementation of `arr` would support it. NEP 18 explicitly says that allowing forwarding could be done, but will not be done at this time. 2. Arguments must only be forwarded if they are passed in: np.mean(cupy_array) ends up as `cupy.mean(cupy_array)` and not: cupy.mean(cupy_array, axis=None, dtype=None, out=None, keepdims=False, where=True) meaning that CuPy does not need to implement all of those kwargs and NumPy can add new ones without breaking anyones code. 3. NumPy should not check the *validity* of the arguments. For example: `np.add.reduce(xarray, axis="long")` should probably work in xarray. (`xarray.DataArray` does not actually implement the above.) But a string cannot be used as an axis in NumPy. Cheers, Sebastian [1] I think `dask` breaks this rule by using an `output_dtypes` keyword. I would just consider this a valid exception and keep allowing it. In fact, `output_dtypes` may very well be a useful argument for NumPy itself. `dtype` looks like it serves that purpose, but it does not have quite the same meaning. This has been discussed also here: https://github.com/numpy/numpy/issues/8892 [2] This is done for performance reasons, although it is entirely avoidable. However, avoiding it might just add a bunch of annoying code unless part of a larger maintenance effort. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From alan.isaac at gmail.com Wed Dec 2 18:56:26 2020 From: alan.isaac at gmail.com (Alan G. Isaac) Date: Wed, 2 Dec 2020 18:56:26 -0500 Subject: [Numpy-discussion] problem with numpy 1.19.4 install via pip on Win 10 In-Reply-To: References: Message-ID: numpy 1.19.3 installs fine. numpy 1.19.4 appears to install but does not work. (Details below. The supplied tinyurl appears relevant.) Alan Isaac PS test> python38 -m pip install -U numpy Collecting numpy Using cached numpy-1.19.4-cp38-cp38-win_amd64.whl (13.0 MB) Installing collected packages: numpy Successfully installed numpy-1.19.4 PS test> python38 Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy ** On entry to DGEBAL parameter number 3 had an illegal value ** On entry to DGEHRD parameter number 2 had an illegal value ** On entry to DORGHR DORGQR parameter number 2 had an illegal value ** On entry to DHSEQR parameter number 4 had an illegal value Traceback (most recent call last): File "", line 1, in File "C:\Program Files\Python38\lib\site-packages\numpy\__init__.py", line 305, in _win_os_check() File "C:\Program Files\Python38\lib\site-packages\numpy\__init__.py", line 302, in _win_os_check raise RuntimeError(msg.format(__file__)) from None RuntimeError: The current Numpy installation ('C:\\Program Files\\Python38\\lib\\site-packages\\numpy\\__init__.py') fails to pass a sanity check due to a bug in the windows runtime. See this issue for more information: https://tinyurl.com/y3dm3h86 >>> From ilhanpolat at gmail.com Wed Dec 2 19:13:10 2020 From: ilhanpolat at gmail.com (Ilhan Polat) Date: Thu, 3 Dec 2020 01:13:10 +0100 Subject: [Numpy-discussion] problem with numpy 1.19.4 install via pip on Win 10 In-Reply-To: References: Message-ID: Yes this is known and we are waiting MS to roll out a solution for this. Here are more details https://developercommunity2.visualstudio.com/t/fmod-after-an-update-to-windows-2004-is-causing-a/1207405 On Thu, Dec 3, 2020 at 12:57 AM Alan G. Isaac wrote: > numpy 1.19.3 installs fine. > numpy 1.19.4 appears to install but does not work. > (Details below. The supplied tinyurl appears relevant.) > Alan Isaac > > PS test> python38 -m pip install -U numpy > Collecting numpy > Using cached numpy-1.19.4-cp38-cp38-win_amd64.whl (13.0 MB) > Installing collected packages: numpy > Successfully installed numpy-1.19.4 > PS test> python38 > Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 > bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > ** On entry to DGEBAL parameter number 3 had an illegal value > ** On entry to DGEHRD parameter number 2 had an illegal value > ** On entry to DORGHR DORGQR parameter number 2 had an illegal value > ** On entry to DHSEQR parameter number 4 had an illegal value > Traceback (most recent call last): > File "", line 1, in > File "C:\Program Files\Python38\lib\site-packages\numpy\__init__.py", > line 305, in > _win_os_check() > File "C:\Program Files\Python38\lib\site-packages\numpy\__init__.py", > line 302, in _win_os_check > raise RuntimeError(msg.format(__file__)) from None > RuntimeError: The current Numpy installation ('C:\\Program > Files\\Python38\\lib\\site-packages\\numpy\\__init__.py') fails to pass a > sanity check due to a bug > in the windows runtime. See this issue for more information: > https://tinyurl.com/y3dm3h86 > >>> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Wed Dec 2 19:37:26 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Wed, 02 Dec 2020 18:37:26 -0600 Subject: [Numpy-discussion] problem with numpy 1.19.4 install via pip on Win 10 In-Reply-To: References: Message-ID: <247f1d089db3a94c2637a82ec0c6f6726bc69842.camel@sipsolutions.net> On Thu, 2020-12-03 at 01:13 +0100, Ilhan Polat wrote: > Yes this is known and we are waiting MS to roll out a solution for > this. > Here are more details > https://developercommunity2.visualstudio.com/t/fmod-after-an-update-to-windows-2004-is-causing-a/1207405 I think one workaround was `pip install numpy==1.19.3`, which uses a different OpenBLAS that avoids the worst of the windows bug. - Sebastian > > On Thu, Dec 3, 2020 at 12:57 AM Alan G. Isaac > wrote: > > > numpy 1.19.3 installs fine. > > numpy 1.19.4 appears to install but does not work. > > (Details below. The supplied tinyurl appears relevant.) > > Alan Isaac > > > > PS test> python38 -m pip install -U numpy > > Collecting numpy > > Using cached numpy-1.19.4-cp38-cp38-win_amd64.whl (13.0 MB) > > Installing collected packages: numpy > > Successfully installed numpy-1.19.4 > > PS test> python38 > > Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC > > v.1916 64 > > bit (AMD64)] on win32 > > Type "help", "copyright", "credits" or "license" for more > > information. > > >>> import numpy > > ** On entry to DGEBAL parameter number 3 had an illegal value > > ** On entry to DGEHRD parameter number 2 had an illegal value > > ** On entry to DORGHR DORGQR parameter number 2 had an illegal > > value > > ** On entry to DHSEQR parameter number 4 had an illegal value > > Traceback (most recent call last): > > File "", line 1, in > > File "C:\Program Files\Python38\lib\site- > > packages\numpy\__init__.py", > > line 305, in > > _win_os_check() > > File "C:\Program Files\Python38\lib\site- > > packages\numpy\__init__.py", > > line 302, in _win_os_check > > raise RuntimeError(msg.format(__file__)) from None > > RuntimeError: The current Numpy installation ('C:\\Program > > Files\\Python38\\lib\\site-packages\\numpy\\__init__.py') fails to > > pass a > > sanity check due to a bug > > in the windows runtime. See this issue for more information: > > https://tinyurl.com/y3dm3h86 > > >>> > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From drtodd13 at comcast.net Wed Dec 2 21:07:07 2020 From: drtodd13 at comcast.net (drtodd13) Date: Wed, 2 Dec 2020 19:07:07 -0700 (MST) Subject: [Numpy-discussion] Precompute shape of ufunc output. Message-ID: <1606961227112-0.post@n7.nabble.com> If I want to provide the "out" kwarg to, for example, a reduce ufunc then I need to know the shape of the output given the other set of inputs. Is there a utility function to take these arguments and just compute what the shape of the output is going to be without actually computing the result? Would be nice if there were a standard kwarg for many functions that just tells you the shape of the output without actually computing it but I don't think that exists. Any ideas appreciated. Thanks. -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From stefanv at berkeley.edu Thu Dec 3 00:07:32 2020 From: stefanv at berkeley.edu (Stefan van der Walt) Date: Wed, 02 Dec 2020 21:07:32 -0800 Subject: [Numpy-discussion] =?utf-8?q?Rules_for_argument_parsing/forwardi?= =?utf-8?b?bmcgaW4gX19hcnJheV9mdW5jdGlvbl9fIGFuZCBfX2FycmF5X3VmdW5jX18=?= In-Reply-To: References: Message-ID: Hi Sebastian, Looking at these three rules, they all seem to stem from one simple question: do we desire for a single code snippet to be runnable on multiple array implementations? On Wed, Dec 2, 2020, at 15:34, Sebastian Berg wrote: > 1. If an argument is invalid in NumPy it is considered and error. > For example: > > np.log(arr, my_weird_argument=True) > > is always an error even if the `__array_function__` implementation > of `arr` would support it. > NEP 18 explicitly says that allowing forwarding could be done, but > will not be done at this time. Relaxing this rule will mean that code working for one array implementation (which has this keyword) may not work for another. > 2. Arguments must only be forwarded if they are passed in: > > np.mean(cupy_array) > > ends up as `cupy.mean(cupy_array)` and not: > > cupy.mean(cupy_array, axis=None, dtype=None, out=None, > keepdims=False, where=True) > > meaning that CuPy does not need to implement all of those kwargs and > NumPy can add new ones without breaking anyones code. This may ultimately make it harder for array implementors (they will only see errors once someone tries to pass in an argument that they forgot to implement). Perhaps better to pass all so they know what they're dealing with? > 3. NumPy should not check the *validity* of the arguments. For example: > `np.add.reduce(xarray, axis="long")` should probably work in xarray. > (`xarray.DataArray` does not actually implement the above.) > But a string cannot be used as an axis in NumPy. Getting back to the original question: if this code is to be run on multiple implementations, we should ensure that no strange values pass through. Personally, I like the idea of a single API that works on multiple backends. As such, I would 1) not pass through unknown arguments, 2) always pass through all arguments, and 3) validate inputs to each call. Best regards, St?fan -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.isaac at gmail.com Thu Dec 3 16:14:08 2020 From: alan.isaac at gmail.com (Alan G. Isaac) Date: Thu, 3 Dec 2020 16:14:08 -0500 Subject: [Numpy-discussion] problem with numpy 1.19.4 install via pip on Win 10 In-Reply-To: References: Message-ID: <7733f968-2586-1e50-3d58-862807fed41b@gmail.com> "current expectation is that this [fix] will be able to be released near the end of January 2021" ! On 12/2/2020 7:13 PM, Ilhan Polat wrote: > Yes this is known and we are waiting MS to roll out a solution for this. Here are more details > https://developercommunity2.visualstudio.com/t/fmod-after-an-update-to-windows-2004-is-causing-a/1207405 > From charlesr.harris at gmail.com Thu Dec 3 16:34:40 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 3 Dec 2020 14:34:40 -0700 Subject: [Numpy-discussion] NumPy 1.20.0rc1 released Message-ID: Hi All, On behalf of the NumPy team I am pleased to announce the release of NumPy 1.20.0rc1. This NumPy release is the largest to date, containing some 654 merged pull requests contributed by 182 people. See the list of highlights below. The Python versions supported for this release are 3.7-3.9, support for Python 3.6 has been dropped. Wheels can be downloaded from PyPI ; source archives, release notes, and wheel hashes are available on Github . Linux users will need pip >= 0.19.3 in order to install manylinux2010 and manylinux2014 wheels. *Highlights* - Annotations for NumPy functions. This work is ongoing and improvements can be expected pending feedback from users. - Wider use of SIMD to increase execution speed of ufuncs. Much work has been done in introducing universal functions that will ease use of modern features across different hardware platforms. This work is ongoing. - Preliminary work in changing the dtype and casting implementations in order to provide an easier path to extending dtypes. This work is ongoing but enough has been done to allow experimentation and feedback. - Extensive documentation improvements comprising some 185 PR merges. This work is ongoing and part of the larger project to improve NumPy's online presence and usefulness to new users. - Further cleanups related to removing Python 2.7. This improves code readability and removes technical debt. - Preliminary support for the upcoming Cython 3.0. *Contributors* A total of 182 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Aaron Meurer + - Abhilash Barigidad + - Abhinav Reddy + - Abhishek Singh + - Al-Baraa El-Hag + - Albert Villanova del Moral + - Alex Leontiev + - Alex Rockhill + - Alex Rogozhnikov - Alexander Belopolsky - Alexander Kuhn-Regnier + - Allen Downey + - Andras Deak - Andrea Olivo andryandrew at gmail.com andryandrew + - Andrew Eckart + - Anirudh Subramanian - Anthony Byuraev + - Antonio Larrosa + - Ashutosh Singh + - Bangcheng Yang + - Bas van Beek + - Ben Derrett + - Ben Elliston + - Ben Nathanson + - Bharat Medasani + - Bharat Raghunathan - Bijesh Mohan + - Bradley Dice + - Brandon David + - Brandt Bucher - Brian Soto + - Brigitta Sipocz - Cameron Blocker + - Carl Leake + - Charles Harris - Chris Brown + - Chris Vavaliaris + - Chunlin Fang - CloseChoice + - Daniel G. A. Smith + - Daniel Hrisca - Daniel Vanzo + - David Pitchford + - Davide Dal Bosco + - Dima Kogan + - Dmitry Kutlenkov + - Douglas Fenstermacher + - Dustin Spicuzza + - E. Madison Bray + - Elia Franzella + - Enrique Mat?as S?nchez (Quique) + - Erfan Nariman | Veneficus + - Eric Larson - Eric Moore - Eric Wieser - Erik M. Bray - EthanCJ-git + - Etienne Guesnet + - Felix Divo - Frankie Robertson + - Ganesh Kathiresan - Gengxin Xie - Gerry Manoim + - Guilherme Leobas - Hassan Kibirige - Hugo Mendes + - Hugo van Kemenade - Ian Thomas + - InessaPawson + - Isabela Presedo-Floyd + - Isuru Fernando - Jakob Jacobson + - Jakob Jakobson + - Jakub Wilk - James Myatt + - Jesse Li + - John Hagen + - John Zwinck - Joseph Fox-Rabinovitz - Josh Wilson - Jovial Joe Jayarson + - Julia Signell + - Jun Kudo + - Karan Dhir + - Kaspar Thommen + - Kerem Halla? - Kevin Moore + - Kevin Sheppard - Klaus Zimmermann + - LSchroefl + - Laurie + - Laurie Stephey + - Levi Stovall + - Lisa Schwetlick + - Lukas Geiger + - Madhulika Jain Chambers + - Matthias Bussonnier - Matti Picus - Melissa Weber Mendon?a - Michael Hirsch - Nick R. Papior - Nikola Forr? - Noman Arshad + - Paul YS Lee + - Pauli Virtanen - Pawe? Redzy?ski + - Peter Andreas Entschev - Peter Bell - Philippe Ombredanne + - Phoenix Meadowlark + - Piotr Gai?ski - Raghav Khanna + - Raghuveer Devulapalli - Rajas Rade + - Rakesh Vasudevan - Ralf Gommers - Raphael Kruse + - Rashmi K A + - Robert Kern - Rohit Sanjay + - Roman Yurchak - Ross Barnowski - Royston E Tauro + - Ryan C Cooper + - Ryan Soklaski - Safouane Chergui + - Sahil Siddiq + - Sarthak Vineet Kumar + - Sayed Adel - Sebastian Berg - Sergei Vorfolomeev + - Seth Troisi - Sidhant Bansal + - Simon Gasse - Simon Graham + - Stefan Appelhoff + - Stefan Behnel + - Stefan van der Walt - Steve Dower - Steve Joachim + - Steven Pitman + - Stuart Archibald - Sturla Molden - Susan Chang + - Takanori H + - Tapajyoti Bose + - Thomas A Caswell - Tina Oberoi - Tirth Patel - Tobias Pitters + - Tyler Reddy - Veniamin Petrenko + - Wansoo Kim + - Warren Weckesser - Wei Yang + - Wojciech Rzadkowski - Yang Hau + - Yogesh Raisinghani + - Yu Feng - Yuya Unno + - Zac Hatfield-Dodds - Zuhair Ali-Khan + - @abhilash42 + - @bernie gray + - @danbeibei + - @dojafrat - @dpitch40 + - @forfun + - @iamsoto + - @jbrockmendel + - @leeyspaul + - @mitch + - @prateek arora + - @qiyu8 + - @serge-sans-paille + - @skywalker + - @stphnlyd + - @xoviat - @??? + - @JMFT + - @Jack + - @Neal C + Cheers, Charles Harris -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Thu Dec 3 19:01:56 2020 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 4 Dec 2020 00:01:56 +0000 Subject: [Numpy-discussion] problem with numpy 1.19.4 install via pip on Win 10 In-Reply-To: <7733f968-2586-1e50-3d58-862807fed41b@gmail.com> References: <7733f968-2586-1e50-3d58-862807fed41b@gmail.com> Message-ID: The final message in the numpy issue suggests a possible fix: https://github.com/numpy/numpy/issues/16744#issuecomment-727098973 """ My preference would be to have a 1.19.5 that uses OpenBLAS 0.3.9 on Linux and 0.3.12 on Windows, but I don't know if this is possible. """ I don't know if that would fix the problem or how hard that would be to implement but I am seeing an increasing flow of problem reports in different fora about this issue and I'm surprised that the plan is to wait for MS to (possibly) release a fix in two month's time. -- Oscar On Thu, 3 Dec 2020 at 21:14, Alan G. Isaac wrote: > > "current expectation is that this [fix] will be able to be released near the end of January 2021" > ! > > > On 12/2/2020 7:13 PM, Ilhan Polat wrote: > > Yes this is known and we are waiting MS to roll out a solution for this. Here are more details > > https://developercommunity2.visualstudio.com/t/fmod-after-an-update-to-windows-2004-is-causing-a/1207405 > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From kevin.k.sheppard at gmail.com Thu Dec 3 19:18:28 2020 From: kevin.k.sheppard at gmail.com (Kevin Sheppard) Date: Fri, 4 Dec 2020 00:18:28 +0000 Subject: [Numpy-discussion] problem with numpy 1.19.4 install via pip on Win 10 In-Reply-To: References: <7733f968-2586-1e50-3d58-862807fed41b@gmail.com>, Message-ID: <8F1609E0-0DF8-44D9-AD3C-D589543C71E2@hxcore.ol> An HTML attachment was scrubbed... URL: From melissawm at gmail.com Fri Dec 4 15:40:19 2020 From: melissawm at gmail.com (=?UTF-8?Q?Melissa_Mendon=C3=A7a?=) Date: Fri, 4 Dec 2020 17:40:19 -0300 Subject: [Numpy-discussion] Documentation Team meeting - Monday December 7 In-Reply-To: References: Message-ID: Hi all! Our next Documentation Team meeting will be on *Monday, December 7* at ***4PM UTC***. All are welcome - you don't need to already be a contributor to join. If you have questions or are curious about what we're doing, we'll be happy to meet you! If you wish to join on Zoom, use this link: https://zoom.us/j/96219574921?pwd=VTRNeGwwOUlrYVNYSENpVVBRRjlkZz09 Here's the permanent hackmd document with the meeting notes (still being updated in the next few days!): https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg Hope to see you around! ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentation+Team+Meeting&iso=20201207T16&p1=1440&ah=1 - Melissa -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Fri Dec 4 16:51:29 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Fri, 04 Dec 2020 15:51:29 -0600 Subject: [Numpy-discussion] Rules for argument parsing/forwarding in __array_function__ and __array_ufunc__ In-Reply-To: References: Message-ID: On Wed, 2020-12-02 at 21:07 -0800, Stefan van der Walt wrote: > Hi Sebastian, > > Looking at these three rules, they all seem to stem from one simple > question: do we desire for a single code snippet to be runnable on > multiple array implementations? > > On Wed, Dec 2, 2020, at 15:34, Sebastian Berg wrote: > > 1. If an argument is invalid in NumPy it is considered and error. > > For example: > > > > np.log(arr, my_weird_argument=True) > > > > is always an error even if the `__array_function__` > > implementation > > of `arr` would support it. > > NEP 18 explicitly says that allowing forwarding could be done, > > but > > will not be done at this time. > > Relaxing this rule will mean that code working for one array > implementation (which has this keyword) may not work for another. Indeed, while NEP 18 mentions it, I personally don't see why we should relax it. (The NEP 13 implementation does so, but this is an unintentional, and not optimal, implementation detail.) > > > 2. Arguments must only be forwarded if they are passed in: > > > > np.mean(cupy_array) > > > > ends up as `cupy.mean(cupy_array)` and not: > > > > cupy.mean(cupy_array, axis=None, dtype=None, out=None, > > keepdims=False, where=True) > > > > meaning that CuPy does not need to implement all of those kwargs > > and > > NumPy can add new ones without breaking anyones code. > > This may ultimately make it harder for array implementors (they will > only see errors once someone tries to pass in an argument that they > forgot to implement). Perhaps better to pass all so they know what > they're dealing with? True, we do this for `np.mean(obj)`, etc. which end up calling `obj.mean()`, but compared to protocols which explicitly ask for NumPy compatibility, those method forwards are not as clearly defined. So maybe we should actually pass on everything (including the default value?), that is actually safer if we ever update the default. The downside would remain that a newer NumPy is likely to cause a break until the project updates (e.g. if we add a keyword argument). If we were open to this (plus an insignificant change in subclass handling), it would be easy to at least half the overhead of Python `__array_function__` dispatching. That is because it would allow us to inline (in python): def function(arg1, arg2, kwarg1=None): dispatched = dispatch((arg1,), arg1, arg2, kwarg1=kwarg1) if dispatched is not NotImplemented: return dispatched # normal code here (some argument validation could come first) This may look strange, but has to go through 1-2 function calls where currently we go through 4. The other change, would also allow us to remove *all* overhead for functions defined in C. > > > 3. NumPy should not check the *validity* of the arguments. For > > example: > > `np.add.reduce(xarray, axis="long")` should probably work in > > xarray. > > (`xarray.DataArray` does not actually implement the above.) > > But a string cannot be used as an axis in NumPy. > > Getting back to the original question: if this code is to be run on > multiple implementations, we should ensure that no strange values > pass through. > > Personally, I like the idea of a single API that works on multiple > backends. As such, I would 1) not pass through unknown arguments, 2) > always pass through all arguments, and 3) validate inputs to each > call. Thanks for the input! I think point 2) is in the sense the most interesting, because the approach `pytorch` takes to remove the overhead of array-function gets very complicated without it. In the end, parsing validity should maybe be considered an implementation detail... I.e. if there is a good reason why validating is a problem, we can stop doing it and otherwise there is no need to worry about it. (Although for ufuncs, I would go the non-validating route for now personally.) Cheers, Sebastian > > Best regards, > St?fan > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From jni at fastmail.com Sat Dec 5 18:31:03 2020 From: jni at fastmail.com (Juan Nunez-Iglesias) Date: Sun, 6 Dec 2020 10:31:03 +1100 Subject: [Numpy-discussion] np.{bool,float,int} deprecation Message-ID: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Hi all, At the prodding [1] of Sebastian, I?m starting a discussion on the decision to deprecate np.{bool,float,int}. This deprecation broke our prerelease testing in scikit-image (which, hooray for rcs!), and resulted in a large amount of code churn to fix [2]. To be honest, I do think *some* sort of deprecation is needed, because for the longest time I thought that np.float was what np.float_ actually is. I think it would be worthwhile to move to *that*, though it?s an even more invasive deprecation than the currently proposed one. Writing `x = np.zeros(5, dtype=int)` is somewhat magical, because someone with a strict typing mindset (there?s an increasing number!) might expect that this is an array of pointers to Python ints. This is why I?ve always preferred to write `dtype=np.int`, resulting in the current code churn. I don?t know what the best answer is, just sparking the discussion Sebastian wants to see. ;) For skimage we?ve already merged a fix (even if it is one of dubious quality, as St?fan points out [3] ;), so I don?t have too much stake in the outcome. Juan. [1]: https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 [2]: https://github.com/scikit-image/scikit-image/pull/5103 [3]: https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 From deak.andris at gmail.com Sat Dec 5 22:10:28 2020 From: deak.andris at gmail.com (Andras Deak) Date: Sun, 6 Dec 2020 04:10:28 +0100 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Sun, Dec 6, 2020 at 12:31 AM Juan Nunez-Iglesias wrote: > > Hi all, > > At the prodding [1] of Sebastian, I?m starting a discussion on the decision to deprecate np.{bool,float,int}. This deprecation broke our prerelease testing in scikit-image (which, hooray for rcs!), and resulted in a large amount of code churn to fix [2]. > > To be honest, I do think *some* sort of deprecation is needed, because for the longest time I thought that np.float was what np.float_ actually is. I think it would be worthwhile to move to *that*, though it?s an even more invasive deprecation than the currently proposed one. Writing `x = np.zeros(5, dtype=int)` is somewhat magical, because someone with a strict typing mindset (there?s an increasing number!) might expect that this is an array of pointers to Python ints. This is why I?ve always preferred to write `dtype=np.int`, resulting in the current code churn. > > I don?t know what the best answer is, just sparking the discussion Sebastian wants to see. ;) For skimage we?ve already merged a fix (even if it is one of dubious quality, as St?fan points out [3] ;), so I don?t have too much stake in the outcome. Hi Juan, Let me start with a disclaimer that I'm an end user, and as such it's very easy for me to be bold when it comes to deprecations :) But I experienced the same thing that you describe in https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739429373 : > [I]t was very surprising to me when I found out that np.float is float. For the longest time I thought that np.float was equivalent to "whatever the default float value is on my platform", and considered it best practice to use that instead of plain float. ? I think that is a common misconception. And I'm pretty sure the vast majority of end users faces this. The proper np.float32 and other types are intuitive enough that people don't go out of their way to read the documentation in detail, and it's highly unexpected that some `np.*` types are mere aliases. Now, this should probably not be a problem as long as people only stick these aliases into `dtype` keyword arguments, because that works as expected (based on the wrong premise). But once you extrapolate from the `dtype=np.int` behaviour to "`np.int` must be my native numpy int type" you can easily get subtle bugs. For instance, you might expect `isinstance(this_type, np.int)` to give you True if `this_type` is the type of an item of an array with `dtype=np.int`. To be fair I'm not sure that I've ever been bitten by this personally... but once you're aware of the pitfall it seems really ominous. I guess one helpful question is this: among all the code churn needed to fix the breakage did you find any bugs that were revealed by the deprecation? If that's the case (in scikit-image or any other large downstream library) then that would be a good argument for going forward with the deprecation. Cheers, Andr?s > Juan. > > [1]: https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > [3]: https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From charlesr.harris at gmail.com Sat Dec 5 22:12:26 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 5 Dec 2020 20:12:26 -0700 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias wrote: > Hi all, > > At the prodding [1] of Sebastian, I?m starting a discussion on the > decision to deprecate np.{bool,float,int}. This deprecation broke our > prerelease testing in scikit-image (which, hooray for rcs!), and resulted > in a large amount of code churn to fix [2]. > > To be honest, I do think *some* sort of deprecation is needed, because for > the longest time I thought that np.float was what np.float_ actually is. I > think it would be worthwhile to move to *that*, though it?s an even more > invasive deprecation than the currently proposed one. Writing `x = > np.zeros(5, dtype=int)` is somewhat magical, because someone with a strict > typing mindset (there?s an increasing number!) might expect that this is an > array of pointers to Python ints. This is why I?ve always preferred to > write `dtype=np.int`, resulting in the current code churn. > > I don?t know what the best answer is, just sparking the discussion > Sebastian wants to see. ;) For skimage we?ve already merged a fix (even if > it is one of dubious quality, as St?fan points out [3] ;), so I don?t have > too much stake in the outcome. > > Juan. > > [1]: > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > [3]: > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > I checked pandas and astropy and both have several uses of the deprecated types but should be easy to fix. I suppose the question is if we want to make them fix things *right now* :) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.harfouche at gmail.com Sun Dec 6 00:22:58 2020 From: mark.harfouche at gmail.com (Mark Harfouche) Date: Sun, 6 Dec 2020 00:22:58 -0500 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: I guess if the answer is to stop people from from numpy import * there is a good fix for that doesn?t involve deprecating dtype=np.int. If the answer is to deprecate np.int(1) == int(1) then one can add a warning to the __init__ of the np.int class, but continue to subclass the python int class. It just doesn?t seem worthwhile to to stop people from using dtype=np.int, which seem to read: ?I want this to be a numpy integer, not necessarily a python integer?. On Sat, Dec 5, 2020 at 10:14 PM Charles R Harris wrote: > > > On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias > wrote: > >> Hi all, >> >> At the prodding [1] of Sebastian, I?m starting a discussion on the >> decision to deprecate np.{bool,float,int}. This deprecation broke our >> prerelease testing in scikit-image (which, hooray for rcs!), and resulted >> in a large amount of code churn to fix [2]. >> >> To be honest, I do think *some* sort of deprecation is needed, because >> for the longest time I thought that np.float was what np.float_ actually >> is. I think it would be worthwhile to move to *that*, though it?s an even >> more invasive deprecation than the currently proposed one. Writing `x = >> np.zeros(5, dtype=int)` is somewhat magical, because someone with a strict >> typing mindset (there?s an increasing number!) might expect that this is an >> array of pointers to Python ints. This is why I?ve always preferred to >> write `dtype=np.int`, resulting in the current code churn. >> >> I don?t know what the best answer is, just sparking the discussion >> Sebastian wants to see. ;) For skimage we?ve already merged a fix (even if >> it is one of dubious quality, as St?fan points out [3] ;), so I don?t have >> too much stake in the outcome. >> >> Juan. >> >> [1]: >> https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 >> [2]: https://github.com/scikit-image/scikit-image/pull/5103 >> [3]: >> https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 >> > > I checked pandas and astropy and both have several uses of the deprecated > types but should be easy to fix. I suppose the question is if we want to > make them fix things *right now* :) > > Chuck > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Sun Dec 6 00:51:13 2020 From: shoyer at gmail.com (Stephan Hoyer) Date: Sat, 5 Dec 2020 21:51:13 -0800 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Sat, Dec 5, 2020 at 9:24 PM Mark Harfouche wrote: > If the answer is to deprecate > > np.int(1) == int(1) > > then one can add a warning to the __init__ of the np.int class, but > continue to subclass the python int class. > > It just doesn?t seem worthwhile to to stop people from using dtype=np.int, > which seem to read: > > ?I want this to be a numpy integer, not necessarily a python integer?. > The problem is that there is assuredly code that inadvertently relies upon this (mis)feature. If we change the behavior of np.int() to create np.int64() objects instead of int() objects, it is likely to result in breaking some user code. Even with a prior warning, this breakage may be surprising and very hard to track down. In contrast, it's much safer to simply remove np.int entirely, because if users ignore the deprecation they end up with an error. This is a general feature for deprecations: it's much safer to remove functionality than it is to change behavior. So on the whole, I think this is the right call. > > On Sat, Dec 5, 2020 at 10:14 PM Charles R Harris < > charlesr.harris at gmail.com> wrote: > >> >> >> On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias >> wrote: >> >>> Hi all, >>> >>> At the prodding [1] of Sebastian, I?m starting a discussion on the >>> decision to deprecate np.{bool,float,int}. This deprecation broke our >>> prerelease testing in scikit-image (which, hooray for rcs!), and resulted >>> in a large amount of code churn to fix [2]. >>> >>> To be honest, I do think *some* sort of deprecation is needed, because >>> for the longest time I thought that np.float was what np.float_ actually >>> is. I think it would be worthwhile to move to *that*, though it?s an even >>> more invasive deprecation than the currently proposed one. Writing `x = >>> np.zeros(5, dtype=int)` is somewhat magical, because someone with a strict >>> typing mindset (there?s an increasing number!) might expect that this is an >>> array of pointers to Python ints. This is why I?ve always preferred to >>> write `dtype=np.int`, resulting in the current code churn. >>> >>> I don?t know what the best answer is, just sparking the discussion >>> Sebastian wants to see. ;) For skimage we?ve already merged a fix (even if >>> it is one of dubious quality, as St?fan points out [3] ;), so I don?t have >>> too much stake in the outcome. >>> >>> Juan. >>> >>> [1]: >>> https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 >>> [2]: https://github.com/scikit-image/scikit-image/pull/5103 >>> [3]: >>> https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 >>> >> >> I checked pandas and astropy and both have several uses of the deprecated >> types but should be easy to fix. I suppose the question is if we want to >> make them fix things *right now* :) >> >> Chuck >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Sun Dec 6 01:05:00 2020 From: shoyer at gmail.com (Stephan Hoyer) Date: Sat, 5 Dec 2020 22:05:00 -0800 Subject: [Numpy-discussion] Rules for argument parsing/forwarding in __array_function__ and __array_ufunc__ In-Reply-To: References: Message-ID: On Wed, Dec 2, 2020 at 3:39 PM Sebastian Berg wrote: > 1. If an argument is invalid in NumPy it is considered and error. > For example: > > np.log(arr, my_weird_argument=True) > > is always an error even if the `__array_function__` implementation > of `arr` would support it. > NEP 18 explicitly says that allowing forwarding could be done, but > will not be done at this time. > >From my perspective, this is a good thing: it ensures that NumPy's API is only used for features that exist in NumPy. Otherwise I can imagine causing considerable confusion. If you want to use my_weird_argument, you can call my_weird_library.log() instead. > 2. Arguments must only be forwarded if they are passed in: > > np.mean(cupy_array) > > ends up as `cupy.mean(cupy_array)` and not: > > cupy.mean(cupy_array, axis=None, dtype=None, out=None, > keepdims=False, where=True) > > meaning that CuPy does not need to implement all of those kwargs and > NumPy can add new ones without breaking anyones code. > My reasoning here was two-fold: 1. To avoid the unfortunate situation for functions like np.mean(), where NumPy jumps through considerable hoops to avoid passing extra arguments in an ad-hoc way to preserve backwards compatibility 2. To make it easy for a library to implement "incomplete" versions of NumPy's API, by simply omitting arguments. The idea was that NumPy's API is open to partial implementations, but not extension. > 3. NumPy should not check the *validity* of the arguments. For example: > `np.add.reduce(xarray, axis="long")` should probably work in xarray. > (`xarray.DataArray` does not actually implement the above.) > But a string cannot be used as an axis in NumPy. > I don't think libraries should be encouraged to abuse NumPy's API to mean something else. Certainly I would not use this in xarray :). If we could check the validity of arguments cheaply, that would be fine by me. But I didn't think it was worth adding overhead to every function call. Perhaps type annotations could be relied on for these sorts of checks? I am pretty happy considering not checking the validity of arguments to be an implementation detail for now. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Sun Dec 6 09:22:55 2020 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 6 Dec 2020 09:22:55 -0500 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Sun, Dec 6, 2020 at 12:52 AM Stephan Hoyer wrote: > On Sat, Dec 5, 2020 at 9:24 PM Mark Harfouche > wrote: > >> If the answer is to deprecate >> >> np.int(1) == int(1) >> >> then one can add a warning to the __init__ of the np.int class, but >> continue to subclass the python int class. >> >> It just doesn?t seem worthwhile to to stop people from using dtype=np.int, >> which seem to read: >> >> ?I want this to be a numpy integer, not necessarily a python integer?. >> > The problem is that there is assuredly code that inadvertently relies upon > this (mis)feature. > > If we change the behavior of np.int() to create np.int64() objects > instead of int() objects, it is likely to result in breaking some user > code. Even with a prior warning, this breakage may be surprising and very > hard to track down. In contrast, it's much safer to simply remove np.int > entirely, because if users ignore the deprecation they end up with an error. > FWIW (and IIRC), *this* was the original misfeature. `np.int`, `np.bool`, and `np.float` were aliases for their corresponding default scalar types in the first numpy releases. However, too many people were doing `from numpy import *` and covering up the builtins. We renamed these aliases with trailing underscores to avoid that problem, but too many people (even in those early days) still had uses of `dtype=np.int`. Making `np.int is int` was the backwards-compatibility hack. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Sun Dec 6 10:18:10 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Sun, 06 Dec 2020 09:18:10 -0600 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: <873f3c94f34b249401692be47ff5a83769292ce7.camel@sipsolutions.net> On Sat, 2020-12-05 at 20:12 -0700, Charles R Harris wrote: > On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias > wrote: > > > Hi all, > > > > At the prodding [1] of Sebastian, I?m starting a discussion on the > > decision to deprecate np.{bool,float,int}. This deprecation broke > > our > > prerelease testing in scikit-image (which, hooray for rcs!), and > > resulted > > in a large amount of code churn to fix [2]. > > > > To be honest, I do think *some* sort of deprecation is needed, > > because for > > the longest time I thought that np.float was what np.float_ > > actually is. I > > think it would be worthwhile to move to *that*, though it?s an even > > more > > invasive deprecation than the currently proposed one. Writing `x = > > np.zeros(5, dtype=int)` is somewhat magical, because someone with a > > strict > > typing mindset (there?s an increasing number!) might expect that > > this is an > > array of pointers to Python ints. This is why I?ve always preferred > > to > > write `dtype=np.int`, resulting in the current code churn. > > > > I don?t know what the best answer is, just sparking the discussion > > Sebastian wants to see. ;) For skimage we?ve already merged a fix > > (even if > > it is one of dubious quality, as St?fan points out [3] ;), so I > > don?t have > > too much stake in the outcome. > > > > Juan. > > > > [1]: > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > > [3]: > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > > > > I checked pandas and astropy and both have several uses of the > deprecated > types but should be easy to fix. I suppose the question is if we want > to > make them fix things *right now* :) > The reason why I thought it might be good to bring this up again is that I am not sure clear on how painful the deprecation is; which should be weighed against the benefit. And the benefit here is only moderate. Thus, with the things now in and a few more people exposed to it, if anyone thinks its a bad idea or that we should delay, I am all ears. Cheers, Sebastian > Chuck > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From sebastian at sipsolutions.net Sun Dec 6 10:41:57 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Sun, 06 Dec 2020 09:41:57 -0600 Subject: [Numpy-discussion] Rules for argument parsing/forwarding in __array_function__ and __array_ufunc__ In-Reply-To: References: Message-ID: On Sat, 2020-12-05 at 22:05 -0800, Stephan Hoyer wrote: > On Wed, Dec 2, 2020 at 3:39 PM Sebastian Berg < > sebastian at sipsolutions.net> > wrote: > > > 1. If an argument is invalid in NumPy it is considered and error. > > For example: > > > > np.log(arr, my_weird_argument=True) > > > > is always an error even if the `__array_function__` > > implementation > > of `arr` would support it. > > NEP 18 explicitly says that allowing forwarding could be done, > > but > > will not be done at this time. > > > > From my perspective, this is a good thing: it ensures that NumPy's > API is > only used for features that exist in NumPy. Otherwise I can imagine > causing > considerable confusion. > > If you want to use my_weird_argument, you can call > my_weird_library.log() > instead. > > > > 2. Arguments must only be forwarded if they are passed in: > > > > np.mean(cupy_array) > > > > ends up as `cupy.mean(cupy_array)` and not: > > > > cupy.mean(cupy_array, axis=None, dtype=None, out=None, > > keepdims=False, where=True) > > > > meaning that CuPy does not need to implement all of those kwargs > > and > > NumPy can add new ones without breaking anyones code. > > > > My reasoning here was two-fold: > 1. To avoid the unfortunate situation for functions like np.mean(), > where > NumPy jumps through considerable hoops to avoid passing extra > arguments in > an ad-hoc way to preserve backwards compatibility > 2. To make it easy for a library to implement "incomplete" versions > of > NumPy's API, by simply omitting arguments. > > The idea was that NumPy's API is open to partial implementations, but > not > extension. > Indeed, changing this allows inlining in Python easier (because you don't need to use `**kwargs` to be able to know which arguments were not passed). I guess the alternative is to force everyone to keep up, but you are of course allowed to raise a NotImplementedError (which is actually nicer for users, probably). > > > 3. NumPy should not check the *validity* of the arguments. For > > example: > > `np.add.reduce(xarray, axis="long")` should probably work in > > xarray. > > (`xarray.DataArray` does not actually implement the above.) > > But a string cannot be used as an axis in NumPy. > > > > I don't think libraries should be encouraged to abuse NumPy's API to > mean > something else. Certainly I would not use this in xarray :). > > If we could check the validity of arguments cheaply, that would be > fine by > me. But I didn't think it was worth adding overhead to every function > call. > Perhaps type annotations could be relied on for these sorts of > checks? I am > pretty happy considering not checking the validity of arguments to be > an > implementation detail for now. The current implementation of __array_function__ makes this hard, because it assumes the call graph is: array_funciton_implementation(...): impl = find_implementation() # impl may be the default return impl() Unlike for __array_ufunc__ where it is: ufunc.__call__(*args, **kwargs): if needs_to_defer(args, kwargs): return defered_result() If you assume that NumPy is the main consumer, especially on the C- side, validating the arguments (e.g. integer axis, NumPy dtypes) can make things more comfortable. "inlining" the dispatching as the second case, makes things quite a bit more comfortable, but is not necessary. However, it requires a small change to the default __array_function__ (i.e. you have to change the meaning to "defaul __array_function__" is the same as no array function.) Cheers, Sebastian > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From ralf.gommers at gmail.com Mon Dec 7 06:39:00 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 7 Dec 2020 11:39:00 +0000 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: <873f3c94f34b249401692be47ff5a83769292ce7.camel@sipsolutions.net> References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> <873f3c94f34b249401692be47ff5a83769292ce7.camel@sipsolutions.net> Message-ID: On Sun, Dec 6, 2020 at 4:23 PM Sebastian Berg wrote: > On Sat, 2020-12-05 at 20:12 -0700, Charles R Harris wrote: > > On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias > > wrote: > > > > > Hi all, > > > > > > At the prodding [1] of Sebastian, I?m starting a discussion on the > > > decision to deprecate np.{bool,float,int}. This deprecation broke > > > our > > > prerelease testing in scikit-image (which, hooray for rcs!), and > > > resulted > > > in a large amount of code churn to fix [2]. > > > > > > To be honest, I do think *some* sort of deprecation is needed, > > > because for > > > the longest time I thought that np.float was what np.float_ > > > actually is. I > > > think it would be worthwhile to move to *that*, though it?s an even > > > more > > > invasive deprecation than the currently proposed one. Writing `x = > > > np.zeros(5, dtype=int)` is somewhat magical, because someone with a > > > strict > > > typing mindset (there?s an increasing number!) might expect that > > > this is an > > > array of pointers to Python ints. This is why I?ve always preferred > > > to > > > write `dtype=np.int`, resulting in the current code churn. > > > > > > I don?t know what the best answer is, just sparking the discussion > > > Sebastian wants to see. ;) For skimage we?ve already merged a fix > > > (even if > > > it is one of dubious quality, as St?fan points out [3] ;), so I > > > don?t have > > > too much stake in the outcome. > > > > > > Juan. > > > > > > [1]: > > > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > > > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > > > [3]: > > > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > > > > > > > I checked pandas and astropy and both have several uses of the > > deprecated > > types but should be easy to fix. I suppose the question is if we want > > to > > make them fix things *right now* :) > > > > > The reason why I thought it might be good to bring this up again is > that I am not sure clear on how painful the deprecation is; which > should be weighed against the benefit. And the benefit here is only > moderate. > It will be painful as in "lots of churn", but the fixes are straightforward. And it's clear many knowledgeable users didn't know they were aliases, so there is something to gain here. Whether or not we revert the deprecation, I'd be in favor of improving the docs to answer the most common questions and pitfalls, like: - What happens when I use Python builtin types with the dtype keyword? - How do I check if something is an integer array? Or a NumPy or Python integer? - What are default integer, float and complex precisions on all platforms? - How do I iterate over all floating point dtypes when writing tests? - Which of the many equivalent dtypes should I prefer? --> use float64, not float_ or double - warning: float128 and float96 do not exist on all platforms - https://github.com/scikit-learn/scikit-learn/wiki/C-integer-types%3A-the-missing-manual Related: it's still easy to have things leak into the namespace unintentionally - `np.sys` and `np.os` exist too. I think we can probably clean those up without a deprecation, but we should write some more public API tests that prevent this kind of thing. Cheers, Ralf > Thus, with the things now in and a few more people exposed to it, if > anyone thinks its a bad idea or that we should delay, I am all ears. > > Cheers, > > Sebastian > > > > Chuck > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieser.eric+numpy at gmail.com Mon Dec 7 06:49:29 2020 From: wieser.eric+numpy at gmail.com (Eric Wieser) Date: Mon, 7 Dec 2020 11:49:29 +0000 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> <873f3c94f34b249401692be47ff5a83769292ce7.camel@sipsolutions.net> Message-ID: If the CI noise in downstream libraries is particularly painful, we could switch to `PendingDeprecationWarning` instead of `DeprecationWarning` to make it easier to add the warnings to an ignore list. I think this might make the warning less visible to end users though, who are the users that this deprecation was really aimed at. Eric On Mon, 7 Dec 2020 at 11:39, Ralf Gommers wrote: > > > On Sun, Dec 6, 2020 at 4:23 PM Sebastian Berg > wrote: > >> On Sat, 2020-12-05 at 20:12 -0700, Charles R Harris wrote: >> > On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias >> > wrote: >> > >> > > Hi all, >> > > >> > > At the prodding [1] of Sebastian, I?m starting a discussion on the >> > > decision to deprecate np.{bool,float,int}. This deprecation broke >> > > our >> > > prerelease testing in scikit-image (which, hooray for rcs!), and >> > > resulted >> > > in a large amount of code churn to fix [2]. >> > > >> > > To be honest, I do think *some* sort of deprecation is needed, >> > > because for >> > > the longest time I thought that np.float was what np.float_ >> > > actually is. I >> > > think it would be worthwhile to move to *that*, though it?s an even >> > > more >> > > invasive deprecation than the currently proposed one. Writing `x = >> > > np.zeros(5, dtype=int)` is somewhat magical, because someone with a >> > > strict >> > > typing mindset (there?s an increasing number!) might expect that >> > > this is an >> > > array of pointers to Python ints. This is why I?ve always preferred >> > > to >> > > write `dtype=np.int`, resulting in the current code churn. >> > > >> > > I don?t know what the best answer is, just sparking the discussion >> > > Sebastian wants to see. ;) For skimage we?ve already merged a fix >> > > (even if >> > > it is one of dubious quality, as St?fan points out [3] ;), so I >> > > don?t have >> > > too much stake in the outcome. >> > > >> > > Juan. >> > > >> > > [1]: >> > > >> https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 >> > > [2]: https://github.com/scikit-image/scikit-image/pull/5103 >> > > [3]: >> > > >> https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 >> > > >> > >> > I checked pandas and astropy and both have several uses of the >> > deprecated >> > types but should be easy to fix. I suppose the question is if we want >> > to >> > make them fix things *right now* :) >> > >> >> >> The reason why I thought it might be good to bring this up again is >> that I am not sure clear on how painful the deprecation is; which >> should be weighed against the benefit. And the benefit here is only >> moderate. >> > > It will be painful as in "lots of churn", but the fixes are > straightforward. And it's clear many knowledgeable users didn't know they > were aliases, so there is something to gain here. > > Whether or not we revert the deprecation, I'd be in favor of improving the > docs to answer the most common questions and pitfalls, like: > > - What happens when I use Python builtin types with the dtype keyword? > - How do I check if something is an integer array? Or a NumPy or Python > integer? > - What are default integer, float and complex precisions on all platforms? > - How do I iterate over all floating point dtypes when writing tests? > - Which of the many equivalent dtypes should I prefer? --> use float64, > not float_ or double > - warning: float128 and float96 do not exist on all platforms > - > https://github.com/scikit-learn/scikit-learn/wiki/C-integer-types%3A-the-missing-manual > > Related: it's still easy to have things leak into the namespace > unintentionally - `np.sys` and `np.os` exist too. I think we can probably > clean those up without a deprecation, but we should write some more public > API tests that prevent this kind of thing. > > Cheers, > Ralf > > > >> Thus, with the things now in and a few more people exposed to it, if >> anyone thinks its a bad idea or that we should delay, I am all ears. >> >> Cheers, >> >> Sebastian >> >> >> > Chuck >> > _______________________________________________ >> > NumPy-Discussion mailing list >> > NumPy-Discussion at python.org >> > https://mail.python.org/mailman/listinfo/numpy-discussion >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Mon Dec 7 14:30:25 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Mon, 07 Dec 2020 13:30:25 -0600 Subject: [Numpy-discussion] NEP 42 being accepted (New and extensible DTypes) Message-ID: Hi all, It has been a very long time since my emails asking to accept the NEP: https://mail.python.org/pipermail/numpy-discussion/2020-October/081038.html Happy to wait for a bit longer, but without any hesitation/points raised, I consider the NEP as accepted. At this time most of the important points have even been included in NumPy itself. That leaves still a lot of freedom to modify the final public API, but that is not the most important part with respect to moving forward. I have opened a PR to that affect: https://github.com/numpy/numpy/pull/17953 Cheers, Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From asmeurer at gmail.com Mon Dec 7 16:18:32 2020 From: asmeurer at gmail.com (Aaron Meurer) Date: Mon, 7 Dec 2020 14:18:32 -0700 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: Regarding np.bool specifically, if you want to deprecate this, you might want to discuss this with us at the array API standard https://github.com/data-apis/array-api (which is currently in RFC stage). The spec uses bool as the name for the boolean dtype. Would it make sense for NumPy to change np.bool to just be the boolean dtype object? Unlike int and float, there is no ambiguity with bool, and NumPy clearly doesn't have any issues with shadowing builtin names in its namespace. Aaron Meurer On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias wrote: > > Hi all, > > At the prodding [1] of Sebastian, I?m starting a discussion on the decision to deprecate np.{bool,float,int}. This deprecation broke our prerelease testing in scikit-image (which, hooray for rcs!), and resulted in a large amount of code churn to fix [2]. > > To be honest, I do think *some* sort of deprecation is needed, because for the longest time I thought that np.float was what np.float_ actually is. I think it would be worthwhile to move to *that*, though it?s an even more invasive deprecation than the currently proposed one. Writing `x = np.zeros(5, dtype=int)` is somewhat magical, because someone with a strict typing mindset (there?s an increasing number!) might expect that this is an array of pointers to Python ints. This is why I?ve always preferred to write `dtype=np.int`, resulting in the current code churn. > > I don?t know what the best answer is, just sparking the discussion Sebastian wants to see. ;) For skimage we?ve already merged a fix (even if it is one of dubious quality, as St?fan points out [3] ;), so I don?t have too much stake in the outcome. > > Juan. > > [1]: https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > [3]: https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From sebastian at sipsolutions.net Tue Dec 8 13:20:43 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 08 Dec 2020 12:20:43 -0600 Subject: [Numpy-discussion] NumPy Community Meeting Wednesday Message-ID: Hi all, There will be a NumPy Community meeting Wednesday December 9th at 12pm Pacific Time (20:00 UTC). Everyone is invited and encouraged to join in and edit the work-in-progress meeting topics and notes at: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From sebastian at sipsolutions.net Wed Dec 9 11:21:30 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Wed, 09 Dec 2020 10:21:30 -0600 Subject: [Numpy-discussion] Fixing/implementing value based Casting Message-ID: <82fe831b56e9322f352c767a5cab70b979529899.camel@sipsolutions.net> Hi all, Sorry that this will again be a bit complicated again :(. In brief: * I would like to pass around scalars in some (partially new) C-API to implement value-based promotion. * There are some subtle commutativity issues with promotion. Commutativity may change in that case (with respect of value based promotion, probably to the better normally). [0] In the past days, I have been looking into implementing value-based promotion in a way that I had done it for Prototype before. The idea was that NEP 42, allows for the creation of DType dynamically, which does allow very powerful value based promotion/casting. But I decided there are too many quirks with creating type instances dynamically (potentially very often) just to pass around one additional piece of information. That approach was far more powerful, but it is power and complexity that we do not require, given that: * Value based promotion is only used for a mix of scalars and arrays (where "scalar" is annoyingly defined as 0-D at the moment) * I assume it is only relevant for `np.result_type` and promotion in ufuncs (which often uses `np.result_type`). `np.can_cast` has such behaviour, but I think it is easier [1]. We could implement more powerful "value based" logic, but I doubt it is worthwhile. * This is already stretching the Python C-API beyond its limits. So I will suggest this instead which *must* modify some (poorly defined) current behaviour: 1. We always evaluate concrete DTypes first in promotion, this means that in rare cases the non-commutativity of promotion may change the result dtype: np.result_type(-1, 2**16, np.float32) The same can also happens when you reorder the normal dtypes: np.result_type(np.int8, np.uint16, np.float32) np.result_type(np.float32, np.int8, np.uint16) in both cases the `np.float32` is moved to the front 2. If we reorder the above operation, we can define that we never promote two "scalar values". Instead we convert both to a concrete one first. This makes it effectively like: np.result_type(np.array(-1).dtype, np.array(2**16).dtype) This means that we never have to deal with promoting two values. 3. We need additional private API (we were always going to need some additional API); That API could become public: * Convert a single value into a concrete dtype, you could say the same as `self.common_dtype(None)`, but a dedicated function seems simpler. A dtype like this will never use `common_dtype()`. * `common_dtype_with_scalar(self, other, scalar)` (note that only one of the DTypes can have a scalar). As a fallback, this function can be implemented by converting to the concrete DType and retrying with the normal `common_dtype`. (At leas the second slot must be made public we are to allow value based promotion for user DTypes. I expect we will, but it is not particularly important to me right now.) 4. Our public API (including new C-API) has to expose and take the scalar values. That means promotion in ufuncs will get DTypes and `scalar_values`, although those should normally be `NULL` (or None). In future python API, this is probably acceptable: np.result_type([t if v is None else v for t, v in zip(dtypes, scalar_values)]) In C, we need to expose a function below `result_type` which accepts both the scalar values and DTypes explicitly. 5. For the future: As said many times, I would like to deprecate using value based promotion for anything except Python core types. That just seems wrong and confusing. My only problem is that while I can warn (possibly sometimes too often) when behaviour will change. I do not have a good idea about silencing that warning. Note that this affects NEP 42 (a little bit). NEP 42 currently makes a nod towards the dynamic type creation, but falls short of actually defining it. So These rules have to be incorporated, but IMO they do not affect the general design choices in the NEP. There is probably even more complexity to be found here, but for now the above seems to be at least good enough to make headway... Any thoughts or clarity remaining that I can try to confuse? :) Cheers, Sebastian [0] We could use the reordering trick also for concrete DTypes, although, that would require introducing some kind of priority... I do not like that much as public API, but it might be something to look at internally or for types deriving from the builtin abstract DTypes: * inexact * other Just evaluating all `inexact` first would probably solve our commutativity issues. [1] NumPy uses `np.can_cast(value, dtype)` also. For example: np.can_cast(np.array(1., dtype=np.float64), np.float32, casting="safe") returns True. My working hypothesis is that `np.can_cast` as above is just a side battle. I.e. we can either: * Flip the switch on it (can-cast does no value based logic, even though we use it internally, we do not need it). * Or, we can implement those cases of `np.can_cast` by using promotion. The first one is tempting, but I assume we should go with the second since it preserves behaviour and is slightly more powerful. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From sebastian at sipsolutions.net Wed Dec 9 11:41:11 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Wed, 09 Dec 2020 10:41:11 -0600 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > Regarding np.bool specifically, if you want to deprecate this, you > might want to discuss this with us at the array API standard > https://github.com/data-apis/array-api (which is currently in RFC > stage). The spec uses bool as the name for the boolean dtype. > > Would it make sense for NumPy to change np.bool to just be the > boolean > dtype object? Unlike int and float, there is no ambiguity with bool, > and NumPy clearly doesn't have any issues with shadowing builtin > names > in its namespace. We could keep the Python alias around (which for `dtype=` is the same as `np.bool_`). I am not sure I like the idea of immediately shadowing the builtin. That is a switch we can avoid flipping (without warning); `np.bool_` and `bool` are fairly different beasts? [1] OTOH, if someone wants to entertain switching... It could be interesting to see how (unfixed) downstream projects react to it. One approach would be: * Go ahead for now (deprecate) * Add a FutureWarning at some point that we _will_ start to export `np.bool` again (but `from numpy import *` is a problem?) * Aim to make `np.bool is np.bool_` at some point in the (far) future. It is multi-step (and I recall opinions that multi-step is bad). Although, I think the main argument against it was to not force users to modify code more than once. And I do not think that happens here. Of course we could use the `FutureWarning` right away, but I don't mind taking it slow. Cheers, Sebastian [1] I admit, probably almost nobody would notice. And usually using a Python `bool` is better... > > Aaron Meurer > > On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias > wrote: > > Hi all, > > > > At the prodding [1] of Sebastian, I?m starting a discussion on the > > decision to deprecate np.{bool,float,int}. This deprecation broke > > our prerelease testing in scikit-image (which, hooray for rcs!), > > and resulted in a large amount of code churn to fix [2]. > > > > To be honest, I do think *some* sort of deprecation is needed, > > because for the longest time I thought that np.float was what > > np.float_ actually is. I think it would be worthwhile to move to > > *that*, though it?s an even more invasive deprecation than the > > currently proposed one. Writing `x = np.zeros(5, dtype=int)` is > > somewhat magical, because someone with a strict typing mindset > > (there?s an increasing number!) might expect that this is an array > > of pointers to Python ints. This is why I?ve always preferred to > > write `dtype=np.int`, resulting in the current code churn. > > > > I don?t know what the best answer is, just sparking the discussion > > Sebastian wants to see. ;) For skimage we?ve already merged a fix > > (even if it is one of dubious quality, as St?fan points out [3] ;), > > so I don?t have too much stake in the outcome. > > > > Juan. > > > > [1]: > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > > [3]: > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From asmeurer at gmail.com Wed Dec 9 16:06:56 2020 From: asmeurer at gmail.com (Aaron Meurer) Date: Wed, 9 Dec 2020 14:06:56 -0700 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg wrote: > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > > Regarding np.bool specifically, if you want to deprecate this, you > > might want to discuss this with us at the array API standard > > https://github.com/data-apis/array-api (which is currently in RFC > > stage). The spec uses bool as the name for the boolean dtype. > > > > Would it make sense for NumPy to change np.bool to just be the > > boolean > > dtype object? Unlike int and float, there is no ambiguity with bool, > > and NumPy clearly doesn't have any issues with shadowing builtin > > names > > in its namespace. > > We could keep the Python alias around (which for `dtype=` is the same > as `np.bool_`). > > I am not sure I like the idea of immediately shadowing the builtin. > That is a switch we can avoid flipping (without warning); `np.bool_` > and `bool` are fairly different beasts? [1] NumPy already shadows a lot of builtins, in many cases, in ways that are incompatible with existing ones. It's not something I would have done personally, but it's been this way for a long time. Aaron Meurer > OTOH, if someone wants to entertain switching... It could be > interesting to see how (unfixed) downstream projects react to it. > > One approach would be: > > * Go ahead for now (deprecate) > * Add a FutureWarning at some point that we _will_ start to export > `np.bool` again (but `from numpy import *` is a problem?) > * Aim to make `np.bool is np.bool_` at some point in the (far) future. > > It is multi-step (and I recall opinions that multi-step is bad). > Although, I think the main argument against it was to not force users > to modify code more than once. And I do not think that happens here. > > Of course we could use the `FutureWarning` right away, but I don't mind > taking it slow. > > Cheers, > > Sebastian > > > > [1] I admit, probably almost nobody would notice. And usually using a > Python `bool` is better... > > > > > > Aaron Meurer > > > > On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias > > wrote: > > > Hi all, > > > > > > At the prodding [1] of Sebastian, I?m starting a discussion on the > > > decision to deprecate np.{bool,float,int}. This deprecation broke > > > our prerelease testing in scikit-image (which, hooray for rcs!), > > > and resulted in a large amount of code churn to fix [2]. > > > > > > To be honest, I do think *some* sort of deprecation is needed, > > > because for the longest time I thought that np.float was what > > > np.float_ actually is. I think it would be worthwhile to move to > > > *that*, though it?s an even more invasive deprecation than the > > > currently proposed one. Writing `x = np.zeros(5, dtype=int)` is > > > somewhat magical, because someone with a strict typing mindset > > > (there?s an increasing number!) might expect that this is an array > > > of pointers to Python ints. This is why I?ve always preferred to > > > write `dtype=np.int`, resulting in the current code churn. > > > > > > I don?t know what the best answer is, just sparking the discussion > > > Sebastian wants to see. ;) For skimage we?ve already merged a fix > > > (even if it is one of dubious quality, as St?fan points out [3] ;), > > > so I don?t have too much stake in the outcome. > > > > > > Juan. > > > > > > [1]: > > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > > > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > > > [3]: > > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion at python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From robert.kern at gmail.com Wed Dec 9 16:27:14 2020 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 9 Dec 2020 16:27:14 -0500 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Wed, Dec 9, 2020 at 4:08 PM Aaron Meurer wrote: > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg > wrote: > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > > > Regarding np.bool specifically, if you want to deprecate this, you > > > might want to discuss this with us at the array API standard > > > https://github.com/data-apis/array-api (which is currently in RFC > > > stage). The spec uses bool as the name for the boolean dtype. > > > > > > Would it make sense for NumPy to change np.bool to just be the > > > boolean > > > dtype object? Unlike int and float, there is no ambiguity with bool, > > > and NumPy clearly doesn't have any issues with shadowing builtin > > > names > > > in its namespace. > > > > We could keep the Python alias around (which for `dtype=` is the same > > as `np.bool_`). > > > > I am not sure I like the idea of immediately shadowing the builtin. > > That is a switch we can avoid flipping (without warning); `np.bool_` > > and `bool` are fairly different beasts? [1] > > NumPy already shadows a lot of builtins, in many cases, in ways that > are incompatible with existing ones. It's not something I would have > done personally, but it's been this way for a long time. > Sometimes, we had the function first before Python added them to the builtins (e.g. sum(), any(), all(), IIRC). I think max() and min() are the main ones that we added after Python did, and we explicitly exclude them from __all__ to avoid clobbering the builtins. Shadowing the types (bool, int, float) historically tended to be more problematic than those functions. The first releases of numpy _did_ have those as the scalar types. That empirically turned out to cause more problems for people than sum() or any(), so we renamed the scalar types to have the trailing underscore. We only left the shadowed names as aliases for the builtins because enough people still had `dtype=np.float` in their code that we didn't want to break. All that said, "from numpy import *" is less common these days. We have been pretty successful at getting people on board with the np campaign. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Wed Dec 9 16:37:37 2020 From: shoyer at gmail.com (Stephan Hoyer) Date: Wed, 9 Dec 2020 13:37:37 -0800 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Wed, Dec 9, 2020 at 1:07 PM Aaron Meurer wrote: > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg > wrote: > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > > > Regarding np.bool specifically, if you want to deprecate this, you > > > might want to discuss this with us at the array API standard > > > https://github.com/data-apis/array-api (which is currently in RFC > > > stage). The spec uses bool as the name for the boolean dtype. > > > > > > Would it make sense for NumPy to change np.bool to just be the > > > boolean > > > dtype object? Unlike int and float, there is no ambiguity with bool, > > > and NumPy clearly doesn't have any issues with shadowing builtin > > > names > > > in its namespace. > > > > We could keep the Python alias around (which for `dtype=` is the same > > as `np.bool_`). > > > > I am not sure I like the idea of immediately shadowing the builtin. > > That is a switch we can avoid flipping (without warning); `np.bool_` > > and `bool` are fairly different beasts? [1] > > NumPy already shadows a lot of builtins, in many cases, in ways that > are incompatible with existing ones. It's not something I would have > done personally, but it's been this way for a long time. > It may be defensible to keep np.bool as an alias for Python's bool even when we remove the other aliases. np.int_ and np.float_ have fixed precision, which makes them somewhat different from the builtin types. NumPy has a whole bunch of different precisions for integer and floats, so this distinction matters. In contrast, there is only one boolean dtype in NumPy, which matches Python's bool. So we wouldn't have to worry, for example, about whether a user has requested a specific precision explicitly. This comes up in issues like type-promotion where libraries like JAX and PyTorch have special case logic for most Python types vs NumPy dtypes (but booleans are the same for both): https://jax.readthedocs.io/en/latest/type_promotion.html > > Aaron Meurer > > > OTOH, if someone wants to entertain switching... It could be > > interesting to see how (unfixed) downstream projects react to it. > > > > One approach would be: > > > > * Go ahead for now (deprecate) > > * Add a FutureWarning at some point that we _will_ start to export > > `np.bool` again (but `from numpy import *` is a problem?) > > * Aim to make `np.bool is np.bool_` at some point in the (far) future. > > > > It is multi-step (and I recall opinions that multi-step is bad). > > Although, I think the main argument against it was to not force users > > to modify code more than once. And I do not think that happens here. > > > > Of course we could use the `FutureWarning` right away, but I don't mind > > taking it slow. > > > > Cheers, > > > > Sebastian > > > > > > > > [1] I admit, probably almost nobody would notice. And usually using a > > Python `bool` is better... > > > > > > > > > > Aaron Meurer > > > > > > On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias > > > wrote: > > > > Hi all, > > > > > > > > At the prodding [1] of Sebastian, I?m starting a discussion on the > > > > decision to deprecate np.{bool,float,int}. This deprecation broke > > > > our prerelease testing in scikit-image (which, hooray for rcs!), > > > > and resulted in a large amount of code churn to fix [2]. > > > > > > > > To be honest, I do think *some* sort of deprecation is needed, > > > > because for the longest time I thought that np.float was what > > > > np.float_ actually is. I think it would be worthwhile to move to > > > > *that*, though it?s an even more invasive deprecation than the > > > > currently proposed one. Writing `x = np.zeros(5, dtype=int)` is > > > > somewhat magical, because someone with a strict typing mindset > > > > (there?s an increasing number!) might expect that this is an array > > > > of pointers to Python ints. This is why I?ve always preferred to > > > > write `dtype=np.int`, resulting in the current code churn. > > > > > > > > I don?t know what the best answer is, just sparking the discussion > > > > Sebastian wants to see. ;) For skimage we?ve already merged a fix > > > > (even if it is one of dubious quality, as St?fan points out [3] ;), > > > > so I don?t have too much stake in the outcome. > > > > > > > > Juan. > > > > > > > > [1]: > > > > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > > > > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > > > > [3]: > > > > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > > > > _______________________________________________ > > > > NumPy-Discussion mailing list > > > > NumPy-Discussion at python.org > > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion at python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fangzh at umich.edu Wed Dec 9 17:22:37 2020 From: fangzh at umich.edu (Fang Zhang) Date: Wed, 9 Dec 2020 14:22:37 -0800 Subject: [Numpy-discussion] Feature request: Alternative representation for arrays with many dimensions Message-ID: By default, the __repr__ and __str__ functions of NumPy arrays summarize long arrays (i.e. omit all items but a few at beginning and end of each dimension), which is a good thing because when debugging, programmers can call print() on arrays with millions of elements without clogging the output or taking up too much CPU/memory (unsurprisingly, the string representation of an array item usually takes more bytes than its binary representation). However, this mechanic does not help when an array has a lot of short dimensions, e.g. np.arange(2 ** 20).reshape((2,) * 20). I often encounter such arrays in my work, and every once in a while I would try to print such an array without flattening it first (usually because I didn't know what shape or even what type the variable I was trying to print is), which has caused incidents ranging from losing everything in my scrollback buffer to crashing my computer by using too much memory. I think it may be a good idea to change the way NumPy pretty prints arrays with such shapes to avoid this situation. Something like "array([ 0, 1, 2, ..., 1048573, 1048574, 1048575]).reshape(2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2)" would be good enough for me. The condition to trigger such a representation can either be a fixed number of dimensions, or when after summarizing the pretty printer would still print more items than the threshold (1000 by default). Since the outputs of __repr__ and __str__ are meant for human eyes rather than computers, I think this should not cause too much of a compatibility problem. What do you all think? Sincerely, Fang Zhang -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Wed Dec 9 18:13:17 2020 From: shoyer at gmail.com (Stephan Hoyer) Date: Wed, 9 Dec 2020 15:13:17 -0800 Subject: [Numpy-discussion] Feature request: Alternative representation for arrays with many dimensions In-Reply-To: References: Message-ID: On Wed, Dec 9, 2020 at 2:24 PM Fang Zhang wrote: > By default, the __repr__ and __str__ functions of NumPy arrays summarize > long arrays (i.e. omit all items but a few at beginning and end of each > dimension), which is a good thing because when debugging, programmers can > call print() on arrays with millions of elements without clogging the > output or taking up too much CPU/memory (unsurprisingly, the string > representation of an array item usually takes more bytes than its binary > representation). > > However, this mechanic does not help when an array has a lot of short > dimensions, e.g. np.arange(2 ** 20).reshape((2,) * 20). I often encounter > such arrays in my work, and every once in a while I would try to print such > an array without flattening it first (usually because I didn't know what > shape or even what type the variable I was trying to print is), which has > caused incidents ranging from losing everything in my scrollback buffer to > crashing my computer by using too much memory. > > I think it may be a good idea to change the way NumPy pretty prints arrays > with such shapes to avoid this situation. Something like "array([ 0, 1, 2, > ..., 1048573, 1048574, 1048575]).reshape(2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, > 2, 2, 2, 2, 2, 2, 2, 2, 2)" would be good enough for me. The condition to > trigger such a representation can either be a fixed number of dimensions, > or when after summarizing the pretty printer would still print more items > than the threshold (1000 by default). Since the outputs of __repr__ and > __str__ are meant for human eyes rather than computers, I think this should > not cause too much of a compatibility problem. > +1, this could use improvement. For high dimensional arrays, the way NumPy prints is way too verbose. In xarray, we automatically decrease "edgeitems" for printing NumPy arrays, to 2 for ndim=3 and 1 for ndim>3: https://github.com/pydata/xarray/blob/9802411b35291a6149d850e8e573cde71a93bfbf/xarray/core/formatting.py#L439-L453 As a last resort, we could consider automatically limiting the maximum number of displayed lines, adding "..." for clipped lines. It is unlikely, for example, that anymore ever wants to print more than ~100 lines of text to the screen, which can easily happen for very high dimensional arrays. > What do you all think? > > Sincerely, > Fang Zhang > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Wed Dec 9 19:17:22 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 9 Dec 2020 17:17:22 -0700 Subject: [Numpy-discussion] Inclusion of licenses Message-ID: Hi All, Currently we append appropriate platform licenses to the LICENSE.txt file when building wheels for release. This means that there are uncommitted changes which shows up in the versioneer version as 'dirty', see the nightly files. This is unfortunate, but accurate :) There are at least two possible solutions to this problem. 1. Patch versioneer to omit the dirty string, very easy to do. 2. Put the platform specific file in the repo or combine them in the LICENSE file. I don't recall why we did things the way we do, but there was a discussion. Patching is easy, but the second option seems preferable. In particular, folks who now build their own NumPy wheels aren't going to have the license files. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Dec 10 00:41:42 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 9 Dec 2020 22:41:42 -0700 Subject: [Numpy-discussion] Inclusion of licenses In-Reply-To: References: Message-ID: On Wed, Dec 9, 2020 at 5:17 PM Charles R Harris wrote: > Hi All, > > Currently we append appropriate platform licenses to the LICENSE.txt file > when building wheels for release. This means that there are > uncommitted changes which shows up in the versioneer version as 'dirty', > see the nightly files. This is unfortunate, but accurate :) There are at > least two possible solutions to this problem. > > 1. Patch versioneer to omit the dirty string, very easy to do. > 2. Put the platform specific file in the repo or combine them in the > LICENSE file. > > I don't recall why we did things the way we do, but there was a > discussion. Patching is easy, but the second option seems preferable. In > particular, folks who now build their own NumPy wheels aren't going to have > the license files. > > Chuck > Note that LICENSES_bundled.txt, excluded from the sdist in MANIFEST.in, is included in the wheel in the dist-info file. charris at fc [numpy.git (master)]$ ls dist/numpy-1.21.0.dev0+135.g26f8b11b6e.dist-info entry_points.txt LICENSES_bundled.txt LICENSE.txt METADATA RECORD top_level.txt WHEEL Looks like any LICENSE* files in the root directory will be included in the wheel. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Thu Dec 10 04:34:28 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Thu, 10 Dec 2020 10:34:28 +0100 Subject: [Numpy-discussion] Inclusion of licenses In-Reply-To: References: Message-ID: On Thu, Dec 10, 2020 at 6:42 AM Charles R Harris wrote: > > > On Wed, Dec 9, 2020 at 5:17 PM Charles R Harris > wrote: > >> Hi All, >> >> Currently we append appropriate platform licenses to the LICENSE.txt >> file when building wheels for release. This means that there are >> uncommitted changes which shows up in the versioneer version as 'dirty', >> see the nightly files. This is unfortunate, but accurate :) There are at >> least two possible solutions to this problem. >> >> 1. Patch versioneer to omit the dirty string, very easy to do. >> 2. Put the platform specific file in the repo or combine them in the >> LICENSE file. >> >> I don't recall why we did things the way we do, but there was a >> discussion. Patching is easy, but the second option seems preferable. In >> particular, folks who now build their own NumPy wheels aren't going to have >> the license files. >> > The reason for that construct is that GitHub won't recognize the license if we add vendored info. As a result, it would not only not display the license in its UI, but also it provides an API to query the license for a repo which then gives the wrong result. That in turn throws off Tidelift, which uses two sources of licensing info in its service (GitHub and libraries.io) and those should match. Please consider this an issue with versioneer, and choose (1) Note that LICENSES_bundled.txt, excluded from the sdist in MANIFEST.in, is > included in the wheel in the dist-info file. > Ah, that needs fixing then. Cheers, Ralf > charris at fc [numpy.git (master)]$ ls > dist/numpy-1.21.0.dev0+135.g26f8b11b6e.dist-info > entry_points.txt LICENSES_bundled.txt LICENSE.txt METADATA RECORD > top_level.txt WHEEL > > Looks like any LICENSE* files in the root directory will be included in > the wheel. > > Chuck > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Dec 10 10:28:28 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 10 Dec 2020 08:28:28 -0700 Subject: [Numpy-discussion] Inclusion of licenses In-Reply-To: References: Message-ID: On Thu, Dec 10, 2020 at 2:35 AM Ralf Gommers wrote: > > > On Thu, Dec 10, 2020 at 6:42 AM Charles R Harris < > charlesr.harris at gmail.com> wrote: > >> >> >> On Wed, Dec 9, 2020 at 5:17 PM Charles R Harris < >> charlesr.harris at gmail.com> wrote: >> >>> Hi All, >>> >>> Currently we append appropriate platform licenses to the LICENSE.txt >>> file when building wheels for release. This means that there are >>> uncommitted changes which shows up in the versioneer version as 'dirty', >>> see the nightly files. This is unfortunate, but accurate :) There are at >>> least two possible solutions to this problem. >>> >>> 1. Patch versioneer to omit the dirty string, very easy to do. >>> 2. Put the platform specific file in the repo or combine them in the >>> LICENSE file. >>> >>> I don't recall why we did things the way we do, but there was a >>> discussion. Patching is easy, but the second option seems preferable. In >>> particular, folks who now build their own NumPy wheels aren't going to have >>> the license files. >>> >> > The reason for that construct is that GitHub won't recognize the license > if we add vendored info. As a result, it would not only not display the > license in its UI, but also it provides an API to query the license for a > repo which then gives the wrong result. That in turn throws off Tidelift, > which uses two sources of licensing info in its service (GitHub and > libraries.io) and those should match. > > Please consider this an issue with versioneer, and choose (1) > > Note that LICENSES_bundled.txt, excluded from the sdist in MANIFEST.in, is >> included in the wheel in the dist-info file. >> > > Ah, that needs fixing then. > > Seems setup can be called with an option to use MANIFEST.in, I'll experiment a bit. Since the bundled license is only included in `dist-info` it may also be a bug in setuptools. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Thu Dec 10 13:19:43 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Thu, 10 Dec 2020 12:19:43 -0600 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Wed, 2020-12-09 at 13:37 -0800, Stephan Hoyer wrote: > On Wed, Dec 9, 2020 at 1:07 PM Aaron Meurer > wrote: > > > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg > > wrote: > > > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > > > > Regarding np.bool specifically, if you want to deprecate this, > > > > you > > > > might want to discuss this with us at the array API standard > > > > https://github.com/data-apis/array-api?(which is currently in > > > > RFC > > > > stage). The spec uses bool as the name for the boolean dtype. > > > > > > > > Would it make sense for NumPy to change np.bool to just be the > > > > boolean > > > > dtype object? Unlike int and float, there is no ambiguity with > > > > bool, > > > > and NumPy clearly doesn't have any issues with shadowing > > > > builtin > > > > names > > > > in its namespace. > > > > > > We could keep the Python alias around (which for `dtype=` is the > > > same > > > as `np.bool_`). > > > > > > I am not sure I like the idea of immediately shadowing the > > > builtin. > > > That is a switch we can avoid flipping (without warning); > > > `np.bool_` > > > and `bool` are fairly different beasts? [1] > > > > NumPy already shadows a lot of builtins, in many cases, in ways > > that > > are incompatible with existing ones. It's not something I would > > have > > done personally, but it's been this way for a long time. > > > > It may be defensible to keep np.bool as an alias for Python's bool > even > when we remove the other aliases. That is true, `int` is probably the most confusing, since it is not at all compatible to a Python integer, but rather the "default" integer (which happens to be the same as C `long` currently). So we could focus on `np.int`, `np.long`. I am a bit unsure whether you would prefer that or are mainly pointing out the possibility? Right now, my main take-away from the discussion is that it would be good to clarify the release notes a bit more. Using `float` for a dtype seems fine to me, but I prefer mentioning `np.float64` over `np.float_`. For integers, I wonder if we should also suggest `np.int64`, even ? or because ? if the default integer on many systems is currently `np.int_`? Cheers, Sebastian > > np.int_ and np.float_ have fixed precision, which makes them somewhat > different from the builtin types. NumPy has a whole bunch of > different > precisions for integer and floats, so this distinction matters. > > In contrast, there is only one boolean dtype in NumPy, which matches > Python's bool. So we wouldn't have to worry, for example, about > whether a > user has requested a specific precision explicitly. This comes up in > issues > like type-promotion where libraries like JAX and PyTorch have special > case > logic for most Python types vs NumPy dtypes (but booleans are the > same for > both): > https://jax.readthedocs.io/en/latest/type_promotion.html > > > > > > > Aaron Meurer > > > > > OTOH, if someone wants to entertain switching... It could be > > > interesting to see how (unfixed) downstream projects react to it. > > > > > > One approach would be: > > > > > > * Go ahead for now (deprecate) > > > * Add a FutureWarning at some point that we _will_ start to > > > export > > > ? `np.bool` again (but `from numpy import *` is a problem?) > > > * Aim to make `np.bool is np.bool_` at some point in the (far) > > > future. > > > > > > It is multi-step (and I recall opinions that multi-step is bad). > > > Although, I think the main argument against it was to not force > > > users > > > to modify code more than once.? And I do not think that happens > > > here. > > > > > > Of course we could use the `FutureWarning` right away, but I > > > don't mind > > > taking it slow. > > > > > > Cheers, > > > > > > Sebastian > > > > > > > > > > > > [1] I admit, probably almost nobody would notice. And usually > > > using a > > > Python `bool` is better... > > > > > > > > > > > > > > Aaron Meurer > > > > > > > > On Sat, Dec 5, 2020 at 4:31 PM Juan Nunez-Iglesias < > > > > jni at fastmail.com> > > > > wrote: > > > > > Hi all, > > > > > > > > > > At the prodding [1] of Sebastian, I?m starting a discussion > > > > > on the > > > > > decision to deprecate np.{bool,float,int}. This deprecation > > > > > broke > > > > > our prerelease testing in scikit-image (which, hooray for > > > > > rcs!), > > > > > and resulted in a large amount of code churn to fix [2]. > > > > > > > > > > To be honest, I do think *some* sort of deprecation is > > > > > needed, > > > > > because for the longest time I thought that np.float was what > > > > > np.float_ actually is. I think it would be worthwhile to move > > > > > to > > > > > *that*, though it?s an even more invasive deprecation than > > > > > the > > > > > currently proposed one. Writing `x = np.zeros(5, dtype=int)` > > > > > is > > > > > somewhat magical, because someone with a strict typing > > > > > mindset > > > > > (there?s an increasing number!) might expect that this is an > > > > > array > > > > > of pointers to Python ints. This is why I?ve always preferred > > > > > to > > > > > write `dtype=np.int`, resulting in the current code churn. > > > > > > > > > > I don?t know what the best answer is, just sparking the > > > > > discussion > > > > > Sebastian wants to see. ;) For skimage we?ve already merged a > > > > > fix > > > > > (even if it is one of dubious quality, as St?fan points out > > > > > [3] ;), > > > > > so I don?t have too much stake in the outcome. > > > > > > > > > > Juan. > > > > > > > > > > [1]: > > > > > > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739334463 > > > > > [2]: https://github.com/scikit-image/scikit-image/pull/5103 > > > > > [3]: > > > > > > > https://github.com/scikit-image/scikit-image/pull/5103#issuecomment-739368765 > > > > > _______________________________________________ > > > > > NumPy-Discussion mailing list > > > > > NumPy-Discussion at python.org > > > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > > > > NumPy-Discussion mailing list > > > > NumPy-Discussion at python.org > > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion at python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From ralf.gommers at gmail.com Thu Dec 10 14:38:56 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Thu, 10 Dec 2020 20:38:56 +0100 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Thu, Dec 10, 2020 at 7:25 PM Sebastian Berg wrote: > On Wed, 2020-12-09 at 13:37 -0800, Stephan Hoyer wrote: > > On Wed, Dec 9, 2020 at 1:07 PM Aaron Meurer > > wrote: > > > > > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg > > > wrote: > > > > > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > > > > > Regarding np.bool specifically, if you want to deprecate this, > > > > > you > > > > > might want to discuss this with us at the array API standard > > > > > https://github.com/data-apis/array-api (which is currently in > > > > > RFC > > > > > stage). The spec uses bool as the name for the boolean dtype. > > > > > > > > > > Would it make sense for NumPy to change np.bool to just be the > > > > > boolean > > > > > dtype object? Unlike int and float, there is no ambiguity with > > > > > bool, > > > > > and NumPy clearly doesn't have any issues with shadowing > > > > > builtin > > > > > names > > > > > in its namespace. > > > > > > > > We could keep the Python alias around (which for `dtype=` is the > > > > same > > > > as `np.bool_`). > > > > > > > > I am not sure I like the idea of immediately shadowing the > > > > builtin. > > > > That is a switch we can avoid flipping (without warning); > > > > `np.bool_` > > > > and `bool` are fairly different beasts? [1] > > > > > > NumPy already shadows a lot of builtins, in many cases, in ways > > > that > > > are incompatible with existing ones. It's not something I would > > > have > > > done personally, but it's been this way for a long time. > > > > > > > It may be defensible to keep np.bool as an alias for Python's bool > > even when we remove the other aliases. > I'd agree with that. > That is true, `int` is probably the most confusing, since it is not at > all compatible to a Python integer, but rather the "default" integer > (which happens to be the same as C `long` currently). > > So we could focus on `np.int`, `np.long`. I am a bit unsure whether > you would prefer that or are mainly pointing out the possibility? > Not sure what you mean with focus, focus on describing in the release notes? Deprecating `np.int` seems like the most beneficial part of this whole exercise. Right now, my main take-away from the discussion is that it would be > good to clarify the release notes a bit more. > > Using `float` for a dtype seems fine to me, but I prefer mentioning > `np.float64` over `np.float_`. > For integers, I wonder if we should also suggest `np.int64`, even ? or > because ? if the default integer on many systems is currently > `np.int_`? > I agree. I think we should recommend sane, descriptive names that do the right thing. So ideally we'd have people spell their dtype specifiers as dtype=bool # or np.bool dtype=np.float64 dtype=np.int64 dtype=np.complex128 The names with underscores at the end make little sense from a UX perspective. And the C equivalents (single/double/etc) made sense 15 years ago, but with the user base of today - the majority of whom will not know C fluently or at all - also don't make too much sense. The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 and 64 bits is likely to be a pitfall much more often than it is what the user actually needs, so shouldn't be recommended and probably deserves a warning in the docs. Cheers, Ralf > > > > > np.int_ and np.float_ have fixed precision, which makes them somewhat > > different from the builtin types. NumPy has a whole bunch of > > different > > precisions for integer and floats, so this distinction matters. > > > > In contrast, there is only one boolean dtype in NumPy, which matches > > Python's bool. So we wouldn't have to worry, for example, about > > whether a > > user has requested a specific precision explicitly. This comes up in > > issues > > like type-promotion where libraries like JAX and PyTorch have special > > case > > logic for most Python types vs NumPy dtypes (but booleans are the > > same for > > both): > > https://jax.readthedocs.io/en/latest/type_promotion.html > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Thu Dec 10 14:59:37 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Thu, 10 Dec 2020 13:59:37 -0600 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Thu, 2020-12-10 at 20:38 +0100, Ralf Gommers wrote: > On Thu, Dec 10, 2020 at 7:25 PM Sebastian Berg < > sebastian at sipsolutions.net> > wrote: > > > On Wed, 2020-12-09 at 13:37 -0800, Stephan Hoyer wrote: > > > On Wed, Dec 9, 2020 at 1:07 PM Aaron Meurer > > > wrote: > > > > > > > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg > > > > wrote: > > > > > > > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > > > > > > Regarding np.bool specifically, if you want to deprecate > > > > > > this, > > > > > > you > > > > > > might want to discuss this with us at the array API > > > > > > standard > > > > > > https://github.com/data-apis/array-api?(which is currently > > > > > > in > > > > > > RFC > > > > > > stage). The spec uses bool as the name for the boolean > > > > > > dtype. > > > > > > > > > > > > Would it make sense for NumPy to change np.bool to just be > > > > > > the > > > > > > boolean > > > > > > dtype object? Unlike int and float, there is no ambiguity > > > > > > with > > > > > > bool, > > > > > > and NumPy clearly doesn't have any issues with shadowing > > > > > > builtin > > > > > > names > > > > > > in its namespace. > > > > > > > > > > We could keep the Python alias around (which for `dtype=` is > > > > > the > > > > > same > > > > > as `np.bool_`). > > > > > > > > > > I am not sure I like the idea of immediately shadowing the > > > > > builtin. > > > > > That is a switch we can avoid flipping (without warning); > > > > > `np.bool_` > > > > > and `bool` are fairly different beasts? [1] > > > > > > > > NumPy already shadows a lot of builtins, in many cases, in ways > > > > that > > > > are incompatible with existing ones. It's not something I would > > > > have > > > > done personally, but it's been this way for a long time. > > > > > > > > > > It may be defensible to keep np.bool as an alias for Python's > > > bool > > > even when we remove the other aliases. > > > > I'd agree with that. > > > > That is true, `int` is probably the most confusing, since it is not > > at > > all compatible to a Python integer, but rather the "default" > > integer > > (which happens to be the same as C `long` currently). > > > > So we could focus on `np.int`, `np.long`.? I am a bit unsure > > whether > > you would prefer that or are mainly pointing out the possibility? > > > > Not sure what you mean with focus, focus on describing in the release > notes? Deprecating `np.int` seems like the most beneficial part of > this > whole exercise. > I meant limiting the current deprecation to `np.int`, maybe `np.long`, and a "carefully chosen" set. To be honest, I don't mind either way, so any stronger opinion will tip the scale for me personally (my default currently is to update the release notes to recommend the more descriptive names). There are probably more doc updates that would be nice, I will suggest updating a separate issue for that. > Right now, my main take-away from the discussion is that it would be > > good to clarify the release notes a bit more. > > > > Using `float` for a dtype seems fine to me, but I prefer mentioning > > `np.float64` over `np.float_`. > > For integers, I wonder if we should also suggest `np.int64`, even ? > > or > > because ? if the default integer on many systems is currently > > `np.int_`? > > > > I agree. I think we should recommend sane, descriptive names that do > the > right thing. So ideally we'd have people spell their dtype specifiers > as > ? dtype=bool? # or np.bool > ? dtype=np.float64 > ? dtype=np.int64 > ? dtype=np.complex128 > The names with underscores at the end make little sense from a UX > perspective. And the C equivalents (single/double/etc) made sense 15 > years > ago, but with the user base of today - the majority of whom will not > know C > fluently or at all - also don't make too much sense. > > The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 and > 64 > bits is likely to be a pitfall much more often than it is what the > user > actually needs, so shouldn't be recommended and probably deserves a > warning > in the docs. Right, there is one slight trickery because `np.intp` is often a great integer dtype to use, because it is the integer that NumPy uses for all things related to indexing and array sizes. (I would be happy to dig out my PR making `np.intp` the default NumPy integer.) Cheers, Sebastian > > Cheers, > Ralf > > > > > > > > > > np.int_ and np.float_ have fixed precision, which makes them > > > somewhat > > > different from the builtin types. NumPy has a whole bunch of > > > different > > > precisions for integer and floats, so this distinction matters. > > > > > > In contrast, there is only one boolean dtype in NumPy, which > > > matches > > > Python's bool. So we wouldn't have to worry, for example, about > > > whether a > > > user has requested a specific precision explicitly. This comes up > > > in > > > issues > > > like type-promotion where libraries like JAX and PyTorch have > > > special > > > case > > > logic for most Python types vs NumPy dtypes (but booleans are the > > > same for > > > both): > > > https://jax.readthedocs.io/en/latest/type_promotion.html > > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From tyler.je.reddy at gmail.com Thu Dec 10 21:49:21 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Thu, 10 Dec 2020 19:49:21 -0700 Subject: [Numpy-discussion] ANN: SciPy 1.6.0rc1 -- please test Message-ID: Hi all, On behalf of the SciPy development team I'm pleased to announce the release candidate SciPy 1.6.0rc1. Please help us test this pre-release. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.6.0rc1 One of a few ways to install the release candidate with pip: pip install scipy==1.6.0rc1 ========================== SciPy 1.6.0 Release Notes ========================== Note: Scipy 1.6.0 is not released yet! SciPy 1.6.0 is the culmination of 6 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Before upgrading, we recommend that users check that their own code does not use deprecated SciPy functionality (to do so, run your code with ``python -Wd`` and check for ``DeprecationWarning`` s). Our development attention will now shift to bug-fix releases on the 1.6.x branch, and on adding new features on the master branch. This release requires Python 3.7+ and NumPy 1.16.5 or greater. For running on PyPy, PyPy3 6.0+ is required. Highlights of this release --------------------------------- - `scipy.ndimage` improvements: Fixes and ehancements to boundary extension modes for interpolation functions. Support for complex-valued inputs in many filtering and interpolation functions. New ``grid_mode`` option for `scipy.ndimage.zoom` to enable results consistent with scikit-image's ``rescale``. - `scipy.optimize.linprog` has fast, new methods for large, sparse problems from the ``HiGHS`` library. - `scipy.stats` improvements including new distributions, a new test, and enhancements to existing distributions and tests New features ============ `scipy.special` improvements ---------------------------------------- `scipy.special` now has improved support for 64-bit ``LAPACK`` backend `scipy.odr` improvements ----------------------------------- `scipy.odr` now has support for 64-bit integer ``BLAS`` `scipy.odr.ODR` has gained an optional ``overwrite`` argument so that existing files may be overwritten. `scipy.integrate` improvements ------------------------------------------ Some renames of functions with poor names were done, with the old names retained without being in the reference guide for backwards compatibility reasons: - ``integrate.simps`` was renamed to ``integrate.simpson`` - ``integrate.trapz`` was renamed to ``integrate.trapezoid`` - ``integrate.cumtrapz`` was renamed to ``integrate.cumulative_trapezoid`` `scipy.cluster` improvements --------------------------------------- `scipy.cluster.hierarchy.DisjointSet` has been added for incremental connectivity queries. `scipy.cluster.hierarchy.dendrogram` return value now also includes leaf color information in `leaves_color_list`. `scipy.interpolate` improvements -------------------------------------------- `scipy.interpolate.interp1d` has a new method ``nearest-up``, similar to the existing method ``nearest`` but rounds half-integers up instead of down. `scipy.io` improvements -------------------------------- Support has been added for reading arbitrary bit depth integer PCM WAV files from 1- to 32-bit, including the commonly-requested 24-bit depth. `scipy.linalg` improvements ------------------------------------- The new function `scipy.linalg.matmul_toeplitz` uses the FFT to compute the product of a Toeplitz matrix with another matrix. `scipy.linalg.sqrtm` and `scipy.linalg.logm` have performance improvements thanks to additional Cython code. Python ``LAPACK`` wrappers have been added for ``pptrf``, ``pptrs``, ``ppsv``, ``pptri``, and ``ppcon``. `scipy.linalg.norm` and the ``svd`` family of functions will now use 64-bit integer backends when available. `scipy.ndimage` improvements ----------------------------------------- `scipy.ndimage.convolve`, `scipy.ndimage.correlate` and their 1d counterparts now accept both complex-valued images and/or complex-valued filter kernels. All convolution-based filters also now accept complex-valued inputs (e.g. ``gaussian_filter``, ``uniform_filter``, etc.). Multiple fixes and enhancements to boundary handling were introduced to `scipy.ndimage` interpolation functions (i.e. ``affine_transform``, ``geometric_transform``, ``map_coordinates``, ``rotate``, ``shift``, ``zoom``). A new boundary mode, ``grid-wrap`` was added which wraps images periodically, using a period equal to the shape of the input image grid. This is in contrast to the existing ``wrap`` mode which uses a period that is one sample smaller than the original signal extent along each dimension. A long-standing bug in the ``reflect`` boundary condition has been fixed and the mode ``grid-mirror`` was introduced as a synonym for ``reflect``. A new boundary mode, ``grid-constant`` is now available. This is similar to the existing ndimage ``constant`` mode, but interpolation will still performed at coordinate values outside of the original image extent. This ``grid-constant`` mode is consistent with OpenCV's ``BORDER_CONSTANT`` mode and scikit-image's ``constant`` mode. Spline pre-filtering (used internally by ``ndimage`` interpolation functions when ``order >= 2``), now supports all boundary modes rather than always defaulting to mirror boundary conditions. The standalone functions ``spline_filter`` and ``spline_filter1d`` have analytical boundary conditions that match modes ``mirror``, ``grid-wrap`` and ``reflect``. `scipy.ndimage` interpolation functions now accept complex-valued inputs. In this case, the interpolation is applied independently to the real and imaginary components. The ``ndimage`` tutorials (https://docs.scipy.org/doc/scipy/reference/tutorial/ndimage.html) have been updated with new figures to better clarify the exact behavior of all of the interpolation boundary modes. `scipy.ndimage.zoom` now has a ``grid_mode`` option that changes the coordinate of the center of the first pixel along an axis from 0 to 0.5. This allows resizing in a manner that is consistent with the behavior of scikit-image's ``resize`` and ``rescale`` functions (and OpenCV's ``cv2.resize``). `scipy.optimize` improvements ----------------------------------------- `scipy.optimize.linprog` has fast, new methods for large, sparse problems from the ``HiGHS`` C++ library. ``method='highs-ds'`` uses a high performance dual revised simplex implementation (HSOL), ``method='highs-ipm'`` uses an interior-point method with crossover, and ``method='highs'`` chooses between the two automatically. These methods are typically much faster and often exceed the accuracy of other ``linprog`` methods, so we recommend explicitly specifying one of these three method values when using ``linprog``. `scipy.optimize.quadratic_assignment` has been added for approximate solution of the quadratic assignment problem. `scipy.optimize.linear_sum_assignment` now has a substantially reduced overhead for small cost matrix sizes `scipy.optimize.least_squares` has improved performance when the user provides the jacobian as a sparse jacobian already in ``csr_matrix`` format `scipy.optimize.linprog` now has an ``rr_method`` argument for specification of the method used for redundancy handling, and a new method for this purpose is available based on the interpolative decomposition approach. `scipy.signal` improvements -------------------------------------- `scipy.signal.gammatone` has been added to design FIR or IIR filters that model the human auditory system. `scipy.signal.iircomb` has been added to design IIR peaking/notching comb filters that can boost/attenuate a frequency from a signal. `scipy.signal.sosfilt` performance has been improved to avoid some previously- observed slowdowns `scipy.signal.windows.taylor` has been added--the Taylor window function is commonly used in radar digital signal processing `scipy.signal.gauss_spline` now supports ``list`` type input for consistency with other related SciPy functions `scipy.signal.correlation_lags` has been added to allow calculation of the lag/ displacement indices array for 1D cross-correlation. `scipy.sparse` improvements --------------------------------------- A solver for the minimum weight full matching problem for bipartite graphs, also known as the linear assignment problem, has been added in `scipy.sparse.csgraph.min_weight_full_bipartite_matching`. In particular, this provides functionality analogous to that of `scipy.optimize.linear_sum_assignment`, but with improved performance for sparse inputs, and the ability to handle inputs whose dense representations would not fit in memory. The time complexity of `scipy.sparse.block_diag` has been improved dramatically from quadratic to linear. `scipy.sparse.linalg` improvements ----------------------------------------------- The vendored version of ``SuperLU`` has been updated `scipy.fft` improvements --------------------------------- The vendored ``pocketfft`` library now supports compiling with ARM neon vector extensions and has improved thread pool behavior. `scipy.spatial` improvements --------------------------------------- The python implementation of ``KDTree`` has been dropped and ``KDTree`` is now implemented in terms of ``cKDTree``. You can now expect ``cKDTree``-like performance by default. This also means ``sys.setrecursionlimit`` no longer needs to be increased for querying large trees. ``transform.Rotation`` has been updated with support for Modified Rodrigues Parameters alongside the existing rotation representations (PR gh-12667). `scipy.spatial.transform.Rotation` has been partially cythonized, with some performance improvements observed `scipy.spatial.distance.cdist` has improved performance with the ``minkowski`` metric, especially for p-norm values of 1 or 2. `scipy.stats` improvements ------------------------------------ New distributions have been added to `scipy.stats`: - The asymmetric Laplace continuous distribution has been added as `scipy.stats.laplace_asymmetric`. - The negative hypergeometric distribution has been added as `scipy.stats.nhypergeom`. - The multivariate t distribution has been added as `scipy.stats.multivariate_t`. - The multivariate hypergeometric distribution has been added as `scipy.stats.multivariate_hypergeom`. The ``fit`` method has been overridden for several distributions (``laplace``, ``pareto``, ``rayleigh``, ``invgauss``, ``logistic``, ``gumbel_l``, ``gumbel_r``); they now use analytical, distribution-specific maximum likelihood estimation results for greater speed and accuracy than the generic (numerical optimization) implementation. The one-sample Cram?r-von Mises test has been added as `scipy.stats.cramervonmises`. An option to compute one-sided p-values was added to `scipy.stats.ttest_1samp`, `scipy.stats.ttest_ind_from_stats`, `scipy.stats.ttest_ind` and `scipy.stats.ttest_rel`. The function `scipy.stats.kendalltau` now has an option to compute Kendall's tau-c (also known as Stuart's tau-c), and support has been added for exact p-value calculations for sample sizes ``> 171``. `stats.trapz` was renamed to `stats.trapezoid`, with the former name retained as an alias for backwards compatibility reasons. The function `scipy.stats.linregress` now includes the standard error of the intercept in its return value. The ``_logpdf``, ``_sf``, and ``_isf`` methods have been added to `scipy.stats.nakagami`; ``_sf`` and ``_isf`` methods also added to `scipy.stats.gumbel_r` The ``sf`` method has been added to `scipy.stats.levy` and `scipy.stats.levy_l` for improved precision. `scipy.stats.binned_statistic_dd` performance improvements for the following computed statistics: ``max``, ``min``, ``median``, and ``std``. We gratefully acknowledge the Chan-Zuckerberg Initiative Essential Open Source Software for Science program for supporting many of these improvements to `scipy.stats`. Deprecated features ================ `scipy.spatial` changes -------------------------------- Calling ``KDTree.query`` with ``k=None`` to find all neighbours is deprecated. Use ``KDTree.query_ball_point`` instead. ``distance.wminkowski`` was deprecated; use ``distance.minkowski`` and supply weights with the ``w`` keyword instead. Backwards incompatible changes ========================== `scipy` changes ---------------------- Using `scipy.fft` as a function aliasing ``numpy.fft.fft`` was removed after being deprecated in SciPy ``1.4.0``. As a result, the `scipy.fft` submodule must be explicitly imported now, in line with other SciPy subpackages. `scipy.signal` changes -------------------------------- The output of ``decimate``, ``lfilter_zi``, ``lfiltic``, ``sos2tf``, and ``sosfilt_zi`` have been changed to match ``numpy.result_type`` of their inputs. The window function ``slepian`` was removed. It had been deprecated since SciPy ``1.1``. `scipy.spatial` changes -------------------------------- ``cKDTree.query`` now returns 64-bit rather than 32-bit integers on Windows, making behaviour consistent between platforms (PR gh-12673). `scipy.stats` changes ----------------------------- The ``frechet_l`` and ``frechet_r`` distributions were removed. They were deprecated since SciPy ``1.0``. Other changes ============= ``setup_requires`` was removed from ``setup.py``. This means that users invoking ``python setup.py install`` without having numpy already installed will now get an error, rather than having numpy installed for them via ``easy_install``. This install method was always fragile and problematic, users are encouraged to use ``pip`` when installing from source. - - Fixed a bug in `scipy.optimize.dual_annealing` ``accept_reject`` calculation that caused uphill jumps to be accepted less frequently. - - The time required for (un)pickling of `scipy.stats.rv_continuous`, `scipy.stats.rv_discrete`, and `scipy.stats.rv_frozen` has been significantly reduced (gh12550). Inheriting subclasses should note that ``__setstate__`` no longer calls ``__init__`` upon unpickling. Authors ======= * @endolith * @vkk800 * aditya + * George Bateman + * Christoph Baumgarten * Peter Bell * Tobias Biester + * Keaton J. Burns + * Evgeni Burovski * R?diger Busche + * Matthias Bussonnier * Dominic C + * Corallus Caninus + * CJ Carey * Thomas A Caswell * chapochn + * Luc?a Cheung * Zach Colbert + * Coloquinte + * Yannick Copin + * Devin Crowley + * Terry Davis + * Micha?l Defferrard + * devonwp + * Didier + * divenex + * Thomas Duvernay + * Eoghan O'Connell + * G?k?en Eraslan * Kristian Eschenburg + * Ralf Gommers * Thomas Grainger + * GreatV + * Gregory Gundersen + * h-vetinari + * Matt Haberland * Mark Harfouche + * He He + * Alex Henrie * Chun-Ming Huang + * Martin James McHugh III + * Alex Izvorski + * Joey + * ST John + * Jonas Jonker + * Julius Bier Kirkegaard * Marcin Konowalczyk + * Konrad0 * Sam Van Kooten + * Sergey Koposov + * Peter Mahler Larsen * Eric Larson * Antony Lee * Gregory R. Lee * Lo?c Est?ve * Jean-Luc Margot + * MarkusKoebis + * Nikolay Mayorov * G. D. McBain * Andrew McCluskey + * Nicholas McKibben * Sturla Molden * Denali Molitor + * Eric Moore * Shashaank N + * Prashanth Nadukandi + * nbelakovski + * Andrew Nelson * Nick + * Nikola Forr? + * odidev * ofirr + * Sambit Panda * Dima Pasechnik * Tirth Patel + * Pawe? Redzy?ski + * Vladimir Philipenko + * Philipp Th?lke + * Ilhan Polat * Eugene Prilepin + * Vladyslav Rachek * Ram Rachum + * Tyler Reddy * Martin Reinecke + * Simon Segerblom Rex + * Lucas Roberts * Benjamin Rowell + * Eli Rykoff + * Atsushi Sakai * Moritz Schulte + * Daniel B. Smith * Steve Smith + * Jan Soedingrekso + * Victor Stinner + * Jose Storopoli + * Diana Sukhoverkhova + * S?ren Fuglede J?rgensen * taoky + * Mike Taves + * Ian Thomas + * Will Tirone + * Frank Torres + * Seth Troisi * Ronald van Elburg + * Hugo van Kemenade * Paul van Mulbregt * Saul Ivan Rivas Vega + * Pauli Virtanen * Jan Vleeshouwers * Samuel Wallan * Warren Weckesser * Ben West + * Eric Wieser * WillTirone + * Levi John Wolf + * Zhiqing Xiao * Rory Yorke + * Yun Wang (Maigo) + * Egor Zemlyanoy + * ZhihuiChen0903 + * Jacob Zhong + A total of 121 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.6.0 ------------------------------- * `#1323 `__: ndimage.shift destroys data from edges (Trac #796) * `#1892 `__: using rptfile= with an existing file causes a Fortran runtime... * `#1903 `__: ndimage.rotate misses some values (Trac #1378) * `#1930 `__: scipy.io.wavfile should be able to read 24 bit signed wave (Trac... * `#3158 `__: Odd casting behaviour of signal.filtfilt * `#3203 `__: interpolation.zoom incorrect output for certain cases * `#3645 `__: BUG: stats: mstats.pearsonr calculation is wrong if the masks... * `#3665 `__: Return Bunch objects from stats functions * `#4922 `__: unexpected zero output values from zoom * `#5202 `__: BUG: stats: Spurious warnings from the pdf method of several... * `#5223 `__: Zoom does not return the same values when resizing a sub-array... * `#5396 `__: scipy.spatial.distance.pdist documention bug * `#5489 `__: ValueError: failed to create intent(cache|hide)|optional array--... * `#6096 `__: loadmat drops dtype of empty arrays when squeeze_me=True * `#6713 `__: sicpy.ndimage.zoom returns artefacts and boundaries in some cases * `#7125 `__: Impossible to know number of dimensions in c function used by... * `#7324 `__: scipy.ndimage.zoom bad interpolation when downsampling (zoom... * `#8131 `__: BUG: geometric_transform wrap mode possible bug * `#8163 `__: LSMR fails on some random values when providing an x0 * `#8210 `__: Why should I choose order > 1 for scipy.ndimage.zoom? * `#8465 `__: Unexpected behavior with reflect mode of ndimage.rotate * `#8776 `__: cdist behavior with Minkowsky and np.inf * `#9168 `__: documentation of pearson3 in scipy.stats unclear * `#9223 `__: Faster implementation of scipy.sparse.block_diag * `#9476 `__: Invalid index in signal.medfilt2d's QUICK_SELECT * `#9857 `__: scipy.odr.Output.sd_beta is not standard error * `#9865 `__: Strange behavior of \`ndimage.shift\` and \`ndimage.affine_transform\` * `#10042 `__: Consider support for multivariate student-t distribution? * `#10134 `__: gausshyper distribution accepts invalid parameters * `#10179 `__: str+bytes concatenation error in test_lapack.py * `#10216 `__: cKDTree.query_ball_point speed regression * `#10463 `__: ENH: vectorize scipy.fft for more CPU architectures * `#10593 `__: Rename \`sum\` ndimage function * `#10595 `__: scipy.stats.ttest_1samp should support alternative hypothesis * `#10610 `__: ndimage.interpolation.spline_filter1d default value of mode * `#10620 `__: ndimage.interpolation.zoom() option to work like skimage.transform.resize() * `#10711 `__: Array Shapes Not Aligned Bug in scipy.optimize._lsq.lsq_linear.py * `#10782 `__: BUG: optimize: methods unknown to \`scipy.optimize.show_options\` * `#10892 `__: Possible typo in an equation of optimize/dual_annealing * `#11020 `__: signal.fftconvolve return a tuple including lag information * `#11093 `__: scipy.interpolate.interp1d can not handle datetime64 * `#11170 `__: Use manylinux2014 to get aarch64/ppc64le support * `#11186 `__: BUG: stats: pearson3 CDF and SF functions incorrect when skew... * `#11366 `__: DeprecationWarning due to invalid escape sequences * `#11403 `__: Optimize raises "ValueError: \`x0\` violates bound constraints"... * `#11558 `__: ENH: IIR comb filter * `#11559 `__: BUG: iirdesign doesn't fail for frequencies above Nyquist * `#11567 `__: scipy.signal.iirdesign doesn't check consistency of wp and ws... * `#11654 `__: ENH: Add Negative Hypergeometric Distribution * `#11720 `__: BUG: stats: wrong shape from median_absolute_deviation for arrays... * `#11746 `__: BUG: stats: pearson3 returns size 1 arrays where other distributions... * `#11756 `__: Improve and fix \*Spline docstrings and code * `#11758 `__: BUG: of scipy.interpolate.CubicSpline when \`bc_type' is set... * `#11925 `__: MAINT: remove character encoding check in CI? * `#11963 `__: Test failures - TestLinprogIPSparseCholmod * `#12102 `__: incorrect first moment of non central t-distribution * `#12113 `__: scipy.stats.poisson docs for rate = 0 * `#12152 `__: ENH: signal.gauss_spline should accept a list * `#12157 `__: BUG: Iteration index initialisation is wrong in scipy.optimize.linesearch.scalar_search_wolfe2 * `#12162 `__: Storing Rotation object in NumPy array returns an array with... * `#12176 `__: cannot modify the slice of an array returned by \`wavfile.read\` * `#12190 `__: retrieve leave colors from dendrogram * `#12196 `__: PERF: scipy.linalg.pinv is very slow compared to numpy.linalg.pinv * `#12222 `__: Interpolating categorical data (interp1d) * `#12231 `__: Is the p-value of the Kruskal-Wallis test two-sided? * `#12249 `__: ENH: least_squares: should not re-instanciate csr_matrix if already... * `#12264 `__: DOC: optimize: linprog method-specific function signature * `#12290 `__: DOC: Convex Hull areas are actually perimeters for 2-dimensional... * `#12308 `__: integrate.solve_ivp with DOP853 method fails when yDot = 0 * `#12326 `__: BUG: stats.exponnorm.pdf returns 0 for small K * `#12337 `__: scipy.sparse.linalg.eigsh documentation is misleading * `#12339 `__: scipy.io.wavfile.write documentation has wrong example * `#12340 `__: sparse.lil_matrix.tocsr() fails silently on matrices with nzn... * `#12350 `__: Create a 2-parameter version of the gamma distribution * `#12369 `__: scipy.signal.correlate has an error in the documentation, examples... * `#12373 `__: interp1d returns incorrect values for step functions * `#12378 `__: interpolate.NearestNDInterpolator.__call__ & LinearNDInterpolator.__call__... * `#12411 `__: scipy.stats.spearmanr mishandles nan variables with "propogate" * `#12413 `__: DOC: Remove the "Basic functions" section from the SciPy tutorial. * `#12415 `__: scipy.stats.dirichlet documentation issue * `#12419 `__: least_squares ValueError with 'lm' method - regression from 1.4.1... * `#12431 `__: Request for Python wrapper for LAPACK's ?pptrf (Cholesky factorization... * `#12458 `__: spearmanr with entire NaN columns produces errors * `#12477 `__: WIP: Addition of MLE for stats.invgauss/wald * `#12483 `__: reading .wav fails when the file is too big on python 3.6.0 * `#12490 `__: BUG: stats: logistic and genlogistic logpdf overflow for large... * `#12499 `__: LinearNDInterpolator raises ValueError when value array has writeable=False... * `#12523 `__: Wrong key in __odrpack.c * `#12547 `__: typo in scipy/cluster/_hierarchy.pyx * `#12549 `__: DOC: least_squares return type is poorly formatted. * `#12578 `__: TST: test_bounds_infeasible_2 failing on wheels repo cron jobs * `#12585 `__: ENH: Add Multivariate Hypergeometric Distribution * `#12604 `__: unintuitive conversion in \`scipy.constants.lambda2nu\` * `#12606 `__: DOC: Invalid syntax in example. * `#12665 `__: List of possible bugs found by automated code analysis * `#12696 `__: scipy.optimize.fminbound, numpy depreciation warning Creating... * `#12699 `__: TestProjections.test_iterative_refinements_dense failure * `#12701 `__: TestDifferentialEvolutionSolver::test_L4 failing * `#12719 `__: Misleading scipy.signal.get_window() docstring with 'exponential'... * `#12740 `__: circstd doesn't handle R = hypot(S, C) > 1 * `#12749 `__: ENH: interp1d Matlab compatibility * `#12773 `__: Meta-issue: ndimage spline boundary handling (NumFOCUS proposal) * `#12813 `__: optimize.root(method="krylov") fails if options["tol_norm"] expects... * `#12815 `__: stats.zscore inconsistent behavior when all values are the same * `#12840 `__: scipy.signal.windows.dpss docstring typo * `#12874 `__: Rotation.random vs stats.special_ortho_group * `#12881 `__: FFT - documentation - examples - linspace construction * `#12904 `__: BUG: parsing in loadarff() * `#12917 `__: GitHub Actions nightly build triggered on forks * `#12919 `__: BUG: numerical precision, use gammaln in nct.mean * `#12924 `__: Rename Sample Based Integration Methods to Comply with Code of... * `#12940 `__: Should the minimum numpy for AIX be bumped to 1.16.5 * `#12951 `__: A possible typo in scipy.stats.weightedtau * `#12952 `__: [Documentation question] Would it be more precise to specify... * `#12970 `__: Documentation presents second order sections as the correct choice... * `#12982 `__: Calculate standard error of the intercept in linregress * `#12985 `__: Possible wrong link in scipy.stats.wilcoxon doc * `#12991 `__: least_squares broken with float32 * `#13001 `__: \`OptimizeResult.message\` from \`L-BFGS-B\` is a bytes, not... * `#13030 `__: BUG: lint_diff.py still fails for backport PRs * `#13077 `__: CI: codecov proper patch diffs * `#13085 `__: Build failing on main branch after HiGHS solver merge * `#13088 `__: BLD, BUG: wheel builds failure with HiGHS/optimize * `#13099 `__: Wrong output format for empty sparse results of kron * `#13108 `__: TST, CI: GitHub Actions MacOS Failures * `#13111 `__: BUG, DOC: refguide check is failing * `#13127 `__: ODR output file writing broken in conda env with system compilers * `#13134 `__: FromTravis migration tracker * `#13140 `__: BUG: signal: \`ss2tf\` incorrectly truncates output to integers. * `#13179 `__: CI: lint is failing because of output to stderr * `#13182 `__: Key appears twice in \`test_optimize.test_show_options\` * `#13191 `__: \`scipy.linalg.lapack.dgesjv\` overwrites original arrays if... * `#13207 `__: TST: Erratic test failure in test_cossin_separate Pull requests for 1.6.0 ------------------------------ * `#8032 `__: ENH: Add in taylor window common in Radar processing * `#8779 `__: CI: Run benchmarks * `#9361 `__: ENH: Add Kendall's tau-a and tau-c variants to scipy.stats.kendalltau() * `#11068 `__: ENH: Adds correlation_lags function to scipy.signal * `#11119 `__: ENH: add Cramer-von-Mises (one-sample) test to scipy.stats * `#11249 `__: ENH: optimize: interpolative decomposition redundancy removal... * `#11346 `__: ENH: add fast toeplitz matrix multiplication using FFT * `#11413 `__: ENH: Multivariate t-distribution (stale) * `#11563 `__: ENH: exact p-value in stats.kendalltau() for sample sizes > 171 * `#11691 `__: ENH: add a stack of reversal functions to linprog * `#12043 `__: ENH: optimize: add HiGHS methods to linprog - continued * `#12061 `__: Check parameter consistensy in signal.iirdesign * `#12067 `__: MAINT: Cleanup OLDAPI in ndimage/src/_ctest.c * `#12069 `__: DOC: Add developer guidelines for implementing the nan_policy... * `#12077 `__: MAINT: malloc return value checks for cython * `#12080 `__: MAINT: Remove suppress_warnings * `#12085 `__: ENH: special: support ILP64 Lapack * `#12086 `__: MAINT: Cleanup PyMODINIT_FUNC used during 2to3 * `#12097 `__: ENH: stats: override stats.rayleigh.fit with analytical MLE * `#12112 `__: DOC: Improve integrate.nquad docstring * `#12125 `__: TST: Add a test for stats.gmean with negative input * `#12139 `__: TST: Reduce flakiness in lsmr test * `#12142 `__: DOC: add a note in poisson distribution when mu=0 and k=0 in... * `#12144 `__: DOC: Update ndimage.morphology.distance_transform\* * `#12154 `__: ENH: scipy.signal: allow lists in gauss_spline * `#12170 `__: ENH: scipy.stats: add negative hypergeometric distribution * `#12177 `__: MAINT: Correctly add input line to ValueError * `#12183 `__: ENH: Use fromfile where possible * `#12186 `__: MAINT: generalize tests in SphericalVoronoi * `#12198 `__: TST: Fix str + bytes error * `#12199 `__: ENH: match np.result_type behaviour in some scipy.signal functions * `#12200 `__: ENH: add FIR and IIR gammatone filters to scipy.signal * `#12204 `__: ENH: Add overwrite argument for odr.ODR() and its test. * `#12206 `__: MAINT:lstsq: Switch to tranposed problem if the array is tall * `#12208 `__: wavfile bugfixes and maintenance * `#12214 `__: DOC: fix docstring of "sd_beta" of odr.Output. * `#12234 `__: MAINT: prevent divide by zero warnings in scipy.optimize BFGS... * `#12235 `__: REL: set version to 1.6.0.dev0 * `#12237 `__: BUG: Fix exit condition for QUICK_SELECT pivot * `#12242 `__: ENH: Rename ndimage.sum to ndimage.sum_labels (keep sum as alias) * `#12243 `__: EHN: Update SuperLU * `#12244 `__: MAINT: stats: avoid spurious warnings in ncx2.pdf * `#12245 `__: DOC: Fixed incorrect default for mode in scipy.ndimage.spline_filter1d * `#12248 `__: MAINT: clean up pavement.py * `#12250 `__: ENH: Replaced csr_matrix() by tocsr() and complemented docstring * `#12253 `__: TST, CI: turn on codecov patch diffs * `#12259 `__: MAINT: Remove duplicated test for import cycles * `#12263 `__: ENH: Rename LocalSearchWrapper bounds * `#12265 `__: BUG optimize: Accept np.matrix in lsq_linear * `#12266 `__: BUG: Fix paren error in dual annealing accept_reject calculation * `#12269 `__: MAINT: Included mismatched shapes in error messages. * `#12279 `__: MAINT: \`__array__\` and array protocols cannot be used in sparse. * `#12281 `__: DOC: update wheel DL docs * `#12283 `__: ENH: odr: ILP64 Blas support in ODR * `#12284 `__: ENH: linalg: support for ILP64 BLAS/LAPACK in f2py wrappers * `#12286 `__: ENH: Cythonize scipy.spatial.transform.Rotation * `#12287 `__: ENH: Read arbitrary bit depth (including 24-bit) WAVs * `#12292 `__: BLD: fix musl compilation * `#12293 `__: MAINT: Fix a DeprecationWarning in validate_runtests_log.py. * `#12296 `__: DOC: Clarify area/volume in scipy.spatial.ConvexHull docstrings * `#12302 `__: CI: Run travis builds on master to keep cache up to date * `#12305 `__: TST: Cleanup print statements in tests * `#12323 `__: ENH: Add a Bunch-like class to use as a backwards compatible... * `#12324 `__: BUG: io: Fix an error that occurs when attempting to raise a... * `#12327 `__: DOC: clarify docstrings of \`query_ball_tree\` and \`query_pairs\` * `#12334 `__: PERF: Improve cKDTree.query_ball_point constant time cython overhead * `#12338 `__: DOC: improve consistency and clarity of docs in linalg and sparse/linalg * `#12341 `__: DOC: add Examples for KDTree query_ball_tree and query_pairs * `#12343 `__: DOC: add examples for special.eval_legendre() * `#12349 `__: BUG: avoid overflow in sum() for 32-bit systems * `#12351 `__: DOC: Fix example wavfile to be 16bit * `#12352 `__: [BUG] Consider 0/0 division in DOP853 error estimation * `#12353 `__: Fix exception causes in vq.py * `#12354 `__: MAINT: Cleanup unneeded void\* cast in setlist.pxd * `#12355 `__: TST: Remove hack for old win-amd64 bug * `#12356 `__: ENH: Faster implementation of scipy.sparse.block_diag (#9411... * `#12357 `__: MAINT,TST: update and run scipy/special/utils/convert.py * `#12358 `__: TST: Check mstat.skewtest pvalue * `#12359 `__: TST: Sparse matrix test with int64 indptr and indices * `#12363 `__: DOC: ref. in CloughTocher2DInterpolator * `#12364 `__: DOC: \`sparse_distance_matrix\` and \`count_neighbors\` examples * `#12371 `__: MAINT, CI: bump to latest stable OpenBLAS * `#12372 `__: MAINT: Minor cleanup of (c)KDTree tests * `#12374 `__: DEP: Deprecate \`distance.wminkowski\` * `#12375 `__: ENH: Add fast path for minkowski distance with p=1,2 and support... * `#12376 `__: Fix exception causes in most of the codebase * `#12377 `__: DOC: Quick fix - adds newline to correlation_lags docstring Examples... * `#12381 `__: BENCH: remove obsolete goal_time param * `#12382 `__: ENH: Replace KDTree with a thin wrapper over cKDTree * `#12385 `__: DOC: improve docstrings of interpolate.NearestNDInterpolator.__call__... * `#12387 `__: DOC/STY: add example to scipy.signal.correlate * `#12393 `__: CI: Replace the existing check for non-ASCII characters with... * `#12394 `__: CI: arm64 numpy now available * `#12395 `__: ENH: improve stats.binned_statistic_dd performance * `#12396 `__: DOC, MAINT: forward port 1.5.0 relnotes * `#12398 `__: API: Disable len() and indexing of Rotation instances with single... * `#12399 `__: MAINT: Replace some Unicode dash-like chars with an ASCII hyphen. * `#12402 `__: update .mailmap * `#12404 `__: MAINT: io: Change the encoding comment of test_mio.py to utf-8. * `#12416 `__: CI: cache mingw, azure pipelines * `#12427 `__: BUG: logic error in loop unrolling (cKDTree) * `#12432 `__: DOC: Remove the "Basic functions" section from the SciPy tutorial. * `#12434 `__: ENH:linalg: Add LAPACK wrappers pptrf/pptrs/ppsv/pptri/ppcon * `#12435 `__: DOC: fix simplex math for scipy.stats.dirichlet documentation * `#12439 `__: DOC: add API methods summary for NdPPoly * `#12443 `__: BUG: stats: Improve calculation of exponnorm.pdf * `#12448 `__: DOC: stats: Add "Examples" to the ansari docstring. * `#12450 `__: ENH: add \`leaves_color_list\` for cluster.dendrogram dictionary. * `#12451 `__: MAINT: remove "blacklist" terminology from code base * `#12452 `__: DOC: clarify the meaning of whitening for cluster.vq.whiten() * `#12455 `__: MAINT: clearer error message in setup.py * `#12457 `__: ENH: stats: override stats.pareto.fit with analytical MLE * `#12460 `__: check if column in spearman rho is entirely NaN or Inf * `#12463 `__: DOC: improve and clean up \*Spline docstrings in fitpack2.py * `#12474 `__: ENH: linalg: speedup _sqrtm_triu by moving tight loop to Cython * `#12476 `__: ENH: add IIR comb filter to scipy.signal * `#12484 `__: Fix documentation for minimize * `#12486 `__: DOC: add a note in poisson distribution when mu=0 and k=0 in... * `#12491 `__: MAINT: forward port 1.5.1 release notes * `#12508 `__: Fix exception causes all over the codebase * `#12514 `__: ENH: stats: override stats.invgauss.fit with analytical MLE * `#12519 `__: PERF: Avoid np.zeros when custom initialization is needed anyway * `#12520 `__: DOC: Minor RST section renaming. * `#12521 `__: MAINT: Remove unused imports * `#12522 `__: PERF: Get rid of unnececssary allocation in VarReader5.cread_fieldnames * `#12524 `__: DOC: special: Set Axes3D rect to avoid clipping labels in plot. * `#12525 `__: Fix large sparse nnz * `#12526 `__: DOC: Remove double section and too long header underline. * `#12527 `__: Improve error message for wrong interpolation type * `#12530 `__: Move redundant logic outside loop for conditional speedup in... * `#12532 `__: ENH: Add norm={"forward", "backward"} to \`scipy.fft\` * `#12535 `__: MAINT: Avoid sphinx deprecated aliases for SeeAlso and Only * `#12540 `__: BUG: fix odr.output.work_ind key bug and add its test. * `#12541 `__: ENH: add solver for minimum weight full bipartite matching * `#12550 `__: PERF: pickling speed of rv\* * `#12551 `__: DOC: fix typo in cluster/_hierarchy.pyx * `#12552 `__: CI: Cleanup travis pip installs * `#12556 `__: BUG: Fix problem with Scipy.integrate.solve_bvp for big problems * `#12557 `__: MAINT: Use extern templates to improve sparsetools compile time * `#12558 `__: MAINT: Remove hack to allow scipy.fft to act like a function * `#12563 `__: MAINT: Remove unused mu0 in special/orthogonal.py * `#12564 `__: DOC: fix return type docstring for least_squares * `#12565 `__: DOC: stats: respond to query about Kruskal-Wallis test being... * `#12566 `__: BUG: Interpolate: use stable sort * `#12568 `__: Updated documentation for as_quat * `#12571 `__: DEP: remove deprecated slepian window * `#12573 `__: DEP: remove \`frechet_l\` and \`frechet_r\` * `#12575 `__: BUG: stats: fix multinomial.pmf NaNs when params sum > 1 * `#12576 `__: MAINT: remove warning from LSQSphereBivariateSpline * `#12582 `__: ENH: Multivariate t-distribution * `#12587 `__: ENH: speed up rvs of gengamma in scipy.stats * `#12588 `__: DOC: add Examples add see also sections for LinearNDInterpolator,... * `#12597 `__: ENH: Add single-sided p-values to t-tests * `#12599 `__: Small update to scipy FFT tutorial * `#12600 `__: ENH: disjoint set data structure * `#12602 `__: BUG: add const for Read-only views in interpnd.pyx * `#12605 `__: BUG: correct \`np.asanyarray\` use in \`scipy.constants.lambda2nu\` * `#12610 `__: MAINT: forward port 1.5.2 release notes * `#12612 `__: MAINT: stats: Use explicit keyword parameters instead of \`\*\*kwds\`. * `#12616 `__: DOC: make explicit docstring that interpolate.interp1d only accepts... * `#12618 `__: DOC: Minor doc formatting. * `#12640 `__: MAINT: stats: fix issues with scipy.stats.pearson3 docs, moment,... * `#12647 `__: TST: Add Boost ellipr[cdfgj]_data test data * `#12648 `__: DOC: Update special/utils/README with instructions * `#12649 `__: DOC: simplified pip quickstart guide * `#12650 `__: DOC: stats: Fix boxcox docstring: lambda can be negative. * `#12655 `__: DOC: update Steering Council members listed in governance docs * `#12659 `__: rv_sample expect bug * `#12663 `__: DOC: optimize: try to fix linprog method-specific documentation * `#12664 `__: BUG: stats: Fix logpdf with large negative values for logistic... * `#12666 `__: MAINT: Fixes from static analysis * `#12667 `__: ENH: Adding Modified Rodrigues Parameters to the Rotation class * `#12670 `__: DOC: Update documentation for Gamma distribution * `#12673 `__: API: Unconditionally return np.intp from cKDTree.query * `#12677 `__: MAINT: Add Autogenerated notice to ufuncs.pyi * `#12682 `__: MAINT: Remove _util._valarray * `#12688 `__: MAINT: add f2py-generated scipy.integrate files to .gitignore * `#12689 `__: BENCH: simplify benchmark setup, remove benchmarks/run.py * `#12694 `__: scipy/stats: Add laplace_asymmetric continuous distribution * `#12695 `__: DOC: update Ubuntu quickstart; conda compilers now work! * `#12698 `__: MAINT: Replace np.max with np.maximum * `#12700 `__: TST: bump test precision for constrained trustregion test * `#12702 `__: TST: bump test tolerance for \`DifferentialEvolutionSolver.test_L4\` * `#12703 `__: BUG: Improve input validation for sepfir2d * `#12708 `__: MAINT: fix a typo in scipy.sparse * `#12709 `__: BUG: bvls can fail catastrophically to converge * `#12711 `__: MAINT: Use platform.python_implementation to determine IS_PYPY * `#12713 `__: TST: Fix flaky test_lgmres * `#12716 `__: DOC: add examples and tutorial links for interpolate functions... * `#12717 `__: DOC: Fix Issue #5396 * `#12725 `__: ENH: Support complex-valued images and kernels for many ndimage... * `#12729 `__: DEP: remove setup_requires * `#12732 `__: BENCH: skip benchmarks instead of hiding them when SCIPY_XSLOW=0 * `#12734 `__: CI: Don't ignore line-length in the lint_diff check. * `#12736 `__: DOC: Fix signal.windows.get_window() 'exponential' docstring * `#12737 `__: ENH: stats: override stats.gumbel_r.fit and stats.gumbel_l.fit... * `#12738 `__: ENH: stats: override stats.logistic.fit with system of equations... * `#12743 `__: BUG: Avoid negative variances in circular statistics * `#12744 `__: Prevent build error on GNU/Hurd * `#12746 `__: TST: parameterize the test cases in test_ndimage.py * `#12752 `__: DOC: Add examples for some root finding functions. * `#12754 `__: MAINT, CI: Azure windows deps multiline * `#12756 `__: ENH: stats: Add an sf method to levy for improved precision in... * `#12757 `__: ENH: stats: Add an sf method to levy_l for improved precision. * `#12765 `__: TST, MAINT: infeasible_2 context * `#12767 `__: Fix spline interpolation boundary handling for modes reflect... * `#12769 `__: DOC: syntax error in scipy.interpolate.bspl * `#12770 `__: ENH: add nearest-up rounding to scipy.interpolate.interp1d * `#12771 `__: TST: fix invalid input unit test for scipy.signal.gammatone * `#12775 `__: ENH: Adds quadratic_assignment with two methods * `#12776 `__: ENH: add grid-constant boundary handling in ndimage interpolation... * `#12777 `__: Add Taylor Window function - Common in Radar DSP * `#12779 `__: ENH: Improvements to pocketfft thread pool and ARM neon vectorization * `#12788 `__: API: Rename cKDTree n_jobs argument to workers * `#12792 `__: DOC: remove THANKS.txt file in favor of scipy.org * `#12793 `__: Add new flag to authors tool * `#12802 `__: BENCH: add scipy.ndimage.interpolation benchmarks * `#12803 `__: Do not pin the version of numpy in unsupported python versions * `#12810 `__: CI: fix 32-bit Linux build failure on Azure CI runs * `#12812 `__: ENH: support interpolation of complex-valued images * `#12814 `__: BUG: nonlin_solve shouldn't pass non-vector dx to tol_norm * `#12818 `__: Update ckdtree.pyx * `#12822 `__: MAINT: simplify directed_hausdorff * `#12827 `__: DOC: Fix wrong name w being used instead of worN in docs. * `#12831 `__: DOC: fix typo in sparse/base.py * `#12835 `__: MAINT: stats: Improve vonmises PDF calculation. * `#12839 `__: ENH: scipy.stats: add multivariate hypergeometric distribution * `#12843 `__: changed M to N in windows.dpss * `#12846 `__: MAINT: update minimum NumPy version to 1.16.5 * `#12847 `__: DOC: Unify the formula in docs of scipy.stats.pearsonr() * `#12849 `__: DOC: polish QAP docs for consistency and readability * `#12852 `__: ENH, MAINT: Bring KDTree interface to feature-parity with cKDTree * `#12858 `__: DOC: use :doi: and :arxiv: directives for references * `#12872 `__: lazily import multiprocessing.Pool in MapWrapper * `#12878 `__: DOC: document ScalarFunction * `#12882 `__: MAINT: stats: Change a test to use <= instead of strictly less... * `#12885 `__: numpy.linspace calls edited to ensure correct spacing. * `#12886 `__: DOC: stats: Add 'versionadded' to cramervonmises docstring. * `#12899 `__: TST: make a couple of tests expected to fail on 32-bit architectures * `#12903 `__: DOC: update Windows build guide and move into contributor guide * `#12907 `__: DOC: clarify which array the precenter option applies to * `#12908 `__: MAINT: spatial: Remove two occurrences of unused variables in... * `#12909 `__: ENH: stats: Add methods gumbel_r._sf and gumbel_r._isf * `#12910 `__: CI: travis: Remove some unnecessary code from .travis.yml. * `#12911 `__: Minor fixes to dendrogram plotting * `#12921 `__: CI: don't run GitHub Actions on fork or in cron job * `#12927 `__: MAINT: rename integrate.simps to simpson * `#12934 `__: MAINT: rename trapz and cumtrapz to (cumulative\_)trapezoid * `#12936 `__: MAINT: fix numerical precision in nct.stats * `#12938 `__: MAINT: fix linter on master * `#12941 `__: Update minimum AIX pinnings to match non AIX builds * `#12955 `__: BUG: Fixed wrong NaNs check in scipy.stats.weightedtau * `#12958 `__: ENH: stats: Implement _logpdf, _sf and _isf for nakagami. * `#12962 `__: Correcting that p should be in [0,1] for a variety of discrete... * `#12964 `__: BUG: added line.strip() to split_data_line() * `#12968 `__: ENH: stats: Use only an analytical formula or scalar root-finding... * `#12971 `__: MAINT: Declare support for Python 3.9 * `#12972 `__: MAINT: Remove redundant Python < 3.6 code * `#12980 `__: DOC: Update documentation on optimize.rosen * `#12983 `__: ENH: improvements to stats.linregress * `#12990 `__: DOC: Clarify that using sos as output type for iirdesign can... * `#12992 `__: DOC: capitalization and formatting in lsmr * `#12995 `__: DOC: stats: Several documentation fixes. * `#12996 `__: BUG: Improve error messages for \`range\` arg of binned_statistic_dd * `#12998 `__: MAINT: approx_derivative with FP32 closes #12991 * `#13004 `__: TST: isinstance(OptimizeResult.message, str) closes #13001 * `#13006 `__: Keep correct dtype when loading empty mat arrays. * `#13009 `__: MAINT: clip SLSQP step within bounds * `#13012 `__: DOC: fix bilinear_zpk example labels * `#13013 `__: ENH: Add \`subset\` and \`subsets\` methods to \`DisjointSet\`... * `#13029 `__: MAINT: basinhopping callback for initial mininmisation * `#13032 `__: DOC: fix docstring errors in in stats.wilcoxon * `#13036 `__: BUG: forward port lint_diff shims * `#13041 `__: MAINT: dogbox ensure x is within bounds closes #11403 * `#13042 `__: MAINT: forward port 1.5.4 release notes * `#13046 `__: DOC: Update optimize.least_squares doc for all tolerance must... * `#13052 `__: Typo fix for cluster documentation * `#13054 `__: BUG: fix \`scipy.optimize.show_options\` for unknown methods.... * `#13056 `__: MAINT: fft: Fix a C++ compiler warning. * `#13057 `__: Minor fixes on doc of function csr_tocsc * `#13058 `__: DOC: stats: Replace np.float with np.float64 in a tutorial file. * `#13059 `__: DOC: stats: Update the "Returns" section of the linregress docstring. * `#13060 `__: MAINT: clip_x_for_func should be private * `#13061 `__: DOC: signal.win -> signal.windows.win in Examples * `#13063 `__: MAINT: Add suite-sparse and sksparse installation check * `#13070 `__: MAINT: stats: Remove a couple obsolete comments. * `#13073 `__: BUG: Fix scalar_search_wolfe2 to resolve #12157 * `#13078 `__: CI, MAINT: migrate Lint to Azure * `#13081 `__: BLD: drop Python 3.6 support (NEP 29) * `#13082 `__: MAINT: update minimum NumPy version to 1.16.5 in a couple more... * `#13083 `__: DOC: update toolchain.rst * `#13086 `__: DOC: Update the Parameters section of the correlation docstring * `#13087 `__: ENH:signal: Speed-up Cython implementation of _sosfilt * `#13089 `__: BLD, BUG: add c99 compiler flag to HiGHS basiclu library * `#13091 `__: BUG: Fix GIL handling in _sosfilt * `#13094 `__: DOC: clarify "location" in docstring of cKDTree.query * `#13095 `__: Zoom resize update * `#13097 `__: BUG: fix CubicSpline(..., bc_type="periodic") #11758 * `#13100 `__: BUG: sparse: Correct output format of kron * `#13107 `__: ENH: faster linear_sum_assignment for small cost matrices * `#13110 `__: CI, MAINT: refguide/asv checks to azure * `#13112 `__: CI: fix MacOS CI * `#13113 `__: CI: Install word list package for refguide-check * `#13115 `__: BUG: add value range check for signal.iirdesign() * `#13116 `__: CI: Don't report name errors after an exception in refguide-check * `#13117 `__: CI: move sdist/pre-release test Azure * `#13119 `__: Improve error message on friedmanchisquare function * `#13121 `__: Fix factorial() for NaN on Python 3.10 * `#13123 `__: BLD: Specify file extension for language standard version tests * `#13128 `__: TST: skip Fortran I/O test for ODR * `#13130 `__: TST: skip factorial() float tests on Python 3.10 * `#13136 `__: CI:Add python dbg run to GH Actions * `#13138 `__: CI: Port coverage, 64-bit BLAS, GCC 4.8 build to azure * `#13139 `__: Fix edge case for mode='nearest' in ndimage.interpolation functions * `#13141 `__: BUG: signal: Fix data type of the numerator returned by ss2tf. * `#13144 `__: MAINT: stats: restrict gausshyper z > -1 * `#13146 `__: typo in csr.py * `#13148 `__: BUG: stats: fix typo in stable rvs per gh-12870 * `#13149 `__: DOC: spatial/stats: cross-ref random rotation matrix functions * `#13151 `__: MAINT: stats: Fix a test and a couple PEP-8 issues. * `#13152 `__: MAINT: stats: Use np.take_along_axis in the private function... * `#13154 `__: ENH: stats: Implement defined handling of constant inputs in... * `#13156 `__: DOC: maintain equal display range for ndimage.zoom example * `#13159 `__: CI: Azure: Don't run tests on merge commits, except for coverage * `#13160 `__: DOC: stats: disambiguate location-shifted/noncentral * `#13161 `__: BUG: DifferentialEvolutionSolver.__del__ can fail in garbage... * `#13163 `__: BUG: stats: fix bug in spearmanr nan propagation * `#13167 `__: MAINT: stats: Fix a test. * `#13169 `__: BUG: stats: Fix handling of misaligned masks in mstats.pearsonr. * `#13178 `__: CI: testing.yml --> macos.yml * `#13181 `__: CI: fix lint * `#13190 `__: BUG: optimize: fix a duplicate key bug for \`test_show_options\` * `#13192 `__: BUG:linalg: Add overwrite option to gejsv wrapper * `#13194 `__: BUG: slsqp should be able to use rel_step * `#13203 `__: fix typos * `#13209 `__: TST:linalg: set the seed for cossin test * `#13212 `__: [DOC] Backtick and directive consistency. Checksums ========= MD5 ~~~ 8c563522f671e5d748e2d1e2a534b359 scipy-1.6.0rc1-cp37-cp37m-macosx_10_9_x86_64.whl fbe44b3f1560a5c028478f62b1564ed4 scipy-1.6.0rc1-cp37-cp37m-manylinux1_i686.whl 2a8e26081fd7c594c9379172a0b6abd7 scipy-1.6.0rc1-cp37-cp37m-manylinux1_x86_64.whl af63e2db90bfda2db424a49c51332094 scipy-1.6.0rc1-cp37-cp37m-manylinux2014_aarch64.whl d3a80c09c6f4690400a79d4a72374812 scipy-1.6.0rc1-cp37-cp37m-win32.whl 3e5dc5cd18831561f48ebc2b0142bda7 scipy-1.6.0rc1-cp37-cp37m-win_amd64.whl dd2e8c978e5efe5624574f3679e15108 scipy-1.6.0rc1-cp38-cp38-macosx_10_9_x86_64.whl dcd9b3d5e38df4e20853c876a4f65bf2 scipy-1.6.0rc1-cp38-cp38-manylinux1_i686.whl c589adc371aad5d78c86973a7eee8173 scipy-1.6.0rc1-cp38-cp38-manylinux1_x86_64.whl be4bd0a23aafda1ab97cead43e36890a scipy-1.6.0rc1-cp38-cp38-manylinux2014_aarch64.whl ee964284fb7c2e5814a3caf0e33621c5 scipy-1.6.0rc1-cp38-cp38-win32.whl 885363bc091a2abbc274f6fb6ef62c53 scipy-1.6.0rc1-cp38-cp38-win_amd64.whl 7aed9c106050211a43958294138747de scipy-1.6.0rc1-cp39-cp39-macosx_10_9_x86_64.whl 6fca7d8e11d486b5295ed604098356e5 scipy-1.6.0rc1-cp39-cp39-manylinux1_i686.whl 96393118e93a621fbc33850497e3daf9 scipy-1.6.0rc1-cp39-cp39-manylinux1_x86_64.whl 3c7493a013406b7374b5f2150d590833 scipy-1.6.0rc1-cp39-cp39-manylinux2014_aarch64.whl 8e5bfac2a1fbb621f9ba2093dcfb2631 scipy-1.6.0rc1-cp39-cp39-win32.whl c07dc01ae349dbd3f6440ffb47255c66 scipy-1.6.0rc1-cp39-cp39-win_amd64.whl e852074d5b39fcb2c625cba9034efeb6 scipy-1.6.0rc1.tar.gz 123a35d8d77f8ab77ce18b512c6c2ff5 scipy-1.6.0rc1.tar.xz c84945633b4aba069d5916950ffb43d5 scipy-1.6.0rc1.zip SHA256 ~~~~~~ 180e8f6db2e9e3f6596ae33e448b1ed1a5d31d3f77310b54cc3a887f12a6664d scipy-1.6.0rc1-cp37-cp37m-macosx_10_9_x86_64.whl 8bbdc8d0bc8e7db55783f48468af538e5c3021e75f9139b0f26ed585ac38dd16 scipy-1.6.0rc1-cp37-cp37m-manylinux1_i686.whl 418bc9e5fe95ba7e8dc3bd1fec441e841a4134f68b74882509007b91f7414d07 scipy-1.6.0rc1-cp37-cp37m-manylinux1_x86_64.whl b5f4a350db2220c324ccc39de1cdcc0e58110e12d0f44fa83755dd5f18fe15d9 scipy-1.6.0rc1-cp37-cp37m-manylinux2014_aarch64.whl 683f9ef98f407f4d47fb4a344b76ef06d695bc34624974d206480088c2675be4 scipy-1.6.0rc1-cp37-cp37m-win32.whl bf38fbef5332561ec80ca1f25d6e7949fd055f9907c233d80706c8651459ec9e scipy-1.6.0rc1-cp37-cp37m-win_amd64.whl 888230f30db1cfd26ddc08f0664ed7188cbf4b53342c4a2508aba23fc4bead44 scipy-1.6.0rc1-cp38-cp38-macosx_10_9_x86_64.whl 300e9fe90c5394fb7efa94cb092a69a33603d103909d0272d8df1d51b08269cd scipy-1.6.0rc1-cp38-cp38-manylinux1_i686.whl c206a17ffcdf85078ab58917ff097cf41e151bcd7415774c09708a6850a456f8 scipy-1.6.0rc1-cp38-cp38-manylinux1_x86_64.whl 658fd197a49895f8a88d47fd45b57d832c7c48c00c2c7ef01349029791f95c90 scipy-1.6.0rc1-cp38-cp38-manylinux2014_aarch64.whl f779c8189dd5ea7c6bf69d514203df65bf6ffda13b885c069a575cdb7eb731c8 scipy-1.6.0rc1-cp38-cp38-win32.whl 6300f24b169bd6060cc83e1ce98071d05f338e5d45d2bd0f5930616bd52034d6 scipy-1.6.0rc1-cp38-cp38-win_amd64.whl b5a72d1267be5a93c678722e22f5ea7b851979b1266a86e23c6a47564fd64bf7 scipy-1.6.0rc1-cp39-cp39-macosx_10_9_x86_64.whl 85f6c3ba2ea021a1f20447e87b726904e731a01b716ae992a1fbe1b3d0f567f5 scipy-1.6.0rc1-cp39-cp39-manylinux1_i686.whl 6b26aa4970d384f5d69c430a67b55fd3be6ac9a7e4797edd9d1532d3bc3cb837 scipy-1.6.0rc1-cp39-cp39-manylinux1_x86_64.whl 9e8b6dd6debbaa8b9b4299e03553e5bbc696a8104e1511c681d7591c00b568b2 scipy-1.6.0rc1-cp39-cp39-manylinux2014_aarch64.whl 7b946bdfc746471bac3d87b82370bb746f53c185f52d8f794abf09201cae2544 scipy-1.6.0rc1-cp39-cp39-win32.whl ff8fc9af8a7c9c96f3f6f0f6621730b63f1b8dc105d9c33c6e843ed4e571440f scipy-1.6.0rc1-cp39-cp39-win_amd64.whl 11c855fb7726b54a0eb34c46a0a073c28983fe4c7665d9ee41656100c150e736 scipy-1.6.0rc1.tar.gz 5ac98e1ba1e49d2e0e11970203e9e08c3f41840b6d486ffe8292a237c9391c18 scipy-1.6.0rc1.tar.xz 1de1ee679cc0393b78c9bd035b333c267a58bf243f44e67e7eabd234c0b5ea8a scipy-1.6.0rc1.zip -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieser.eric+numpy at gmail.com Fri Dec 11 03:46:19 2020 From: wieser.eric+numpy at gmail.com (Eric Wieser) Date: Fri, 11 Dec 2020 08:46:19 +0000 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: > you might want to discuss this with us at the array API standard > https://github.com/data-apis/array-api (which is currently in RFC > stage). The spec uses bool as the name for the boolean dtype. I don't fully understand this argument - `np.bool` is already not the boolean dtype. Either: * The spec is suggesting that `pkg.bool` be some arbitrary object that can be passed into a dtype argument and will produce a boolean array. If this is the case, the spec could also just require that `dtype=builtins.bool` have this behavior. * The spec is suggesting that `pkg.bool` is some rich dtype object. Ignoring the question of whether this should be `np.bool_` or `np.dtype(np.bool_)`, it's currently neither, and changing it will break users relying on `np.bool(True) is True`. That's not to say this isn't a sensible thing for the specification to have, it's just something that numpy can't conform to without breaking code. While it would be great if `np.bool_` could be spelt `np.bool`, I really don't think we can make that change without a long deprecation first (if at all). Eric On Thu, 10 Dec 2020 at 20:00, Sebastian Berg wrote: > On Thu, 2020-12-10 at 20:38 +0100, Ralf Gommers wrote: > > On Thu, Dec 10, 2020 at 7:25 PM Sebastian Berg < > > sebastian at sipsolutions.net> > > wrote: > > > > > On Wed, 2020-12-09 at 13:37 -0800, Stephan Hoyer wrote: > > > > On Wed, Dec 9, 2020 at 1:07 PM Aaron Meurer > > > > wrote: > > > > > > > > > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg > > > > > wrote: > > > > > > > > > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > > > > > > > Regarding np.bool specifically, if you want to deprecate > > > > > > > this, > > > > > > > you > > > > > > > might want to discuss this with us at the array API > > > > > > > standard > > > > > > > https://github.com/data-apis/array-api (which is currently > > > > > > > in > > > > > > > RFC > > > > > > > stage). The spec uses bool as the name for the boolean > > > > > > > dtype. > > > > > > > > > > > > > > Would it make sense for NumPy to change np.bool to just be > > > > > > > the > > > > > > > boolean > > > > > > > dtype object? Unlike int and float, there is no ambiguity > > > > > > > with > > > > > > > bool, > > > > > > > and NumPy clearly doesn't have any issues with shadowing > > > > > > > builtin > > > > > > > names > > > > > > > in its namespace. > > > > > > > > > > > > We could keep the Python alias around (which for `dtype=` is > > > > > > the > > > > > > same > > > > > > as `np.bool_`). > > > > > > > > > > > > I am not sure I like the idea of immediately shadowing the > > > > > > builtin. > > > > > > That is a switch we can avoid flipping (without warning); > > > > > > `np.bool_` > > > > > > and `bool` are fairly different beasts? [1] > > > > > > > > > > NumPy already shadows a lot of builtins, in many cases, in ways > > > > > that > > > > > are incompatible with existing ones. It's not something I would > > > > > have > > > > > done personally, but it's been this way for a long time. > > > > > > > > > > > > > It may be defensible to keep np.bool as an alias for Python's > > > > bool > > > > even when we remove the other aliases. > > > > > > > I'd agree with that. > > > > > > > That is true, `int` is probably the most confusing, since it is not > > > at > > > all compatible to a Python integer, but rather the "default" > > > integer > > > (which happens to be the same as C `long` currently). > > > > > > So we could focus on `np.int`, `np.long`. I am a bit unsure > > > whether > > > you would prefer that or are mainly pointing out the possibility? > > > > > > > Not sure what you mean with focus, focus on describing in the release > > notes? Deprecating `np.int` seems like the most beneficial part of > > this > > whole exercise. > > > > I meant limiting the current deprecation to `np.int`, maybe `np.long`, > and a "carefully chosen" set. > To be honest, I don't mind either way, so any stronger opinion will tip > the scale for me personally (my default currently is to update the > release notes to recommend the more descriptive names). > > There are probably more doc updates that would be nice, I will suggest > updating a separate issue for that. > > > > Right now, my main take-away from the discussion is that it would be > > > good to clarify the release notes a bit more. > > > > > > Using `float` for a dtype seems fine to me, but I prefer mentioning > > > `np.float64` over `np.float_`. > > > For integers, I wonder if we should also suggest `np.int64`, even ? > > > or > > > because ? if the default integer on many systems is currently > > > `np.int_`? > > > > > > > I agree. I think we should recommend sane, descriptive names that do > > the > > right thing. So ideally we'd have people spell their dtype specifiers > > as > > dtype=bool # or np.bool > > dtype=np.float64 > > dtype=np.int64 > > dtype=np.complex128 > > The names with underscores at the end make little sense from a UX > > perspective. And the C equivalents (single/double/etc) made sense 15 > > years > > ago, but with the user base of today - the majority of whom will not > > know C > > fluently or at all - also don't make too much sense. > > > > The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 and > > 64 > > bits is likely to be a pitfall much more often than it is what the > > user > > actually needs, so shouldn't be recommended and probably deserves a > > warning > > in the docs. > > Right, there is one slight trickery because `np.intp` is often a great > integer dtype to use, because it is the integer that NumPy uses for all > things related to indexing and array sizes. > (I would be happy to dig out my PR making `np.intp` the default NumPy > integer.) > > Cheers, > > Sebastian > > > > > > Cheers, > > Ralf > > > > > > > > > > > > > > > np.int_ and np.float_ have fixed precision, which makes them > > > > somewhat > > > > different from the builtin types. NumPy has a whole bunch of > > > > different > > > > precisions for integer and floats, so this distinction matters. > > > > > > > > In contrast, there is only one boolean dtype in NumPy, which > > > > matches > > > > Python's bool. So we wouldn't have to worry, for example, about > > > > whether a > > > > user has requested a specific precision explicitly. This comes up > > > > in > > > > issues > > > > like type-promotion where libraries like JAX and PyTorch have > > > > special > > > > case > > > > logic for most Python types vs NumPy dtypes (but booleans are the > > > > same for > > > > both): > > > > https://jax.readthedocs.io/en/latest/type_promotion.html > > > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Fri Dec 11 05:29:30 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Fri, 11 Dec 2020 11:29:30 +0100 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Fri, Dec 11, 2020 at 9:47 AM Eric Wieser wrote: > > you might want to discuss this with us at the array API standard > > https://github.com/data-apis/array-api (which is currently in RFC > > stage). The spec uses bool as the name for the boolean dtype. > > I don't fully understand this argument - `np.bool` is already not the > boolean dtype. Either: > > * The spec is suggesting that `pkg.bool` be some arbitrary object that can > be passed into a dtype argument and will produce a boolean array. > If this is the case, the spec could also just require that > `dtype=builtins.bool` have this behavior. > Yes, this. * The spec is suggesting that `pkg.bool` is some rich dtype object. > Ignoring the question of whether this should be `np.bool_` or > `np.dtype(np.bool_)`, it's currently neither, and changing it will break > users relying on `np.bool(True) is True`. > That's not to say this isn't a sensible thing for the specification to > have, it's just something that numpy can't conform to without breaking code. > It can have richer behaviour, there's no constraints there - but it's not necessary. > While it would be great if `np.bool_` could be spelt `np.bool`, I really > don't think we can make that change without a long deprecation first (if at > all). > Given that that standard API would be in a new namespace (given backwards compat we can't possibly introduce it in the main namespace), there `bool` can be the numpy boolean dtype (if desired). The key point is that `bool_` is a terrible name, and keeping `np.bool` that you can use as a dtype specifier is desirable. Cheers, Ralf > Eric > > On Thu, 10 Dec 2020 at 20:00, Sebastian Berg > wrote: > >> On Thu, 2020-12-10 at 20:38 +0100, Ralf Gommers wrote: >> > On Thu, Dec 10, 2020 at 7:25 PM Sebastian Berg < >> > sebastian at sipsolutions.net> >> > wrote: >> > >> > > On Wed, 2020-12-09 at 13:37 -0800, Stephan Hoyer wrote: >> > > > On Wed, Dec 9, 2020 at 1:07 PM Aaron Meurer >> > > > wrote: >> > > > >> > > > > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg >> > > > > wrote: >> > > > > > >> > > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: >> > > > > > > Regarding np.bool specifically, if you want to deprecate >> > > > > > > this, >> > > > > > > you >> > > > > > > might want to discuss this with us at the array API >> > > > > > > standard >> > > > > > > https://github.com/data-apis/array-api (which is currently >> > > > > > > in >> > > > > > > RFC >> > > > > > > stage). The spec uses bool as the name for the boolean >> > > > > > > dtype. >> > > > > > > >> > > > > > > Would it make sense for NumPy to change np.bool to just be >> > > > > > > the >> > > > > > > boolean >> > > > > > > dtype object? Unlike int and float, there is no ambiguity >> > > > > > > with >> > > > > > > bool, >> > > > > > > and NumPy clearly doesn't have any issues with shadowing >> > > > > > > builtin >> > > > > > > names >> > > > > > > in its namespace. >> > > > > > >> > > > > > We could keep the Python alias around (which for `dtype=` is >> > > > > > the >> > > > > > same >> > > > > > as `np.bool_`). >> > > > > > >> > > > > > I am not sure I like the idea of immediately shadowing the >> > > > > > builtin. >> > > > > > That is a switch we can avoid flipping (without warning); >> > > > > > `np.bool_` >> > > > > > and `bool` are fairly different beasts? [1] >> > > > > >> > > > > NumPy already shadows a lot of builtins, in many cases, in ways >> > > > > that >> > > > > are incompatible with existing ones. It's not something I would >> > > > > have >> > > > > done personally, but it's been this way for a long time. >> > > > > >> > > > >> > > > It may be defensible to keep np.bool as an alias for Python's >> > > > bool >> > > > even when we remove the other aliases. >> > > >> > >> > I'd agree with that. >> > >> > >> > > That is true, `int` is probably the most confusing, since it is not >> > > at >> > > all compatible to a Python integer, but rather the "default" >> > > integer >> > > (which happens to be the same as C `long` currently). >> > > >> > > So we could focus on `np.int`, `np.long`. I am a bit unsure >> > > whether >> > > you would prefer that or are mainly pointing out the possibility? >> > > >> > >> > Not sure what you mean with focus, focus on describing in the release >> > notes? Deprecating `np.int` seems like the most beneficial part of >> > this >> > whole exercise. >> > >> >> I meant limiting the current deprecation to `np.int`, maybe `np.long`, >> and a "carefully chosen" set. >> To be honest, I don't mind either way, so any stronger opinion will tip >> the scale for me personally (my default currently is to update the >> release notes to recommend the more descriptive names). >> >> There are probably more doc updates that would be nice, I will suggest >> updating a separate issue for that. >> >> >> > Right now, my main take-away from the discussion is that it would be >> > > good to clarify the release notes a bit more. >> > > >> > > Using `float` for a dtype seems fine to me, but I prefer mentioning >> > > `np.float64` over `np.float_`. >> > > For integers, I wonder if we should also suggest `np.int64`, even ? >> > > or >> > > because ? if the default integer on many systems is currently >> > > `np.int_`? >> > > >> > >> > I agree. I think we should recommend sane, descriptive names that do >> > the >> > right thing. So ideally we'd have people spell their dtype specifiers >> > as >> > dtype=bool # or np.bool >> > dtype=np.float64 >> > dtype=np.int64 >> > dtype=np.complex128 >> > The names with underscores at the end make little sense from a UX >> > perspective. And the C equivalents (single/double/etc) made sense 15 >> > years >> > ago, but with the user base of today - the majority of whom will not >> > know C >> > fluently or at all - also don't make too much sense. >> > >> > The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 and >> > 64 >> > bits is likely to be a pitfall much more often than it is what the >> > user >> > actually needs, so shouldn't be recommended and probably deserves a >> > warning >> > in the docs. >> >> Right, there is one slight trickery because `np.intp` is often a great >> integer dtype to use, because it is the integer that NumPy uses for all >> things related to indexing and array sizes. >> (I would be happy to dig out my PR making `np.intp` the default NumPy >> integer.) >> >> Cheers, >> >> Sebastian >> >> >> > >> > Cheers, >> > Ralf >> > >> > >> > > >> > > > >> > > > np.int_ and np.float_ have fixed precision, which makes them >> > > > somewhat >> > > > different from the builtin types. NumPy has a whole bunch of >> > > > different >> > > > precisions for integer and floats, so this distinction matters. >> > > > >> > > > In contrast, there is only one boolean dtype in NumPy, which >> > > > matches >> > > > Python's bool. So we wouldn't have to worry, for example, about >> > > > whether a >> > > > user has requested a specific precision explicitly. This comes up >> > > > in >> > > > issues >> > > > like type-promotion where libraries like JAX and PyTorch have >> > > > special >> > > > case >> > > > logic for most Python types vs NumPy dtypes (but booleans are the >> > > > same for >> > > > both): >> > > > https://jax.readthedocs.io/en/latest/type_promotion.html >> > > >> > > >> > _______________________________________________ >> > NumPy-Discussion mailing list >> > NumPy-Discussion at python.org >> > https://mail.python.org/mailman/listinfo/numpy-discussion >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Fri Dec 11 05:35:30 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Fri, 11 Dec 2020 11:35:30 +0100 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Thu, Dec 10, 2020 at 9:00 PM Sebastian Berg wrote: > On Thu, 2020-12-10 at 20:38 +0100, Ralf Gommers wrote: > > On Thu, Dec 10, 2020 at 7:25 PM Sebastian Berg < > > sebastian at sipsolutions.net> > > wrote: > > > > > On Wed, 2020-12-09 at 13:37 -0800, Stephan Hoyer wrote: > > > > On Wed, Dec 9, 2020 at 1:07 PM Aaron Meurer > > > > wrote: > > > > > > > > > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg > > > > > wrote: > > > > > > > > > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: > > > > > > > Regarding np.bool specifically, if you want to deprecate > > > > > > > this, > > > > > > > you > > > > > > > might want to discuss this with us at the array API > > > > > > > standard > > > > > > > https://github.com/data-apis/array-api (which is currently > > > > > > > in > > > > > > > RFC > > > > > > > stage). The spec uses bool as the name for the boolean > > > > > > > dtype. > > > > > > > > > > > > > > Would it make sense for NumPy to change np.bool to just be > > > > > > > the > > > > > > > boolean > > > > > > > dtype object? Unlike int and float, there is no ambiguity > > > > > > > with > > > > > > > bool, > > > > > > > and NumPy clearly doesn't have any issues with shadowing > > > > > > > builtin > > > > > > > names > > > > > > > in its namespace. > > > > > > > > > > > > We could keep the Python alias around (which for `dtype=` is > > > > > > the > > > > > > same > > > > > > as `np.bool_`). > > > > > > > > > > > > I am not sure I like the idea of immediately shadowing the > > > > > > builtin. > > > > > > That is a switch we can avoid flipping (without warning); > > > > > > `np.bool_` > > > > > > and `bool` are fairly different beasts? [1] > > > > > > > > > > NumPy already shadows a lot of builtins, in many cases, in ways > > > > > that > > > > > are incompatible with existing ones. It's not something I would > > > > > have > > > > > done personally, but it's been this way for a long time. > > > > > > > > > > > > > It may be defensible to keep np.bool as an alias for Python's > > > > bool > > > > even when we remove the other aliases. > > > > > > > I'd agree with that. > > > > > > > That is true, `int` is probably the most confusing, since it is not > > > at > > > all compatible to a Python integer, but rather the "default" > > > integer > > > (which happens to be the same as C `long` currently). > > > > > > So we could focus on `np.int`, `np.long`. I am a bit unsure > > > whether > > > you would prefer that or are mainly pointing out the possibility? > > > > > > > Not sure what you mean with focus, focus on describing in the release > > notes? Deprecating `np.int` seems like the most beneficial part of > > this > > whole exercise. > > > > I meant limiting the current deprecation to `np.int`, maybe `np.long`, > and a "carefully chosen" set. > Just deprecation `np.int` may make sense. That will already raise awareness, and leaving `np.float` as-is may prevent a lot of churn. And we could then still deprecate `np.float` later. I also don't feel strongly about `float` either way though. I'm not sure why you'd specifically touch `long`, it's not really relevant and it's not a builtin. Cheers, Ralf To be honest, I don't mind either way, so any stronger opinion will tip > the scale for me personally (my default currently is to update the > release notes to recommend the more descriptive names). > > There are probably more doc updates that would be nice, I will suggest > updating a separate issue for that. > > > > Right now, my main take-away from the discussion is that it would be > > > good to clarify the release notes a bit more. > > > > > > Using `float` for a dtype seems fine to me, but I prefer mentioning > > > `np.float64` over `np.float_`. > > > For integers, I wonder if we should also suggest `np.int64`, even ? > > > or > > > because ? if the default integer on many systems is currently > > > `np.int_`? > > > > > > > I agree. I think we should recommend sane, descriptive names that do > > the > > right thing. So ideally we'd have people spell their dtype specifiers > > as > > dtype=bool # or np.bool > > dtype=np.float64 > > dtype=np.int64 > > dtype=np.complex128 > > The names with underscores at the end make little sense from a UX > > perspective. And the C equivalents (single/double/etc) made sense 15 > > years > > ago, but with the user base of today - the majority of whom will not > > know C > > fluently or at all - also don't make too much sense. > > > > The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 and > > 64 > > bits is likely to be a pitfall much more often than it is what the > > user > > actually needs, so shouldn't be recommended and probably deserves a > > warning > > in the docs. > > Right, there is one slight trickery because `np.intp` is often a great > integer dtype to use, because it is the integer that NumPy uses for all > things related to indexing and array sizes. > (I would be happy to dig out my PR making `np.intp` the default NumPy > integer.) > > Cheers, > > Sebastian > > > > > > Cheers, > > Ralf > > > > > > > > > > > > > > > np.int_ and np.float_ have fixed precision, which makes them > > > > somewhat > > > > different from the builtin types. NumPy has a whole bunch of > > > > different > > > > precisions for integer and floats, so this distinction matters. > > > > > > > > In contrast, there is only one boolean dtype in NumPy, which > > > > matches > > > > Python's bool. So we wouldn't have to worry, for example, about > > > > whether a > > > > user has requested a specific precision explicitly. This comes up > > > > in > > > > issues > > > > like type-promotion where libraries like JAX and PyTorch have > > > > special > > > > case > > > > logic for most Python types vs NumPy dtypes (but booleans are the > > > > same for > > > > both): > > > > https://jax.readthedocs.io/en/latest/type_promotion.html > > > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Fri Dec 11 06:09:28 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Fri, 11 Dec 2020 12:09:28 +0100 Subject: [Numpy-discussion] Fixing/implementing value based Casting In-Reply-To: <82fe831b56e9322f352c767a5cab70b979529899.camel@sipsolutions.net> References: <82fe831b56e9322f352c767a5cab70b979529899.camel@sipsolutions.net> Message-ID: On Wed, Dec 9, 2020 at 5:22 PM Sebastian Berg wrote: > Hi all, > > Sorry that this will again be a bit complicated again :(. In brief: > > * I would like to pass around scalars in some (partially new) C-API > to implement value-based promotion. > * There are some subtle commutativity issues with promotion. > Commutativity may change in that case (with respect of value based > promotion, probably to the better normally). [0] > > > In the past days, I have been looking into implementing value-based > promotion in a way that I had done it for Prototype before. > The idea was that NEP 42, allows for the creation of DType dynamically, > which does allow very powerful value based promotion/casting. > > But I decided there are too many quirks with creating type instances > dynamically (potentially very often) just to pass around one additional > piece of information. > That approach was far more powerful, but it is power and complexity > that we do not require, given that: > > * Value based promotion is only used for a mix of scalars and arrays > (where "scalar" is annoyingly defined as 0-D at the moment) > * I assume it is only relevant for `np.result_type` and promotion > in ufuncs (which often uses `np.result_type`). > `np.can_cast` has such behaviour, but I think it is easier [1]. > We could implement more powerful "value based" logic, but I doubt > it is worthwhile. > * This is already stretching the Python C-API beyond its limits. > > > So I will suggest this instead which *must* modify some (poorly > defined) current behaviour: > > 1. We always evaluate concrete DTypes first in promotion, this means > that in rare cases the non-commutativity of promotion may change > the result dtype: > > np.result_type(-1, 2**16, np.float32) > > The same can also happens when you reorder the normal dtypes: > > np.result_type(np.int8, np.uint16, np.float32) > np.result_type(np.float32, np.int8, np.uint16) > > in both cases the `np.float32` is moved to the front > > 2. If we reorder the above operation, we can define that we never > promote two "scalar values". Instead we convert both to a > concrete one first. This makes it effectively like: > > np.result_type(np.array(-1).dtype, np.array(2**16).dtype) > > This means that we never have to deal with promoting two values. > > 3. We need additional private API (we were always going to need some > additional API); That API could become public: > > * Convert a single value into a concrete dtype, you could say > the same as `self.common_dtype(None)`, but a dedicated function > seems simpler. A dtype like this will never use `common_dtype()`. > * `common_dtype_with_scalar(self, other, scalar)` (note that > only one of the DTypes can have a scalar). > As a fallback, this function can be implemented by converting > to the concrete DType and retrying with the normal `common_dtype`. > > (At leas the second slot must be made public we are to allow value > based promotion for user DTypes. I expect we will, but it is not > particularly important to me right now.) > > 4. Our public API (including new C-API) has to expose and take the > scalar values. That means promotion in ufuncs will get DTypes and > `scalar_values`, although those should normally be `NULL` (or None). > > In future python API, this is probably acceptable: > > np.result_type([t if v is None else v for t, v in zip(dtypes, > scalar_values)]) > > In C, we need to expose a function below `result_type` which > accepts both the scalar values and DTypes explicitly. > > 5. For the future: As said many times, I would like to deprecate > using value based promotion for anything except Python core types. > That just seems wrong and confusing. > I agree with this. Value-based promotion was never a great idea, so let's try to keep it as minimal as possible. I'm not even sure what kind of value-based promotion for non Python builtin types is happening now (?). My only problem is that while I can warn (possibly sometimes too > often) when behaviour will change. I do not have a good idea about > silencing that warning. > Do you see a real issue with this somewhere, or is it all just corner cases? In that case no warning seems okay. > > Note that this affects NEP 42 (a little bit). NEP 42 currently makes a > nod towards the dynamic type creation, but falls short of actually > defining it. > So These rules have to be incorporated, but IMO they do not affect the > general design choices in the NEP. > > > There is probably even more complexity to be found here, but for now > the above seems to be at least good enough to make headway... > > > Any thoughts or clarity remaining that I can try to confuse? :) > My main question is why you're considering both deprecating and expanding public API (in points 3 and 4). If you have a choice, keep everything private I'd say. My other question is: this is a complex story, it all sounds reasonable but do you need more feedback than "sounds reasonable"? Cheers, Ralf > Cheers, > > Sebastian > > > > [0] We could use the reordering trick also for concrete DTypes, > although, that would require introducing some kind of priority... I do > not like that much as public API, but it might be something to look at > internally or for types deriving from the builtin abstract DTypes: > * inexact > * other > > Just evaluating all `inexact` first would probably solve our > commutativity issues. > > [1] NumPy uses `np.can_cast(value, dtype)` also. For example: > > np.can_cast(np.array(1., dtype=np.float64), np.float32, casting="safe") > > returns True. My working hypothesis is that `np.can_cast` as above is > just a side battle. I.e. we can either: > > * Flip the switch on it (can-cast does no value based logic, even > though we use it internally, we do not need it). > * Or, we can implement those cases of `np.can_cast` by using promotion. > > The first one is tempting, but I assume we should go with the second > since it preserves behaviour and is slightly more powerful. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Fri Dec 11 11:31:08 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Fri, 11 Dec 2020 10:31:08 -0600 Subject: [Numpy-discussion] Fixing/implementing value based Casting In-Reply-To: References: <82fe831b56e9322f352c767a5cab70b979529899.camel@sipsolutions.net> Message-ID: <7f43f037246e1f332345abc6adf91c91ec72cb52.camel@sipsolutions.net> On Fri, 2020-12-11 at 12:09 +0100, Ralf Gommers wrote: > On Wed, Dec 9, 2020 at 5:22 PM Sebastian Berg < > sebastian at sipsolutions.net> > wrote: > > > Hi all, > > > > Sorry that this will again be a bit complicated again :(. In brief: > > > > * I would like to pass around scalars in some (partially new) C-API > > ? to implement value-based promotion. > > * There are some subtle commutativity issues with promotion. > > ? Commutativity may change in that case (with respect of value > > based > > ? promotion, probably to the better normally). [0] > > > > > > In the past days, I have been looking into implementing value-based > > promotion in a way that I had done it for Prototype before. > > The idea was that NEP 42, allows for the creation of DType > > dynamically, > > which does allow very powerful value based promotion/casting. > > > > But I decided there are too many quirks with creating type > > instances > > dynamically (potentially very often) just to pass around one > > additional > > piece of information. > > That approach was far more powerful, but it is power and complexity > > that we do not require, given that: > > > > * Value based promotion is only used for a mix of scalars and > > arrays > > ? (where "scalar" is annoyingly defined as 0-D at the moment) > > * I assume it is only relevant for `np.result_type` and promotion > > ? in ufuncs (which often uses `np.result_type`). > > ? `np.can_cast` has such behaviour, but I think it is easier [1]. > > ? We could implement more powerful "value based" logic, but I doubt > > ? it is worthwhile. > > * This is already stretching the Python C-API beyond its limits. > > > > > > So I will suggest this instead which *must* modify some (poorly > > defined) current behaviour: > > > > 1. We always evaluate concrete DTypes first in promotion, this > > means > > ?? that in rare cases the non-commutativity of promotion may change > > ?? the result dtype: > > > > ?????? np.result_type(-1, 2**16, np.float32) > > > > ?? The same can also happens when you reorder the normal dtypes: > > > > ?????? np.result_type(np.int8, np.uint16, np.float32) > > ?????? np.result_type(np.float32, np.int8, np.uint16) > > > > ?? in both cases the `np.float32` is moved to the front > > > > 2. If we reorder the above operation, we can define that we never > > ?? promote two "scalar values". Instead we convert both to a > > ?? concrete one first.? This makes it effectively like: > > > > ?????? np.result_type(np.array(-1).dtype, np.array(2**16).dtype) > > > > ?? This means that we never have to deal with promoting two values. > > > > 3. We need additional private API (we were always going to need > > some > > ?? additional API); That API could become public: > > > > ?? * Convert a single value into a concrete dtype, you could say > > ???? the same as `self.common_dtype(None)`, but a dedicated > > function > > ???? seems simpler. A dtype like this will never use > > `common_dtype()`. > > ?? * `common_dtype_with_scalar(self, other, scalar)` (note that > > ???? only one of the DTypes can have a scalar). > > ???? As a fallback, this function can be implemented by converting > > ???? to the concrete DType and retrying with the normal > > `common_dtype`. > > > > ?? (At leas the second slot must be made public we are to allow > > value > > ?? based promotion for user DTypes. I expect we will, but it is not > > ?? particularly important to me right now.) > > > > 4. Our public API (including new C-API) has to expose and take the > > ?? scalar values. That means promotion in ufuncs will get DTypes > > and > > ?? `scalar_values`, although those should normally be `NULL` (or > > None). > > > > ?? In future python API, this is probably acceptable: > > > > ??????? np.result_type([t if v is None else v for t, v in > > zip(dtypes, > > scalar_values)]) > > > > ?? In C, we need to expose a function below `result_type` which > > ?? accepts both the scalar values and DTypes explicitly. > > > > 5. For the future: As said many times, I would like to deprecate > > ?? using value based promotion for anything except Python core > > types. > > ?? That just seems wrong and confusing. > > > > I agree with this.? It is tempting to wonder what would happen if we dropped it entirely, but I fear my current assumption is that it should keep working largely unchanged with careful deprecations hopefully added soon... > Value-based promotion was never a great idea, so let's > try to keep it as minimal as possible. I'm not even sure what kind of > value-based promotion for non Python builtin types is happening now > (?). It (roughly?) identical for all zero dimensional objects: arr1 = np.array(1, dtype=np.int64) arr2 = np.array([1, 2], dtype=np.int32) (arr1 + arr2).dtype == np.int32 (1 + arr2).dtype == np.int32 In the first addition `arr1` behaves like the Python `1` even though it has a dtype attached. The reason for this probably that our entry-points greedily convert arrays. And it shows one caveat: If we/SciPy call `np.asarray` on a Python integer input we would lose value-based behaviour, this may actually be a bigger pain point (see below for example). > > ?? My only problem is that while I can warn (possibly sometimes too > > ?? often) when behaviour will change.? I do not have a good idea > > about > > ?? silencing that warning. > > > > Do you see a real issue with this somewhere, or is it all just corner > cases? In that case no warning seems okay. > Probably it is mostly corner cases, if you do: arr_uint16 + int32(1) + 1. We would warn for the first case, but not for: arr_uint16 + (int32(1) + 1.) even though it gives identical results. The same might happen in `np.concatenate` where all arguments are passed at once. I can think of one bigger pain point for this type of function: def function(arr1, arr2): arr1 = np.asarray(arr1) arr2 = np.asarray(arr2) return arr1 + arr2 # some complex code we could add a cast to the function I guess. But for the end-user it might be tricky to realize that they need to cast the input to that function. And those type of functions are abundant... > > > > > Note that this affects NEP 42 (a little bit). NEP 42 currently > > makes a > > nod towards the dynamic type creation, but falls short of actually > > defining it. > > > So These rules have to be incorporated, but IMO they do not affect > the > > general design choices in the NEP. > > > > > > There is probably even more complexity to be found here, but for > > now > > the above seems to be at least good enough to make headway... > > > > > > Any thoughts or clarity remaining that I can try to confuse? :) > > > > My main question is why you're considering both deprecating and > expanding > public API (in points 3 and 4). If you have a choice, keep everything > private I'd say. > I had to realize that the non-associativity is trickier to solve. Still digging into that... But, I guess we can probably live with it if user DTypes can show some non-associativity even in a single call to `np.result_type` or `np.concatenate`. Generally I don't have much squirms, as long as things don't get worse (they are already broken). The premise for requiring some new public API is that for us: int16(1) + 1 == int16(2) # value based for Python 1 A user implements int24, if it is to fit in perfectly we would like:? int24(1) + 1 == int24(2) Which requires some way to pass `int24` the information that it is a Python `1` in some form (there are probably many options for how to pass it). Exposure in promotion might be interesting for weirdly complex ufuncs, like `scipy.special.eval_jacobi` which have mixed type inputs. Again a corner case of a corner case, but would prefer if there was a (possible) future solution. Cheers, Sebastian > My other question is: this is a complex story, it all sounds > reasonable but > do you need more feedback than "sounds reasonable"? > > Cheers, > Ralf > > > > > Cheers, > > > > Sebastian > > > > > > > > [0] We could use the reordering trick also for concrete DTypes, > > although, that would require introducing some kind of priority... I > > do > > not like that much as public API, but it might be something to look > > at > > internally or for types deriving from the builtin abstract DTypes: > > ??? * inexact > > ??? * other > > > > Just evaluating all `inexact` first would probably solve our > > commutativity issues. > > > > [1] NumPy uses `np.can_cast(value, dtype)` also. For example: > > > > ??? np.can_cast(np.array(1., dtype=np.float64), np.float32, > > casting="safe") > > > > returns True. My working hypothesis is that `np.can_cast` as above > > is > > just a side battle.? I.e. we can either: > > > > * Flip the switch on it (can-cast does no value based logic, even > > though we use it internally, we do not need it). > > * Or, we can implement those cases of `np.can_cast` by using > > promotion. > > > > The first one is tempting, but I assume we should go with the > > second > > since it preserves behaviour and is slightly more powerful. > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From sebastian at sipsolutions.net Fri Dec 11 12:03:25 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Fri, 11 Dec 2020 11:03:25 -0600 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Fri, 2020-12-11 at 11:35 +0100, Ralf Gommers wrote: > On Thu, Dec 10, 2020 at 9:00 PM Sebastian Berg < > sebastian at sipsolutions.net> > wrote: > > Just deprecation `np.int` may make sense. That will already raise > awareness, and leaving `np.float` as-is may prevent a lot of churn. > And we > could then still deprecate `np.float` later. I also don't feel > strongly > about `float` either way though. > > I'm not sure why you'd specifically touch `long`, it's not really > relevant > and it's not a builtin. > `np.long is np.int is int` as it was a builtin on Python 2. But it looks like a C-long. In `dtype=` usage it actually ends up being a C-long (but it might even be nice to consider modifying the default `int` on windows at some point. At that point the "long" alias would be very confusing). OTOH, right now the only way to spell C-long is with `np.int_` which doesn't help. Cheers, Sebastian > Cheers, > Ralf > > To be honest, I don't mind either way, so any stronger opinion will > tip > > the scale for me personally (my default currently is to update the > > release notes to recommend the more descriptive names). > > > > There are probably more doc updates that would be nice, I will > > suggest > > updating a separate issue for that. > > > > > > > Right now, my main take-away from the discussion is that it would > > > be > > > > good to clarify the release notes a bit more. > > > > > > > > Using `float` for a dtype seems fine to me, but I prefer > > > > mentioning > > > > `np.float64` over `np.float_`. > > > > For integers, I wonder if we should also suggest `np.int64`, > > > > even ? > > > > or > > > > because ? if the default integer on many systems is currently > > > > `np.int_`? > > > > > > > > > > I agree. I think we should recommend sane, descriptive names that > > > do > > > the > > > right thing. So ideally we'd have people spell their dtype > > > specifiers > > > as > > > ? dtype=bool? # or np.bool > > > ? dtype=np.float64 > > > ? dtype=np.int64 > > > ? dtype=np.complex128 > > > The names with underscores at the end make little sense from a UX > > > perspective. And the C equivalents (single/double/etc) made sense > > > 15 > > > years > > > ago, but with the user base of today - the majority of whom will > > > not > > > know C > > > fluently or at all - also don't make too much sense. > > > > > > The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 > > > and > > > 64 > > > bits is likely to be a pitfall much more often than it is what > > > the > > > user > > > actually needs, so shouldn't be recommended and probably deserves > > > a > > > warning > > > in the docs. > > > > Right, there is one slight trickery because `np.intp` is often a > > great > > integer dtype to use, because it is the integer that NumPy uses for > > all > > things related to indexing and array sizes. > > (I would be happy to dig out my PR making `np.intp` the default > > NumPy > > integer.) > > > > Cheers, > > > > Sebastian > > > > > > > > > > Cheers, > > > Ralf > > > > > > > > > > > > > > > > > > > > np.int_ and np.float_ have fixed precision, which makes them > > > > > somewhat > > > > > different from the builtin types. NumPy has a whole bunch of > > > > > different > > > > > precisions for integer and floats, so this distinction > > > > > matters. > > > > > > > > > > In contrast, there is only one boolean dtype in NumPy, which > > > > > matches > > > > > Python's bool. So we wouldn't have to worry, for example, > > > > > about > > > > > whether a > > > > > user has requested a specific precision explicitly. This > > > > > comes up > > > > > in > > > > > issues > > > > > like type-promotion where libraries like JAX and PyTorch have > > > > > special > > > > > case > > > > > logic for most Python types vs NumPy dtypes (but booleans are > > > > > the > > > > > same for > > > > > both): > > > > > https://jax.readthedocs.io/en/latest/type_promotion.html > > > > > > > > > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion at python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From asmeurer at gmail.com Fri Dec 11 13:10:31 2020 From: asmeurer at gmail.com (Aaron Meurer) Date: Fri, 11 Dec 2020 11:10:31 -0700 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Fri, Dec 11, 2020 at 1:47 AM Eric Wieser wrote: > > > you might want to discuss this with us at the array API standard > > https://github.com/data-apis/array-api (which is currently in RFC > > stage). The spec uses bool as the name for the boolean dtype. > > I don't fully understand this argument - `np.bool` is already not the boolean dtype. Either: The spec does deviate from what NumPy currently does in some places. If we wanted to just copy NumPy exactly, there wouldn't be a need for a specification. > > * The spec is suggesting that `pkg.bool` be some arbitrary object that can be passed into a dtype argument and will produce a boolean array. > If this is the case, the spec could also just require that `dtype=builtins.bool` have this behavior. > * The spec is suggesting that `pkg.bool` is some rich dtype object. > Ignoring the question of whether this should be `np.bool_` or `np.dtype(np.bool_)`, it's currently neither, and changing it will break users relying on `np.bool(True) is True`. > That's not to say this isn't a sensible thing for the specification to have, it's just something that numpy can't conform to without breaking code. This what it currently says (https://data-apis.github.io/array-api/latest/API_specification/data_types.html) Data types (?dtypes?) are objects that can be used as dtype specifiers in functions and methods (e.g., zeros((2, 3), dtype=float32) ). A conforming implementation may add methods or attributes to data type objects; however, these methods and attributes are not included in this specification. So basically, np.bool just needs to be something that can be used as a dtype. The dtype objects names don't have any requirements on them. A library could have float64 == 'f8', for example. It isn't written there presently but really the only thing that needs to work for the dtype objects is == comparison (or at least, it will be impossible for the test suite to test dtype behavior if a.dtype == float64 doesn't work). So np.bool == builtins.bool is actually fine. My concern here was that the discussion was about deprecating np.bool, meaning it would be removed from the namespace, which goes against what is currently in the spec. Aaron Meurer > > While it would be great if `np.bool_` could be spelt `np.bool`, I really don't think we can make that change without a long deprecation first (if at all). > > Eric > > On Thu, 10 Dec 2020 at 20:00, Sebastian Berg wrote: >> >> On Thu, 2020-12-10 at 20:38 +0100, Ralf Gommers wrote: >> > On Thu, Dec 10, 2020 at 7:25 PM Sebastian Berg < >> > sebastian at sipsolutions.net> >> > wrote: >> > >> > > On Wed, 2020-12-09 at 13:37 -0800, Stephan Hoyer wrote: >> > > > On Wed, Dec 9, 2020 at 1:07 PM Aaron Meurer >> > > > wrote: >> > > > >> > > > > On Wed, Dec 9, 2020 at 9:41 AM Sebastian Berg >> > > > > wrote: >> > > > > > >> > > > > > On Mon, 2020-12-07 at 14:18 -0700, Aaron Meurer wrote: >> > > > > > > Regarding np.bool specifically, if you want to deprecate >> > > > > > > this, >> > > > > > > you >> > > > > > > might want to discuss this with us at the array API >> > > > > > > standard >> > > > > > > https://github.com/data-apis/array-api (which is currently >> > > > > > > in >> > > > > > > RFC >> > > > > > > stage). The spec uses bool as the name for the boolean >> > > > > > > dtype. >> > > > > > > >> > > > > > > Would it make sense for NumPy to change np.bool to just be >> > > > > > > the >> > > > > > > boolean >> > > > > > > dtype object? Unlike int and float, there is no ambiguity >> > > > > > > with >> > > > > > > bool, >> > > > > > > and NumPy clearly doesn't have any issues with shadowing >> > > > > > > builtin >> > > > > > > names >> > > > > > > in its namespace. >> > > > > > >> > > > > > We could keep the Python alias around (which for `dtype=` is >> > > > > > the >> > > > > > same >> > > > > > as `np.bool_`). >> > > > > > >> > > > > > I am not sure I like the idea of immediately shadowing the >> > > > > > builtin. >> > > > > > That is a switch we can avoid flipping (without warning); >> > > > > > `np.bool_` >> > > > > > and `bool` are fairly different beasts? [1] >> > > > > >> > > > > NumPy already shadows a lot of builtins, in many cases, in ways >> > > > > that >> > > > > are incompatible with existing ones. It's not something I would >> > > > > have >> > > > > done personally, but it's been this way for a long time. >> > > > > >> > > > >> > > > It may be defensible to keep np.bool as an alias for Python's >> > > > bool >> > > > even when we remove the other aliases. >> > > >> > >> > I'd agree with that. >> > >> > >> > > That is true, `int` is probably the most confusing, since it is not >> > > at >> > > all compatible to a Python integer, but rather the "default" >> > > integer >> > > (which happens to be the same as C `long` currently). >> > > >> > > So we could focus on `np.int`, `np.long`. I am a bit unsure >> > > whether >> > > you would prefer that or are mainly pointing out the possibility? >> > > >> > >> > Not sure what you mean with focus, focus on describing in the release >> > notes? Deprecating `np.int` seems like the most beneficial part of >> > this >> > whole exercise. >> > >> >> I meant limiting the current deprecation to `np.int`, maybe `np.long`, >> and a "carefully chosen" set. >> To be honest, I don't mind either way, so any stronger opinion will tip >> the scale for me personally (my default currently is to update the >> release notes to recommend the more descriptive names). >> >> There are probably more doc updates that would be nice, I will suggest >> updating a separate issue for that. >> >> >> > Right now, my main take-away from the discussion is that it would be >> > > good to clarify the release notes a bit more. >> > > >> > > Using `float` for a dtype seems fine to me, but I prefer mentioning >> > > `np.float64` over `np.float_`. >> > > For integers, I wonder if we should also suggest `np.int64`, even ? >> > > or >> > > because ? if the default integer on many systems is currently >> > > `np.int_`? >> > > >> > >> > I agree. I think we should recommend sane, descriptive names that do >> > the >> > right thing. So ideally we'd have people spell their dtype specifiers >> > as >> > dtype=bool # or np.bool >> > dtype=np.float64 >> > dtype=np.int64 >> > dtype=np.complex128 >> > The names with underscores at the end make little sense from a UX >> > perspective. And the C equivalents (single/double/etc) made sense 15 >> > years >> > ago, but with the user base of today - the majority of whom will not >> > know C >> > fluently or at all - also don't make too much sense. >> > >> > The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 and >> > 64 >> > bits is likely to be a pitfall much more often than it is what the >> > user >> > actually needs, so shouldn't be recommended and probably deserves a >> > warning >> > in the docs. >> >> Right, there is one slight trickery because `np.intp` is often a great >> integer dtype to use, because it is the integer that NumPy uses for all >> things related to indexing and array sizes. >> (I would be happy to dig out my PR making `np.intp` the default NumPy >> integer.) >> >> Cheers, >> >> Sebastian >> >> >> > >> > Cheers, >> > Ralf >> > >> > >> > > >> > > > >> > > > np.int_ and np.float_ have fixed precision, which makes them >> > > > somewhat >> > > > different from the builtin types. NumPy has a whole bunch of >> > > > different >> > > > precisions for integer and floats, so this distinction matters. >> > > > >> > > > In contrast, there is only one boolean dtype in NumPy, which >> > > > matches >> > > > Python's bool. So we wouldn't have to worry, for example, about >> > > > whether a >> > > > user has requested a specific precision explicitly. This comes up >> > > > in >> > > > issues >> > > > like type-promotion where libraries like JAX and PyTorch have >> > > > special >> > > > case >> > > > logic for most Python types vs NumPy dtypes (but booleans are the >> > > > same for >> > > > both): >> > > > https://jax.readthedocs.io/en/latest/type_promotion.html >> > > >> > > >> > _______________________________________________ >> > NumPy-Discussion mailing list >> > NumPy-Discussion at python.org >> > https://mail.python.org/mailman/listinfo/numpy-discussion >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From robert.kern at gmail.com Fri Dec 11 14:01:01 2020 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 11 Dec 2020 14:01:01 -0500 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: On Fri, Dec 11, 2020 at 1:12 PM Aaron Meurer wrote: > On Fri, Dec 11, 2020 at 1:47 AM Eric Wieser > wrote: > > > > > you might want to discuss this with us at the array API standard > > > https://github.com/data-apis/array-api (which is currently in RFC > > > stage). The spec uses bool as the name for the boolean dtype. > > > > I don't fully understand this argument - `np.bool` is already not the > boolean dtype. Either: > > The spec does deviate from what NumPy currently does in some places. > If we wanted to just copy NumPy exactly, there wouldn't be a need for > a specification. I wouldn't take that as a premise. Specifying a subset of the vast existing NumPy API would be a quite valuable specification in its own right. I find the motivation for deviation laid out in the Purpose and Scope section to be reasonably convincing that deviation might be needed *somewhere*. The question then is, is *this* deviation supporting that stated motivation, or is it taking the opportunity of a redesign to rationalize the names more to our current tastes? Given the mode of adopting the standard (a separate subpackage), that's a reasonable choice to make, but let's be clear about the motivation. I submit that keeping the name `bool_` does not make it any harder for other array APIs to adopt the standard. It's just that few people would design a new API with that name if they were designing a greenfield API. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From lianyuanz at gmail.com Fri Dec 11 14:34:24 2020 From: lianyuanz at gmail.com (Lianyuan Zheng) Date: Fri, 11 Dec 2020 14:34:24 -0500 Subject: [Numpy-discussion] Installing numpy Message-ID: Hello, On my linux server, I downloaded the NUMPY package from GitHub (git clone https://github.com/numpy/numpy.git) and then accessed the directory "numpy". When I typed command "python setup.py install", it shows the following message: File "setup.py", line 51 f"NumPy {VERSION} may not yet support Python " ^ SyntaxError: invalid syntax Obviously the installation failed. Which python version is needed for this numpy package? The python version installed is version 2.7.5. Thanks, Lianyuan -------------- next part -------------- An HTML attachment was scrubbed... URL: From sseibert at anaconda.com Fri Dec 11 14:49:38 2020 From: sseibert at anaconda.com (Stanley Seibert) Date: Fri, 11 Dec 2020 13:49:38 -0600 Subject: [Numpy-discussion] Installing numpy In-Reply-To: References: Message-ID: The development version of NumPy from Github requires Python 3.7 or later. On Fri, Dec 11, 2020 at 1:35 PM Lianyuan Zheng wrote: > Hello, > > On my linux server, I downloaded the NUMPY package from GitHub (git clone > https://github.com/numpy/numpy.git) and then accessed the directory > "numpy". When I typed command "python setup.py install", it shows the > following message: > > File "setup.py", line 51 > f"NumPy {VERSION} may not yet support Python " > ^ > SyntaxError: invalid syntax > > Obviously the installation failed. Which python version is needed for > this numpy package? The python version installed is version 2.7.5. > > Thanks, > Lianyuan > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lianyuanz at gmail.com Fri Dec 11 15:26:15 2020 From: lianyuanz at gmail.com (Lianyuan Zheng) Date: Fri, 11 Dec 2020 15:26:15 -0500 Subject: [Numpy-discussion] Installing numpy In-Reply-To: References: Message-ID: Hi Stanley, Thank you! Lianyuan On Fri, Dec 11, 2020 at 2:49 PM Stanley Seibert wrote: > The development version of NumPy from Github requires Python 3.7 or later. > > On Fri, Dec 11, 2020 at 1:35 PM Lianyuan Zheng > wrote: > >> Hello, >> >> On my linux server, I downloaded the NUMPY package from GitHub (git clone >> https://github.com/numpy/numpy.git) and then accessed the directory >> "numpy". When I typed command "python setup.py install", it shows the >> following message: >> >> File "setup.py", line 51 >> f"NumPy {VERSION} may not yet support Python " >> ^ >> SyntaxError: invalid syntax >> >> Obviously the installation failed. Which python version is needed for >> this numpy package? The python version installed is version 2.7.5. >> >> Thanks, >> Lianyuan >> >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jni at fastmail.com Fri Dec 11 20:34:30 2020 From: jni at fastmail.com (Juan Nunez-Iglesias) Date: Sat, 12 Dec 2020 12:34:30 +1100 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> Message-ID: <815619FF-F2C0-40A1-811E-AE8BC2965F10@fastmail.com> > I agree. I think we should recommend sane, descriptive names that do the right thing. So ideally we'd have people spell their dtype specifiers as > dtype=bool # or np.bool > dtype=np.float64 > dtype=np.int64 > dtype=np.complex128 > The names with underscores at the end make little sense from a UX perspective. And the C equivalents (single/double/etc) made sense 15 years ago, but with the user base of today - the majority of whom will not know C fluently or at all - also don't make too much sense. > > The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 and 64 bits is likely to be a pitfall much more often than it is what the user actually needs, so shouldn't be recommended and probably deserves a warning in the docs. I kinda disagree with this. I want to have a way to say, give me an array of the same type as the default NumPy type (for either ints or floats). This will prevent casting back and forth as different arrays are combined. In other words, as long as NumPy itself flips back and forth (depending on locale), I think users will in many cases want to flip back and forth with it? Juan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Sat Dec 12 14:25:26 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Sat, 12 Dec 2020 13:25:26 -0600 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: <815619FF-F2C0-40A1-811E-AE8BC2965F10@fastmail.com> References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> <815619FF-F2C0-40A1-811E-AE8BC2965F10@fastmail.com> Message-ID: <986e3901393695b9f083f40db65ee9dc31bbe5fa.camel@sipsolutions.net> On Sat, 2020-12-12 at 12:34 +1100, Juan Nunez-Iglesias wrote: > > > I agree. I think we should recommend sane, descriptive names that > > do the right thing. So ideally we'd have people spell their dtype > > specifiers as > > ? dtype=bool? # or np.bool > > ? dtype=np.float64 > > ? dtype=np.int64 > > ? dtype=np.complex128 > > The names with underscores at the end make little sense from a UX > > perspective. And the C equivalents (single/double/etc) made sense > > 15 years ago, but with the user base of today - the majority of > > whom will not know C fluently or at all - also don't make too much > > sense. > > > > The `dtype=int` or `dtype=np.int_` behaviour flopping between 32 > > and 64 bits is likely to be a pitfall much more often than it is > > what the user actually needs, so shouldn't be recommended and > > probably deserves a warning in the docs. > > I kinda disagree with this. I want to have a way to say, give me an > array of the same type as the default NumPy type (for either ints or > floats). This will prevent casting back and forth as different arrays > are combined. In other words, as long as NumPy itself flips back and > forth (depending on locale), I think users will in many cases want to > flip back and forth with it? But "default" in NumPy really doesn't mean a whole lot? I can think of three places where "defaults" exists: 1. `np.array([1])` will default to a C-long (as will `np.uint8(1) + 1`) 2. Sum and product upcast to C-long (and pretty much only those): np.sum(np.arange(10, dtype=np.int8)) np.product(np.arange(10, dtype=np.int8)) 3. NumPy uses `np.intp` for all indexing operations internally and some functions many functions which return integers related to indexing (e.g. `np.nonzero()`). [1] The first two points have no logic at all besides: windows thinks long is always 32bit and others think long is 64bit on 64bit systems. The last point does have some logic. Generally, the only reason to stick to a certain type would be that mixing types can be slower (using a non `intp` to index or doing math with a mix of 32bit and 64bit integers). From a library perspective, I wonder how often you actually expect a "default integer" input, as opposed to 32bit or 64bit depending on the whims of the user; or `intp` because it is "indexing related". It would be interesting to see if we can change the default at some point. It might also be tricky: There may be quite a bit of code expecting `long` (e.g. Cython extensions or `scipy.special` may or may not notice such a change). Cheers, Sebastian [1] intp is technically intptr_t in C, while indexing only requires an ssize_t I think. That probably matters on no currently supported systems, but system where it matters do exist (OpenVMS is one that just came up, and we may support in the future). > > Juan. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From jni at fastmail.com Sun Dec 13 03:00:05 2020 From: jni at fastmail.com (Juan Nunez-Iglesias) Date: Sun, 13 Dec 2020 19:00:05 +1100 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: <986e3901393695b9f083f40db65ee9dc31bbe5fa.camel@sipsolutions.net> References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> <815619FF-F2C0-40A1-811E-AE8BC2965F10@fastmail.com> <986e3901393695b9f083f40db65ee9dc31bbe5fa.camel@sipsolutions.net> Message-ID: > On 13 Dec 2020, at 6:25 am, Sebastian Berg wrote: > > But "default" in NumPy really doesn't mean a whole lot? I can think of > three places where "defaults" exists: Huh? There are platform-specific defaults for literally every array creation function in NumPy? In [1]: np.array([4, 9]).dtype Out[1]: dtype('int64') In [2]: np.array([3., 0.]).dtype Out[2]: dtype('float64') In [3]: np.arange(5).dtype Out[3]: dtype('int64') In [4]: np.arange(5.).dtype Out[4]: dtype('float64') In [5]: np.empty(5).dtype Out[5]: dtype('float64') In [6]: np.zeros(5).dtype Out[6]: dtype('float64') In [7]: np.full(5, 5).dtype Out[7]: dtype('int64') In [8]: np.full(5, 5.).dtype Out[8]: dtype('float64?) The list goes on? And, indeed, mixing types can cause implicit casting, and thus both slowness and unexpected type promotion, which brings with it its own bugs? Again, I think it is valuable to have syntax to express `np.zeros(?, dtype=)`. Juan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Sun Dec 13 13:27:36 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Sun, 13 Dec 2020 12:27:36 -0600 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> <815619FF-F2C0-40A1-811E-AE8BC2965F10@fastmail.com> <986e3901393695b9f083f40db65ee9dc31bbe5fa.camel@sipsolutions.net> Message-ID: <590dbddcc6bc84fa5c3ecd09c068b60349d91f59.camel@sipsolutions.net> On Sun, 2020-12-13 at 19:00 +1100, Juan Nunez-Iglesias wrote: > > > > On 13 Dec 2020, at 6:25 am, Sebastian Berg < > > sebastian at sipsolutions.net> wrote: > > > > But "default" in NumPy really doesn't mean a whole lot?? I can > > think of > > three places where "defaults" exists: > > Huh? There are platform-specific defaults for literally every array > creation function in NumPy? > > In [1]: np.array([4, 9]).dtype > Out[1]: dtype('int64') > The list goes on? > I should have been more clear about this and my opinion on it: 1. The whole list comes down to my point 1: when confronted with a Python integer, NumPy will typically use a C-long [1]. Additionally, `dtype=int` is always the same as long: `np.dtype(int) == np.dtype("long")`. The reason why I see that as a single point, is that it is defined in a single place in C [1]. (The `np.dtype(int)` is a second place.) 2. I agree with Ralf that this is "random". On the same computer you can easily get a wrong result for the identical code because you boot into windows instead of linux [2]. `long` is not a good default! It is 32bit on windows and 64bit on (64bit) linux! That should confuse the majority of our users (and probably many who are aware of C integer types). Good defaults are awesome, but I just can't see how `long` is a good default. There were good reasons for it on Python 2, but that is not relevant anymore. 3. I think that `intp` would be a much saner default for most code. It gives a system dependent result, but two points are in its favor: * NumPy generates `intp` in quite a lot of places * It is always safe (and fast) to index arrays with `intp` > And, indeed, mixing types can cause implicit casting, and thus both > slowness and unexpected type promotion, which brings with it its own > bugs? Again, I think it is valuable to have syntax to express > `np.zeros(?, dtype= data>)`. Yes, it is valuable, but I am unsure we should advise to use it... Cheers, Sebastian [1] Currently defined here: https://github.com/numpy/numpy/blob/7a42940e610b77cee2f98eb88aed5e66ef6d8c2a/numpy/core/src/multiarray/abstractdtypes.c#L16-L45 Which will use `long` normally, but `long long` (64bit) if that fails and even `unsigned long long` if *that* fails also. [2] I would not be surprised if there are quite a few libraries with bugs for very large arrays, that are simply not found yet, because nobody tried to run the code on very large arrays on a windows workstation yet. > Juan. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From ralf.gommers at gmail.com Sun Dec 13 15:31:58 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 13 Dec 2020 21:31:58 +0100 Subject: [Numpy-discussion] np.{bool,float,int} deprecation In-Reply-To: <590dbddcc6bc84fa5c3ecd09c068b60349d91f59.camel@sipsolutions.net> References: <381967EE-1438-40A6-8D76-EF6BD82F7A85@fastmail.com> <815619FF-F2C0-40A1-811E-AE8BC2965F10@fastmail.com> <986e3901393695b9f083f40db65ee9dc31bbe5fa.camel@sipsolutions.net> <590dbddcc6bc84fa5c3ecd09c068b60349d91f59.camel@sipsolutions.net> Message-ID: On Sun, Dec 13, 2020 at 7:29 PM Sebastian Berg wrote: > On Sun, 2020-12-13 at 19:00 +1100, Juan Nunez-Iglesias wrote: > > > > > > > On 13 Dec 2020, at 6:25 am, Sebastian Berg < > > > sebastian at sipsolutions.net> wrote: > > > > > > But "default" in NumPy really doesn't mean a whole lot? I can > > > think of > > > three places where "defaults" exists: > > > > Huh? There are platform-specific defaults for literally every array > > creation function in NumPy? > > > > In [1]: np.array([4, 9]).dtype > > Out[1]: dtype('int64') > > > The list goes on? > > > > I should have been more clear about this and my opinion on it: > > 1. The whole list comes down to my point 1: when confronted with a > Python integer, NumPy will typically use a C-long [1]. > Additionally, `dtype=int` is always the same as long: > `np.dtype(int) == np.dtype("long")`. > > The reason why I see that as a single point, is that it is defined in a > single place in C [1]. (The `np.dtype(int)` is a second place.) > > > 2. I agree with Ralf that this is "random". On the same computer you > can easily get a wrong result for the identical code because you boot > into windows instead of linux [2]. `long` is not a good default! It is > 32bit on windows and 64bit on (64bit) linux! That should confuse the > majority of our users (and probably many who are aware of C integer > types). > Good defaults are awesome, but I just can't see how `long` is a good > default. There were good reasons for it on Python 2, but that is not > relevant anymore. > > > 3. I think that `intp` would be a much saner default for most code. It > gives a system dependent result, but two points are in its favor: > > * NumPy generates `intp` in quite a lot of places > * It is always safe (and fast) to index arrays with `intp` > > > > And, indeed, mixing types can cause implicit casting, and thus both > > slowness and unexpected type promotion, which brings with it its own > > bugs? Again, I think it is valuable to have syntax to express > > `np.zeros(?, dtype= > data>)`. > > Yes, it is valuable, but I am unsure we should advise to use it... > Agreed, it should be possible for people who know that's what they want, but an "always int64" default would be way better. Before we had 32-bit CI, I developed on 32-bit Linux on purpose, and found multiple newly-introduced bugs in NumPy and Scipy each release cycle. Risking correctness issues like overflows is far worse than possible sub-optimal performance. For that same reason, float96/float128 are very annoying. Users don't realize that those aren't portable. Cheers, Ralf > Cheers, > > Sebastian > > > > [1] Currently defined here: > > https://github.com/numpy/numpy/blob/7a42940e610b77cee2f98eb88aed5e66ef6d8c2a/numpy/core/src/multiarray/abstractdtypes.c#L16-L45 > Which will use `long` normally, but `long long` (64bit) if that fails > and even `unsigned long long` if *that* fails also. > > > [2] I would not be surprised if there are quite a few libraries with > bugs for very large arrays, that are simply not found yet, because > nobody tried to run the code on very large arrays on a windows > workstation yet. > > > > > Juan. > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Mon Dec 14 08:37:02 2020 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Mon, 14 Dec 2020 16:37:02 +0300 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? Message-ID: Hi, What would be the correct way of locking the bit generator of np.random.Generator in cython's nogil context? (This is threading 101, surely, so please forgive my ignorance). The docs for extending np.random.Generator in cython (https://numpy.org/doc/stable/reference/random/extending.html#cython) recommend the following idiom for generating uniform variates, where the GIL is released and a Generator-specific lock is held: x = PCG64() rng = PyCapsule_GetPointer(x.capsule, capsule_name) with nogil, x.lock: rng.next_double(rng.state) What is the correct way of locking it when already in the nogil section (so that x.lock is not accessible)? The use case is a long-running MC process which generates random variates in a tight loop (so the loop is all nogil). In case it matters, I probably won't be using python threads, but may use multiprocessing. Basically, cdef double uniform(self) nogil: if self.idx >= self.buf.shape[0]: self._fill() cdef double value = self.buf[self.idx] self.idx += 1 return value cdef void _fill(self) nogil: self.idx = 0 # HERE: Lock ? for i in range(self.buf.shape[0]): self.buf[i] = self.rng.next_double(self.rng.state) Thanks, Evgeni P.S. The full cdef class, for completeness: cdef class RndmWrapper(): cdef: double[::1] buf Py_ssize_t idx bitgen_t *rng object py_gen # keep the garbage collector away def __init__(self, seed=(1234, 0), buf_size=4096, bitgen_kind=None): if bitgen_kind is None: bitgen_kind = PCG64 # cf Numpy-discussion list, K.~Sheppard, R.~Kern, June 29, 2020 and below # https://mail.python.org/pipermail/numpy-discussion/2020-June/080794.html entropy, num = seed seed_seq = SeedSequence(entropy, spawn_key=(num,)) py_gen = bitgen_kind(seed_seq) # store the python object to avoid it being garbage collected self.py_gen = py_gen capsule = py_gen.capsule self.rng = PyCapsule_GetPointer(capsule, capsule_name) if not PyCapsule_IsValid(capsule, capsule_name): raise ValueError("Invalid pointer to anon_func_state") self.buf = np.empty(buf_size, dtype='float64') self._fill() @cython.boundscheck(False) @cython.wraparound(False) cdef void _fill(self) nogil: self.idx = 0 for i in range(self.buf.shape[0]): self.buf[i] = self.rng.next_double(self.rng.state) @cython.boundscheck(False) @cython.wraparound(False) cdef double uniform(self) nogil: if self.idx >= self.buf.shape[0]: self._fill() cdef double value = self.buf[self.idx] self.idx += 1 return value From kevin.k.sheppard at gmail.com Mon Dec 14 08:45:52 2020 From: kevin.k.sheppard at gmail.com (Kevin Sheppard) Date: Mon, 14 Dec 2020 13:45:52 +0000 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: References: Message-ID: You need to reacquire the gil, then you can get the lock and rerelease the gil. I think this works (on phone, so untested) with gil: with nogil, lock: ... Kevin On Mon, Dec 14, 2020, 13:37 Evgeni Burovski wrote: > Hi, > > What would be the correct way of locking the bit generator of > np.random.Generator in cython's nogil context? > (This is threading 101, surely, so please forgive my ignorance). > > The docs for extending np.random.Generator in cython > (https://numpy.org/doc/stable/reference/random/extending.html#cython) > recommend the following idiom for generating uniform variates, where > the GIL is released and a Generator-specific lock is held: > > x = PCG64() > rng = PyCapsule_GetPointer(x.capsule, capsule_name) > with nogil, x.lock: > rng.next_double(rng.state) > > What is the correct way of locking it when already in the nogil > section (so that x.lock is not accessible)? > > The use case is a long-running MC process which generates random > variates in a tight loop (so the loop is all nogil). In case it > matters, I probably won't be using python threads, but may use > multiprocessing. > > Basically, > > cdef double uniform(self) nogil: > if self.idx >= self.buf.shape[0]: > self._fill() > cdef double value = self.buf[self.idx] > self.idx += 1 > return value > > cdef void _fill(self) nogil: > self.idx = 0 > # HERE: Lock ? > for i in range(self.buf.shape[0]): > self.buf[i] = self.rng.next_double(self.rng.state) > > > Thanks, > Evgeni > > > P.S. The full cdef class, for completeness: > > cdef class RndmWrapper(): > cdef: > double[::1] buf > Py_ssize_t idx > bitgen_t *rng > object py_gen # keep the garbage collector away > > def __init__(self, seed=(1234, 0), buf_size=4096, bitgen_kind=None): > if bitgen_kind is None: > bitgen_kind = PCG64 > > # cf Numpy-discussion list, K.~Sheppard, R.~Kern, June 29, > 2020 and below > # > https://mail.python.org/pipermail/numpy-discussion/2020-June/080794.html > entropy, num = seed > seed_seq = SeedSequence(entropy, spawn_key=(num,)) > py_gen = bitgen_kind(seed_seq) > > # store the python object to avoid it being garbage collected > self.py_gen = py_gen > > capsule = py_gen.capsule > self.rng = PyCapsule_GetPointer(capsule, capsule_name) > if not PyCapsule_IsValid(capsule, capsule_name): > raise ValueError("Invalid pointer to anon_func_state") > > self.buf = np.empty(buf_size, dtype='float64') > self._fill() > > @cython.boundscheck(False) > @cython.wraparound(False) > cdef void _fill(self) nogil: > self.idx = 0 > for i in range(self.buf.shape[0]): > self.buf[i] = self.rng.next_double(self.rng.state) > > @cython.boundscheck(False) > @cython.wraparound(False) > cdef double uniform(self) nogil: > if self.idx >= self.buf.shape[0]: > self._fill() > cdef double value = self.buf[self.idx] > self.idx += 1 > return value > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Mon Dec 14 09:09:10 2020 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Mon, 14 Dec 2020 17:09:10 +0300 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: References: Message-ID: On Mon, Dec 14, 2020 at 4:46 PM Kevin Sheppard wrote: > > You need to reacquire the gil, then you can get the lock and rerelease the gil. > > I think this works (on phone, so untested) > > with gil: > with nogil, lock: > ... Thanks Kevin. This surely works, but feels seriously weird. Is this the only / the recommended way? I can of course adjust the buffer size to amortize the time for the GIL manipulation, but this really starts looking like a code smell. My original motivation was to have inner simulation loops python-free. Most likely, the issue is that I'm not using the Generator correctly? Evgeni > On Mon, Dec 14, 2020, 13:37 Evgeni Burovski wrote: >> >> Hi, >> >> What would be the correct way of locking the bit generator of >> np.random.Generator in cython's nogil context? >> (This is threading 101, surely, so please forgive my ignorance). >> >> The docs for extending np.random.Generator in cython >> (https://numpy.org/doc/stable/reference/random/extending.html#cython) >> recommend the following idiom for generating uniform variates, where >> the GIL is released and a Generator-specific lock is held: >> >> x = PCG64() >> rng = PyCapsule_GetPointer(x.capsule, capsule_name) >> with nogil, x.lock: >> rng.next_double(rng.state) >> >> What is the correct way of locking it when already in the nogil >> section (so that x.lock is not accessible)? >> >> The use case is a long-running MC process which generates random >> variates in a tight loop (so the loop is all nogil). In case it >> matters, I probably won't be using python threads, but may use >> multiprocessing. >> >> Basically, >> >> cdef double uniform(self) nogil: >> if self.idx >= self.buf.shape[0]: >> self._fill() >> cdef double value = self.buf[self.idx] >> self.idx += 1 >> return value >> >> cdef void _fill(self) nogil: >> self.idx = 0 >> # HERE: Lock ? >> for i in range(self.buf.shape[0]): >> self.buf[i] = self.rng.next_double(self.rng.state) >> >> >> Thanks, >> Evgeni >> >> >> P.S. The full cdef class, for completeness: >> >> cdef class RndmWrapper(): >> cdef: >> double[::1] buf >> Py_ssize_t idx >> bitgen_t *rng >> object py_gen # keep the garbage collector away >> >> def __init__(self, seed=(1234, 0), buf_size=4096, bitgen_kind=None): >> if bitgen_kind is None: >> bitgen_kind = PCG64 >> >> # cf Numpy-discussion list, K.~Sheppard, R.~Kern, June 29, >> 2020 and below >> # https://mail.python.org/pipermail/numpy-discussion/2020-June/080794.html >> entropy, num = seed >> seed_seq = SeedSequence(entropy, spawn_key=(num,)) >> py_gen = bitgen_kind(seed_seq) >> >> # store the python object to avoid it being garbage collected >> self.py_gen = py_gen >> >> capsule = py_gen.capsule >> self.rng = PyCapsule_GetPointer(capsule, capsule_name) >> if not PyCapsule_IsValid(capsule, capsule_name): >> raise ValueError("Invalid pointer to anon_func_state") >> >> self.buf = np.empty(buf_size, dtype='float64') >> self._fill() >> >> @cython.boundscheck(False) >> @cython.wraparound(False) >> cdef void _fill(self) nogil: >> self.idx = 0 >> for i in range(self.buf.shape[0]): >> self.buf[i] = self.rng.next_double(self.rng.state) >> >> @cython.boundscheck(False) >> @cython.wraparound(False) >> cdef double uniform(self) nogil: >> if self.idx >= self.buf.shape[0]: >> self._fill() >> cdef double value = self.buf[self.idx] >> self.idx += 1 >> return value >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From lianyuanz at gmail.com Mon Dec 14 09:14:23 2020 From: lianyuanz at gmail.com (Lianyuan Zheng) Date: Mon, 14 Dec 2020 09:14:23 -0500 Subject: [Numpy-discussion] Installing numpy In-Reply-To: References: Message-ID: Hello, When I run the command "python numpy install", it shows the following error: Traceback (most recent call last): File "setup.py", line 27, in import versioneer File "./numpy/versioneer.py", line 1739 file=sys.stderr) ^ SyntaxError: invalid syntax Is this caused by the python version installed too old (v2.7.17) or other? Thanks in advance! Lianyuan On Fri, Dec 11, 2020 at 2:49 PM Stanley Seibert wrote: > The development version of NumPy from Github requires Python 3.7 or later. > > On Fri, Dec 11, 2020 at 1:35 PM Lianyuan Zheng > wrote: > >> Hello, >> >> On my linux server, I downloaded the NUMPY package from GitHub (git clone >> https://github.com/numpy/numpy.git) and then accessed the directory >> "numpy". When I typed command "python setup.py install", it shows the >> following message: >> >> File "setup.py", line 51 >> f"NumPy {VERSION} may not yet support Python " >> ^ >> SyntaxError: invalid syntax >> >> Obviously the installation failed. Which python version is needed for >> this numpy package? The python version installed is version 2.7.5. >> >> Thanks, >> Lianyuan >> >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.k.sheppard at gmail.com Mon Dec 14 09:25:10 2020 From: kevin.k.sheppard at gmail.com (Kevin Sheppard) Date: Mon, 14 Dec 2020 14:25:10 +0000 Subject: [Numpy-discussion] Installing numpy In-Reply-To: References: , Message-ID: <35657BDA-CA4D-4D09-946A-3CD73E28FF08@hxcore.ol> An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Mon Dec 14 09:26:03 2020 From: matti.picus at gmail.com (Matti Picus) Date: Mon, 14 Dec 2020 16:26:03 +0200 Subject: [Numpy-discussion] Installing numpy In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From kevin.k.sheppard at gmail.com Mon Dec 14 09:39:59 2020 From: kevin.k.sheppard at gmail.com (Kevin Sheppard) Date: Mon, 14 Dec 2020 14:39:59 +0000 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: References: , Message-ID: <2011C308-3387-4142-93D4-2D63F64FEC48@hxcore.ol> An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Mon Dec 14 09:44:59 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 14 Dec 2020 15:44:59 +0100 Subject: [Numpy-discussion] Installing numpy In-Reply-To: References: Message-ID: On Mon, Dec 14, 2020 at 3:26 PM Matti Picus wrote: > NumPy HEAD does not support python2. You should use v1.16.6 which was the > last release to support python2. > That said, we should raise a comprehensible error. As well as write a test that checks we avoid py27 syntax errors like f-strings in setup.py Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Mon Dec 14 15:26:29 2020 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Mon, 14 Dec 2020 23:26:29 +0300 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: <2011C308-3387-4142-93D4-2D63F64FEC48@hxcore.ol> References: <2011C308-3387-4142-93D4-2D63F64FEC48@hxcore.ol> Message-ID: > I also think that the lock only matters for Multithreaded code not Multiprocess. I believe the latter pickles and unpickles any Generator object (and the underying BitGenerator) and so each process has its own version. Note that when multiprocessing the recommended procedure is to use spawn() to generate a sequence of BitGenerators and to use a distinct BitGenerator in each process. If you do this then you are free from the lock. Thanks. Just to confirm: does using SeedSequence spawn_key arg generate distinct BitGenerators? As in cdef class Wrapper(): def __init__(self, seed): entropy, num = seed py_gen = PCG64(SeedSequence(entropy, spawn_key=(spawn_key,))) self.rng = py_gen.capsule.PyCapsule_GetPointer(capsule, "BitGenerator") # <--- this cdef Wrapper rng_0 = Wrapper(seed=(123, 0)) cdef Wrapper rng_1 = Wrapper(seed=(123, 1)) And then,of these two objects, do they have distinct BitGenerators? Evgeni > > > Kevin > > > > > > From: Evgeni Burovski > Sent: Monday, December 14, 2020 2:10 PM > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] locking np.random.Generator in a cython nogil context? > > > > On Mon, Dec 14, 2020 at 4:46 PM Kevin Sheppard > > wrote: > > > > > > You need to reacquire the gil, then you can get the lock and rerelease the gil. > > > > > > I think this works (on phone, so untested) > > > > > > with gil: > > > with nogil, lock: > > > .. > > > > Thanks Kevin. > > This surely works, but feels seriously weird. Is this the only / the > > recommended way? > > I can of course adjust the buffer size to amortize the time for the > > GIL manipulation, but this really starts looking like a code smell. > > My original motivation was to have inner simulation loops python-free. > > Most likely, the issue is that I'm not using the Generator correctly? > > > > Evgeni > > > > > > > On Mon, Dec 14, 2020, 13:37 Evgeni Burovski wrote: > > >> > > >> Hi, > > >> > > >> What would be the correct way of locking the bit generator of > > >> np.random.Generator in cython's nogil context? > > >> (This is threading 101, surely, so please forgive my ignorance). > > >> > > >> The docs for extending np.random.Generator in cython > > >> (https://numpy.org/doc/stable/reference/random/extending.html#cython) > > >> recommend the following idiom for generating uniform variates, where > > >> the GIL is released and a Generator-specific lock is held: > > >> > > >> x = PCG64() > > >> rng = PyCapsule_GetPointer(x.capsule, capsule_name) > > >> with nogil, x.lock: > > >> rng.next_double(rng.state) > > >> > > >> What is the correct way of locking it when already in the nogil > > >> section (so that x.lock is not accessible)? > > >> > > >> The use case is a long-running MC process which generates random > > >> variates in a tight loop (so the loop is all nogil). In case it > > >> matters, I probably won't be using python threads, but may use > > >> multiprocessing. > > >> > > >> Basically, > > >> > > >> cdef double uniform(self) nogil: > > >> if self.idx >= self.buf.shape[0]: > > >> self._fill() > > >> cdef double value = self.buf[self.idx] > > >> self.idx += 1 > > >> return value > > >> > > >> cdef void _fill(self) nogil: > > >> self.idx = 0 > > >> # HERE: Lock ? > > >> for i in range(self.buf.shape[0]): > > >> self.buf[i] = self.rng.next_double(self.rng.state) > > >> > > >> > > >> Thanks, > > >> Evgeni > > >> > > >> > > >> P.S. The full cdef class, for completeness: > > >> > > >> cdef class RndmWrapper(): > > >> cdef: > > >> double[::1] buf > > >> Py_ssize_t idx > > >> bitgen_t *rng > > >> object py_gen # keep the garbage collector away > > >> > > >> def __init__(self, seed=(1234, 0), buf_size=4096, bitgen_kind=None): > > >> if bitgen_kind is None: > > >> bitgen_kind = PCG64 > > >> > > >> # cf Numpy-discussion list, K.~Sheppard, R.~Kern, June 29, > > >> 2020 and below > > >> # https://mail.python.org/pipermail/numpy-discussion/2020-June/080794.html > > >> entropy, num = seed > > >> seed_seq = SeedSequence(entropy, spawn_key=(num,)) > > >> py_gen = bitgen_kind(seed_seq) > > >> > > >> # store the python object to avoid it being garbage collected > > >> self.py_gen = py_gen > > >> > > >> capsule = py_gen.capsule > > >> self.rng = PyCapsule_GetPointer(capsule, capsule_name) > > >> if not PyCapsule_IsValid(capsule, capsule_name): > > >> raise ValueError("Invalid pointer to anon_func_state") > > >> > > >> self.buf = np.empty(buf_size, dtype='float64') > > >> self._fill() > > >> > > >> @cython.boundscheck(False) > > >> @cython.wraparound(False) > > >> cdef void _fill(self) nogil: > > >> self.idx = 0 > > >> for i in range(self.buf.shape[0]): > > >> self.buf[i] = self.rng.next_double(self.rng.state) > > >> > > >> @cython.boundscheck(False) > > >> @cython.wraparound(False) > > >> cdef double uniform(self) nogil: > > >> if self.idx >= self.buf.shape[0]: > > >> self._fill() > > >> cdef double value = self.buf[self.idx] > > >> self.idx += 1 > > >> return value > > >> _______________________________________________ > > >> NumPy-Discussion mailing list > > >> NumPy-Discussion at python.org > > >> https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion at python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From robert.kern at gmail.com Mon Dec 14 16:59:23 2020 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 14 Dec 2020 16:59:23 -0500 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: References: <2011C308-3387-4142-93D4-2D63F64FEC48@hxcore.ol> Message-ID: On Mon, Dec 14, 2020 at 3:27 PM Evgeni Burovski wrote: > > > > I also think that the lock only matters for Multithreaded code not > Multiprocess. I believe the latter pickles and unpickles any Generator > object (and the underying BitGenerator) and so each process has its own > version. Note that when multiprocessing the recommended procedure is to > use spawn() to generate a sequence of BitGenerators and to use a distinct > BitGenerator in each process. If you do this then you are free from the > lock. > > Thanks. Just to confirm: does using SeedSequence spawn_key arg > generate distinct BitGenerators? As in > > cdef class Wrapper(): > def __init__(self, seed): > entropy, num = seed > py_gen = PCG64(SeedSequence(entropy, spawn_key=(spawn_key,))) > self.rng = > py_gen.capsule.PyCapsule_GetPointer(capsule, "BitGenerator") # <--- > this > > cdef Wrapper rng_0 = Wrapper(seed=(123, 0)) > cdef Wrapper rng_1 = Wrapper(seed=(123, 1)) > > And then,of these two objects, do they have distinct BitGenerators? > The code you wrote doesn't work (`spawn_key` is never assigned). I can guess what you meant to write, though, and yes, you would get distinct `BitGenerator`s. However, I do not recommend using `spawn_key` explicitly. The `SeedSequence.spawn()` method internally keeps track of how many children it has spawned and uses that to construct the `spawn_key`s for its subsequent children. If you play around with making your own `spawn_key`s, then the parent `SeedSequence(entropy)` might spawn identical `SeedSequence`s to the ones you constructed. If you don't want to use the `spawn()` API to construct the separate `SeedSequence`s but still want to incorporate some per-process information into the seeds (e.g. the 0 and 1 in your example), then note that a tuple of integers is a valid value for the `entropy` argument. You can have the first item be the same (i.e. per-run information) and the second item be a per-process ID or counter. cdef class Wrapper(): def __init__(self, seed): py_gen = PCG64(SeedSequence(seed)) self.rng = py_gen.capsule.PyCapsule_GetPointer(capsule, "BitGenerator") cdef Wrapper rng_0 = Wrapper(seed=(123, 0)) cdef Wrapper rng_1 = Wrapper(seed=(123, 1)) -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From scratchmex at gmail.com Mon Dec 14 22:30:08 2020 From: scratchmex at gmail.com (Ivan Gonzalez) Date: Mon, 14 Dec 2020 22:30:08 -0500 Subject: [Numpy-discussion] Assign shifted diagonal values with `np.fill_diagonal` and similar behaviour as `np.diag` Message-ID: As the subject says it will be great if `np.fill_diagonal` had a k-ith diagonal argument as `np.diag` does. The behavior expected (and a hack for the solution) is better explained in the following StackOverflow questions by me: https://stackoverflow.com/questions/65299295/assign-shifted-diagonal-values-with-np-diag/65299483 I had posted an issue on numpy repo as well: https://github.com/numpy/numpy/issues/18000 Hope to hear your suggestions. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Tue Dec 15 14:45:42 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 15 Dec 2020 13:45:42 -0600 Subject: [Numpy-discussion] NumPy Development Meeting Wednesday - Triage Focus Message-ID: Hi all, Our bi-weekly triage-focused NumPy development meeting is today (Wednesday, November 18th) at 11 am Pacific Time (19:00 UTC). Everyone is invited to join in and edit the work-in-progress meeting topics and notes: https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg I encourage everyone to notify us of issues or PRs that you feel should be prioritized, discussed, or reviewed. Best regards Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From sebastian at sipsolutions.net Tue Dec 15 14:46:55 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 15 Dec 2020 13:46:55 -0600 Subject: [Numpy-discussion] NumPy Development Meeting Wednesday - Triage Focus In-Reply-To: References: Message-ID: <1d2d852473e45357f609a73937b351e64db1aa3d.camel@sipsolutions.net> On Tue, 2020-12-15 at 13:45 -0600, Sebastian Berg wrote: > Hi all, > > Our bi-weekly triage-focused NumPy development meeting is today > (Wednesday, November 18th) at 11 am Pacific Time (19:00 UTC). Maybe I won't forget to update this next time. Of course tomorrow December 16th. > Everyone is invited to join in and edit the work-in-progress meeting > topics and notes: > https://hackmd.io/68i_JvOYQfy9ERiHgXMPvg > > I encourage everyone to notify us of issues or PRs that you feel > should > be prioritized, discussed, or reviewed. > > Best regards > > Sebastian > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From symphoni.bush at noaa.gov Wed Dec 16 11:24:48 2020 From: symphoni.bush at noaa.gov (Symphoni Bush - NOAA Affiliate) Date: Wed, 16 Dec 2020 11:24:48 -0500 Subject: [Numpy-discussion] NumPy EOL Versions? Message-ID: I am trying to find out if there are any end-of-life versions for NumPy, and if so, when do these versions typically become EOL/unsupported? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Thu Dec 17 04:47:11 2020 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Thu, 17 Dec 2020 12:47:11 +0300 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: References: <2011C308-3387-4142-93D4-2D63F64FEC48@hxcore.ol> Message-ID: On Tue, Dec 15, 2020 at 1:00 AM Robert Kern wrote: > > On Mon, Dec 14, 2020 at 3:27 PM Evgeni Burovski wrote: >> >> >> >> > I also think that the lock only matters for Multithreaded code not Multiprocess. I believe the latter pickles and unpickles any Generator object (and the underying BitGenerator) and so each process has its own version. Note that when multiprocessing the recommended procedure is to use spawn() to generate a sequence of BitGenerators and to use a distinct BitGenerator in each process. If you do this then you are free from the lock. >> >> Thanks. Just to confirm: does using SeedSequence spawn_key arg >> generate distinct BitGenerators? As in >> >> cdef class Wrapper(): >> def __init__(self, seed): >> entropy, num = seed >> py_gen = PCG64(SeedSequence(entropy, spawn_key=(spawn_key,))) >> self.rng = >> py_gen.capsule.PyCapsule_GetPointer(capsule, "BitGenerator") # <--- >> this >> >> cdef Wrapper rng_0 = Wrapper(seed=(123, 0)) >> cdef Wrapper rng_1 = Wrapper(seed=(123, 1)) >> >> And then,of these two objects, do they have distinct BitGenerators? > > > The code you wrote doesn't work (`spawn_key` is never assigned). I can guess what you meant to write, though, and yes, you would get distinct `BitGenerator`s. However, I do not recommend using `spawn_key` explicitly. The `SeedSequence.spawn()` method internally keeps track of how many children it has spawned and uses that to construct the `spawn_key`s for its subsequent children. If you play around with making your own `spawn_key`s, then the parent `SeedSequence(entropy)` might spawn identical `SeedSequence`s to the ones you constructed. > > If you don't want to use the `spawn()` API to construct the separate `SeedSequence`s but still want to incorporate some per-process information into the seeds (e.g. the 0 and 1 in your example), then note that a tuple of integers is a valid value for the `entropy` argument. You can have the first item be the same (i.e. per-run information) and the second item be a per-process ID or counter. > > cdef class Wrapper(): > def __init__(self, seed): > py_gen = PCG64(SeedSequence(seed)) > self.rng = py_gen.capsule.PyCapsule_GetPointer(capsule, "BitGenerator") > > cdef Wrapper rng_0 = Wrapper(seed=(123, 0)) > cdef Wrapper rng_1 = Wrapper(seed=(123, 1)) Thanks Robert! I indeed typo'd the spawn_key, and indeed the intention is exactly to include a worker_id into a seed to make sure each worker gets a separate stream. The use of the spawn_key was --- as I now finally realize --- a misunderstanding of your and Kevin's previous replies in https://mail.python.org/pipermail/numpy-discussion/2020-July/080833.html So I'm moving my project to use the `SeedSequence((base_seed, worker_id))` API --- thanks! Just as a side note, this is not very prominent in the docs, and I'm ready to volunteer to send a doc PR --- I'm only not sure which part of the docs, and would appreciate a pointer. From matti.picus at gmail.com Thu Dec 17 05:01:10 2020 From: matti.picus at gmail.com (Matti Picus) Date: Thu, 17 Dec 2020 12:01:10 +0200 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: References: <2011C308-3387-4142-93D4-2D63F64FEC48@hxcore.ol> Message-ID: <29a1304e-2f90-1cd4-2009-1d42a6563c05@gmail.com> On 12/17/20 11:47 AM, Evgeni Burovski wrote: > Just as a side note, this is not very prominent in the docs, and I'm > ready to volunteer to send a doc PR --- I'm only not sure which part > of the docs, and would appreciate a pointer. Maybe here https://numpy.org/devdocs/reference/random/bit_generators/index.html#seeding-and-entropy which is here in the sources https://github.com/numpy/numpy/blob/master/doc/source/reference/random/bit_generators/index.rst#seeding-and-entropy And/or in the SeedSequence docstring documentation https://numpy.org/devdocs/reference/random/bit_generators/generated/numpy.random.SeedSequence.html#numpy.random.SeedSequence which is here in the sources https://github.com/numpy/numpy/blob/master/numpy/random/bit_generator.pyx#L255 Matti From zcbtgy4 at ucl.ac.uk Thu Dec 17 08:18:47 2020 From: zcbtgy4 at ucl.ac.uk (k1o0) Date: Thu, 17 Dec 2020 06:18:47 -0700 (MST) Subject: [Numpy-discussion] datetime64: Remove deprecation warning when constructing with timezone In-Reply-To: References: Message-ID: <1608211127336-0.post@n7.nabble.com> Noam Yorav-Raphael wrote > The solution is simple, and is what datetime64 used to do before the > change > - have a type that just represents a moment in time. It's not "in UTC" - > it > just stores the number of seconds that passed since an agreed moment in > time (which is usually 1970-01-01 02:00+0200, which is more commonly > referred to as 1970-01-01 00:00Z - it's the exact same moment). I agree with this. I understand the issue of parsing arbitrary timestamps with incomplete information, however it's not clear to me why it has become more difficult to work with ISO 8601 timestamps. For example, we use numpy.genfromtxt to load an array with UTC offset timestamps e.g. `2020-08-19T12:42:57.7903616-04:00`. If loading this array took 0.0352s without having to convert, it now takes 0.8615s with the following converter: >>> lambda x: >>> dateutil.parser.parse(x).astimezone(timezone.utc).replace(tzinfo=None) That's a huge performance hit to do something that should be considered a standard operation, namely loading ISO compliant data. There may be more efficient converters out there but it seems strange to employ an external function to remove precision from an ISO datatype. As an aside, with or without the converter, numpy.genfromtxt is consistently faster than numpy.loadtxt, despite the documentation stating otherwise. I feel there's a lack of guidance in the documentation on this issue. In most threads I've encountered on this the first recommendation is to use pandas. The most effective way to crack a nut should not be to use a sledgehammer. The purpose of introducing standards should be to make these sorts of operations trivial and efficient. Perhaps I'm missing the solution here... -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From evgeny.burovskiy at gmail.com Thu Dec 17 09:55:32 2020 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Thu, 17 Dec 2020 17:55:32 +0300 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: <29a1304e-2f90-1cd4-2009-1d42a6563c05@gmail.com> References: <2011C308-3387-4142-93D4-2D63F64FEC48@hxcore.ol> <29a1304e-2f90-1cd4-2009-1d42a6563c05@gmail.com> Message-ID: On Thu, Dec 17, 2020 at 1:01 PM Matti Picus wrote: > > > On 12/17/20 11:47 AM, Evgeni Burovski wrote: > > Just as a side note, this is not very prominent in the docs, and I'm > > ready to volunteer to send a doc PR --- I'm only not sure which part > > of the docs, and would appreciate a pointer. > > Maybe here > > https://numpy.org/devdocs/reference/random/bit_generators/index.html#seeding-and-entropy > > which is here in the sources > > https://github.com/numpy/numpy/blob/master/doc/source/reference/random/bit_generators/index.rst#seeding-and-entropy > > > And/or in the SeedSequence docstring documentation > > https://numpy.org/devdocs/reference/random/bit_generators/generated/numpy.random.SeedSequence.html#numpy.random.SeedSequence > > which is here in the sources > > https://github.com/numpy/numpy/blob/master/numpy/random/bit_generator.pyx#L255 Here's the PR, https://github.com/numpy/numpy/pull/18014 Two minor comments, both OT for the PR: 1. The recommendation to seed the generators from the OS --- I've been bitten by exactly this once. That was a rather exotic combination of a vendor RNG and a batch queueing system, and some of my runs did end up with identical random streams. Given that the recommendation is what it is, it probably means that experience is a singular point and it no longer happens with modern generators. 2. Robert's comment that `SeedSequence(..., spawn_key=(num,))` is not equivalent to `SeedSequence(...).spawn(num)[num]` and that the former is not recommended. I'm not questioning the recommendation, but then __repr__ seems to suggest the equivalence: In [2]: from numpy.random import PCG64, SeedSequence In [3]: base_seq = SeedSequence(1234) In [4]: base_seq.spawn(8) Out[4]: [SeedSequence( entropy=1234, spawn_key=(0,), ), Evgeni From robert.kern at gmail.com Thu Dec 17 10:17:55 2020 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 17 Dec 2020 10:17:55 -0500 Subject: [Numpy-discussion] locking np.random.Generator in a cython nogil context? In-Reply-To: References: <2011C308-3387-4142-93D4-2D63F64FEC48@hxcore.ol> <29a1304e-2f90-1cd4-2009-1d42a6563c05@gmail.com> Message-ID: On Thu, Dec 17, 2020 at 9:56 AM Evgeni Burovski wrote: > On Thu, Dec 17, 2020 at 1:01 PM Matti Picus wrote: > > > > > > On 12/17/20 11:47 AM, Evgeni Burovski wrote: > > > Just as a side note, this is not very prominent in the docs, and I'm > > > ready to volunteer to send a doc PR --- I'm only not sure which part > > > of the docs, and would appreciate a pointer. > > > > Maybe here > > > > > https://numpy.org/devdocs/reference/random/bit_generators/index.html#seeding-and-entropy > > > > which is here in the sources > > > > > https://github.com/numpy/numpy/blob/master/doc/source/reference/random/bit_generators/index.rst#seeding-and-entropy > > > > > > And/or in the SeedSequence docstring documentation > > > > > https://numpy.org/devdocs/reference/random/bit_generators/generated/numpy.random.SeedSequence.html#numpy.random.SeedSequence > > > > which is here in the sources > > > > > https://github.com/numpy/numpy/blob/master/numpy/random/bit_generator.pyx#L255 > > > Here's the PR, https://github.com/numpy/numpy/pull/18014 > > Two minor comments, both OT for the PR: > > 1. The recommendation to seed the generators from the OS --- I've been > bitten by exactly this once. That was a rather exotic combination of a > vendor RNG and a batch queueing system, and some of my runs did end up > with identical random streams. Given that the recommendation is what > it is, it probably means that experience is a singular point and it no > longer happens with modern generators. > I suspect the vendor RNG was rolling its own entropy using time. We use `secrets.getrandbits()`, which ultimately uses the best cryptographic entropy source available. And if there is no cryptographic entropy source available, I think we fail hard instead of falling back to less reliable things like time. I'm not entirely sure that's a feature, but it is safe! > 2. Robert's comment that `SeedSequence(..., spawn_key=(num,))` is not > equivalent to `SeedSequence(...).spawn(num)[num]` and that the former > is not recommended. I'm not questioning the recommendation, but then > __repr__ seems to suggest the equivalence: > I was saying that they were equivalent. That's precisely why it's not recommended: it's too easy to do both and get identical streams inadvertently. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From tinchochalela at gmail.com Fri Dec 18 09:58:31 2020 From: tinchochalela at gmail.com (=?UTF-8?Q?Mart=C3=ADn_Chalela?=) Date: Fri, 18 Dec 2020 11:58:31 -0300 Subject: [Numpy-discussion] Optimized np.digitize for equidistant bins Message-ID: Hi all! I was wondering if there is a way around to using np.digitize when dealing with equidistant bins. For example: bins = np.linspace(0, 1, 20) The main problem I encountered is that digitize calls np.searchsorted. This is the correct way, I think, for generic bins, i.e. bins that have different widths. However, in the special, but not uncommon, case of equidistant bins, the searchsorted call can be very expensive and unnecessary. One can perform a simple calculation like the following: def digitize_eqbins(x, bins): """ Return the indices of the bins to which each value in input array belongs. Assumes equidistant bins. """ nbins = len(bins) - 1 digit = (nbins * (x - bins[0]) / (bins[-1] - bins[0])).astype(np.int) return digit + 1 Is there a better way of computing this for equidistant bins? Thank you! Martin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jfoxrabinovitz at gmail.com Fri Dec 18 12:15:43 2020 From: jfoxrabinovitz at gmail.com (Joseph Fox-Rabinovitz) Date: Fri, 18 Dec 2020 12:15:43 -0500 Subject: [Numpy-discussion] Optimized np.digitize for equidistant bins In-Reply-To: References: Message-ID: Bin index is just value floor divided by the bin size. On Fri, Dec 18, 2020, 09:59 Mart?n Chalela wrote: > Hi all! I was wondering if there is a way around to using np.digitize when > dealing with equidistant bins. For example: > bins = np.linspace(0, 1, 20) > > The main problem I encountered is that digitize calls np.searchsorted. > This is the correct way, I think, for generic bins, i.e. bins that have > different widths. However, in the special, but not uncommon, case of > equidistant bins, the searchsorted call can be very expensive and > unnecessary. One can perform a simple calculation like the following: > > def digitize_eqbins(x, bins): > """ > Return the indices of the bins to which each value in input array belongs. > Assumes equidistant bins. > """ > nbins = len(bins) - 1 > digit = (nbins * (x - bins[0]) / (bins[-1] - bins[0])).astype(np.int) > return digit + 1 > > Is there a better way of computing this for equidistant bins? > > Thank you! > Martin. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tinchochalela at gmail.com Fri Dec 18 14:36:57 2020 From: tinchochalela at gmail.com (=?UTF-8?Q?Mart=C3=ADn_Chalela?=) Date: Fri, 18 Dec 2020 16:36:57 -0300 Subject: [Numpy-discussion] Optimized np.digitize for equidistant bins In-Reply-To: References: Message-ID: Right! I just thought there would/should be a "digitize" function that did this. El vie, 18 dic 2020 a las 14:16, Joseph Fox-Rabinovitz (< jfoxrabinovitz at gmail.com>) escribi?: > Bin index is just value floor divided by the bin size. > > On Fri, Dec 18, 2020, 09:59 Mart?n Chalela > wrote: > >> Hi all! I was wondering if there is a way around to using np.digitize >> when dealing with equidistant bins. For example: >> bins = np.linspace(0, 1, 20) >> >> The main problem I encountered is that digitize calls np.searchsorted. >> This is the correct way, I think, for generic bins, i.e. bins that have >> different widths. However, in the special, but not uncommon, case of >> equidistant bins, the searchsorted call can be very expensive and >> unnecessary. One can perform a simple calculation like the following: >> >> def digitize_eqbins(x, bins): >> """ >> Return the indices of the bins to which each value in input array belongs >> . >> Assumes equidistant bins. >> """ >> nbins = len(bins) - 1 >> digit = (nbins * (x - bins[0]) / (bins[-1] - bins[0])).astype(np.int) >> return digit + 1 >> >> Is there a better way of computing this for equidistant bins? >> >> Thank you! >> Martin. >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jfoxrabinovitz at gmail.com Fri Dec 18 14:56:16 2020 From: jfoxrabinovitz at gmail.com (Joseph Fox-Rabinovitz) Date: Fri, 18 Dec 2020 14:56:16 -0500 Subject: [Numpy-discussion] Optimized np.digitize for equidistant bins In-Reply-To: References: Message-ID: There is: np.floor_divide. On Fri, Dec 18, 2020, 14:38 Mart?n Chalela wrote: > Right! I just thought there would/should be a "digitize" function that did > this. > > El vie, 18 dic 2020 a las 14:16, Joseph Fox-Rabinovitz (< > jfoxrabinovitz at gmail.com>) escribi?: > >> Bin index is just value floor divided by the bin size. >> >> On Fri, Dec 18, 2020, 09:59 Mart?n Chalela >> wrote: >> >>> Hi all! I was wondering if there is a way around to using np.digitize >>> when dealing with equidistant bins. For example: >>> bins = np.linspace(0, 1, 20) >>> >>> The main problem I encountered is that digitize calls np.searchsorted. >>> This is the correct way, I think, for generic bins, i.e. bins that have >>> different widths. However, in the special, but not uncommon, case of >>> equidistant bins, the searchsorted call can be very expensive and >>> unnecessary. One can perform a simple calculation like the following: >>> >>> def digitize_eqbins(x, bins): >>> """ >>> Return the indices of the bins to which each value in input array belongs >>> . >>> Assumes equidistant bins. >>> """ >>> nbins = len(bins) - 1 >>> digit = (nbins * (x - bins[0]) / (bins[-1] - bins[0])).astype(np.int) >>> return digit + 1 >>> >>> Is there a better way of computing this for equidistant bins? >>> >>> Thank you! >>> Martin. >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion at python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tinchochalela at gmail.com Fri Dec 18 16:19:49 2020 From: tinchochalela at gmail.com (=?UTF-8?Q?Mart=C3=ADn_Chalela?=) Date: Fri, 18 Dec 2020 18:19:49 -0300 Subject: [Numpy-discussion] Optimized np.digitize for equidistant bins In-Reply-To: References: Message-ID: Thank you Joseph El vie, 18 dic 2020 a las 16:56, Joseph Fox-Rabinovitz (< jfoxrabinovitz at gmail.com>) escribi?: > There is: np.floor_divide. > > On Fri, Dec 18, 2020, 14:38 Mart?n Chalela > wrote: > >> Right! I just thought there would/should be a "digitize" function that >> did this. >> >> El vie, 18 dic 2020 a las 14:16, Joseph Fox-Rabinovitz (< >> jfoxrabinovitz at gmail.com>) escribi?: >> >>> Bin index is just value floor divided by the bin size. >>> >>> On Fri, Dec 18, 2020, 09:59 Mart?n Chalela >>> wrote: >>> >>>> Hi all! I was wondering if there is a way around to using np.digitize >>>> when dealing with equidistant bins. For example: >>>> bins = np.linspace(0, 1, 20) >>>> >>>> The main problem I encountered is that digitize calls np.searchsorted. >>>> This is the correct way, I think, for generic bins, i.e. bins that have >>>> different widths. However, in the special, but not uncommon, case of >>>> equidistant bins, the searchsorted call can be very expensive and >>>> unnecessary. One can perform a simple calculation like the following: >>>> >>>> def digitize_eqbins(x, bins): >>>> """ >>>> Return the indices of the bins to which each value in input array >>>> belongs. >>>> Assumes equidistant bins. >>>> """ >>>> nbins = len(bins) - 1 >>>> digit = (nbins * (x - bins[0]) / (bins[-1] - bins[0])).astype(np.int) >>>> return digit + 1 >>>> >>>> Is there a better way of computing this for equidistant bins? >>>> >>>> Thank you! >>>> Martin. >>>> _______________________________________________ >>>> NumPy-Discussion mailing list >>>> NumPy-Discussion at python.org >>>> https://mail.python.org/mailman/listinfo/numpy-discussion >>>> >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion at python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Tue Dec 22 17:35:41 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Tue, 22 Dec 2020 15:35:41 -0700 Subject: [Numpy-discussion] ANN: SciPy 1.6.0rc2 -- please test Message-ID: Hi all, On behalf of the SciPy development team I'm pleased to announce the release candidate SciPy 1.6.0rc2. Please help us test this pre-release. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.6.0rc2 One of a few ways to install the release candidate with pip: pip install scipy==1.6.0rc2 ========================== SciPy 1.6.0 Release Notes ========================== Note: Scipy 1.6.0 is not released yet! SciPy 1.6.0 is the culmination of 6 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Before upgrading, we recommend that users check that their own code does not use deprecated SciPy functionality (to do so, run your code with ``python -Wd`` and check for ``DeprecationWarning`` s). Our development attention will now shift to bug-fix releases on the 1.6.x branch, and on adding new features on the master branch. This release requires Python 3.7+ and NumPy 1.16.5 or greater. For running on PyPy, PyPy3 6.0+ is required. Highlights of this release --------------------------------- - `scipy.ndimage` improvements: Fixes and ehancements to boundary extension modes for interpolation functions. Support for complex-valued inputs in many filtering and interpolation functions. New ``grid_mode`` option for `scipy.ndimage.zoom` to enable results consistent with scikit-image's ``rescale``. - `scipy.optimize.linprog` has fast, new methods for large, sparse problems from the ``HiGHS`` library. - `scipy.stats` improvements including new distributions, a new test, and enhancements to existing distributions and tests New features ============ `scipy.special` improvements --------------------------------------- `scipy.special` now has improved support for 64-bit ``LAPACK`` backend `scipy.odr` improvements ---------------------------------- `scipy.odr` now has support for 64-bit integer ``BLAS`` `scipy.odr.ODR` has gained an optional ``overwrite`` argument so that existing files may be overwritten. `scipy.integrate` improvements ------------------------------------------ Some renames of functions with poor names were done, with the old names retained without being in the reference guide for backwards compatibility reasons: - ``integrate.simps`` was renamed to ``integrate.simpson`` - ``integrate.trapz`` was renamed to ``integrate.trapezoid`` - ``integrate.cumtrapz`` was renamed to ``integrate.cumulative_trapezoid`` `scipy.cluster` improvements --------------------------------------- `scipy.cluster.hierarchy.DisjointSet` has been added for incremental connectivity queries. `scipy.cluster.hierarchy.dendrogram` return value now also includes leaf color information in `leaves_color_list`. `scipy.interpolate` improvements -------------------------------------------- `scipy.interpolate.interp1d` has a new method ``nearest-up``, similar to the existing method ``nearest`` but rounds half-integers up instead of down. `scipy.io` improvements -------------------------------- Support has been added for reading arbitrary bit depth integer PCM WAV files from 1- to 32-bit, including the commonly-requested 24-bit depth. `scipy.linalg` improvements ------------------------------------- The new function `scipy.linalg.matmul_toeplitz` uses the FFT to compute the product of a Toeplitz matrix with another matrix. `scipy.linalg.sqrtm` and `scipy.linalg.logm` have performance improvements thanks to additional Cython code. Python ``LAPACK`` wrappers have been added for ``pptrf``, ``pptrs``, ``ppsv``, ``pptri``, and ``ppcon``. `scipy.linalg.norm` and the ``svd`` family of functions will now use 64-bit integer backends when available. `scipy.ndimage` improvements ----------------------------------------- `scipy.ndimage.convolve`, `scipy.ndimage.correlate` and their 1d counterparts now accept both complex-valued images and/or complex-valued filter kernels. All convolution-based filters also now accept complex-valued inputs (e.g. ``gaussian_filter``, ``uniform_filter``, etc.). Multiple fixes and enhancements to boundary handling were introduced to `scipy.ndimage` interpolation functions (i.e. ``affine_transform``, ``geometric_transform``, ``map_coordinates``, ``rotate``, ``shift``, ``zoom``). A new boundary mode, ``grid-wrap`` was added which wraps images periodically, using a period equal to the shape of the input image grid. This is in contrast to the existing ``wrap`` mode which uses a period that is one sample smaller than the original signal extent along each dimension. A long-standing bug in the ``reflect`` boundary condition has been fixed and the mode ``grid-mirror`` was introduced as a synonym for ``reflect``. A new boundary mode, ``grid-constant`` is now available. This is similar to the existing ndimage ``constant`` mode, but interpolation will still performed at coordinate values outside of the original image extent. This ``grid-constant`` mode is consistent with OpenCV's ``BORDER_CONSTANT`` mode and scikit-image's ``constant`` mode. Spline pre-filtering (used internally by ``ndimage`` interpolation functions when ``order >= 2``), now supports all boundary modes rather than always defaulting to mirror boundary conditions. The standalone functions ``spline_filter`` and ``spline_filter1d`` have analytical boundary conditions that match modes ``mirror``, ``grid-wrap`` and ``reflect``. `scipy.ndimage` interpolation functions now accept complex-valued inputs. In this case, the interpolation is applied independently to the real and imaginary components. The ``ndimage`` tutorials (https://docs.scipy.org/doc/scipy/reference/tutorial/ndimage.html) have been updated with new figures to better clarify the exact behavior of all of the interpolation boundary modes. `scipy.ndimage.zoom` now has a ``grid_mode`` option that changes the coordinate of the center of the first pixel along an axis from 0 to 0.5. This allows resizing in a manner that is consistent with the behavior of scikit-image's ``resize`` and ``rescale`` functions (and OpenCV's ``cv2.resize``). `scipy.optimize` improvements ----------------------------------------- `scipy.optimize.linprog` has fast, new methods for large, sparse problems from the ``HiGHS`` C++ library. ``method='highs-ds'`` uses a high performance dual revised simplex implementation (HSOL), ``method='highs-ipm'`` uses an interior-point method with crossover, and ``method='highs'`` chooses between the two automatically. These methods are typically much faster and often exceed the accuracy of other ``linprog`` methods, so we recommend explicitly specifying one of these three method values when using ``linprog``. `scipy.optimize.quadratic_assignment` has been added for approximate solution of the quadratic assignment problem. `scipy.optimize.linear_sum_assignment` now has a substantially reduced overhead for small cost matrix sizes `scipy.optimize.least_squares` has improved performance when the user provides the jacobian as a sparse jacobian already in ``csr_matrix`` format `scipy.optimize.linprog` now has an ``rr_method`` argument for specification of the method used for redundancy handling, and a new method for this purpose is available based on the interpolative decomposition approach. `scipy.signal` improvements -------------------------------------- `scipy.signal.gammatone` has been added to design FIR or IIR filters that model the human auditory system. `scipy.signal.iircomb` has been added to design IIR peaking/notching comb filters that can boost/attenuate a frequency from a signal. `scipy.signal.sosfilt` performance has been improved to avoid some previously- observed slowdowns `scipy.signal.windows.taylor` has been added--the Taylor window function is commonly used in radar digital signal processing `scipy.signal.gauss_spline` now supports ``list`` type input for consistency with other related SciPy functions `scipy.signal.correlation_lags` has been added to allow calculation of the lag/ displacement indices array for 1D cross-correlation. `scipy.sparse` improvements --------------------------------------- A solver for the minimum weight full matching problem for bipartite graphs, also known as the linear assignment problem, has been added in `scipy.sparse.csgraph.min_weight_full_bipartite_matching`. In particular, this provides functionality analogous to that of `scipy.optimize.linear_sum_assignment`, but with improved performance for sparse inputs, and the ability to handle inputs whose dense representations would not fit in memory. The time complexity of `scipy.sparse.block_diag` has been improved dramatically from quadratic to linear. `scipy.sparse.linalg` improvements ----------------------------------------------- The vendored version of ``SuperLU`` has been updated `scipy.fft` improvements -------------------------------- The vendored ``pocketfft`` library now supports compiling with ARM neon vector extensions and has improved thread pool behavior. `scipy.spatial` improvements -------------------------------------- The python implementation of ``KDTree`` has been dropped and ``KDTree`` is now implemented in terms of ``cKDTree``. You can now expect ``cKDTree``-like performance by default. This also means ``sys.setrecursionlimit`` no longer needs to be increased for querying large trees. ``transform.Rotation`` has been updated with support for Modified Rodrigues Parameters alongside the existing rotation representations (PR gh-12667). `scipy.spatial.transform.Rotation` has been partially cythonized, with some performance improvements observed `scipy.spatial.distance.cdist` has improved performance with the ``minkowski`` metric, especially for p-norm values of 1 or 2. `scipy.stats` improvements ------------------------------------ New distributions have been added to `scipy.stats`: - The asymmetric Laplace continuous distribution has been added as `scipy.stats.laplace_asymmetric`. - The negative hypergeometric distribution has been added as `scipy.stats.nhypergeom`. - The multivariate t distribution has been added as `scipy.stats.multivariate_t`. - The multivariate hypergeometric distribution has been added as `scipy.stats.multivariate_hypergeom`. The ``fit`` method has been overridden for several distributions (``laplace``, ``pareto``, ``rayleigh``, ``invgauss``, ``logistic``, ``gumbel_l``, ``gumbel_r``); they now use analytical, distribution-specific maximum likelihood estimation results for greater speed and accuracy than the generic (numerical optimization) implementation. The one-sample Cram?r-von Mises test has been added as `scipy.stats.cramervonmises`. An option to compute one-sided p-values was added to `scipy.stats.ttest_1samp`, `scipy.stats.ttest_ind_from_stats`, `scipy.stats.ttest_ind` and `scipy.stats.ttest_rel`. The function `scipy.stats.kendalltau` now has an option to compute Kendall's tau-c (also known as Stuart's tau-c), and support has been added for exact p-value calculations for sample sizes ``> 171``. `stats.trapz` was renamed to `stats.trapezoid`, with the former name retained as an alias for backwards compatibility reasons. The function `scipy.stats.linregress` now includes the standard error of the intercept in its return value. The ``_logpdf``, ``_sf``, and ``_isf`` methods have been added to `scipy.stats.nakagami`; ``_sf`` and ``_isf`` methods also added to `scipy.stats.gumbel_r` The ``sf`` method has been added to `scipy.stats.levy` and `scipy.stats.levy_l` for improved precision. `scipy.stats.binned_statistic_dd` performance improvements for the following computed statistics: ``max``, ``min``, ``median``, and ``std``. We gratefully acknowledge the Chan-Zuckerberg Initiative Essential Open Source Software for Science program for supporting many of these improvements to `scipy.stats`. Deprecated features ================ `scipy.spatial` changes ------------------------------- Calling ``KDTree.query`` with ``k=None`` to find all neighbours is deprecated. Use ``KDTree.query_ball_point`` instead. ``distance.wminkowski`` was deprecated; use ``distance.minkowski`` and supply weights with the ``w`` keyword instead. Backwards incompatible changes ========================== `scipy` changes ---------------------- Using `scipy.fft` as a function aliasing ``numpy.fft.fft`` was removed after being deprecated in SciPy ``1.4.0``. As a result, the `scipy.fft` submodule must be explicitly imported now, in line with other SciPy subpackages. `scipy.interpolate` changes ------------------------------------ `scipy.linalg` changes ----------------------------- `scipy.signal` changes ----------------------------- The output of ``decimate``, ``lfilter_zi``, ``lfiltic``, ``sos2tf``, and ``sosfilt_zi`` have been changed to match ``numpy.result_type`` of their inputs. The window function ``slepian`` was removed. It had been deprecated since SciPy ``1.1``. `scipy.spatial` changes ------------------------------- ``cKDTree.query`` now returns 64-bit rather than 32-bit integers on Windows, making behaviour consistent between platforms (PR gh-12673). `scipy.stats` changes ----------------------------- The ``frechet_l`` and ``frechet_r`` distributions were removed. They were deprecated since SciPy ``1.0``. Other changes ============= ``setup_requires`` was removed from ``setup.py``. This means that users invoking ``python setup.py install`` without having numpy already installed will now get an error, rather than having numpy installed for them via ``easy_install``. This install method was always fragile and problematic, users are encouraged to use ``pip`` when installing from source. - Fixed a bug in `scipy.optimize.dual_annealing` ``accept_reject`` calculation that caused uphill jumps to be accepted less frequently. - The time required for (un)pickling of `scipy.stats.rv_continuous`, `scipy.stats.rv_discrete`, and `scipy.stats.rv_frozen` has been significantly reduced (gh12550). Inheriting subclasses should note that ``__setstate__`` no longer calls ``__init__`` upon unpickling. Authors ======= * @endolith * @vkk800 * aditya + * George Bateman + * Christoph Baumgarten * Peter Bell * Tobias Biester + * Keaton J. Burns + * Evgeni Burovski * R?diger Busche + * Matthias Bussonnier * Dominic C + * Corallus Caninus + * CJ Carey * Thomas A Caswell * chapochn + * Luc?a Cheung * Zach Colbert + * Coloquinte + * Yannick Copin + * Devin Crowley + * Terry Davis + * Micha?l Defferrard + * devonwp + * Didier + * divenex + * Thomas Duvernay + * Eoghan O'Connell + * G?k?en Eraslan * Kristian Eschenburg + * Ralf Gommers * Thomas Grainger + * GreatV + * Gregory Gundersen + * h-vetinari + * Matt Haberland * Mark Harfouche + * He He + * Alex Henrie * Chun-Ming Huang + * Martin James McHugh III + * Alex Izvorski + * Joey + * ST John + * Jonas Jonker + * Julius Bier Kirkegaard * Marcin Konowalczyk + * Konrad0 * Sam Van Kooten + * Sergey Koposov + * Peter Mahler Larsen * Eric Larson * Antony Lee * Gregory R. Lee * Lo?c Est?ve * Jean-Luc Margot + * MarkusKoebis + * Nikolay Mayorov * G. D. McBain * Andrew McCluskey + * Nicholas McKibben * Sturla Molden * Denali Molitor + * Eric Moore * Shashaank N + * Prashanth Nadukandi + * nbelakovski + * Andrew Nelson * Nick + * Nikola Forr? + * odidev * ofirr + * Sambit Panda * Dima Pasechnik * Tirth Patel + * Matti Picus * Pawe? Redzy?ski + * Vladimir Philipenko + * Philipp Th?lke + * Ilhan Polat * Eugene Prilepin + * Vladyslav Rachek * Ram Rachum + * Tyler Reddy * Martin Reinecke + * Simon Segerblom Rex + * Lucas Roberts * Benjamin Rowell + * Eli Rykoff + * Atsushi Sakai * Moritz Schulte + * Daniel B. Smith * Steve Smith + * Jan Soedingrekso + * Victor Stinner + * Jose Storopoli + * Diana Sukhoverkhova + * S?ren Fuglede J?rgensen * taoky + * Mike Taves + * Ian Thomas + * Will Tirone + * Frank Torres + * Seth Troisi * Ronald van Elburg + * Hugo van Kemenade * Paul van Mulbregt * Saul Ivan Rivas Vega + * Pauli Virtanen * Jan Vleeshouwers * Samuel Wallan * Warren Weckesser * Ben West + * Eric Wieser * WillTirone + * Levi John Wolf + * Zhiqing Xiao * Rory Yorke + * Yun Wang (Maigo) + * Egor Zemlyanoy + * ZhihuiChen0903 + * Jacob Zhong + A total of 122 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.6.0 ------------------------------ * `#1323 `__: ndimage.shift destroys data from edges (Trac #796) * `#1892 `__: using rptfile= with an existing file causes a Fortran runtime... * `#1903 `__: ndimage.rotate misses some values (Trac #1378) * `#1930 `__: scipy.io.wavfile should be able to read 24 bit signed wave (Trac... * `#3158 `__: Odd casting behaviour of signal.filtfilt * `#3203 `__: interpolation.zoom incorrect output for certain cases * `#3645 `__: BUG: stats: mstats.pearsonr calculation is wrong if the masks... * `#3665 `__: Return Bunch objects from stats functions * `#4922 `__: unexpected zero output values from zoom * `#5202 `__: BUG: stats: Spurious warnings from the pdf method of several... * `#5223 `__: Zoom does not return the same values when resizing a sub-array... * `#5396 `__: scipy.spatial.distance.pdist documention bug * `#5489 `__: ValueError: failed to create intent(cache|hide)|optional array--... * `#6096 `__: loadmat drops dtype of empty arrays when squeeze_me=True * `#6713 `__: sicpy.ndimage.zoom returns artefacts and boundaries in some cases * `#7125 `__: Impossible to know number of dimensions in c function used by... * `#7324 `__: scipy.ndimage.zoom bad interpolation when downsampling (zoom... * `#8131 `__: BUG: geometric_transform wrap mode possible bug * `#8163 `__: LSMR fails on some random values when providing an x0 * `#8210 `__: Why should I choose order > 1 for scipy.ndimage.zoom? * `#8465 `__: Unexpected behavior with reflect mode of ndimage.rotate * `#8776 `__: cdist behavior with Minkowsky and np.inf * `#9168 `__: documentation of pearson3 in scipy.stats unclear * `#9223 `__: Faster implementation of scipy.sparse.block_diag * `#9476 `__: Invalid index in signal.medfilt2d's QUICK_SELECT * `#9857 `__: scipy.odr.Output.sd_beta is not standard error * `#9865 `__: Strange behavior of \`ndimage.shift\` and \`ndimage.affine_transform\` * `#10042 `__: Consider support for multivariate student-t distribution? * `#10134 `__: gausshyper distribution accepts invalid parameters * `#10179 `__: str+bytes concatenation error in test_lapack.py * `#10216 `__: cKDTree.query_ball_point speed regression * `#10463 `__: ENH: vectorize scipy.fft for more CPU architectures * `#10593 `__: Rename \`sum\` ndimage function * `#10595 `__: scipy.stats.ttest_1samp should support alternative hypothesis * `#10610 `__: ndimage.interpolation.spline_filter1d default value of mode * `#10620 `__: ndimage.interpolation.zoom() option to work like skimage.transform.resize() * `#10711 `__: Array Shapes Not Aligned Bug in scipy.optimize._lsq.lsq_linear.py * `#10782 `__: BUG: optimize: methods unknown to \`scipy.optimize.show_options\` * `#10892 `__: Possible typo in an equation of optimize/dual_annealing * `#11020 `__: signal.fftconvolve return a tuple including lag information * `#11093 `__: scipy.interpolate.interp1d can not handle datetime64 * `#11170 `__: Use manylinux2014 to get aarch64/ppc64le support * `#11186 `__: BUG: stats: pearson3 CDF and SF functions incorrect when skew... * `#11366 `__: DeprecationWarning due to invalid escape sequences * `#11403 `__: Optimize raises "ValueError: \`x0\` violates bound constraints"... * `#11558 `__: ENH: IIR comb filter * `#11559 `__: BUG: iirdesign doesn't fail for frequencies above Nyquist * `#11567 `__: scipy.signal.iirdesign doesn't check consistency of wp and ws... * `#11654 `__: ENH: Add Negative Hypergeometric Distribution * `#11720 `__: BUG: stats: wrong shape from median_absolute_deviation for arrays... * `#11746 `__: BUG: stats: pearson3 returns size 1 arrays where other distributions... * `#11756 `__: Improve and fix \*Spline docstrings and code * `#11758 `__: BUG: of scipy.interpolate.CubicSpline when \`bc_type' is set... * `#11925 `__: MAINT: remove character encoding check in CI? * `#11963 `__: Test failures - TestLinprogIPSparseCholmod * `#12102 `__: incorrect first moment of non central t-distribution * `#12113 `__: scipy.stats.poisson docs for rate = 0 * `#12152 `__: ENH: signal.gauss_spline should accept a list * `#12157 `__: BUG: Iteration index initialisation is wrong in scipy.optimize.linesearch.scalar_search_wolfe2 * `#12162 `__: Storing Rotation object in NumPy array returns an array with... * `#12176 `__: cannot modify the slice of an array returned by \`wavfile.read\` * `#12190 `__: retrieve leave colors from dendrogram * `#12196 `__: PERF: scipy.linalg.pinv is very slow compared to numpy.linalg.pinv * `#12222 `__: Interpolating categorical data (interp1d) * `#12231 `__: Is the p-value of the Kruskal-Wallis test two-sided? * `#12249 `__: ENH: least_squares: should not re-instanciate csr_matrix if already... * `#12264 `__: DOC: optimize: linprog method-specific function signature * `#12290 `__: DOC: Convex Hull areas are actually perimeters for 2-dimensional... * `#12308 `__: integrate.solve_ivp with DOP853 method fails when yDot = 0 * `#12326 `__: BUG: stats.exponnorm.pdf returns 0 for small K * `#12337 `__: scipy.sparse.linalg.eigsh documentation is misleading * `#12339 `__: scipy.io.wavfile.write documentation has wrong example * `#12340 `__: sparse.lil_matrix.tocsr() fails silently on matrices with nzn... * `#12350 `__: Create a 2-parameter version of the gamma distribution * `#12369 `__: scipy.signal.correlate has an error in the documentation, examples... * `#12373 `__: interp1d returns incorrect values for step functions * `#12378 `__: interpolate.NearestNDInterpolator.__call__ & LinearNDInterpolator.__call__... * `#12411 `__: scipy.stats.spearmanr mishandles nan variables with "propogate" * `#12413 `__: DOC: Remove the "Basic functions" section from the SciPy tutorial. * `#12415 `__: scipy.stats.dirichlet documentation issue * `#12419 `__: least_squares ValueError with 'lm' method - regression from 1.4.1... * `#12431 `__: Request for Python wrapper for LAPACK's ?pptrf (Cholesky factorization... * `#12458 `__: spearmanr with entire NaN columns produces errors * `#12477 `__: WIP: Addition of MLE for stats.invgauss/wald * `#12483 `__: reading .wav fails when the file is too big on python 3.6.0 * `#12490 `__: BUG: stats: logistic and genlogistic logpdf overflow for large... * `#12499 `__: LinearNDInterpolator raises ValueError when value array has writeable=False... * `#12523 `__: Wrong key in __odrpack.c * `#12547 `__: typo in scipy/cluster/_hierarchy.pyx * `#12549 `__: DOC: least_squares return type is poorly formatted. * `#12578 `__: TST: test_bounds_infeasible_2 failing on wheels repo cron jobs * `#12585 `__: ENH: Add Multivariate Hypergeometric Distribution * `#12604 `__: unintuitive conversion in \`scipy.constants.lambda2nu\` * `#12606 `__: DOC: Invalid syntax in example. * `#12665 `__: List of possible bugs found by automated code analysis * `#12696 `__: scipy.optimize.fminbound, numpy depreciation warning Creating... * `#12699 `__: TestProjections.test_iterative_refinements_dense failure * `#12701 `__: TestDifferentialEvolutionSolver::test_L4 failing * `#12719 `__: Misleading scipy.signal.get_window() docstring with 'exponential'... * `#12740 `__: circstd doesn't handle R = hypot(S, C) > 1 * `#12749 `__: ENH: interp1d Matlab compatibility * `#12773 `__: Meta-issue: ndimage spline boundary handling (NumFOCUS proposal) * `#12813 `__: optimize.root(method="krylov") fails if options["tol_norm"] expects... * `#12815 `__: stats.zscore inconsistent behavior when all values are the same * `#12840 `__: scipy.signal.windows.dpss docstring typo * `#12874 `__: Rotation.random vs stats.special_ortho_group * `#12881 `__: FFT - documentation - examples - linspace construction * `#12904 `__: BUG: parsing in loadarff() * `#12917 `__: GitHub Actions nightly build triggered on forks * `#12919 `__: BUG: numerical precision, use gammaln in nct.mean * `#12924 `__: Rename Sample Based Integration Methods to Comply with Code of... * `#12940 `__: Should the minimum numpy for AIX be bumped to 1.16.5 * `#12951 `__: A possible typo in scipy.stats.weightedtau * `#12952 `__: [Documentation question] Would it be more precise to specify... * `#12970 `__: Documentation presents second order sections as the correct choice... * `#12982 `__: Calculate standard error of the intercept in linregress * `#12985 `__: Possible wrong link in scipy.stats.wilcoxon doc * `#12991 `__: least_squares broken with float32 * `#13001 `__: \`OptimizeResult.message\` from \`L-BFGS-B\` is a bytes, not... * `#13030 `__: BUG: lint_diff.py still fails for backport PRs * `#13077 `__: CI: codecov proper patch diffs * `#13085 `__: Build failing on main branch after HiGHS solver merge * `#13088 `__: BLD, BUG: wheel builds failure with HiGHS/optimize * `#13099 `__: Wrong output format for empty sparse results of kron * `#13108 `__: TST, CI: GitHub Actions MacOS Failures * `#13111 `__: BUG, DOC: refguide check is failing * `#13127 `__: ODR output file writing broken in conda env with system compilers * `#13134 `__: FromTravis migration tracker * `#13140 `__: BUG: signal: \`ss2tf\` incorrectly truncates output to integers. * `#13179 `__: CI: lint is failing because of output to stderr * `#13182 `__: Key appears twice in \`test_optimize.test_show_options\` * `#13191 `__: \`scipy.linalg.lapack.dgesjv\` overwrites original arrays if... * `#13207 `__: TST: Erratic test failure in test_cossin_separate * `#13221 `__: BUG: pavement.py glitch * `#13248 `__: ndimage: improper cval handling for complex-valued inputs Pull requests for 1.6.0 ------------------------------ * `#8032 `__: ENH: Add in taylor window common in Radar processing * `#8779 `__: CI: Run benchmarks * `#9361 `__: ENH: Add Kendall's tau-a and tau-c variants to scipy.stats.kendalltau() * `#11068 `__: ENH: Adds correlation_lags function to scipy.signal * `#11119 `__: ENH: add Cramer-von-Mises (one-sample) test to scipy.stats * `#11249 `__: ENH: optimize: interpolative decomposition redundancy removal... * `#11346 `__: ENH: add fast toeplitz matrix multiplication using FFT * `#11413 `__: ENH: Multivariate t-distribution (stale) * `#11563 `__: ENH: exact p-value in stats.kendalltau() for sample sizes > 171 * `#11691 `__: ENH: add a stack of reversal functions to linprog * `#12043 `__: ENH: optimize: add HiGHS methods to linprog - continued * `#12061 `__: Check parameter consistensy in signal.iirdesign * `#12067 `__: MAINT: Cleanup OLDAPI in ndimage/src/_ctest.c * `#12069 `__: DOC: Add developer guidelines for implementing the nan_policy... * `#12077 `__: MAINT: malloc return value checks for cython * `#12080 `__: MAINT: Remove suppress_warnings * `#12085 `__: ENH: special: support ILP64 Lapack * `#12086 `__: MAINT: Cleanup PyMODINIT_FUNC used during 2to3 * `#12097 `__: ENH: stats: override stats.rayleigh.fit with analytical MLE * `#12112 `__: DOC: Improve integrate.nquad docstring * `#12125 `__: TST: Add a test for stats.gmean with negative input * `#12139 `__: TST: Reduce flakiness in lsmr test * `#12142 `__: DOC: add a note in poisson distribution when mu=0 and k=0 in... * `#12144 `__: DOC: Update ndimage.morphology.distance_transform\* * `#12154 `__: ENH: scipy.signal: allow lists in gauss_spline * `#12170 `__: ENH: scipy.stats: add negative hypergeometric distribution * `#12177 `__: MAINT: Correctly add input line to ValueError * `#12183 `__: ENH: Use fromfile where possible * `#12186 `__: MAINT: generalize tests in SphericalVoronoi * `#12198 `__: TST: Fix str + bytes error * `#12199 `__: ENH: match np.result_type behaviour in some scipy.signal functions * `#12200 `__: ENH: add FIR and IIR gammatone filters to scipy.signal * `#12204 `__: ENH: Add overwrite argument for odr.ODR() and its test. * `#12206 `__: MAINT:lstsq: Switch to tranposed problem if the array is tall * `#12208 `__: wavfile bugfixes and maintenance * `#12214 `__: DOC: fix docstring of "sd_beta" of odr.Output. * `#12234 `__: MAINT: prevent divide by zero warnings in scipy.optimize BFGS... * `#12235 `__: REL: set version to 1.6.0.dev0 * `#12237 `__: BUG: Fix exit condition for QUICK_SELECT pivot * `#12242 `__: ENH: Rename ndimage.sum to ndimage.sum_labels (keep sum as alias) * `#12243 `__: EHN: Update SuperLU * `#12244 `__: MAINT: stats: avoid spurious warnings in ncx2.pdf * `#12245 `__: DOC: Fixed incorrect default for mode in scipy.ndimage.spline_filter1d * `#12248 `__: MAINT: clean up pavement.py * `#12250 `__: ENH: Replaced csr_matrix() by tocsr() and complemented docstring * `#12253 `__: TST, CI: turn on codecov patch diffs * `#12259 `__: MAINT: Remove duplicated test for import cycles * `#12263 `__: ENH: Rename LocalSearchWrapper bounds * `#12265 `__: BUG optimize: Accept np.matrix in lsq_linear * `#12266 `__: BUG: Fix paren error in dual annealing accept_reject calculation * `#12269 `__: MAINT: Included mismatched shapes in error messages. * `#12279 `__: MAINT: \`__array__\` and array protocols cannot be used in sparse. * `#12281 `__: DOC: update wheel DL docs * `#12283 `__: ENH: odr: ILP64 Blas support in ODR * `#12284 `__: ENH: linalg: support for ILP64 BLAS/LAPACK in f2py wrappers * `#12286 `__: ENH: Cythonize scipy.spatial.transform.Rotation * `#12287 `__: ENH: Read arbitrary bit depth (including 24-bit) WAVs * `#12292 `__: BLD: fix musl compilation * `#12293 `__: MAINT: Fix a DeprecationWarning in validate_runtests_log.py. * `#12296 `__: DOC: Clarify area/volume in scipy.spatial.ConvexHull docstrings * `#12302 `__: CI: Run travis builds on master to keep cache up to date * `#12305 `__: TST: Cleanup print statements in tests * `#12323 `__: ENH: Add a Bunch-like class to use as a backwards compatible... * `#12324 `__: BUG: io: Fix an error that occurs when attempting to raise a... * `#12327 `__: DOC: clarify docstrings of \`query_ball_tree\` and \`query_pairs\` * `#12334 `__: PERF: Improve cKDTree.query_ball_point constant time cython overhead * `#12338 `__: DOC: improve consistency and clarity of docs in linalg and sparse/linalg * `#12341 `__: DOC: add Examples for KDTree query_ball_tree and query_pairs * `#12343 `__: DOC: add examples for special.eval_legendre() * `#12349 `__: BUG: avoid overflow in sum() for 32-bit systems * `#12351 `__: DOC: Fix example wavfile to be 16bit * `#12352 `__: [BUG] Consider 0/0 division in DOP853 error estimation * `#12353 `__: Fix exception causes in vq.py * `#12354 `__: MAINT: Cleanup unneeded void\* cast in setlist.pxd * `#12355 `__: TST: Remove hack for old win-amd64 bug * `#12356 `__: ENH: Faster implementation of scipy.sparse.block_diag (#9411... * `#12357 `__: MAINT,TST: update and run scipy/special/utils/convert.py * `#12358 `__: TST: Check mstat.skewtest pvalue * `#12359 `__: TST: Sparse matrix test with int64 indptr and indices * `#12363 `__: DOC: ref. in CloughTocher2DInterpolator * `#12364 `__: DOC: \`sparse_distance_matrix\` and \`count_neighbors\` examples * `#12371 `__: MAINT, CI: bump to latest stable OpenBLAS * `#12372 `__: MAINT: Minor cleanup of (c)KDTree tests * `#12374 `__: DEP: Deprecate \`distance.wminkowski\` * `#12375 `__: ENH: Add fast path for minkowski distance with p=1,2 and support... * `#12376 `__: Fix exception causes in most of the codebase * `#12377 `__: DOC: Quick fix - adds newline to correlation_lags docstring Examples... * `#12381 `__: BENCH: remove obsolete goal_time param * `#12382 `__: ENH: Replace KDTree with a thin wrapper over cKDTree * `#12385 `__: DOC: improve docstrings of interpolate.NearestNDInterpolator.__call__... * `#12387 `__: DOC/STY: add example to scipy.signal.correlate * `#12393 `__: CI: Replace the existing check for non-ASCII characters with... * `#12394 `__: CI: arm64 numpy now available * `#12395 `__: ENH: improve stats.binned_statistic_dd performance * `#12396 `__: DOC, MAINT: forward port 1.5.0 relnotes * `#12398 `__: API: Disable len() and indexing of Rotation instances with single... * `#12399 `__: MAINT: Replace some Unicode dash-like chars with an ASCII hyphen. * `#12402 `__: update .mailmap * `#12404 `__: MAINT: io: Change the encoding comment of test_mio.py to utf-8. * `#12416 `__: CI: cache mingw, azure pipelines * `#12427 `__: BUG: logic error in loop unrolling (cKDTree) * `#12432 `__: DOC: Remove the "Basic functions" section from the SciPy tutorial. * `#12434 `__: ENH:linalg: Add LAPACK wrappers pptrf/pptrs/ppsv/pptri/ppcon * `#12435 `__: DOC: fix simplex math for scipy.stats.dirichlet documentation * `#12439 `__: DOC: add API methods summary for NdPPoly * `#12443 `__: BUG: stats: Improve calculation of exponnorm.pdf * `#12448 `__: DOC: stats: Add "Examples" to the ansari docstring. * `#12450 `__: ENH: add \`leaves_color_list\` for cluster.dendrogram dictionary. * `#12451 `__: MAINT: remove "blacklist" terminology from code base * `#12452 `__: DOC: clarify the meaning of whitening for cluster.vq.whiten() * `#12455 `__: MAINT: clearer error message in setup.py * `#12457 `__: ENH: stats: override stats.pareto.fit with analytical MLE * `#12460 `__: check if column in spearman rho is entirely NaN or Inf * `#12463 `__: DOC: improve and clean up \*Spline docstrings in fitpack2.py * `#12474 `__: ENH: linalg: speedup _sqrtm_triu by moving tight loop to Cython * `#12476 `__: ENH: add IIR comb filter to scipy.signal * `#12484 `__: Fix documentation for minimize * `#12486 `__: DOC: add a note in poisson distribution when mu=0 and k=0 in... * `#12491 `__: MAINT: forward port 1.5.1 release notes * `#12508 `__: Fix exception causes all over the codebase * `#12514 `__: ENH: stats: override stats.invgauss.fit with analytical MLE * `#12519 `__: PERF: Avoid np.zeros when custom initialization is needed anyway * `#12520 `__: DOC: Minor RST section renaming. * `#12521 `__: MAINT: Remove unused imports * `#12522 `__: PERF: Get rid of unnececssary allocation in VarReader5.cread_fieldnames * `#12524 `__: DOC: special: Set Axes3D rect to avoid clipping labels in plot. * `#12525 `__: Fix large sparse nnz * `#12526 `__: DOC: Remove double section and too long header underline. * `#12527 `__: Improve error message for wrong interpolation type * `#12530 `__: Move redundant logic outside loop for conditional speedup in... * `#12532 `__: ENH: Add norm={"forward", "backward"} to \`scipy.fft\` * `#12535 `__: MAINT: Avoid sphinx deprecated aliases for SeeAlso and Only * `#12540 `__: BUG: fix odr.output.work_ind key bug and add its test. * `#12541 `__: ENH: add solver for minimum weight full bipartite matching * `#12550 `__: PERF: pickling speed of rv\* * `#12551 `__: DOC: fix typo in cluster/_hierarchy.pyx * `#12552 `__: CI: Cleanup travis pip installs * `#12556 `__: BUG: Fix problem with Scipy.integrate.solve_bvp for big problems * `#12557 `__: MAINT: Use extern templates to improve sparsetools compile time * `#12558 `__: MAINT: Remove hack to allow scipy.fft to act like a function * `#12563 `__: MAINT: Remove unused mu0 in special/orthogonal.py * `#12564 `__: DOC: fix return type docstring for least_squares * `#12565 `__: DOC: stats: respond to query about Kruskal-Wallis test being... * `#12566 `__: BUG: Interpolate: use stable sort * `#12568 `__: Updated documentation for as_quat * `#12571 `__: DEP: remove deprecated slepian window * `#12573 `__: DEP: remove \`frechet_l\` and \`frechet_r\` * `#12575 `__: BUG: stats: fix multinomial.pmf NaNs when params sum > 1 * `#12576 `__: MAINT: remove warning from LSQSphereBivariateSpline * `#12582 `__: ENH: Multivariate t-distribution * `#12587 `__: ENH: speed up rvs of gengamma in scipy.stats * `#12588 `__: DOC: add Examples add see also sections for LinearNDInterpolator,... * `#12597 `__: ENH: Add single-sided p-values to t-tests * `#12599 `__: Small update to scipy FFT tutorial * `#12600 `__: ENH: disjoint set data structure * `#12602 `__: BUG: add const for Read-only views in interpnd.pyx * `#12605 `__: BUG: correct \`np.asanyarray\` use in \`scipy.constants.lambda2nu\` * `#12610 `__: MAINT: forward port 1.5.2 release notes * `#12612 `__: MAINT: stats: Use explicit keyword parameters instead of \`\*\*kwds\`. * `#12616 `__: DOC: make explicit docstring that interpolate.interp1d only accepts... * `#12618 `__: DOC: Minor doc formatting. * `#12640 `__: MAINT: stats: fix issues with scipy.stats.pearson3 docs, moment,... * `#12647 `__: TST: Add Boost ellipr[cdfgj]_data test data * `#12648 `__: DOC: Update special/utils/README with instructions * `#12649 `__: DOC: simplified pip quickstart guide * `#12650 `__: DOC: stats: Fix boxcox docstring: lambda can be negative. * `#12655 `__: DOC: update Steering Council members listed in governance docs * `#12659 `__: rv_sample expect bug * `#12663 `__: DOC: optimize: try to fix linprog method-specific documentation * `#12664 `__: BUG: stats: Fix logpdf with large negative values for logistic... * `#12666 `__: MAINT: Fixes from static analysis * `#12667 `__: ENH: Adding Modified Rodrigues Parameters to the Rotation class * `#12670 `__: DOC: Update documentation for Gamma distribution * `#12673 `__: API: Unconditionally return np.intp from cKDTree.query * `#12677 `__: MAINT: Add Autogenerated notice to ufuncs.pyi * `#12682 `__: MAINT: Remove _util._valarray * `#12688 `__: MAINT: add f2py-generated scipy.integrate files to .gitignore * `#12689 `__: BENCH: simplify benchmark setup, remove benchmarks/run.py * `#12694 `__: scipy/stats: Add laplace_asymmetric continuous distribution * `#12695 `__: DOC: update Ubuntu quickstart; conda compilers now work! * `#12698 `__: MAINT: Replace np.max with np.maximum * `#12700 `__: TST: bump test precision for constrained trustregion test * `#12702 `__: TST: bump test tolerance for \`DifferentialEvolutionSolver.test_L4\` * `#12703 `__: BUG: Improve input validation for sepfir2d * `#12708 `__: MAINT: fix a typo in scipy.sparse * `#12709 `__: BUG: bvls can fail catastrophically to converge * `#12711 `__: MAINT: Use platform.python_implementation to determine IS_PYPY * `#12713 `__: TST: Fix flaky test_lgmres * `#12716 `__: DOC: add examples and tutorial links for interpolate functions... * `#12717 `__: DOC: Fix Issue #5396 * `#12725 `__: ENH: Support complex-valued images and kernels for many ndimage... * `#12729 `__: DEP: remove setup_requires * `#12732 `__: BENCH: skip benchmarks instead of hiding them when SCIPY_XSLOW=0 * `#12734 `__: CI: Don't ignore line-length in the lint_diff check. * `#12736 `__: DOC: Fix signal.windows.get_window() 'exponential' docstring * `#12737 `__: ENH: stats: override stats.gumbel_r.fit and stats.gumbel_l.fit... * `#12738 `__: ENH: stats: override stats.logistic.fit with system of equations... * `#12743 `__: BUG: Avoid negative variances in circular statistics * `#12744 `__: Prevent build error on GNU/Hurd * `#12746 `__: TST: parameterize the test cases in test_ndimage.py * `#12752 `__: DOC: Add examples for some root finding functions. * `#12754 `__: MAINT, CI: Azure windows deps multiline * `#12756 `__: ENH: stats: Add an sf method to levy for improved precision in... * `#12757 `__: ENH: stats: Add an sf method to levy_l for improved precision. * `#12765 `__: TST, MAINT: infeasible_2 context * `#12767 `__: Fix spline interpolation boundary handling for modes reflect... * `#12769 `__: DOC: syntax error in scipy.interpolate.bspl * `#12770 `__: ENH: add nearest-up rounding to scipy.interpolate.interp1d * `#12771 `__: TST: fix invalid input unit test for scipy.signal.gammatone * `#12775 `__: ENH: Adds quadratic_assignment with two methods * `#12776 `__: ENH: add grid-constant boundary handling in ndimage interpolation... * `#12777 `__: Add Taylor Window function - Common in Radar DSP * `#12779 `__: ENH: Improvements to pocketfft thread pool and ARM neon vectorization * `#12788 `__: API: Rename cKDTree n_jobs argument to workers * `#12792 `__: DOC: remove THANKS.txt file in favor of scipy.org * `#12793 `__: Add new flag to authors tool * `#12802 `__: BENCH: add scipy.ndimage.interpolation benchmarks * `#12803 `__: Do not pin the version of numpy in unsupported python versions * `#12810 `__: CI: fix 32-bit Linux build failure on Azure CI runs * `#12812 `__: ENH: support interpolation of complex-valued images * `#12814 `__: BUG: nonlin_solve shouldn't pass non-vector dx to tol_norm * `#12818 `__: Update ckdtree.pyx * `#12822 `__: MAINT: simplify directed_hausdorff * `#12827 `__: DOC: Fix wrong name w being used instead of worN in docs. * `#12831 `__: DOC: fix typo in sparse/base.py * `#12835 `__: MAINT: stats: Improve vonmises PDF calculation. * `#12839 `__: ENH: scipy.stats: add multivariate hypergeometric distribution * `#12843 `__: changed M to N in windows.dpss * `#12846 `__: MAINT: update minimum NumPy version to 1.16.5 * `#12847 `__: DOC: Unify the formula in docs of scipy.stats.pearsonr() * `#12849 `__: DOC: polish QAP docs for consistency and readability * `#12852 `__: ENH, MAINT: Bring KDTree interface to feature-parity with cKDTree * `#12858 `__: DOC: use :doi: and :arxiv: directives for references * `#12872 `__: lazily import multiprocessing.Pool in MapWrapper * `#12878 `__: DOC: document ScalarFunction * `#12882 `__: MAINT: stats: Change a test to use <= instead of strictly less... * `#12885 `__: numpy.linspace calls edited to ensure correct spacing. * `#12886 `__: DOC: stats: Add 'versionadded' to cramervonmises docstring. * `#12899 `__: TST: make a couple of tests expected to fail on 32-bit architectures * `#12903 `__: DOC: update Windows build guide and move into contributor guide * `#12907 `__: DOC: clarify which array the precenter option applies to * `#12908 `__: MAINT: spatial: Remove two occurrences of unused variables in... * `#12909 `__: ENH: stats: Add methods gumbel_r._sf and gumbel_r._isf * `#12910 `__: CI: travis: Remove some unnecessary code from .travis.yml. * `#12911 `__: Minor fixes to dendrogram plotting * `#12921 `__: CI: don't run GitHub Actions on fork or in cron job * `#12927 `__: MAINT: rename integrate.simps to simpson * `#12934 `__: MAINT: rename trapz and cumtrapz to (cumulative\_)trapezoid * `#12936 `__: MAINT: fix numerical precision in nct.stats * `#12938 `__: MAINT: fix linter on master * `#12941 `__: Update minimum AIX pinnings to match non AIX builds * `#12955 `__: BUG: Fixed wrong NaNs check in scipy.stats.weightedtau * `#12958 `__: ENH: stats: Implement _logpdf, _sf and _isf for nakagami. * `#12962 `__: Correcting that p should be in [0,1] for a variety of discrete... * `#12964 `__: BUG: added line.strip() to split_data_line() * `#12968 `__: ENH: stats: Use only an analytical formula or scalar root-finding... * `#12971 `__: MAINT: Declare support for Python 3.9 * `#12972 `__: MAINT: Remove redundant Python < 3.6 code * `#12980 `__: DOC: Update documentation on optimize.rosen * `#12983 `__: ENH: improvements to stats.linregress * `#12990 `__: DOC: Clarify that using sos as output type for iirdesign can... * `#12992 `__: DOC: capitalization and formatting in lsmr * `#12995 `__: DOC: stats: Several documentation fixes. * `#12996 `__: BUG: Improve error messages for \`range\` arg of binned_statistic_dd * `#12998 `__: MAINT: approx_derivative with FP32 closes #12991 * `#13004 `__: TST: isinstance(OptimizeResult.message, str) closes #13001 * `#13006 `__: Keep correct dtype when loading empty mat arrays. * `#13009 `__: MAINT: clip SLSQP step within bounds * `#13012 `__: DOC: fix bilinear_zpk example labels * `#13013 `__: ENH: Add \`subset\` and \`subsets\` methods to \`DisjointSet\`... * `#13029 `__: MAINT: basinhopping callback for initial mininmisation * `#13032 `__: DOC: fix docstring errors in in stats.wilcoxon * `#13036 `__: BUG: forward port lint_diff shims * `#13041 `__: MAINT: dogbox ensure x is within bounds closes #11403 * `#13042 `__: MAINT: forward port 1.5.4 release notes * `#13046 `__: DOC: Update optimize.least_squares doc for all tolerance must... * `#13052 `__: Typo fix for cluster documentation * `#13054 `__: BUG: fix \`scipy.optimize.show_options\` for unknown methods.... * `#13056 `__: MAINT: fft: Fix a C++ compiler warning. * `#13057 `__: Minor fixes on doc of function csr_tocsc * `#13058 `__: DOC: stats: Replace np.float with np.float64 in a tutorial file. * `#13059 `__: DOC: stats: Update the "Returns" section of the linregress docstring. * `#13060 `__: MAINT: clip_x_for_func should be private * `#13061 `__: DOC: signal.win -> signal.windows.win in Examples * `#13063 `__: MAINT: Add suite-sparse and sksparse installation check * `#13070 `__: MAINT: stats: Remove a couple obsolete comments. * `#13073 `__: BUG: Fix scalar_search_wolfe2 to resolve #12157 * `#13078 `__: CI, MAINT: migrate Lint to Azure * `#13081 `__: BLD: drop Python 3.6 support (NEP 29) * `#13082 `__: MAINT: update minimum NumPy version to 1.16.5 in a couple more... * `#13083 `__: DOC: update toolchain.rst * `#13086 `__: DOC: Update the Parameters section of the correlation docstring * `#13087 `__: ENH:signal: Speed-up Cython implementation of _sosfilt * `#13089 `__: BLD, BUG: add c99 compiler flag to HiGHS basiclu library * `#13091 `__: BUG: Fix GIL handling in _sosfilt * `#13094 `__: DOC: clarify "location" in docstring of cKDTree.query * `#13095 `__: Zoom resize update * `#13097 `__: BUG: fix CubicSpline(..., bc_type="periodic") #11758 * `#13100 `__: BUG: sparse: Correct output format of kron * `#13107 `__: ENH: faster linear_sum_assignment for small cost matrices * `#13110 `__: CI, MAINT: refguide/asv checks to azure * `#13112 `__: CI: fix MacOS CI * `#13113 `__: CI: Install word list package for refguide-check * `#13115 `__: BUG: add value range check for signal.iirdesign() * `#13116 `__: CI: Don't report name errors after an exception in refguide-check * `#13117 `__: CI: move sdist/pre-release test Azure * `#13119 `__: Improve error message on friedmanchisquare function * `#13121 `__: Fix factorial() for NaN on Python 3.10 * `#13123 `__: BLD: Specify file extension for language standard version tests * `#13128 `__: TST: skip Fortran I/O test for ODR * `#13130 `__: TST: skip factorial() float tests on Python 3.10 * `#13136 `__: CI:Add python dbg run to GH Actions * `#13138 `__: CI: Port coverage, 64-bit BLAS, GCC 4.8 build to azure * `#13139 `__: Fix edge case for mode='nearest' in ndimage.interpolation functions * `#13141 `__: BUG: signal: Fix data type of the numerator returned by ss2tf. * `#13144 `__: MAINT: stats: restrict gausshyper z > -1 * `#13146 `__: typo in csr.py * `#13148 `__: BUG: stats: fix typo in stable rvs per gh-12870 * `#13149 `__: DOC: spatial/stats: cross-ref random rotation matrix functions * `#13151 `__: MAINT: stats: Fix a test and a couple PEP-8 issues. * `#13152 `__: MAINT: stats: Use np.take_along_axis in the private function... * `#13154 `__: ENH: stats: Implement defined handling of constant inputs in... * `#13156 `__: DOC: maintain equal display range for ndimage.zoom example * `#13159 `__: CI: Azure: Don't run tests on merge commits, except for coverage * `#13160 `__: DOC: stats: disambiguate location-shifted/noncentral * `#13161 `__: BUG: DifferentialEvolutionSolver.__del__ can fail in garbage... * `#13163 `__: BUG: stats: fix bug in spearmanr nan propagation * `#13167 `__: MAINT: stats: Fix a test. * `#13169 `__: BUG: stats: Fix handling of misaligned masks in mstats.pearsonr. * `#13178 `__: CI: testing.yml --> macos.yml * `#13181 `__: CI: fix lint * `#13190 `__: BUG: optimize: fix a duplicate key bug for \`test_show_options\` * `#13192 `__: BUG:linalg: Add overwrite option to gejsv wrapper * `#13194 `__: BUG: slsqp should be able to use rel_step * `#13199 `__: [skip travis] DOC: 1.6.0 release notes * `#13203 `__: fix typos * `#13209 `__: TST:linalg: set the seed for cossin test * `#13212 `__: [DOC] Backtick and directive consistency. * `#13217 `__: REL: add necessary setuptools and numpy version pins in pyproject.toml... * `#13226 `__: BUG: pavement.py file handle fixes * `#13249 `__: Handle cval correctly for ndimage functions with complex-valued... * `#13253 `__: BUG,MAINT: Ensure all Pool objects are closed * `#13260 `__: CI: fix macOS testing * `#13269 `__: CI: github actions: In the linux dbg tests, update apt before... Checksums ========= MD5 ~~~ 3a8f443339b8df566a7583832db25218 scipy-1.6.0rc2-cp37-cp37m-macosx_10_9_x86_64.whl 8c9deb6b70832062825cd5c6850f2098 scipy-1.6.0rc2-cp37-cp37m-manylinux1_i686.whl c0502194b53dd294aa0b6f5114f6ac0b scipy-1.6.0rc2-cp37-cp37m-manylinux1_x86_64.whl 695baabc39cfd43ff611325836d006a2 scipy-1.6.0rc2-cp37-cp37m-manylinux2014_aarch64.whl acb6d59865bf8f90c79f018df77b18be scipy-1.6.0rc2-cp37-cp37m-win32.whl da2b7084b4a82a496e2e7df831a716fc scipy-1.6.0rc2-cp37-cp37m-win_amd64.whl ce1ebeabac7efe9ecc76d56137e7cbc5 scipy-1.6.0rc2-cp38-cp38-macosx_10_9_x86_64.whl bda240e1185696b3ba5959628751234c scipy-1.6.0rc2-cp38-cp38-manylinux1_i686.whl 0518fbf374805086b96bc28296909f98 scipy-1.6.0rc2-cp38-cp38-manylinux1_x86_64.whl 7b74fbab9e5b2091ac28b82824e2ca50 scipy-1.6.0rc2-cp38-cp38-manylinux2014_aarch64.whl 14d548ce783fef31df3efc846447651f scipy-1.6.0rc2-cp38-cp38-win32.whl d552d0418f9c1f67f1727767a438f90d scipy-1.6.0rc2-cp38-cp38-win_amd64.whl 3e1f3ca93ee57473850dc1c0cf9065ad scipy-1.6.0rc2-cp39-cp39-macosx_10_9_x86_64.whl 3551faf10137c117c98be3d48ab51c7c scipy-1.6.0rc2-cp39-cp39-manylinux1_i686.whl 2ded7da06d3d386e9cc3a4b5b5a3e635 scipy-1.6.0rc2-cp39-cp39-manylinux1_x86_64.whl fcd47232b2d3d15c403658d49fcf3cf6 scipy-1.6.0rc2-cp39-cp39-manylinux2014_aarch64.whl 8607bfbfffe8a94b0b43c0f018c08655 scipy-1.6.0rc2-cp39-cp39-win32.whl 3cd505a118f399cf6470b515a64387b2 scipy-1.6.0rc2-cp39-cp39-win_amd64.whl d884a448b0296f3c20c527d4d703b755 scipy-1.6.0rc2.tar.gz a78bcc3948156bd957f172ee8481fdc5 scipy-1.6.0rc2.tar.xz edb2c09f204963bde45b28cc97a3a88f scipy-1.6.0rc2.zip SHA256 ~~~~~~ c00707043198d3480989c82ae5b7e821ee72c1089558dce0d8531de18979fd93 scipy-1.6.0rc2-cp37-cp37m-macosx_10_9_x86_64.whl e0ee8d470eafc83b3408eca90e46c77d47fdf50df956ed7b6dd87d9b9a229520 scipy-1.6.0rc2-cp37-cp37m-manylinux1_i686.whl a3f7219545343a7d61e453ec7626994933a97f5b24b9d35f2ff8372fd235feb7 scipy-1.6.0rc2-cp37-cp37m-manylinux1_x86_64.whl 8c61dfafd3d0b77b67ecf8a713b45e36c5a699bc838081ff2cdec1cf53c284a2 scipy-1.6.0rc2-cp37-cp37m-manylinux2014_aarch64.whl 77c696f369e27dd22f15ce283e0bbf10bc9402282a09078bd40ba3bebf1647bd scipy-1.6.0rc2-cp37-cp37m-win32.whl f9eaad9c026f24e6520aae7183b783a0a81c5844df66197b02d8bdd0af381076 scipy-1.6.0rc2-cp37-cp37m-win_amd64.whl 76b64c1c32861e2ecf33d62c394280b3e04abf4aa5e4ff9e13dd0d28d5564868 scipy-1.6.0rc2-cp38-cp38-macosx_10_9_x86_64.whl cee0a503e741442ea75c1688368b29bcf9f718e8e267d0d7bc51087367b77fbd scipy-1.6.0rc2-cp38-cp38-manylinux1_i686.whl 919747cfcf91e17924aec3052490e7c5336698b39d8c727b79426c32aed17049 scipy-1.6.0rc2-cp38-cp38-manylinux1_x86_64.whl 5f4108817ab45d88c412aba21dc408459e6d90fb534738595ffe56955e3447f1 scipy-1.6.0rc2-cp38-cp38-manylinux2014_aarch64.whl 43b2092876c815e6e11ca005580dca4eeb778a01785f2a7e49d332f89c20f50f scipy-1.6.0rc2-cp38-cp38-win32.whl 5ddc8345d842f82e09350a95ef23f1b3b46b14883aaf0a8e5c79474c711ca02d scipy-1.6.0rc2-cp38-cp38-win_amd64.whl a20c1b62cbf9d1cf8e3c58100e7ad27a745ba298f7507c6e95d9611a250aba17 scipy-1.6.0rc2-cp39-cp39-macosx_10_9_x86_64.whl ee419d8f519af24b89410e340eb9e71539e0c55f76767d70b7015aaa2624d046 scipy-1.6.0rc2-cp39-cp39-manylinux1_i686.whl 7383b26c288e5fc6cd7138314f20bcdf2956a6876ba3c3b5173f196d2b806dcf scipy-1.6.0rc2-cp39-cp39-manylinux1_x86_64.whl e0fdca6a071a71ad2038f7b4d80d924c2d2c6a743bd0fb9eb55ac7d17b504342 scipy-1.6.0rc2-cp39-cp39-manylinux2014_aarch64.whl 9b860becc2e5b216c55ff5e1628f3d629954f5455d7293d54bc97401f685229f scipy-1.6.0rc2-cp39-cp39-win32.whl 8adb99a5c2a907091c014c31f68dea78dee66585a84262b2f7e88d8f6ffddcf7 scipy-1.6.0rc2-cp39-cp39-win_amd64.whl 0999932000281bf5b8e19faf13ef883c5d957ef1f8acd021cbc33ad023504d0e scipy-1.6.0rc2.tar.gz e5f61a8b6e2f88ed3fb6691a9c8c99102d43c56451cd1060146373119196f4eb scipy-1.6.0rc2.tar.xz bd44218ba55f8519c84e1477be3bf3d60e4c1332df7e62a8c6c0046729a450c0 scipy-1.6.0rc2.zip -------------- next part -------------- An HTML attachment was scrubbed... URL: From rypolley at gmail.com Wed Dec 23 13:04:09 2020 From: rypolley at gmail.com (rpolley) Date: Wed, 23 Dec 2020 11:04:09 -0700 (MST) Subject: [Numpy-discussion] Deprecate np.testing.dec Message-ID: <1608746649771-0.post@n7.nabble.com> Saw some discussion in this issue that since the decorators in np.testing.dec are there to support the nose testing framework, and since we use pytest mainly for testing now, that the decorators there should be deprecated. I created a pull request that does this, and wanted to get the mailing list's opinion before it went forward. Any thoughts, suggestions, or objections? -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.k.sheppard at gmail.com Wed Dec 23 13:08:57 2020 From: kevin.k.sheppard at gmail.com (Kevin Sheppard) Date: Wed, 23 Dec 2020 18:08:57 +0000 Subject: [Numpy-discussion] Deprecate np.testing.dec In-Reply-To: <1608746649771-0.post@n7.nabble.com> References: <1608746649771-0.post@n7.nabble.com> Message-ID: <4099625D-A5D9-4274-9640-1BD7F9CE5CED@hxcore.ol> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: E6C3ED9F5F55406C9DB14A6CCA258363.png Type: image/png Size: 140 bytes Desc: not available URL: From tom at swirly.com Wed Dec 23 15:05:55 2020 From: tom at swirly.com (Tom Swirly) Date: Wed, 23 Dec 2020 21:05:55 +0100 Subject: [Numpy-discussion] Suggestion: add an pad parameter to np.memmap Message-ID: Hello, and thanks for all your work on the amazing numpy. I'm using np.memmap() with great success for memory mapping WAV audio files, with one tiny blemish - the WAV spec wants me to sometimes put one byte after the list of samples that I'm memory mapping, which I can't do easily when writing a new file. This little patch is probably backward compatible and would accomplish what I need: https://github.com/rec/numpy/commit/4dd855b14bed5947f2f9120979c9ee936c17158f If there were interest, I could figure out numpy's test system and write tests for it. (For the foreseeable future, I'm intercepting the call to mmap.mmap() to add the padding, so I'm not blocked, though the code has aspects of ugliness.) -- /t PGP Key: https://flowcrypt.com/pub/tom.ritchford at gmail.com *https://tom.ritchford.com * *https://tom.swirly.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Wed Dec 23 20:17:58 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 23 Dec 2020 18:17:58 -0700 Subject: [Numpy-discussion] Deprecate np.testing.dec In-Reply-To: <4099625D-A5D9-4274-9640-1BD7F9CE5CED@hxcore.ol> References: <1608746649771-0.post@n7.nabble.com> <4099625D-A5D9-4274-9640-1BD7F9CE5CED@hxcore.ol> Message-ID: On Wed, Dec 23, 2020 at 11:09 AM Kevin Sheppard wrote: > Have you performed a search on GitHub to look for use of the decorators? > I think external use is more of a concern than internal. > > > > Kevin > Note that nose is no longer supported and needs (minor) patching to work with recent Python. Most distributions have maintained patched versions, but nose users should move to nose2 . The nose decorators are like the NumPy matrix module and financial functions, but I'm not convinced they are worth keeping around when folks can move to nose2. Deprecation would be a good first step and then we can see who complains. Chuck > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Dec 24 22:54:35 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 24 Dec 2020 20:54:35 -0700 Subject: [Numpy-discussion] NumPy 1.20.0rc2 released Message-ID: Hi All, On behalf of the NumPy team I am pleased to announce the release of NumPy 1.20.0rc2. This NumPy release is the largest to date, containing some 670 merged pull requests contributed by 184 people. See the list of highlights below. The Python versions supported for this release are 3.7-3.9, support for Python 3.6 has been dropped. Wheels can be downloaded from PyPI ; source archives, release notes, and wheel hashes are available on Github . Linux users will need pip >= 0.19.3 in order to install manylinux2010 and manylinux2014 wheels. *Highlights* - Annotations for NumPy functions. This work is ongoing and improvements can be expected pending feedback from users. - Wider use of SIMD to increase execution speed of ufuncs. Much work has been done in introducing universal functions that will ease use of modern features across different hardware platforms. This work is ongoing. - Preliminary work in changing the dtype and casting implementations in order to provide an easier path to extending dtypes. This work is ongoing but enough has been done to allow experimentation and feedback. - Extensive documentation improvements comprising some 185 PR merges. This work is ongoing and part of the larger project to improve NumPy's online presence and usefulness to new users. - Further cleanups related to removing Python 2.7. This improves code readability and removes technical debt. - Preliminary support for the upcoming Cython 3.0. *Contributors* A total of 184 people contributed to this release. People with a "+" by their names contributed a patch for the first time. * Aaron Meurer + * Abhilash Barigidad + * Abhinav Reddy + * Abhishek Singh + * Al-Baraa El-Hag + * Albert Villanova del Moral + * Alex Leontiev + * Alex Rockhill + * Alex Rogozhnikov * Alexander Belopolsky * Alexander Kuhn-Regnier + * Allen Downey + * Andras Deak * Andrea Olivo + * Andrew Eckart + * Anirudh Subramanian * Anthony Byuraev + * Antonio Larrosa + * Ashutosh Singh + * Bangcheng Yang + * Bas van Beek + * Ben Derrett + * Ben Elliston + * Ben Nathanson + * Bernie Gray + * Bharat Medasani + * Bharat Raghunathan * Bijesh Mohan + * Bradley Dice + * Brandon David + * Brandt Bucher * Brian Soto + * Brigitta Sipocz * Cameron Blocker + * Carl Leake + * Charles Harris * Chris Brown + * Chris Vavaliaris + * Christoph Gohlke * Chunlin Fang * CloseChoice + * Daniel G. A. Smith + * Daniel Hrisca * Daniel Vanzo + * David Pitchford + * Davide Dal Bosco + * Derek Homeier * Dima Kogan + * Dmitry Kutlenkov + * Douglas Fenstermacher + * Dustin Spicuzza + * E. Madison Bray + * Elia Franzella + * Enrique Mat?as S?nchez + * Erfan Nariman | Veneficus + * Eric Larson * Eric Moore * Eric Wieser * Erik M. Bray * EthanCJ-git + * Etienne Guesnet + * FX Coudert + * Felix Divo * Frankie Robertson + * Ganesh Kathiresan * Gengxin Xie * Gerry Manoim + * Guilherme Leobas * Hassan Kibirige * Hugo Mendes + * Hugo van Kemenade * Ian Thomas + * InessaPawson + * Isabela Presedo-Floyd + * Isuru Fernando * Jakob Jakobson + * Jakub Wilk * James Myatt + * Jesse Li + * John Hagen + * John Zwinck * Joseph Fox-Rabinovitz * Josh Wilson * Jovial Joe Jayarson + * Julia Signell + * Jun Kudo + * Karan Dhir + * Kaspar Thommen + * Kerem Halla? * Kevin Moore + * Kevin Sheppard * Klaus Zimmermann + * LSchroefl + * Laurie + * Laurie Stephey + * Levi Stovall + * Lisa Schwetlick + * Lukas Geiger + * Madhulika Jain Chambers + * Matthias Bussonnier * Matti Picus * Melissa Weber Mendon?a * Michael Hirsch * Nick R. Papior * Nikola Forr? * Noman Arshad + * Paul YS Lee + * Pauli Virtanen * Pawe? Redzy?ski + * Peter Andreas Entschev * Peter Bell * Philippe Ombredanne + * Phoenix Meadowlark + * Piotr Gai?ski * Raghav Khanna + * Raghuveer Devulapalli * Rajas Rade + * Rakesh Vasudevan * Ralf Gommers * Raphael Kruse + * Rashmi K A + * Robert Kern * Rohit Sanjay + * Roman Yurchak * Ross Barnowski * Royston E Tauro + * Ryan C Cooper + * Ryan Soklaski * Safouane Chergui + * Sahil Siddiq + * Sarthak Vineet Kumar + * Sayed Adel * Sebastian Berg * Sergei Vorfolomeev + * Seth Troisi * Sidhant Bansal + * Simon Gasse * Simon Graham + * Stefan Appelhoff + * Stefan Behnel + * Stefan van der Walt * Steve Dower * Steve Joachim + * Steven Pitman + * Stuart Archibald * Sturla Molden * Susan Chang + * Takanori H + * Tapajyoti Bose + * Thomas A Caswell * Tina Oberoi * Tirth Patel * Tobias Pitters + * Tomoki, Karatsu + * Tyler Reddy * Veniamin Petrenko + * Wansoo Kim + * Warren Weckesser * Wei Yang + * Wojciech Rzadkowski * Yang Hau + * Yogesh Raisinghani + * Yu Feng * Yuya Unno + * Zac Hatfield-Dodds * Zuhair Ali-Khan + * @abhilash42 + * @danbeibei + * @dojafrat * @dpitch40 + * @forfun + * @iamsoto + * @jbrockmendel + * @leeyspaul + * @mitch + * @prateek arora + * @serge-sans-paille + * @skywalker + * @stphnlyd + * @xoviat * @??? + * @JMFT + * @Jack + * @Neal C + Cheers, Charles Harris -------------- next part -------------- An HTML attachment was scrubbed... URL: From cimrman3 at ntc.zcu.cz Sat Dec 26 18:54:50 2020 From: cimrman3 at ntc.zcu.cz (Robert Cimrman) Date: Sun, 27 Dec 2020 00:54:50 +0100 Subject: [Numpy-discussion] ANN: SfePy 2020.4 Message-ID: <282bf9a7-a3dc-af92-c7ed-9725fa88ca8e@ntc.zcu.cz> I am pleased to announce the release of SfePy 2020.4. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by finite element methods. It is distributed under the new BSD license. Home page: https://sfepy.org Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/ Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -------------------------- - Ogden hyperelastic term - serendipity finite element basis of orders 1-3 For full release notes see [1]. Cheers, Robert Cimrman [1] http://docs.sfepy.org/doc/release_notes.html#id1 --- Contributors to this release in alphabetical order: Robert Cimrman Jan Heczko Vladimir Lukes From BLKZOL001 at myuct.ac.za Sun Dec 27 06:04:14 2020 From: BLKZOL001 at myuct.ac.za (Zolisa Bleki) Date: Sun, 27 Dec 2020 11:04:14 +0000 Subject: [Numpy-discussion] Addition of new distributions: Polya-gamma Message-ID: Hi All, I would like to know if Numpy accepts addition of new distributions since the implementation of the Generator interface. If so, what is the criteria for a particular distribution to be accepted? The reason why i'm asking is because I would like to propose adding the Polya-gamma distribution to numpy, for the following reasons: 1) Polya-gamma random variables are commonly used as auxiliary variables during data augmentation in Bayesian sampling algorithms, which have wide-spread usage in Statistics and recently, Machine learning. 2) Since this distribution is mostly useful for random sampling, it since appropriate to have it in numpy and not projects like scipy [1]. 3) The only python/C++ implementation of the sampler available is licensed under GPLv3 which I believe limits copying into packages that choose to use a different license [2]. 4) Numpy's random API makes adding the distribution painless. I have done preliminary work on this by implementing the distribution sampler as decribed in [3]; see: https://github.com/numpy/numpy/compare/master...zoj613:polyagamma . There is a more efficient sampling algorithm described in a later paper [4], but I chose not to start with that one unless I know it is worth investing time in. I would appreciate your thoughts on this proposal. Regards, Zolisa Refs: [1] https://github.com/scipy/scipy/issues/11009 [2] https://github.com/slinderman/pypolyagamma [3] https://arxiv.org/pdf/1205.0310v1.pdf [4] https://arxiv.org/pdf/1405.0506.pdf Disclaimer - University of Cape Town This email is subject to UCT policies and email disclaimer published on our website at http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 9111. If this email is not related to the business of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via https://csirt.uct.ac.za/page/report-an-incident.php. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom at swirly.com Sun Dec 27 06:27:13 2020 From: tom at swirly.com (Tom Swirly) Date: Sun, 27 Dec 2020 12:27:13 +0100 Subject: [Numpy-discussion] Addition of new distributions: Polya-gamma In-Reply-To: References: Message-ID: I'm just a lurker, but I spent a minute or two to look at that commit, which looks to be high quality. While I personally have not used this distribution, people I know use it all the time (for ML). A quibble: #define NPY_PI 3.141592653589793238462643383279502884 /* pi */ and the following defines which appear in numpy/random/src/distributions/random_polyagamma.c are already defined in numpy/core/include/numpy/npy_math.h Probably it would be better to include that file instead, if it isn't already included. DISCLAIMER: I checked none of the math other than passing my eyes over it. On Sun, Dec 27, 2020 at 12:05 PM Zolisa Bleki wrote: > Hi All, > > I would like to know if Numpy accepts addition of new distributions since > the implementation of the Generator interface. If so, what is the criteria > for a particular distribution to be accepted? The reason why i'm asking is > because I would like to propose adding the Polya-gamma distribution to > numpy, for the following reasons: > > 1) Polya-gamma random variables are commonly used as auxiliary variables > during data augmentation in Bayesian sampling algorithms, which have > wide-spread usage in Statistics and recently, Machine learning. > 2) Since this distribution is mostly useful for random sampling, it since > appropriate to have it in numpy and not projects like scipy [1]. > 3) The only python/C++ implementation of the sampler available is licensed > under GPLv3 which I believe limits copying into packages that choose to use > a different license [2]. > 4) Numpy's random API makes adding the distribution painless. > > I have done preliminary work on this by implementing the distribution > sampler as decribed in [3]; see: > https://github.com/numpy/numpy/compare/master...zoj613:polyagamma . > There is a more efficient sampling algorithm described in a later paper > [4], but I chose not to start with that one unless I know it is worth > investing time in. > > I would appreciate your thoughts on this proposal. > > Regards, > Zolisa > > > Refs: > [1] https://github.com/scipy/scipy/issues/11009 > [2] https://github.com/slinderman/pypolyagamma > [3] https://arxiv.org/pdf/1205.0310v1.pdf > [4] https://arxiv.org/pdf/1405.0506.pdf > > > > Disclaimer - University of Cape Town This email is subject to UCT policies > and email disclaimer published on our website at > http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 > 9111. If this email is not related to the business of UCT, it is sent by > the sender in an individual capacity. Please report security incidents or > abuse via https://csirt.uct.ac.za/page/report-an-incident.php. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -- /t PGP Key: https://flowcrypt.com/pub/tom.ritchford at gmail.com *https://tom.ritchford.com * *https://tom.swirly.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Mon Dec 28 02:51:55 2020 From: shoyer at gmail.com (Stephan Hoyer) Date: Sun, 27 Dec 2020 23:51:55 -0800 Subject: [Numpy-discussion] Addition of new distributions: Polya-gamma In-Reply-To: References: Message-ID: Thanks for putting together this clean implementation! My concern is that Polya-Gamma is not popular enough to warrant inclusion in NumPy, which tries very hard to limit scope these days. For example, Polya-Gamma isn?t implemented in scioy.stats and doesn?t have a Wikipedia page, both of which are generally much more inclusive than NumPy. On Sun, Dec 27, 2020 at 3:29 AM Tom Swirly wrote: > I'm just a lurker, but I spent a minute or two to look at that commit, > which looks to be high quality. While I personally have not used this > distribution, people I know use it all the time (for ML). > > > A quibble: > > #define NPY_PI 3.141592653589793238462643383279502884 /* pi */ > > and the following defines which appear > in numpy/random/src/distributions/random_polyagamma.c are already defined > in numpy/core/include/numpy/npy_math.h > > Probably it would be better to include that file instead, if it isn't > already included. > > > DISCLAIMER: I checked none of the math other than passing my eyes over it. > > > > On Sun, Dec 27, 2020 at 12:05 PM Zolisa Bleki > wrote: > >> Hi All, >> >> I would like to know if Numpy accepts addition of new distributions since >> the implementation of the Generator interface. If so, what is the criteria >> for a particular distribution to be accepted? The reason why i'm asking is >> because I would like to propose adding the Polya-gamma distribution to >> numpy, for the following reasons: >> >> 1) Polya-gamma random variables are commonly used as auxiliary variables >> during data augmentation in Bayesian sampling algorithms, which have >> wide-spread usage in Statistics and recently, Machine learning. >> 2) Since this distribution is mostly useful for random sampling, it since >> appropriate to have it in numpy and not projects like scipy [1]. >> 3) The only python/C++ implementation of the sampler available is >> licensed under GPLv3 which I believe limits copying into packages that >> choose to use a different license [2]. >> 4) Numpy's random API makes adding the distribution painless. >> >> I have done preliminary work on this by implementing the distribution >> sampler as decribed in [3]; see: >> https://github.com/numpy/numpy/compare/master...zoj613:polyagamma . >> There is a more efficient sampling algorithm described in a later paper >> [4], but I chose not to start with that one unless I know it is worth >> investing time in. >> >> I would appreciate your thoughts on this proposal. >> >> Regards, >> Zolisa >> >> >> Refs: >> [1] https://github.com/scipy/scipy/issues/11009 >> [2] https://github.com/slinderman/pypolyagamma >> [3] https://arxiv.org/pdf/1205.0310v1.pdf >> [4] https://arxiv.org/pdf/1405.0506.pdf >> >> >> >> Disclaimer - University of Cape Town This email is subject to UCT >> policies and email disclaimer published on our website at >> http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 >> 9111. If this email is not related to the business of UCT, it is sent by >> the sender in an individual capacity. Please report security incidents or >> abuse via https://csirt.uct.ac.za/page/report-an-incident.php. >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > > > -- > /t > > PGP Key: https://flowcrypt.com/pub/tom.ritchford at gmail.com > *https://tom.ritchford.com * > *https://tom.swirly.com * > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Mon Dec 28 13:06:33 2020 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 28 Dec 2020 13:06:33 -0500 Subject: [Numpy-discussion] Addition of new distributions: Polya-gamma In-Reply-To: References: Message-ID: My view is that we will not add more non-uniform distribution (i.e. "named" statistical probability distributions like Polya-Gamma) methods to `Generator`. I think that we might add a couple more methods to handle some more fundamental issues (like sampling from the unit interval with control over whether each boundary is open or closed, maybe one more variation on shuffling) that helps write randomized algorithms. Now that we have the C and Cython APIs which allow one to implement non-uniform distributions in other packages, we strongly encourage that. As I commented on the linked PR, `scipy.stats` would be a reasonable place for a Polya-Gamma sampling function, even if it's not feasible to implement an `rv_continuous` class for it. You have convinced me that the nature of the Polya-Gamma distribution warrants this. The only issue is that scipy still depends on a pre-`Generator` version of numpy. So I recommend implementing this function in your own package with an eye towards contributing it to scipy later. On Sun, Dec 27, 2020 at 6:05 AM Zolisa Bleki wrote: > Hi All, > > I would like to know if Numpy accepts addition of new distributions since > the implementation of the Generator interface. If so, what is the criteria > for a particular distribution to be accepted? The reason why i'm asking is > because I would like to propose adding the Polya-gamma distribution to > numpy, for the following reasons: > > 1) Polya-gamma random variables are commonly used as auxiliary variables > during data augmentation in Bayesian sampling algorithms, which have > wide-spread usage in Statistics and recently, Machine learning. > 2) Since this distribution is mostly useful for random sampling, it since > appropriate to have it in numpy and not projects like scipy [1]. > 3) The only python/C++ implementation of the sampler available is licensed > under GPLv3 which I believe limits copying into packages that choose to use > a different license [2]. > 4) Numpy's random API makes adding the distribution painless. > > I have done preliminary work on this by implementing the distribution > sampler as decribed in [3]; see: > https://github.com/numpy/numpy/compare/master...zoj613:polyagamma . > There is a more efficient sampling algorithm described in a later paper > [4], but I chose not to start with that one unless I know it is worth > investing time in. > > I would appreciate your thoughts on this proposal. > > Regards, > Zolisa > > > Refs: > [1] https://github.com/scipy/scipy/issues/11009 > [2] https://github.com/slinderman/pypolyagamma > [3] https://arxiv.org/pdf/1205.0310v1.pdf > [4] https://arxiv.org/pdf/1405.0506.pdf > > > > Disclaimer - University of Cape Town This email is subject to UCT policies > and email disclaimer published on our website at > http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 > 9111. If this email is not related to the business of UCT, it is sent by > the sender in an individual capacity. Please report security incidents or > abuse via https://csirt.uct.ac.za/page/report-an-incident.php. > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From amardeepjk at gmail.com Tue Dec 29 00:55:03 2020 From: amardeepjk at gmail.com (Amardeep Singh) Date: Tue, 29 Dec 2020 13:55:03 +0800 Subject: [Numpy-discussion] Help needed GDB Message-ID: Hi All, I am trying to debug c code of numpy via gdb.Can someone help me with this? i am getting " Python scripting is not supported in this copy of GDB". How to install python supported gdb on win10? https://numpy.org/doc/stable/dev/development_environment.html I am following the steps in the docs. machine is windows 10. Debugging Another frequently asked question is ?How do I debug C code inside NumPy??. First, ensure that you have gdb installed on your system with the Python extensions (often the default on Linux). You can see which version of Python is running inside gdb to verify your setup: (gdb) python>import sys>print(sys.version_info)>endsys.version_info(major=3, minor=7, micro=0, releaselevel='final', serial=0) $ gdb -v GNU gdb (GDB) 7.6.1 This GDB was configured as "mingw32". $ gdb (gdb) python >import sys >print(sys.version_info) >end (gdb) Python scripting is not supported in this copy of GDB. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robbmcleod at gmail.com Tue Dec 29 01:38:07 2020 From: robbmcleod at gmail.com (Robert McLeod) Date: Mon, 28 Dec 2020 22:38:07 -0800 Subject: [Numpy-discussion] ANN: NumExpr 2.7.2 Message-ID: ======================== Announcing NumExpr 2.7.2 ======================== Hi everyone, It's been awhile since the last update to NumExpr, mostly as the existing scientific Python tool chain for building wheels on PyPi became defunct and we have had to redevelop a new one based on `cibuildwheel` and GitHub Actions. This release also brings us support (and wheels for) Python 3.9. There have been a number of changes to enhance how NumExpr works when NumPy uses MKL as a backend. Project documentation is available at: http://numexpr.readthedocs.io/ Changes from 2.7.1 to 2.7.2 --------------------------- - Support for Python 2.7 and 3.5 is deprecated and will be discontinued when `cibuildwheels` and/or GitHub Actions no longer support these versions. - Wheels are now provided for Python 3.7, 3.5, 3.6, 3.7, 3.8, and 3.9 via GitHub Actions. - The block size is now exported into the namespace as `numexpr.__BLOCK_SIZE1__` as a read-only value. - If using MKL, the number of threads for VML is no longer forced to 1 on loading the module. Testing has shown that VML never runs in multi-threaded mode for the default BLOCKSIZE1 of 1024 elements, and forcing to 1 can have deleterious effects on NumPy functions when built with MKL. See issue #355 for details. - Use of `ndarray.tostring()` in tests has been switch to `ndarray.tobytes()` for future-proofing deprecation of `.tostring()`, if the version of NumPy is greater than 1.9. - Added a utility method `get_num_threads` that returns the (maximum) number of threads currently in use by the virtual machine. The functionality of `set_num_threads` whereby it returns the previous value has been deprecated and will be removed in 2.8.X. What's Numexpr? --------------- Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It has multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ------------------------- The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Documentation is hosted at: http://numexpr.readthedocs.io/en/latest/ Share your experience --------------------- Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Robert McLeod robbmcleod at gmail.com robert.mcleod at hitachi-hhtc.ca -------------- next part -------------- An HTML attachment was scrubbed... URL: From amardeepjk at gmail.com Tue Dec 29 03:04:13 2020 From: amardeepjk at gmail.com (Amardeep Singh) Date: Tue, 29 Dec 2020 16:04:13 +0800 Subject: [Numpy-discussion] Help needed GDB In-Reply-To: References: Message-ID: Hi All, I was able to fix it but one thing i am not getting. it is building with python 2. I need python 3 enabled gdb. ./configure --with-python. --> how to make it to use python 3 installed on my machine? thx On Tue, Dec 29, 2020 at 1:55 PM Amardeep Singh wrote: > Hi All, > > I am trying to debug c code of numpy via gdb.Can someone help me with this? > i am getting " Python scripting is not supported in this copy of GDB". How > to install python supported gdb on win10? > > > https://numpy.org/doc/stable/dev/development_environment.html > > I am following the steps in the docs. machine is windows 10. > > Debugging > > > Another frequently asked question is ?How do I debug C code inside > NumPy??. First, ensure that you have gdb installed on your system with the > Python extensions (often the default on Linux). You can see which version > of Python is running inside gdb to verify your setup: > > (gdb) python>import sys>print(sys.version_info)>endsys.version_info(major=3, minor=7, micro=0, releaselevel='final', serial=0) > > > > > $ gdb -v > GNU gdb (GDB) 7.6.1 > This GDB was configured as "mingw32". > > $ gdb > (gdb) python > >import sys > >print(sys.version_info) > >end > (gdb) Python scripting is not supported in this copy of GDB. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amardeepjk at gmail.com Tue Dec 29 03:27:26 2020 From: amardeepjk at gmail.com (Amardeep Singh) Date: Tue, 29 Dec 2020 16:27:26 +0800 Subject: [Numpy-discussion] Help needed GDB In-Reply-To: References: Message-ID: Hi All when i. try to use python3 installed on my macbook i get this. checking for libmpfr... no configure: WARNING: MPFR is missing or unusable; some features may be unavailable. checking whether to use python... /usr/local/bin/python3 checking for python... no configure: error: no usable python found at /usr/local/bin/python3 These are the commands i ran 1) /Users/amardeepsingh/Downloads/gdb-9.2/configure --with-python=/usr/local/bin/python3 2) make my mac % which python3 /Library/Frameworks/Python.framework/Versions/3.9/bin/python3 % ls -lrt /usr/local/bin/python3 lrwxr-xr-x 1 root wheel 69 Dec 29 14:37 /usr/local/bin/python3 -> ../../../Library/Frameworks/Python.framework/Versions/3.9/bin/python3 On Tue, Dec 29, 2020 at 4:04 PM Amardeep Singh wrote: > Hi All, > > I was able to fix it but one thing i am not getting. > it is building with python 2. > > I need python 3 enabled gdb. > > ./configure --with-python. --> how to make it to use python 3 installed > on my machine? > > thx > > > > > On Tue, Dec 29, 2020 at 1:55 PM Amardeep Singh > wrote: > >> Hi All, >> >> I am trying to debug c code of numpy via gdb.Can someone help me with >> this? >> i am getting " Python scripting is not supported in this copy of GDB". >> How to install python supported gdb on win10? >> >> >> https://numpy.org/doc/stable/dev/development_environment.html >> >> I am following the steps in the docs. machine is windows 10. >> >> Debugging >> >> >> Another frequently asked question is ?How do I debug C code inside >> NumPy??. First, ensure that you have gdb installed on your system with the >> Python extensions (often the default on Linux). You can see which version >> of Python is running inside gdb to verify your setup: >> >> (gdb) python>import sys>print(sys.version_info)>endsys.version_info(major=3, minor=7, micro=0, releaselevel='final', serial=0) >> >> >> >> >> $ gdb -v >> GNU gdb (GDB) 7.6.1 >> This GDB was configured as "mingw32". >> >> $ gdb >> (gdb) python >> >import sys >> >print(sys.version_info) >> >end >> (gdb) Python scripting is not supported in this copy of GDB. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Tue Dec 29 06:27:24 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Tue, 29 Dec 2020 12:27:24 +0100 Subject: [Numpy-discussion] proposal for NumPy sponsorship guidelines (NEP 46) Message-ID: Hi all, I just opened https://github.com/numpy/numpy/pull/18084, "NumPy Sponsorship Guidelines". Below are the most important parts for review (for Related Work, References, etc. see the PR). Please bring up broader points here, and small/textual feedback on the PR. Cheers, Ralf Abstract -------- This NEP provides guidelines on how the NumPy project will acknowledge financial and in-kind support. Motivation and Scope -------------------- In the past few years the NumPy project has gotten significant financial support, as well as dedicated work time for maintainers to work on NumPy. There is a need to acknowledge that support - funders and organizations expect or require it, it's helpful when looking for new funding, and it's the right thing to do. Furthermore, having a clear policy for how NumPy acknowledges support is helpful when searching for new support. This NEP is aimed at both the NumPy community - who can use it when looking for support and acknowledging existing support - and at past, current and prospective sponsors, who often want or need to know what they get in return for their support (other than a healthier NumPy). The scope of this proposal includes: - direct financial support, employers providing paid time for NumPy maintainers and regular contributors, and in-kind support such as free hardware resources or services. - where and how NumPy acknowledges support (e.g., logo placement on the website). - the amount and duration of support which leads to acknowledgement. - who in the NumPy project is responsible for sponsorship related topics, and how to contact them. How NumPy will acknowledge support ---------------------------------- There will be two different ways to acknowledge financial and in-kind support, one to recognize significant active support, and another one to recognize support received in the past and smaller amounts of support. Entities who fall under "significant active supporter" we'll call Sponsor. The minimum level of support given to NumPy to be considered a Sponsor are: - $30,000/yr for unrestricted financial contributions - $60,000/yr for financial contributions for a particular purpose - $100,000/yr for in-kind contributions The rationale for the above levels is that unrestricted financial contributions are typically the most valuable for the project, and the hardest to obtain. The opposite is true for in-kind contributions. The dollar value of the levels also reflect that NumPy's needs have grown to the point where we need at least a few paid developers in order to effectively support our user base and continue to move the project forward. Financial support at or above these levels is needed to be able to make a significant difference. Sponsors will get acknowledged through: - a small logo displayed on the front page of the NumPy website - prominent logo placement on https://numpy.org/about/ - logos displayed in talks about NumPy by maintainers - announcements of the sponsorship on the NumPy mailing list and the numpy-team Twitter account In addition to Sponsors, we already have the concept of Institutional Partner (defined in NumPy's `governance document `__), for entities who employ a NumPy maintainer and let them work on NumPy as part of their official duties. The governance document doesn't currently define a minimum amount of paid maintainer time needed to be considered for partnership. Therefore we propose that level here, roughly in line with the sponsorship levels: - 6 person-months/yr of paid work time for one or more NumPy maintainers or regular contributors Institutional Partners get the same benefits as Sponsors, in addition to what is specified in the NumPy governance document. Finally, a new page on the website (https://numpy.org/funding/, linked from the About page) will be added to acknowledge all current and previous sponsors, partners, and any other entities and individuals who provided $5,000 or more of financial or in-kind support. This page will include relevant details of support (dates, amounts, names and purpose); no logos will be used on this page. The rationale for the $5,000 minimum level is to keep the amount of work maintaining the page reasonable; the level is the equivalent of, e.g., one GSoC or a person-week's worth of engineering time in a Western country, which seems like a reasonable lower limit. Implementation -------------- The following content changes need to be made: - Add a section with small logos towards the bottom of the `numpy.org `__ website. - Create a full list of historical and current support and deploy it to https://numpy.org/funding. - Update the NumPy governance document for changes to Institutional Partner eligibility requirements and benefits. - Update https://numpy.org/about with details on how to get in touch with the NumPy project about sponsorship related matters (see next section). A NumPy Funding Team ~~~~~~~~~~~~~~~~~~~~ At the moment NumPy has only one official body, the Steering Council, and no good way to get in touch with either that body or any person or group responsible for funding and sponsorship related matters. The way this is typically done now is to somehow find the personal email of a maintainer, and email them in private. There is a need to organize this more transparently - a potential sponsor isn't likely to inquire through the mailing list, nor is it easy for a potential sponsor to know if they're reaching out to the right person in private. https://numpy.org/about/ already says that NumPy has a "funding and grants" team, however that is not the case. We propose to organize this team, name team members on it, and add the names of those team members plus a dedicated email address for the team to the About page. Status before this proposal --------------------------- Acknowledgement of support ~~~~~~~~~~~~~~~~~~~~~~~~~~ At the time of writing (Dec 2020), the logos of the four largest financial sponsors and two institutional partners are displayed on https://numpy.org/about/. The `Nature paper about NumPy < https://www.nature.com/articles/s41586-020-2649-2>`__ mentions some early funding. No comprehensive list of received funding and in-kind support is published anywhere. Decisions on which logos to list on the website have been made mostly by the website team. Decisions on which entities to recognize as Institutional Partner have been made by the NumPy Steering Council. NumPy governance, decision-making and financial oversight ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *This section is meant as context for the reader, to help put the rest of this NEP in perspective, and perhaps answer questions the reader has when reading this as a potential sponsor.* NumPy has a formal governance structure defined in `this governance document < https://numpy.org/devdocs/dev/governance/index.html>`__). Decisions are made by consensus among all active participants in a discussion (typically on the mailing list), and if consensus cannot be reached then the Steering Council takes the decision (also by consensus). NumPy is a sponsored project of NumFOCUS, a US-based 501(c)3 nonprofit. NumFOCUS administers NumPy funds, and ensures they are spent in accordance with its mission and nonprofit status. In practice, NumPy has a NumFOCUS subcommittee (with its members named in the NumPy governance document) who can authorize financial transactions. Those transactions, for example paying a contractor for a particular activity or deliverable, are decided on by the NumPy Steering Council. Alternatives ------------ *Tiered sponsorship levels.* We considered using tiered sponsorship levels, and rejected this alternative because it would be more complex, and not necessarily communicate the right intent - the minimum levels are for us to determine how to acknowledge support that we receive, not a commercial value proposition. Entities typically will support NumPy because they rely on the project or want to help advance it, and not to get brand awareness through logo placement. *Listing all donations*. Note that in the past we have received many smaller donations, mostly from individuals through NumFOCUS. It would be great to list all of those contributions, but given the way we receive information on those donations right now, that would be quite labor-intensive. If we manage to move to a more suitable platform, such as `Open Collective < https://opencollective.com/>`__, in the future, we should reconsider listing all individual donations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amardeepjk at gmail.com Tue Dec 29 08:38:21 2020 From: amardeepjk at gmail.com (Amardeep Singh) Date: Tue, 29 Dec 2020 21:38:21 +0800 Subject: [Numpy-discussion] Help needed GDB In-Reply-To: References: Message-ID: Hi All I was able to fix. >import sys >print(sys.version_info) >end sys.version_info(major=3, minor=9, micro=1, releaselevel='final', serial=0) (gdb) quit https://github.com/crosstool-ng/crosstool-ng/issues/1308 this was the issue. The solution is to open /gdb/python/python-config.py and comment out these two lines: if getvar('LINKFORSHARED') is not None: libs.extend(getvar('LINKFORSHARED').split()) On Tue, Dec 29, 2020 at 4:27 PM Amardeep Singh wrote: > Hi All > > when i. try to use python3 installed on my macbook i get this. > > checking for libmpfr... no > > configure: WARNING: MPFR is missing or unusable; some features may be > unavailable. > > checking whether to use python... /usr/local/bin/python3 > > checking for python... no > > configure: error: no usable python found at /usr/local/bin/python3 > > > > These are the commands i ran > > > 1) /Users/amardeepsingh/Downloads/gdb-9.2/configure > --with-python=/usr/local/bin/python3 > > > 2) make > > > > > my mac > > > % which python3 > > > /Library/Frameworks/Python.framework/Versions/3.9/bin/python3 > > % ls -lrt /usr/local/bin/python3 > > lrwxr-xr-x 1 root wheel 69 Dec 29 14:37 /usr/local/bin/python3 -> > ../../../Library/Frameworks/Python.framework/Versions/3.9/bin/python3 > > > > > > > > > On Tue, Dec 29, 2020 at 4:04 PM Amardeep Singh > wrote: > >> Hi All, >> >> I was able to fix it but one thing i am not getting. >> it is building with python 2. >> >> I need python 3 enabled gdb. >> >> ./configure --with-python. --> how to make it to use python 3 installed >> on my machine? >> >> thx >> >> >> >> >> On Tue, Dec 29, 2020 at 1:55 PM Amardeep Singh >> wrote: >> >>> Hi All, >>> >>> I am trying to debug c code of numpy via gdb.Can someone help me with >>> this? >>> i am getting " Python scripting is not supported in this copy of GDB". >>> How to install python supported gdb on win10? >>> >>> >>> https://numpy.org/doc/stable/dev/development_environment.html >>> >>> I am following the steps in the docs. machine is windows 10. >>> >>> Debugging >>> >>> >>> Another frequently asked question is ?How do I debug C code inside >>> NumPy??. First, ensure that you have gdb installed on your system with the >>> Python extensions (often the default on Linux). You can see which version >>> of Python is running inside gdb to verify your setup: >>> >>> (gdb) python>import sys>print(sys.version_info)>endsys.version_info(major=3, minor=7, micro=0, releaselevel='final', serial=0) >>> >>> >>> >>> >>> $ gdb -v >>> GNU gdb (GDB) 7.6.1 >>> This GDB was configured as "mingw32". >>> >>> $ gdb >>> (gdb) python >>> >import sys >>> >print(sys.version_info) >>> >end >>> (gdb) Python scripting is not supported in this copy of GDB. >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From amardeepjk at gmail.com Tue Dec 29 11:54:41 2020 From: amardeepjk at gmail.com (Amardeep Singh) Date: Wed, 30 Dec 2020 00:54:41 +0800 Subject: [Numpy-discussion] Help needed GDB In-Reply-To: References: Message-ID: Hi Apologies Once last issue is pending.Any pointers are helpful. I am following the below docs. Next you need to write a Python script that invokes the C code whose execution you want to debug. For instance mytest.py: import numpy as npx = np.arange(5)np.empty_like(x) Now, you can run: $ gdb --args python runtests.py -g --python mytest.py And then in the debugger: (gdb) break array_empty_like(gdb) run. --> once i type run and enter nothing happens it just gets struck up there. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (array_empty_like) pending. (gdb) run Starting program: /usr/bin/python mytest.py [New Thread 0x1e03 of process 16513] [New Thread 0x2303 of process 16513] thx On Tue, Dec 29, 2020 at 9:38 PM Amardeep Singh wrote: > Hi All > > I was able to fix. > > >import sys > > >print(sys.version_info) > > >end > > sys.version_info(major=3, minor=9, micro=1, releaselevel='final', serial=0) > > (gdb) quit > > > > > https://github.com/crosstool-ng/crosstool-ng/issues/1308 > > > this was the issue. > > > The solution is to open /gdb/python/python-config.py and comment out these > two lines: > > if getvar('LINKFORSHARED') is not None: > libs.extend(getvar('LINKFORSHARED').split()) > > > > > > > > > > > On Tue, Dec 29, 2020 at 4:27 PM Amardeep Singh > wrote: > >> Hi All >> >> when i. try to use python3 installed on my macbook i get this. >> >> checking for libmpfr... no >> >> configure: WARNING: MPFR is missing or unusable; some features may be >> unavailable. >> >> checking whether to use python... /usr/local/bin/python3 >> >> checking for python... no >> >> configure: error: no usable python found at /usr/local/bin/python3 >> >> >> >> These are the commands i ran >> >> >> 1) /Users/amardeepsingh/Downloads/gdb-9.2/configure >> --with-python=/usr/local/bin/python3 >> >> >> 2) make >> >> >> >> >> my mac >> >> >> % which python3 >> >> >> /Library/Frameworks/Python.framework/Versions/3.9/bin/python3 >> >> % ls -lrt /usr/local/bin/python3 >> >> lrwxr-xr-x 1 root wheel 69 Dec 29 14:37 /usr/local/bin/python3 -> >> ../../../Library/Frameworks/Python.framework/Versions/3.9/bin/python3 >> >> >> >> >> >> >> >> >> On Tue, Dec 29, 2020 at 4:04 PM Amardeep Singh >> wrote: >> >>> Hi All, >>> >>> I was able to fix it but one thing i am not getting. >>> it is building with python 2. >>> >>> I need python 3 enabled gdb. >>> >>> ./configure --with-python. --> how to make it to use python 3 installed >>> on my machine? >>> >>> thx >>> >>> >>> >>> >>> On Tue, Dec 29, 2020 at 1:55 PM Amardeep Singh >>> wrote: >>> >>>> Hi All, >>>> >>>> I am trying to debug c code of numpy via gdb.Can someone help me with >>>> this? >>>> i am getting " Python scripting is not supported in this copy of GDB". >>>> How to install python supported gdb on win10? >>>> >>>> >>>> https://numpy.org/doc/stable/dev/development_environment.html >>>> >>>> I am following the steps in the docs. machine is windows 10. >>>> >>>> Debugging >>>> >>>> >>>> Another frequently asked question is ?How do I debug C code inside >>>> NumPy??. First, ensure that you have gdb installed on your system with the >>>> Python extensions (often the default on Linux). You can see which version >>>> of Python is running inside gdb to verify your setup: >>>> >>>> (gdb) python>import sys>print(sys.version_info)>endsys.version_info(major=3, minor=7, micro=0, releaselevel='final', serial=0) >>>> >>>> >>>> >>>> >>>> $ gdb -v >>>> GNU gdb (GDB) 7.6.1 >>>> This GDB was configured as "mingw32". >>>> >>>> $ gdb >>>> (gdb) python >>>> >import sys >>>> >print(sys.version_info) >>>> >end >>>> (gdb) Python scripting is not supported in this copy of GDB. >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From theintrocode at gmail.com Tue Dec 29 17:34:25 2020 From: theintrocode at gmail.com (Brian Soto) Date: Tue, 29 Dec 2020 14:34:25 -0800 Subject: [Numpy-discussion] NumPy-Discussion Digest, Vol 171, Issue 39 In-Reply-To: References: Message-ID: I'm still learning proper mailing list etiquette so I'm not sure if this is where I should respond. But users just getting into debugging might also benefit from knowing this: You can turn off optimizations when compiling numpy by passing CFLAGS to setup.py like so: *CFLAGS="-O0 -g3" python setup.py build_ext -i * **Assuming you have the source code and setup.py available * This will remove optimizations while compiling and will make it easier to see more variables. That took me a long time to figure out so I wanted to share the knowledge Thanks! On Mon, Dec 28, 2020 at 10:38 PM wrote: > Send NumPy-Discussion mailing list submissions to > numpy-discussion at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-request at python.org > > You can reach the person managing the list at > numpy-discussion-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > > > Today's Topics: > > 1. Re: Addition of new distributions: Polya-gamma (Robert Kern) > 2. Help needed GDB (Amardeep Singh) > 3. ANN: NumExpr 2.7.2 (Robert McLeod) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 28 Dec 2020 13:06:33 -0500 > From: Robert Kern > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] Addition of new distributions: > Polya-gamma > Message-ID: > < > CAF6FJivqpLsXqvyUQcAUL67VHQw0viQWTry1+L1pq4GipfDdhw at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > My view is that we will not add more non-uniform distribution (i.e. "named" > statistical probability distributions like Polya-Gamma) methods to > `Generator`. I think that we might add a couple more methods to handle some > more fundamental issues (like sampling from the unit interval with control > over whether each boundary is open or closed, maybe one more variation on > shuffling) that helps write randomized algorithms. Now that we have the C > and Cython APIs which allow one to implement non-uniform distributions in > other packages, we strongly encourage that. > > As I commented on the linked PR, `scipy.stats` would be a reasonable place > for a Polya-Gamma sampling function, even if it's not feasible to implement > an `rv_continuous` class for it. You have convinced me that the nature of > the Polya-Gamma distribution warrants this. The only issue is that scipy > still depends on a pre-`Generator` version of numpy. So I recommend > implementing this function in your own package with an eye towards > contributing it to scipy later. > > On Sun, Dec 27, 2020 at 6:05 AM Zolisa Bleki > wrote: > > > Hi All, > > > > I would like to know if Numpy accepts addition of new distributions since > > the implementation of the Generator interface. If so, what is the > criteria > > for a particular distribution to be accepted? The reason why i'm asking > is > > because I would like to propose adding the Polya-gamma distribution to > > numpy, for the following reasons: > > > > 1) Polya-gamma random variables are commonly used as auxiliary variables > > during data augmentation in Bayesian sampling algorithms, which have > > wide-spread usage in Statistics and recently, Machine learning. > > 2) Since this distribution is mostly useful for random sampling, it since > > appropriate to have it in numpy and not projects like scipy [1]. > > 3) The only python/C++ implementation of the sampler available is > licensed > > under GPLv3 which I believe limits copying into packages that choose to > use > > a different license [2]. > > 4) Numpy's random API makes adding the distribution painless. > > > > I have done preliminary work on this by implementing the distribution > > sampler as decribed in [3]; see: > > https://github.com/numpy/numpy/compare/master...zoj613:polyagamma . > > There is a more efficient sampling algorithm described in a later paper > > [4], but I chose not to start with that one unless I know it is worth > > investing time in. > > > > I would appreciate your thoughts on this proposal. > > > > Regards, > > Zolisa > > > > > > Refs: > > [1] https://github.com/scipy/scipy/issues/11009 > > [2] https://github.com/slinderman/pypolyagamma > > [3] https://arxiv.org/pdf/1205.0310v1.pdf > > [4] https://arxiv.org/pdf/1405.0506.pdf > > > > > > > > Disclaimer - University of Cape Town This email is subject to UCT > policies > > and email disclaimer published on our website at > > http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 > > 9111. If this email is not related to the business of UCT, it is sent by > > the sender in an individual capacity. Please report security incidents or > > abuse via https://csirt.uct.ac.za/page/report-an-incident.php. > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > -- > Robert Kern > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://mail.python.org/pipermail/numpy-discussion/attachments/20201228/f4bbd564/attachment-0001.html > > > > ------------------------------ > > Message: 2 > Date: Tue, 29 Dec 2020 13:55:03 +0800 > From: Amardeep Singh > To: numpy-discussion at python.org > Subject: [Numpy-discussion] Help needed GDB > Message-ID: > < > CAJmcdX6dgoh_FKdHKxQB1_GPLdYw3+0PMG_ne_T5BBhEzMmGmg at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi All, > > I am trying to debug c code of numpy via gdb.Can someone help me with this? > i am getting " Python scripting is not supported in this copy of GDB". How > to install python supported gdb on win10? > > > https://numpy.org/doc/stable/dev/development_environment.html > > I am following the steps in the docs. machine is windows 10. > > Debugging > > > Another frequently asked question is ?How do I debug C code inside NumPy??. > First, ensure that you have gdb installed on your system with the Python > extensions (often the default on Linux). You can see which version of > Python is running inside gdb to verify your setup: > > (gdb) python>import > sys>print(sys.version_info)>endsys.version_info(major=3, minor=7, > micro=0, releaselevel='final', serial=0) > > > > > $ gdb -v > GNU gdb (GDB) 7.6.1 > This GDB was configured as "mingw32". > > $ gdb > (gdb) python > >import sys > >print(sys.version_info) > >end > (gdb) Python scripting is not supported in this copy of GDB. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://mail.python.org/pipermail/numpy-discussion/attachments/20201229/268f82ab/attachment-0001.html > > > > ------------------------------ > > Message: 3 > Date: Mon, 28 Dec 2020 22:38:07 -0800 > From: Robert McLeod > To: python-announce-list at python.org, Discussion of Numerical Python > , pydata at googlegroups.com > Subject: [Numpy-discussion] ANN: NumExpr 2.7.2 > Message-ID: > < > CAEFUWWV0T8ACcZbS7N56zXtPZEBgMFVQQ7whjhgxVFttgO6DTQ at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > ======================== > Announcing NumExpr 2.7.2 > ======================== > > Hi everyone, > > It's been awhile since the last update to NumExpr, mostly as the existing > scientific > Python tool chain for building wheels on PyPi became defunct and we have > had to > redevelop a new one based on `cibuildwheel` and GitHub Actions. This > release also > brings us support (and wheels for) Python 3.9. > > There have been a number of changes to enhance how NumExpr works when NumPy > uses MKL as a backend. > > Project documentation is available at: > > http://numexpr.readthedocs.io/ > > Changes from 2.7.1 to 2.7.2 > --------------------------- > > - Support for Python 2.7 and 3.5 is deprecated and will be discontinued > when > `cibuildwheels` and/or GitHub Actions no longer support these versions. > - Wheels are now provided for Python 3.7, 3.5, 3.6, 3.7, 3.8, and 3.9 via > GitHub Actions. > - The block size is now exported into the namespace as > `numexpr.__BLOCK_SIZE1__` > as a read-only value. > - If using MKL, the number of threads for VML is no longer forced to 1 on > loading > the module. Testing has shown that VML never runs in multi-threaded mode > for > the default BLOCKSIZE1 of 1024 elements, and forcing to 1 can have > deleterious > effects on NumPy functions when built with MKL. See issue #355 for > details. > - Use of `ndarray.tostring()` in tests has been switch to > `ndarray.tobytes()` > for future-proofing deprecation of `.tostring()`, if the version of NumPy > is > greater than 1.9. > - Added a utility method `get_num_threads` that returns the (maximum) > number of > threads currently in use by the virtual machine. The functionality of > `set_num_threads` whereby it returns the previous value has been > deprecated > and will be removed in 2.8.X. > > What's Numexpr? > --------------- > > Numexpr is a fast numerical expression evaluator for NumPy. With it, > expressions that operate on arrays (like "3*a+4*b") are accelerated > and use less memory than doing the same calculation in Python. > > It has multi-threaded capabilities, as well as support for Intel's > MKL (Math Kernel Library), which allows an extremely fast evaluation > of transcendental functions (sin, cos, tan, exp, log...) while > squeezing the last drop of performance out of your multi-core > processors. Look here for a some benchmarks of numexpr using MKL: > > https://github.com/pydata/numexpr/wiki/NumexprMKL > > Its only dependency is NumPy (MKL is optional), so it works well as an > easy-to-deploy, easy-to-use, computational engine for projects that > don't want to adopt other solutions requiring more heavy dependencies. > > Where I can find Numexpr? > ------------------------- > > The project is hosted at GitHub in: > > https://github.com/pydata/numexpr > > You can get the packages from PyPI as well (but not for RC releases): > > http://pypi.python.org/pypi/numexpr > > Documentation is hosted at: > > http://numexpr.readthedocs.io/en/latest/ > > Share your experience > --------------------- > > Let us know of any bugs, suggestions, gripes, kudos, etc. you may > have. > > Enjoy data! > > > -- > Robert McLeod > robbmcleod at gmail.com > robert.mcleod at hitachi-hhtc.ca > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://mail.python.org/pipermail/numpy-discussion/attachments/20201228/f4240dab/attachment.html > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > > ------------------------------ > > End of NumPy-Discussion Digest, Vol 171, Issue 39 > ************************************************* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Tue Dec 29 17:55:12 2020 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 29 Dec 2020 17:55:12 -0500 Subject: [Numpy-discussion] NumPy-Discussion Digest, Vol 171, Issue 39 In-Reply-To: References: Message-ID: On Tue, Dec 29, 2020 at 5:35 PM Brian Soto wrote: > I'm still learning proper mailing list etiquette so I'm not sure if this > is where I should respond. > FWIW, if you want to reply to conversations regularly, it's more ergonomic all around to subscribe regularly and not use the digests. But if you only occasionally want to jump in, using the digest is fine. It helps everyone if you change the subject line back to that of the thread that you are replying to and trimming the digest to just the email you are responding to. Thank you! I hope this is helpful. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Wed Dec 30 09:05:41 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Wed, 30 Dec 2020 15:05:41 +0100 Subject: [Numpy-discussion] updated backwards compatibility and deprecation policy NEP Message-ID: Hi all, Here is a long overdue update of the draft NEP about backwards compatibility and deprecation policy: https://github.com/numpy/numpy/pull/18097 - This is NEP 23: https://numpy.org/neps/nep-0023-backwards-compatibility.html - Link to the previous mailing list discussion: https://mail.python.org/pipermail/numpy-discussion/2018-July/078432.html It would be nice to get this NEP to Accepted status. Main changes are: - Removed all examples that people objected to - Removed all content regarding versioning - Restructured sections, and added "Strategies related to deprecations" (using suggestions by @njsmith and @shoyer). - Added concrete examples of deprecations, and a more thorough description of how to go about adding warnings incl. Sphinx directives, using `stacklevel`, etc. As always, feedback here or on the PR is very welcome! Cheers, Ralf Abstract -------- In this NEP we describe NumPy's approach to backwards compatibility, its deprecation and removal policy, and the trade-offs and decision processes for individual cases where breaking backwards compatibility is considered. Motivation and Scope -------------------- NumPy has a very large user base. Those users rely on NumPy being stable and the code they write that uses NumPy functionality to keep working. NumPy is also actively maintained and improved -- and sometimes improvements require, or are made much easier by, breaking backwards compatibility. Finally, there are trade-offs in stability for existing users vs. avoiding errors or having a better user experience for new users. These competing needs often give rise to long debates and to delays in accepting or rejecting contributions. This NEP tries to address that by providing a policy as well as examples and rationales for when it is or isn't a good idea to break backwards compatibility. In scope for this NEP are: - Principles of NumPy's approach to backwards compatibility. - How to deprecate functionality, and when to remove already deprecated functionality. - Decision making process for deprecations and removals. Out of scope are: - Making concrete decisions about deprecations of particular functionality. - NumPy's versioning scheme. General principles ------------------ When considering proposed changes that are backwards incompatible, the main principles the NumPy developers use when making a decision are: 1. Changes need to benefit users more than they harm them. 2. NumPy is widely used so breaking changes should by default be assumed to be fairly harmful. 3. Decisions should be based on data and actual effects on users and downstream packages rather than, e.g., appealing to the docs or for stylistic reasons. 4. Silently getting a wrong answer is much worse than getting a loud error. When assessing the costs of proposed changes, keep in mind that most users do not read the mailing list, do not look at deprecation warnings, and sometimes wait more than one or two years before upgrading from their old version. And that NumPy has millions of users, so "no one will do or use this" is very likely incorrect. Benefits include improved functionality, usability and performance, as well as lower maintenance cost and improved future extensibility. Fixes for clear bugs are exempt from this backwards compatibility policy. However in case of serious impact on users (e.g. a downstream library doesn't build anymore or would start giving incorrect results), even bug fixes may have to be delayed for one or more releases. Strategies related to deprecations ---------------------------------- Getting hard data on the impact of a deprecation of often difficult. Strategies that can be used to assess such impact include: - Use a code search engine ([1]_) or static ([2]_) or dynamic ([3]_) code analysis tools to determine where and how the functionality is used. - Testing prominent downstream libraries against a development build of NumPy containing the proposed change to get real-world data on its impact. - Making a change in master and reverting it, if needed, before a release. We do encourage other packages to test against NumPy's master branch, so this often turns up issues quickly. If the impact is unclear or significant, it is often good to consider alternatives to deprecations. For example discouraging use in documentation only, or moving the documentation for the functionality to a less prominent place or even removing it completely. Commenting on open issues related to it that they are low-prio or labeling them as "wontfix" will also be a signal to users, and reduce the maintenance effort needing to be spent. Implementing deprecations and removals -------------------------------------- Deprecation warnings are necessary in all cases where functionality will eventually be removed. If there is no intent to remove functionality, then it should not be deprecated either. A "please don't use this" in the documentation or other type of warning should be used instead. Deprecations: - shall include the version number of the release in which the functionality was deprecated. - shall include information on alternatives to the deprecated functionality, or a reason for the deprecation if no clear alternative is available. - shall use ``VisibleDeprecationWarning`` rather than ``DeprecationWarning`` for cases of relevance to end users. For cases only relevant to downstream libraries, a regular ``DeprecationWarning`` is fine. *Rationale: regular deprecation warnings are invisible by default; library authors should be aware how deprecations work and test for them, but we can't expect this from all users.* - shall be listed in the release notes of the release where the deprecation is first present. - shall set a ``stacklevel``, so the warning appears to come from the correct place. - shall be mentioned in the documentation for the functionality. A ``.. deprecated::`` directive can be used for this. Examples of good deprecation warnings: .. code-block:: python warnings.warn('np.asscalar(a) is deprecated since NumPy 1.16.0, use ' 'a.item() instead', DeprecationWarning, stacklevel=3) warnings.warn("Importing from numpy.testing.utils is deprecated " "since 1.15.0, import from numpy.testing instead.", DeprecationWarning, stacklevel=2) # A change in NumPy 1.14.0 for Python 3 loadtxt/genfromtext, slightly # tweaked in this NEP (original didn't have version number). warnings.warn( "Reading unicode strings without specifying the encoding " "argument is deprecated since NumPy 1.14.0. Set the encoding, " "use None for the system default.", np.VisibleDeprecationWarning, stacklevel=2) Removal of deprecated functionality: - shall be done after at least 2 releases (assuming the current 6-monthly release cycle; if that changes, there shall be at least 1 year between deprecation and removal). - shall be listed in the release notes of the release where the removal happened. - can be done in any minor (but not bugfix) release. For backwards incompatible changes that aren't "deprecate and remove" but for which code will start behaving differently, a ``FutureWarning`` should be used. Release notes, mentioning version number and using ``stacklevel`` should be done in the same way as for deprecation warnings. A ``.. versionchanged::`` directive can be used in the documentation to indicate when the behavior changed: .. code-block:: python def argsort(self, axis=np._NoValue, ...): """ Parameters ---------- axis : int, optional Axis along which to sort. If None, the default, the flattened array is used. .. versionchanged:: 1.13.0 Previously, the default was documented to be -1, but that was in error. At some future date, the default will change to -1, as originally intended. Until then, the axis should be given explicitly when ``arr.ndim > 1``, to avoid a FutureWarning. """ ... warnings.warn( "In the future the default for argsort will be axis=-1, not the " "current None, to match its documentation and np.argsort. " "Explicitly pass -1 or None to silence this warning.", MaskedArrayFutureWarning, stacklevel=3) Decision making ~~~~~~~~~~~~~~~ In concrete cases where this policy needs to be applied, decisions are made according to the `NumPy governance model `_. All deprecations must be proposed on the mailing list, in order to give everyone with an interest in NumPy development to be able to comment. Removal of deprecated functionality does not need discussion on the mailing list. Functionality with more strict deprecation policies ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ``numpy.random`` has its own backwards compatibility policy, see `NEP 19 `_. - The file format for ``.npy`` and ``.npz`` files must not be changed in a backwards incompatible way. Example cases ------------- We now discuss a few concrete examples from NumPy's history to illustrate typical issues and trade-offs. **Changing the behavior of a function** ``np.histogram`` is probably the most infamous example. First, a new keyword ``new=False`` was introduced, this was then switched over to None one release later, and finally it was removed again. Also, it has a ``normed`` keyword that had behavior that could be considered either suboptimal or broken (depending on ones opinion on the statistics). A new keyword ``density`` was introduced to replace it; ``normed`` started giving ``DeprecationWarning`` only in v.1.15.0. Evolution of ``histogram``:: def histogram(a, bins=10, range=None, normed=False): # v1.0.0 def histogram(a, bins=10, range=None, normed=False, weights=None, new=False): #v1.1.0 def histogram(a, bins=10, range=None, normed=False, weights=None, new=None): #v1.2.0 def histogram(a, bins=10, range=None, normed=False, weights=None): #v1.5.0 def histogram(a, bins=10, range=None, normed=False, weights=None, density=None): #v1.6.0 def histogram(a, bins=10, range=None, normed=None, weights=None, density=None): #v1.15.0 # v1.15.0 was the first release where `normed` started emitting # DeprecationWarnings The ``new`` keyword was planned from the start to be temporary. Such a plan forces users to change their code more than once, which is almost never the right thing to do. Instead, a better approach here would have been to deprecate ``histogram`` and introduce a new function ``hist`` in its place. **Disallowing indexing with floats** Indexing an array with floats is asking for something ambiguous, and can be a sign of a bug in user code. After some discussion, it was deemed a good idea to deprecate indexing with floats. This was first tried for the v1.8.0 release, however in pre-release testing it became clear that this would break many libraries that depend on NumPy. Therefore it was reverted before release, to give those libraries time to fix their code first. It was finally introduced for v1.11.0 and turned into a hard error for v1.12.0. This change was disruptive, however it did catch real bugs in, e.g., SciPy and scikit-learn. Overall the change was worth the cost, and introducing it in master first to allow testing, then removing it again before a release, is a useful strategy. Similar deprecations that also look like good examples of cleanups/improvements: - removing deprecated boolean indexing (in 2016, see `gh-8312 < https://github.com/numpy/numpy/pull/8312>`__) - deprecating truth testing on empty arrays (in 2017, see `gh-9718 < https://github.com/numpy/numpy/pull/9718>`__) **Removing the financial functions** The financial functions (e.g. ``np.pmt``) had short non-descriptive names, were present in the main NumPy namespace, and didn't really fit well within NumPy's scope. They were added in 2008 after `a discussion < https://mail.python.org/pipermail/numpy-discussion/2008-April/032353.html>`_ on the mailing list where opinion was divided (but a majority in favor). The financial functions didn't cause a lot of overhead, however there were still multiple issues and PRs a year for them which cost maintainer time to deal with. And they cluttered up the ``numpy`` namespace. Discussion on removing them happened in 2013 (gh-2880, rejected) and then again in 2019 (:ref:`NEP32`, accepted without significant complaints). Given that they were clearly outside of NumPy's scope, moving them to a separate ``numpy-financial`` package and removing them from NumPy after a deprecation period made sense. Alternatives ------------ **Being more aggressive with deprecations.** The goal of being more aggressive is to allow NumPy to move forward faster. This would avoid others inventing their own solutions (often in multiple places), as well as be a benefit to users without a legacy code base. We reject this alternative because of the place NumPy has in the scientific Python ecosystem - being fairly conservative is required in order to not increase the extra maintenance for downstream libraries and end users to an unacceptable level. Discussion ---------- - `Mailing list discussion on the first version of this NEP in 2018 < https://mail.python.org/pipermail/numpy-discussion/2018-July/078432.html>`__ References and Footnotes ------------------------ - `Issue requesting semantic versioning < https://github.com/numpy/numpy/issues/10156>`__ .. [1] https://searchcode.com/ .. [2] https://github.com/Quansight-Labs/python-api-inspect .. [3] https://github.com/data-apis/python-record-api -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilhanpolat at gmail.com Wed Dec 30 10:04:51 2020 From: ilhanpolat at gmail.com (Ilhan Polat) Date: Wed, 30 Dec 2020 16:04:51 +0100 Subject: [Numpy-discussion] updated backwards compatibility and deprecation policy NEP In-Reply-To: References: Message-ID: Hi Ralf, This reads really nice. Thanks to everyone who contributed. Before nitpicking here and there, and sticking my head for others, is this is a finished discussion and only stylistic feedback is expected? Also is it preferred here or in the PR? GitHub is really not designed for extended discussions and here it if there are two subjects are discussed simultaneously it just becomes difficult to follow (maybe it's a bias due to my dislike of mailing lists). One of the less mentioned point is about what the tipping point is for the benefits outweighing the compatibility breakage sin and how to get a feeling for it. Because for a typical user, every break is just a break. Nobody will squint their eyes to see the reasoning behind it downstream. Thus this is more of a declaration of "yes as maintainers we are ready for facing the consequences but it had to be done because such and such". I am not asking to initiate a power discussion ala "who has the mod hammer" but rather what constitutes as a valid business case for a breakage proposal. A few generic lines about that would go a long way. Because we are in the same situation with scipy.linalg in which, what to do is crystal clear but how to do it without breaking anything is herding the cats hence I am genuinely curious how to go about this. Best, ilhan On Wed, Dec 30, 2020 at 3:07 PM Ralf Gommers wrote: > Hi all, > > Here is a long overdue update of the draft NEP about backwards > compatibility and deprecation policy: > https://github.com/numpy/numpy/pull/18097 > > - This is NEP 23: > https://numpy.org/neps/nep-0023-backwards-compatibility.html > - Link to the previous mailing list discussion: > https://mail.python.org/pipermail/numpy-discussion/2018-July/078432.html > > It would be nice to get this NEP to Accepted status. Main changes are: > > - Removed all examples that people objected to > - Removed all content regarding versioning > - Restructured sections, and added "Strategies related to deprecations" > (using suggestions by @njsmith and @shoyer). > - Added concrete examples of deprecations, and a more thorough description > of how to go about adding warnings incl. Sphinx directives, using > `stacklevel`, etc. > > As always, feedback here or on the PR is very welcome! > > Cheers, > Ralf > > > Abstract > -------- > > In this NEP we describe NumPy's approach to backwards compatibility, > its deprecation and removal policy, and the trade-offs and decision > processes for individual cases where breaking backwards compatibility > is considered. > > > Motivation and Scope > -------------------- > > NumPy has a very large user base. Those users rely on NumPy being stable > and the code they write that uses NumPy functionality to keep working. > NumPy is also actively maintained and improved -- and sometimes > improvements > require, or are made much easier by, breaking backwards compatibility. > Finally, there are trade-offs in stability for existing users vs. avoiding > errors or having a better user experience for new users. These competing > needs often give rise to long debates and to delays in accepting or > rejecting > contributions. This NEP tries to address that by providing a policy as > well > as examples and rationales for when it is or isn't a good idea to break > backwards compatibility. > > In scope for this NEP are: > > - Principles of NumPy's approach to backwards compatibility. > - How to deprecate functionality, and when to remove already deprecated > functionality. > - Decision making process for deprecations and removals. > > Out of scope are: > > - Making concrete decisions about deprecations of particular functionality. > - NumPy's versioning scheme. > > > General principles > ------------------ > > When considering proposed changes that are backwards incompatible, the > main principles the NumPy developers use when making a decision are: > > 1. Changes need to benefit users more than they harm them. > 2. NumPy is widely used so breaking changes should by default be assumed > to be > fairly harmful. > 3. Decisions should be based on data and actual effects on users and > downstream > packages rather than, e.g., appealing to the docs or for stylistic > reasons. > 4. Silently getting a wrong answer is much worse than getting a loud error. > > When assessing the costs of proposed changes, keep in mind that most users > do > not read the mailing list, do not look at deprecation warnings, and > sometimes > wait more than one or two years before upgrading from their old version. > And > that NumPy has millions of users, so "no one will do or use this" is very > likely incorrect. > > Benefits include improved functionality, usability and performance, as > well as > lower maintenance cost and improved future extensibility. > > Fixes for clear bugs are exempt from this backwards compatibility policy. > However in case of serious impact on users (e.g. a downstream library > doesn't > build anymore or would start giving incorrect results), even bug fixes may > have > to be delayed for one or more releases. > > > Strategies related to deprecations > ---------------------------------- > > Getting hard data on the impact of a deprecation of often difficult. > Strategies > that can be used to assess such impact include: > > - Use a code search engine ([1]_) or static ([2]_) or dynamic ([3]_) code > analysis tools to determine where and how the functionality is used. > - Testing prominent downstream libraries against a development build of > NumPy > containing the proposed change to get real-world data on its impact. > - Making a change in master and reverting it, if needed, before a release. > We > do encourage other packages to test against NumPy's master branch, so > this > often turns up issues quickly. > > If the impact is unclear or significant, it is often good to consider > alternatives to deprecations. For example discouraging use in documentation > only, or moving the documentation for the functionality to a less prominent > place or even removing it completely. Commenting on open issues related to > it > that they are low-prio or labeling them as "wontfix" will also be a signal > to > users, and reduce the maintenance effort needing to be spent. > > > Implementing deprecations and removals > -------------------------------------- > > Deprecation warnings are necessary in all cases where functionality > will eventually be removed. If there is no intent to remove functionality, > then it should not be deprecated either. A "please don't use this" in the > documentation or other type of warning should be used instead. > > Deprecations: > > - shall include the version number of the release in which the > functionality > was deprecated. > - shall include information on alternatives to the deprecated > functionality, or a > reason for the deprecation if no clear alternative is available. > - shall use ``VisibleDeprecationWarning`` rather than > ``DeprecationWarning`` > for cases of relevance to end users. For cases only relevant to > downstream libraries, a regular ``DeprecationWarning`` is fine. > *Rationale: regular deprecation warnings are invisible by default; > library > authors should be aware how deprecations work and test for them, but we > can't > expect this from all users.* > - shall be listed in the release notes of the release where the > deprecation is > first present. > - shall set a ``stacklevel``, so the warning appears to come from the > correct > place. > - shall be mentioned in the documentation for the functionality. A > ``.. deprecated::`` directive can be used for this. > > Examples of good deprecation warnings: > > .. code-block:: python > > warnings.warn('np.asscalar(a) is deprecated since NumPy 1.16.0, use ' > 'a.item() instead', DeprecationWarning, stacklevel=3) > > warnings.warn("Importing from numpy.testing.utils is deprecated " > "since 1.15.0, import from numpy.testing instead.", > DeprecationWarning, stacklevel=2) > > # A change in NumPy 1.14.0 for Python 3 loadtxt/genfromtext, slightly > # tweaked in this NEP (original didn't have version number). > warnings.warn( > "Reading unicode strings without specifying the encoding " > "argument is deprecated since NumPy 1.14.0. Set the encoding, " > "use None for the system default.", > np.VisibleDeprecationWarning, stacklevel=2) > > Removal of deprecated functionality: > > - shall be done after at least 2 releases (assuming the current 6-monthly > release cycle; if that changes, there shall be at least 1 year between > deprecation and removal). > - shall be listed in the release notes of the release where the removal > happened. > - can be done in any minor (but not bugfix) release. > > For backwards incompatible changes that aren't "deprecate and remove" but > for > which code will start behaving differently, a ``FutureWarning`` should be > used. Release notes, mentioning version number and using ``stacklevel`` > should > be done in the same way as for deprecation warnings. A ``.. > versionchanged::`` > directive can be used in the documentation to indicate when the behavior > changed: > > .. code-block:: python > > def argsort(self, axis=np._NoValue, ...): > """ > Parameters > ---------- > axis : int, optional > Axis along which to sort. If None, the default, the flattened > array > is used. > > .. versionchanged:: 1.13.0 > Previously, the default was documented to be -1, but that > was > in error. At some future date, the default will change to > -1, as > originally intended. > Until then, the axis should be given explicitly when > ``arr.ndim > 1``, to avoid a FutureWarning. > """ > ... > warnings.warn( > "In the future the default for argsort will be axis=-1, not > the " > "current None, to match its documentation and np.argsort. " > "Explicitly pass -1 or None to silence this warning.", > MaskedArrayFutureWarning, stacklevel=3) > > > Decision making > ~~~~~~~~~~~~~~~ > > In concrete cases where this policy needs to be applied, decisions are > made according > to the `NumPy governance model > `_. > > All deprecations must be proposed on the mailing list, in order to give > everyone > with an interest in NumPy development to be able to comment. Removal of > deprecated functionality does not need discussion on the mailing list. > > > Functionality with more strict deprecation policies > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > - ``numpy.random`` has its own backwards compatibility policy, > see `NEP 19 `_. > - The file format for ``.npy`` and ``.npz`` files must not be changed in a > backwards > incompatible way. > > > Example cases > ------------- > > We now discuss a few concrete examples from NumPy's history to illustrate > typical issues and trade-offs. > > **Changing the behavior of a function** > > ``np.histogram`` is probably the most infamous example. > First, a new keyword ``new=False`` was introduced, this was then switched > over to None one release later, and finally it was removed again. > Also, it has a ``normed`` keyword that had behavior that could be > considered > either suboptimal or broken (depending on ones opinion on the statistics). > A new keyword ``density`` was introduced to replace it; ``normed`` started > giving > ``DeprecationWarning`` only in v.1.15.0. Evolution of ``histogram``:: > > def histogram(a, bins=10, range=None, normed=False): # v1.0.0 > > def histogram(a, bins=10, range=None, normed=False, weights=None, > new=False): #v1.1.0 > > def histogram(a, bins=10, range=None, normed=False, weights=None, > new=None): #v1.2.0 > > def histogram(a, bins=10, range=None, normed=False, weights=None): > #v1.5.0 > > def histogram(a, bins=10, range=None, normed=False, weights=None, > density=None): #v1.6.0 > > def histogram(a, bins=10, range=None, normed=None, weights=None, > density=None): #v1.15.0 > # v1.15.0 was the first release where `normed` started emitting > # DeprecationWarnings > > The ``new`` keyword was planned from the start to be temporary. Such a > plan > forces users to change their code more than once, which is almost never the > right thing to do. Instead, a better approach here would have been to > deprecate ``histogram`` and introduce a new function ``hist`` in its place. > > > **Disallowing indexing with floats** > > Indexing an array with floats is asking for something ambiguous, and can > be a > sign of a bug in user code. After some discussion, it was deemed a good > idea > to deprecate indexing with floats. This was first tried for the v1.8.0 > release, however in pre-release testing it became clear that this would > break > many libraries that depend on NumPy. Therefore it was reverted before > release, > to give those libraries time to fix their code first. It was finally > introduced for v1.11.0 and turned into a hard error for v1.12.0. > > This change was disruptive, however it did catch real bugs in, e.g., SciPy > and > scikit-learn. Overall the change was worth the cost, and introducing it in > master first to allow testing, then removing it again before a release, is > a > useful strategy. > > Similar deprecations that also look like good examples of > cleanups/improvements: > > - removing deprecated boolean indexing (in 2016, see `gh-8312 < > https://github.com/numpy/numpy/pull/8312>`__) > - deprecating truth testing on empty arrays (in 2017, see `gh-9718 < > https://github.com/numpy/numpy/pull/9718>`__) > > > **Removing the financial functions** > > The financial functions (e.g. ``np.pmt``) had short non-descriptive names, > were > present in the main NumPy namespace, and didn't really fit well within > NumPy's > scope. They were added in 2008 after > `a discussion < > https://mail.python.org/pipermail/numpy-discussion/2008-April/032353.html > >`_ > on the mailing list where opinion was divided (but a majority in favor). > The financial functions didn't cause a lot of overhead, however there were > still multiple issues and PRs a year for them which cost maintainer time to > deal with. And they cluttered up the ``numpy`` namespace. Discussion on > removing them happened in 2013 (gh-2880, rejected) and then again in 2019 > (:ref:`NEP32`, accepted without significant complaints). > > Given that they were clearly outside of NumPy's scope, moving them to a > separate ``numpy-financial`` package and removing them from NumPy after a > deprecation period made sense. > > > Alternatives > ------------ > > **Being more aggressive with deprecations.** > > The goal of being more aggressive is to allow NumPy to move forward faster. > This would avoid others inventing their own solutions (often in multiple > places), as well as be a benefit to users without a legacy code base. We > reject this alternative because of the place NumPy has in the scientific > Python > ecosystem - being fairly conservative is required in order to not increase > the > extra maintenance for downstream libraries and end users to an unacceptable > level. > > > Discussion > ---------- > > - `Mailing list discussion on the first version of this NEP in 2018 < > https://mail.python.org/pipermail/numpy-discussion/2018-July/078432.html > >`__ > > > References and Footnotes > ------------------------ > > - `Issue requesting semantic versioning < > https://github.com/numpy/numpy/issues/10156>`__ > > .. [1] https://searchcode.com/ > > .. [2] https://github.com/Quansight-Labs/python-api-inspect > > .. [3] https://github.com/data-apis/python-record-api > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Wed Dec 30 10:27:38 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Wed, 30 Dec 2020 16:27:38 +0100 Subject: [Numpy-discussion] updated backwards compatibility and deprecation policy NEP In-Reply-To: References: Message-ID: On Wed, Dec 30, 2020 at 4:05 PM Ilhan Polat wrote: > Hi Ralf, > > This reads really nice. Thanks to everyone who contributed. > > Before nitpicking here and there, and sticking my head for others, is this > is a finished discussion and only stylistic feedback is expected? > It's not. I removed everything that was controversial last time around and only added things that are basically the way we already do things, so I don't really expect major issues. But it is a big rewrite, so some discussion is certainly expected. Also is it preferred here or in the PR? GitHub is really not designed for > extended discussions and here it if there are two subjects are discussed > simultaneously it just becomes difficult to follow (maybe it's a bias due > to my dislike of mailing lists). > Agreed. The idea is we post NEPs to the mailing list, and major issues get discussed here. If there's smaller comments like stylistic things or small errors, commenting on the PR for those is much easier. > One of the less mentioned point is about what the tipping point is for the > benefits outweighing the compatibility breakage sin and how to get a > feeling for it. Because for a typical user, every break is just a break. > Nobody will squint their eyes to see the reasoning behind it downstream. > Thus this is more of a declaration of "yes as maintainers we are ready for > facing the consequences but it had to be done because such and such". > That's very hard to describe, since it relies so much on previous experience and qualitative judgements. That's the main reason why I had more examples before, but they just led to more discussion about those examples - so that didn't quite have the intended effect. I am not asking to initiate a power discussion ala "who has the mod hammer" > but rather what constitutes as a valid business case for a breakage > proposal. A few generic lines about that would go a long way. Because we > are in the same situation with scipy.linalg in which, what to do is crystal > clear but how to do it without breaking anything is herding the cats hence > I am genuinely curious how to go about this. > If anyone has a good proposal, that'd be great. But I find it hard to come up with those few lines right now. Cheers, Ralf > Best, > ilhan > > > On Wed, Dec 30, 2020 at 3:07 PM Ralf Gommers > wrote: > >> Hi all, >> >> Here is a long overdue update of the draft NEP about backwards >> compatibility and deprecation policy: >> https://github.com/numpy/numpy/pull/18097 >> >> - This is NEP 23: >> https://numpy.org/neps/nep-0023-backwards-compatibility.html >> - Link to the previous mailing list discussion: >> https://mail.python.org/pipermail/numpy-discussion/2018-July/078432.html >> >> It would be nice to get this NEP to Accepted status. Main changes are: >> >> - Removed all examples that people objected to >> - Removed all content regarding versioning >> - Restructured sections, and added "Strategies related to deprecations" >> (using suggestions by @njsmith and @shoyer). >> - Added concrete examples of deprecations, and a more thorough >> description of how to go about adding warnings incl. Sphinx directives, >> using `stacklevel`, etc. >> >> As always, feedback here or on the PR is very welcome! >> >> Cheers, >> Ralf >> >> >> Abstract >> -------- >> >> In this NEP we describe NumPy's approach to backwards compatibility, >> its deprecation and removal policy, and the trade-offs and decision >> processes for individual cases where breaking backwards compatibility >> is considered. >> >> >> Motivation and Scope >> -------------------- >> >> NumPy has a very large user base. Those users rely on NumPy being stable >> and the code they write that uses NumPy functionality to keep working. >> NumPy is also actively maintained and improved -- and sometimes >> improvements >> require, or are made much easier by, breaking backwards compatibility. >> Finally, there are trade-offs in stability for existing users vs. avoiding >> errors or having a better user experience for new users. These competing >> needs often give rise to long debates and to delays in accepting or >> rejecting >> contributions. This NEP tries to address that by providing a policy as >> well >> as examples and rationales for when it is or isn't a good idea to break >> backwards compatibility. >> >> In scope for this NEP are: >> >> - Principles of NumPy's approach to backwards compatibility. >> - How to deprecate functionality, and when to remove already deprecated >> functionality. >> - Decision making process for deprecations and removals. >> >> Out of scope are: >> >> - Making concrete decisions about deprecations of particular >> functionality. >> - NumPy's versioning scheme. >> >> >> General principles >> ------------------ >> >> When considering proposed changes that are backwards incompatible, the >> main principles the NumPy developers use when making a decision are: >> >> 1. Changes need to benefit users more than they harm them. >> 2. NumPy is widely used so breaking changes should by default be assumed >> to be >> fairly harmful. >> 3. Decisions should be based on data and actual effects on users and >> downstream >> packages rather than, e.g., appealing to the docs or for stylistic >> reasons. >> 4. Silently getting a wrong answer is much worse than getting a loud >> error. >> >> When assessing the costs of proposed changes, keep in mind that most >> users do >> not read the mailing list, do not look at deprecation warnings, and >> sometimes >> wait more than one or two years before upgrading from their old version. >> And >> that NumPy has millions of users, so "no one will do or use this" is very >> likely incorrect. >> >> Benefits include improved functionality, usability and performance, as >> well as >> lower maintenance cost and improved future extensibility. >> >> Fixes for clear bugs are exempt from this backwards compatibility policy. >> However in case of serious impact on users (e.g. a downstream library >> doesn't >> build anymore or would start giving incorrect results), even bug fixes >> may have >> to be delayed for one or more releases. >> >> >> Strategies related to deprecations >> ---------------------------------- >> >> Getting hard data on the impact of a deprecation of often difficult. >> Strategies >> that can be used to assess such impact include: >> >> - Use a code search engine ([1]_) or static ([2]_) or dynamic ([3]_) code >> analysis tools to determine where and how the functionality is used. >> - Testing prominent downstream libraries against a development build of >> NumPy >> containing the proposed change to get real-world data on its impact. >> - Making a change in master and reverting it, if needed, before a >> release. We >> do encourage other packages to test against NumPy's master branch, so >> this >> often turns up issues quickly. >> >> If the impact is unclear or significant, it is often good to consider >> alternatives to deprecations. For example discouraging use in >> documentation >> only, or moving the documentation for the functionality to a less >> prominent >> place or even removing it completely. Commenting on open issues related >> to it >> that they are low-prio or labeling them as "wontfix" will also be a >> signal to >> users, and reduce the maintenance effort needing to be spent. >> >> >> Implementing deprecations and removals >> -------------------------------------- >> >> Deprecation warnings are necessary in all cases where functionality >> will eventually be removed. If there is no intent to remove >> functionality, >> then it should not be deprecated either. A "please don't use this" in the >> documentation or other type of warning should be used instead. >> >> Deprecations: >> >> - shall include the version number of the release in which the >> functionality >> was deprecated. >> - shall include information on alternatives to the deprecated >> functionality, or a >> reason for the deprecation if no clear alternative is available. >> - shall use ``VisibleDeprecationWarning`` rather than >> ``DeprecationWarning`` >> for cases of relevance to end users. For cases only relevant to >> downstream libraries, a regular ``DeprecationWarning`` is fine. >> *Rationale: regular deprecation warnings are invisible by default; >> library >> authors should be aware how deprecations work and test for them, but we >> can't >> expect this from all users.* >> - shall be listed in the release notes of the release where the >> deprecation is >> first present. >> - shall set a ``stacklevel``, so the warning appears to come from the >> correct >> place. >> - shall be mentioned in the documentation for the functionality. A >> ``.. deprecated::`` directive can be used for this. >> >> Examples of good deprecation warnings: >> >> .. code-block:: python >> >> warnings.warn('np.asscalar(a) is deprecated since NumPy 1.16.0, use ' >> 'a.item() instead', DeprecationWarning, stacklevel=3) >> >> warnings.warn("Importing from numpy.testing.utils is deprecated " >> "since 1.15.0, import from numpy.testing instead.", >> DeprecationWarning, stacklevel=2) >> >> # A change in NumPy 1.14.0 for Python 3 loadtxt/genfromtext, slightly >> # tweaked in this NEP (original didn't have version number). >> warnings.warn( >> "Reading unicode strings without specifying the encoding " >> "argument is deprecated since NumPy 1.14.0. Set the encoding, " >> "use None for the system default.", >> np.VisibleDeprecationWarning, stacklevel=2) >> >> Removal of deprecated functionality: >> >> - shall be done after at least 2 releases (assuming the current 6-monthly >> release cycle; if that changes, there shall be at least 1 year between >> deprecation and removal). >> - shall be listed in the release notes of the release where the removal >> happened. >> - can be done in any minor (but not bugfix) release. >> >> For backwards incompatible changes that aren't "deprecate and remove" but >> for >> which code will start behaving differently, a ``FutureWarning`` should be >> used. Release notes, mentioning version number and using ``stacklevel`` >> should >> be done in the same way as for deprecation warnings. A ``.. >> versionchanged::`` >> directive can be used in the documentation to indicate when the behavior >> changed: >> >> .. code-block:: python >> >> def argsort(self, axis=np._NoValue, ...): >> """ >> Parameters >> ---------- >> axis : int, optional >> Axis along which to sort. If None, the default, the flattened >> array >> is used. >> >> .. versionchanged:: 1.13.0 >> Previously, the default was documented to be -1, but that >> was >> in error. At some future date, the default will change to >> -1, as >> originally intended. >> Until then, the axis should be given explicitly when >> ``arr.ndim > 1``, to avoid a FutureWarning. >> """ >> ... >> warnings.warn( >> "In the future the default for argsort will be axis=-1, not >> the " >> "current None, to match its documentation and np.argsort. " >> "Explicitly pass -1 or None to silence this warning.", >> MaskedArrayFutureWarning, stacklevel=3) >> >> >> Decision making >> ~~~~~~~~~~~~~~~ >> >> In concrete cases where this policy needs to be applied, decisions are >> made according >> to the `NumPy governance model >> `_. >> >> All deprecations must be proposed on the mailing list, in order to give >> everyone >> with an interest in NumPy development to be able to comment. Removal of >> deprecated functionality does not need discussion on the mailing list. >> >> >> Functionality with more strict deprecation policies >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> - ``numpy.random`` has its own backwards compatibility policy, >> see `NEP 19 `_. >> - The file format for ``.npy`` and ``.npz`` files must not be changed in >> a backwards >> incompatible way. >> >> >> Example cases >> ------------- >> >> We now discuss a few concrete examples from NumPy's history to illustrate >> typical issues and trade-offs. >> >> **Changing the behavior of a function** >> >> ``np.histogram`` is probably the most infamous example. >> First, a new keyword ``new=False`` was introduced, this was then switched >> over to None one release later, and finally it was removed again. >> Also, it has a ``normed`` keyword that had behavior that could be >> considered >> either suboptimal or broken (depending on ones opinion on the statistics). >> A new keyword ``density`` was introduced to replace it; ``normed`` >> started giving >> ``DeprecationWarning`` only in v.1.15.0. Evolution of ``histogram``:: >> >> def histogram(a, bins=10, range=None, normed=False): # v1.0.0 >> >> def histogram(a, bins=10, range=None, normed=False, weights=None, >> new=False): #v1.1.0 >> >> def histogram(a, bins=10, range=None, normed=False, weights=None, >> new=None): #v1.2.0 >> >> def histogram(a, bins=10, range=None, normed=False, weights=None): >> #v1.5.0 >> >> def histogram(a, bins=10, range=None, normed=False, weights=None, >> density=None): #v1.6.0 >> >> def histogram(a, bins=10, range=None, normed=None, weights=None, >> density=None): #v1.15.0 >> # v1.15.0 was the first release where `normed` started emitting >> # DeprecationWarnings >> >> The ``new`` keyword was planned from the start to be temporary. Such a >> plan >> forces users to change their code more than once, which is almost never >> the >> right thing to do. Instead, a better approach here would have been to >> deprecate ``histogram`` and introduce a new function ``hist`` in its >> place. >> >> >> **Disallowing indexing with floats** >> >> Indexing an array with floats is asking for something ambiguous, and can >> be a >> sign of a bug in user code. After some discussion, it was deemed a good >> idea >> to deprecate indexing with floats. This was first tried for the v1.8.0 >> release, however in pre-release testing it became clear that this would >> break >> many libraries that depend on NumPy. Therefore it was reverted before >> release, >> to give those libraries time to fix their code first. It was finally >> introduced for v1.11.0 and turned into a hard error for v1.12.0. >> >> This change was disruptive, however it did catch real bugs in, e.g., >> SciPy and >> scikit-learn. Overall the change was worth the cost, and introducing it >> in >> master first to allow testing, then removing it again before a release, >> is a >> useful strategy. >> >> Similar deprecations that also look like good examples of >> cleanups/improvements: >> >> - removing deprecated boolean indexing (in 2016, see `gh-8312 < >> https://github.com/numpy/numpy/pull/8312>`__) >> - deprecating truth testing on empty arrays (in 2017, see `gh-9718 < >> https://github.com/numpy/numpy/pull/9718>`__) >> >> >> **Removing the financial functions** >> >> The financial functions (e.g. ``np.pmt``) had short non-descriptive >> names, were >> present in the main NumPy namespace, and didn't really fit well within >> NumPy's >> scope. They were added in 2008 after >> `a discussion < >> https://mail.python.org/pipermail/numpy-discussion/2008-April/032353.html >> >`_ >> on the mailing list where opinion was divided (but a majority in favor). >> The financial functions didn't cause a lot of overhead, however there were >> still multiple issues and PRs a year for them which cost maintainer time >> to >> deal with. And they cluttered up the ``numpy`` namespace. Discussion on >> removing them happened in 2013 (gh-2880, rejected) and then again in 2019 >> (:ref:`NEP32`, accepted without significant complaints). >> >> Given that they were clearly outside of NumPy's scope, moving them to a >> separate ``numpy-financial`` package and removing them from NumPy after a >> deprecation period made sense. >> >> >> Alternatives >> ------------ >> >> **Being more aggressive with deprecations.** >> >> The goal of being more aggressive is to allow NumPy to move forward >> faster. >> This would avoid others inventing their own solutions (often in multiple >> places), as well as be a benefit to users without a legacy code base. We >> reject this alternative because of the place NumPy has in the scientific >> Python >> ecosystem - being fairly conservative is required in order to not >> increase the >> extra maintenance for downstream libraries and end users to an >> unacceptable >> level. >> >> >> Discussion >> ---------- >> >> - `Mailing list discussion on the first version of this NEP in 2018 < >> https://mail.python.org/pipermail/numpy-discussion/2018-July/078432.html >> >`__ >> >> >> References and Footnotes >> ------------------------ >> >> - `Issue requesting semantic versioning < >> https://github.com/numpy/numpy/issues/10156>`__ >> >> .. [1] https://searchcode.com/ >> >> .. [2] https://github.com/Quansight-Labs/python-api-inspect >> >> .. [3] https://github.com/data-apis/python-record-api >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Wed Dec 30 12:43:55 2020 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Wed, 30 Dec 2020 11:43:55 -0600 Subject: [Numpy-discussion] updated backwards compatibility and deprecation policy NEP In-Reply-To: References: Message-ID: On Wed, 2020-12-30 at 16:27 +0100, Ralf Gommers wrote: > On Wed, Dec 30, 2020 at 4:05 PM Ilhan Polat > wrote: > > > Hi Ralf, > > > > This reads really nice. Thanks to everyone who contributed. > > > > Before nitpicking here and there, and sticking my head for others, > > is this > > is a finished discussion and only stylistic feedback is expected? > > Thanks Ralf, I will look at it more carefully only next year probably. > > It's not. I removed everything that was controversial last time > around and > only added things that are basically the way we already do things, so > I > don't really expect major issues. But it is a big rewrite, so some > discussion is certainly expected. > > > > That's very hard to describe, since it relies so much on previous > experience and qualitative judgements. That's the main reason why I > had > more examples before, but they just led to more discussion about > those > examples - so that didn't quite have the intended effect. One thing I thought could be useful here is to use quality management/assurance techniques that are typical (at least in Europe to my knowledge) for pretty much every product development (I do _not_ mean software specific Q&A, which has a different ISO, that I doubt helps). I only took a short course and used this very little. I am sure there are many here with industry experience where the use of Q&A is every day work. One concept from there is to create a risk/danger and probability assessment, which can be ad-hoc for your product. An example just to make something up: Developing a chair, possibility: leg breaks. Likelyhood (based on current design): [Moderate] someone puts something too heavy on it Danger: Serious risk of large injury [high] You then say that (rows and columns are danger and likelyhood): low moderate high low OK OK not OK moderate OK not OK not OK high not OK not OK not OK (low danger could for example be a splinter, its OK if it happens sometimes, but you don't want it to happen to many customers.) Now in the above case, you get into the "not OK" column, so you will try to mitigate (reinforce the chair, print a maximum weight on it, maybe you also have to discuss it away as an unavoidable risk). For us, this would translate to number of users affected and how badly they (can be) affected, probably. And since I don't like a "this change can never happen", the lower part of the triangle would probably just be "requires a NEP" (In my opinion, I realize that some things are probably truly impossible, but in that case a NEP won't fly either). This is just an idea that I think could very much be helpful. The table needs to be filled with rough examples, but it would be completely fine to not even fill it completely IMO. There are also tricky things like a two release policy (which could be part of a "mitigation", lowering the likelyhood or danger but I am not certain it fits well). (I think the example tables usually had 4 columns/rows, but I don't remember) This felt very ad-hoc to me when I first learned about it and of course it is not always clear if something is low or moderate risk. But I do like that it gives *some* formalization. Note that IIRC the ISO standard does not even attempt to say what categories a specific product development should use. (I think this is all ISO 9000, but I am did not double check and just to note, ISO norms are fairly expensive unless you live in India.) Cheers, Sebastian > > I am not asking to initiate a power discussion ala "who has the mod > hammer" > > but rather what constitutes as a valid business case for a breakage > > proposal. A few generic lines about that would go a long way. > > Because we > > are in the same situation with scipy.linalg in which, what to do is > > crystal > > clear but how to do it without breaking anything is herding the > > cats hence > > I am genuinely curious how to go about this. > > > > If anyone has a good proposal, that'd be great. But I find it hard to > come > up with those few lines right now. > > Cheers, > Ralf > > > > > Best, > > ilhan > > > > > > On Wed, Dec 30, 2020 at 3:07 PM Ralf Gommers < > > ralf.gommers at gmail.com> > > wrote: > > > > > Hi all, > > > > > > Here is a long overdue update of the draft NEP about backwards > > > compatibility and deprecation policy: > > > https://github.com/numpy/numpy/pull/18097 > > > > > > - This is NEP 23: > > > https://numpy.org/neps/nep-0023-backwards-compatibility.html > > > - Link to the previous mailing list discussion: > > > https://mail.python.org/pipermail/numpy-discussion/2018-July/078432.html > > > > > > It would be nice to get this NEP to Accepted status. Main changes > > > are: > > > > > > - Removed all examples that people objected to > > > - Removed all content regarding versioning > > > - Restructured sections, and added "Strategies related to > > > deprecations" > > > (using suggestions by @njsmith and @shoyer). > > > - Added concrete examples of deprecations, and a more thorough > > > description of how to go about adding warnings incl. Sphinx > > > directives, > > > using `stacklevel`, etc. > > > > > > As always, feedback here or on the PR is very welcome! > > > > > > Cheers, > > > Ralf > > > > > > > > > Abstract > > > -------- > > > > > > In this NEP we describe NumPy's approach to backwards > > > compatibility, > > > its deprecation and removal policy, and the trade-offs and > > > decision > > > processes for individual cases where breaking backwards > > > compatibility > > > is considered. > > > > > > > > > Motivation and Scope > > > -------------------- > > > > > > NumPy has a very large user base.? Those users rely on NumPy > > > being stable > > > and the code they write that uses NumPy functionality to keep > > > working. > > > NumPy is also actively maintained and improved -- and sometimes > > > improvements > > > require, or are made much easier by, breaking backwards > > > compatibility. > > > Finally, there are trade-offs in stability for existing users vs. > > > avoiding > > > errors or having a better user experience for new users.? These > > > competing > > > needs often give rise to long debates and to delays in accepting > > > or > > > rejecting > > > contributions.? This NEP tries to address that by providing a > > > policy as > > > well > > > as examples and rationales for when it is or isn't a good idea to > > > break > > > backwards compatibility. > > > > > > In scope for this NEP are: > > > > > > - Principles of NumPy's approach to backwards compatibility. > > > - How to deprecate functionality, and when to remove already > > > deprecated > > > ? functionality. > > > - Decision making process for deprecations and removals. > > > > > > Out of scope are: > > > > > > - Making concrete decisions about deprecations of particular > > > functionality. > > > - NumPy's versioning scheme. > > > > > > > > > General principles > > > ------------------ > > > > > > When considering proposed changes that are backwards > > > incompatible, the > > > main principles the NumPy developers use when making a decision > > > are: > > > > > > 1. Changes need to benefit users more than they harm them. > > > 2. NumPy is widely used so breaking changes should by default be > > > assumed > > > to be > > > ?? fairly harmful. > > > 3. Decisions should be based on data and actual effects on users > > > and > > > downstream > > > ?? packages rather than, e.g., appealing to the docs or for > > > stylistic > > > reasons. > > > 4. Silently getting a wrong answer is much worse than getting a > > > loud > > > error. > > > > > > When assessing the costs of proposed changes, keep in mind that > > > most > > > users do > > > not read the mailing list, do not look at deprecation warnings, > > > and > > > sometimes > > > wait more than one or two years before upgrading from their old > > > version. > > > And > > > that NumPy has millions of users, so "no one will do or use this" > > > is very > > > likely incorrect. > > > > > > Benefits include improved functionality, usability and > > > performance, as > > > well as > > > lower maintenance cost and improved future extensibility. > > > > > > Fixes for clear bugs are exempt from this backwards compatibility > > > policy. > > > However in case of serious impact on users (e.g. a downstream > > > library > > > doesn't > > > build anymore or would start giving incorrect results), even bug > > > fixes > > > may have > > > to be delayed for one or more releases. > > > > > > > > > Strategies related to deprecations > > > ---------------------------------- > > > > > > Getting hard data on the impact of a deprecation of often > > > difficult. > > > Strategies > > > that can be used to assess such impact include: > > > > > > - Use a code search engine ([1]_) or static ([2]_) or dynamic > > > ([3]_) code > > > ? analysis tools to determine where and how the functionality is > > > used. > > > - Testing prominent downstream libraries against a development > > > build of > > > NumPy > > > ? containing the proposed change to get real-world data on its > > > impact. > > > - Making a change in master and reverting it, if needed, before a > > > release. We > > > ? do encourage other packages to test against NumPy's master > > > branch, so > > > this > > > ? often turns up issues quickly. > > > > > > If the impact is unclear or significant, it is often good to > > > consider > > > alternatives to deprecations. For example discouraging use in > > > documentation > > > only, or moving the documentation for the functionality to a less > > > prominent > > > place or even removing it completely. Commenting on open issues > > > related > > > to it > > > that they are low-prio or labeling them as "wontfix" will also be > > > a > > > signal to > > > users, and reduce the maintenance effort needing to be spent. > > > > > > > > > Implementing deprecations and removals > > > -------------------------------------- > > > > > > Deprecation warnings are necessary in all cases where > > > functionality > > > will eventually be removed.? If there is no intent to remove > > > functionality, > > > then it should not be deprecated either. A "please don't use > > > this" in the > > > documentation or other type of warning should be used instead. > > > > > > Deprecations: > > > > > > - shall include the version number of the release in which the > > > functionality > > > ? was deprecated. > > > - shall include information on alternatives to the deprecated > > > functionality, or a > > > ? reason for the deprecation if no clear alternative is > > > available. > > > - shall use ``VisibleDeprecationWarning`` rather than > > > ``DeprecationWarning`` > > > ? for cases of relevance to end users. For cases only relevant to > > > ? downstream libraries, a regular ``DeprecationWarning`` is fine. > > > ? *Rationale: regular deprecation warnings are invisible by > > > default; > > > library > > > ? authors should be aware how deprecations work and test for > > > them, but we > > > can't > > > ? expect this from all users.* > > > - shall be listed in the release notes of the release where the > > > deprecation is > > > ? first present. > > > - shall set a ``stacklevel``, so the warning appears to come from > > > the > > > correct > > > ? place. > > > - shall be mentioned in the documentation for the functionality. > > > A > > > ? ``.. deprecated::`` directive can be used for this. > > > > > > Examples of good deprecation warnings: > > > > > > .. code-block:: python > > > > > > ??? warnings.warn('np.asscalar(a) is deprecated since NumPy > > > 1.16.0, use ' > > > ????????????????? 'a.item() instead', DeprecationWarning, > > > stacklevel=3) > > > > > > ??? warnings.warn("Importing from numpy.testing.utils is > > > deprecated " > > > ????????????????? "since 1.15.0, import from numpy.testing > > > instead.", > > > ????????????????? DeprecationWarning, stacklevel=2) > > > > > > ??? # A change in NumPy 1.14.0 for Python 3 loadtxt/genfromtext, > > > slightly > > > ??? # tweaked in this NEP (original didn't have version number). > > > ??? warnings.warn( > > > ??????? "Reading unicode strings without specifying the encoding > > > " > > > ??????? "argument is deprecated since NumPy 1.14.0. Set the > > > encoding, " > > > ??????? "use None for the system default.", > > > ??????? np.VisibleDeprecationWarning, stacklevel=2) > > > > > > Removal of deprecated functionality: > > > > > > - shall be done after at least 2 releases (assuming the current > > > 6-monthly > > > ? release cycle; if that changes, there shall be at least 1 year > > > between > > > ? deprecation and removal). > > > - shall be listed in the release notes of the release where the > > > removal > > > happened. > > > - can be done in any minor (but not bugfix) release. > > > > > > For backwards incompatible changes that aren't "deprecate and > > > remove" but > > > for > > > which code will start behaving differently, a ``FutureWarning`` > > > should be > > > used. Release notes, mentioning version number and using > > > ``stacklevel`` > > > should > > > be done in the same way as for deprecation warnings. A ``.. > > > versionchanged::`` > > > directive can be used in the documentation to indicate when the > > > behavior > > > changed: > > > > > > .. code-block:: python > > > > > > ??? def argsort(self, axis=np._NoValue, ...): > > > ??????? """ > > > ??????? Parameters > > > ??????? ---------- > > > ??????? axis : int, optional > > > ??????????? Axis along which to sort. If None, the default, the > > > flattened > > > array > > > ??????????? is used. > > > > > > ??????????? ..? versionchanged:: 1.13.0 > > > ??????????????? Previously, the default was documented to be -1, > > > but that > > > was > > > ??????????????? in error. At some future date, the default will > > > change to > > > -1, as > > > ??????????????? originally intended. > > > ??????????????? Until then, the axis should be given explicitly > > > when > > > ??????????????? ``arr.ndim > 1``, to avoid a FutureWarning. > > > ??????? """ > > > ??????? ... > > > ??????? warnings.warn( > > > ??????????? "In the future the default for argsort will be axis=- > > > 1, not > > > the " > > > ??????????? "current None, to match its documentation and > > > np.argsort. " > > > ??????????? "Explicitly pass -1 or None to silence this > > > warning.", > > > ??????????? MaskedArrayFutureWarning, stacklevel=3) > > > > > > > > > Decision making > > > ~~~~~~~~~~~~~~~ > > > > > > In concrete cases where this policy needs to be applied, > > > decisions are > > > made according > > > to the `NumPy governance model > > > `_. > > > > > > All deprecations must be proposed on the mailing list, in order > > > to give > > > everyone > > > with an interest in NumPy development to be able to comment. > > > Removal of > > > deprecated functionality does not need discussion on the mailing > > > list. > > > > > > > > > Functionality with more strict deprecation policies > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > > > - ``numpy.random`` has its own backwards compatibility policy, > > > ? see `NEP 19 > > > `_. > > > - The file format for ``.npy`` and ``.npz`` files must not be > > > changed in > > > a backwards > > > ? incompatible way. > > > > > > > > > Example cases > > > ------------- > > > > > > We now discuss a few concrete examples from NumPy's history to > > > illustrate > > > typical issues and trade-offs. > > > > > > **Changing the behavior of a function** > > > > > > ``np.histogram`` is probably the most infamous example. > > > First, a new keyword ``new=False`` was introduced, this was then > > > switched > > > over to None one release later, and finally it was removed again. > > > Also, it has a ``normed`` keyword that had behavior that could be > > > considered > > > either suboptimal or broken (depending on ones opinion on the > > > statistics). > > > A new keyword ``density`` was introduced to replace it; > > > ``normed`` > > > started giving > > > ``DeprecationWarning`` only in v.1.15.0.? Evolution of > > > ``histogram``:: > > > > > > ??? def histogram(a, bins=10, range=None, normed=False):? # > > > v1.0.0 > > > > > > ??? def histogram(a, bins=10, range=None, normed=False, > > > weights=None, > > > new=False):? #v1.1.0 > > > > > > ??? def histogram(a, bins=10, range=None, normed=False, > > > weights=None, > > > new=None):? #v1.2.0 > > > > > > ??? def histogram(a, bins=10, range=None, normed=False, > > > weights=None): > > > ?#v1.5.0 > > > > > > ??? def histogram(a, bins=10, range=None, normed=False, > > > weights=None, > > > density=None):? #v1.6.0 > > > > > > ??? def histogram(a, bins=10, range=None, normed=None, > > > weights=None, > > > density=None):? #v1.15.0 > > > ??????? # v1.15.0 was the first release where `normed` started > > > emitting > > > ??????? # DeprecationWarnings > > > > > > The ``new`` keyword was planned from the start to be temporary.? > > > Such a > > > plan > > > forces users to change their code more than once, which is almost > > > never > > > the > > > right thing to do.? Instead, a better approach here would have > > > been to > > > deprecate ``histogram`` and introduce a new function ``hist`` in > > > its > > > place. > > > > > > > > > **Disallowing indexing with floats** > > > > > > Indexing an array with floats is asking for something ambiguous, > > > and can > > > be a > > > sign of a bug in user code.? After some discussion, it was deemed > > > a good > > > idea > > > to deprecate indexing with floats.? This was first tried for the > > > v1.8.0 > > > release, however in pre-release testing it became clear that this > > > would > > > break > > > many libraries that depend on NumPy.? Therefore it was reverted > > > before > > > release, > > > to give those libraries time to fix their code first.? It was > > > finally > > > introduced for v1.11.0 and turned into a hard error for v1.12.0. > > > > > > This change was disruptive, however it did catch real bugs in, > > > e.g., > > > SciPy and > > > scikit-learn.? Overall the change was worth the cost, and > > > introducing it > > > in > > > master first to allow testing, then removing it again before a > > > release, > > > is a > > > useful strategy. > > > > > > Similar deprecations that also look like good examples of > > > cleanups/improvements: > > > > > > - removing deprecated boolean indexing (in 2016, see `gh-8312 < > > > https://github.com/numpy/numpy/pull/8312>`__) > > > - deprecating truth testing on empty arrays (in 2017, see `gh- > > > 9718 < > > > https://github.com/numpy/numpy/pull/9718>`__) > > > > > > > > > **Removing the financial functions** > > > > > > The financial functions (e.g. ``np.pmt``) had short non- > > > descriptive > > > names, were > > > present in the main NumPy namespace, and didn't really fit well > > > within > > > NumPy's > > > scope.? They were added in 2008 after > > > `a discussion < > > > https://mail.python.org/pipermail/numpy-discussion/2008-April/032353.html > > > > `_ > > > on the mailing list where opinion was divided (but a majority in > > > favor). > > > The financial functions didn't cause a lot of overhead, however > > > there were > > > still multiple issues and PRs a year for them which cost > > > maintainer time > > > to > > > deal with.? And they cluttered up the ``numpy`` namespace.? > > > Discussion on > > > removing them happened in 2013 (gh-2880, rejected) and then again > > > in 2019 > > > (:ref:`NEP32`, accepted without significant complaints). > > > > > > Given that they were clearly outside of NumPy's scope, moving > > > them to a > > > separate ``numpy-financial`` package and removing them from NumPy > > > after a > > > deprecation period made sense. > > > > > > > > > Alternatives > > > ------------ > > > > > > **Being more aggressive with deprecations.** > > > > > > The goal of being more aggressive is to allow NumPy to move > > > forward > > > faster. > > > This would avoid others inventing their own solutions (often in > > > multiple > > > places), as well as be a benefit to users without a legacy code > > > base.? We > > > reject this alternative because of the place NumPy has in the > > > scientific > > > Python > > > ecosystem - being fairly conservative is required in order to not > > > increase the > > > extra maintenance for downstream libraries and end users to an > > > unacceptable > > > level. > > > > > > > > > Discussion > > > ---------- > > > > > > - `Mailing list discussion on the first version of this NEP in > > > 2018 < > > > https://mail.python.org/pipermail/numpy-discussion/2018-July/078432.html > > > > `__ > > > > > > > > > References and Footnotes > > > ------------------------ > > > > > > - `Issue requesting semantic versioning < > > > https://github.com/numpy/numpy/issues/10156>`__ > > > > > > .. [1] https://searchcode.com/ > > > > > > .. [2] https://github.com/Quansight-Labs/python-api-inspect > > > > > > .. [3] https://github.com/data-apis/python-record-api > > > > > > > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion at python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From blkzol001 at myuct.ac.za Thu Dec 31 08:26:34 2020 From: blkzol001 at myuct.ac.za (zoj613) Date: Thu, 31 Dec 2020 06:26:34 -0700 (MST) Subject: [Numpy-discussion] Addition of new distributions: Polya-gamma In-Reply-To: References: Message-ID: <1609421194605-0.post@n7.nabble.com> Thanks for the insightful comments. I totally understand the concerns with maintenance. After some thought, I agree that maybe it would be best to have it as a separate package and hopefully get it added to scipy in the future? I will continue development at https://github.com/zoj613/polya-gamma for those interested in using the code or contributing. -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ From tyler.je.reddy at gmail.com Thu Dec 31 10:58:17 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Thu, 31 Dec 2020 08:58:17 -0700 Subject: [Numpy-discussion] ANN: SciPy 1.6.0 Message-ID: Hi all, On behalf of the SciPy development team I'm pleased to announce the release of SciPy 1.6.0. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.6.0 One of a few ways to install this release with pip: pip install scipy==1.6.0 ===================== SciPy 1.6.0 Release Notes ===================== SciPy 1.6.0 is the culmination of 6 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Before upgrading, we recommend that users check that their own code does not use deprecated SciPy functionality (to do so, run your code with ``python -Wd`` and check for ``DeprecationWarning`` s). Our development attention will now shift to bug-fix releases on the 1.6.x branch, and on adding new features on the master branch. This release requires Python 3.7+ and NumPy 1.16.5 or greater. For running on PyPy, PyPy3 6.0+ is required. Highlights of this release --------------------------------- - `scipy.ndimage` improvements: Fixes and ehancements to boundary extension modes for interpolation functions. Support for complex-valued inputs in many filtering and interpolation functions. New ``grid_mode`` option for `scipy.ndimage.zoom` to enable results consistent with scikit-image's ``rescale``. - `scipy.optimize.linprog` has fast, new methods for large, sparse problems from the ``HiGHS`` library. - `scipy.stats` improvements including new distributions, a new test, and enhancements to existing distributions and tests New features ========== `scipy.special` improvements --------------------------------------- `scipy.special` now has improved support for 64-bit ``LAPACK`` backend `scipy.odr` improvements ---------------------------------- `scipy.odr` now has support for 64-bit integer ``BLAS`` `scipy.odr.ODR` has gained an optional ``overwrite`` argument so that existing files may be overwritten. `scipy.integrate` improvements ------------------------------------------ Some renames of functions with poor names were done, with the old names retained without being in the reference guide for backwards compatibility reasons: - ``integrate.simps`` was renamed to ``integrate.simpson`` - ``integrate.trapz`` was renamed to ``integrate.trapezoid`` - ``integrate.cumtrapz`` was renamed to ``integrate.cumulative_trapezoid`` `scipy.cluster` improvements --------------------------------------- `scipy.cluster.hierarchy.DisjointSet` has been added for incremental connectivity queries. `scipy.cluster.hierarchy.dendrogram` return value now also includes leaf color information in `leaves_color_list`. `scipy.interpolate` improvements -------------------------------------------- `scipy.interpolate.interp1d` has a new method ``nearest-up``, similar to the existing method ``nearest`` but rounds half-integers up instead of down. `scipy.io` improvements -------------------------------- Support has been added for reading arbitrary bit depth integer PCM WAV files from 1- to 32-bit, including the commonly-requested 24-bit depth. `scipy.linalg` improvements ------------------------------------- The new function `scipy.linalg.matmul_toeplitz` uses the FFT to compute the product of a Toeplitz matrix with another matrix. `scipy.linalg.sqrtm` and `scipy.linalg.logm` have performance improvements thanks to additional Cython code. Python ``LAPACK`` wrappers have been added for ``pptrf``, ``pptrs``, ``ppsv``, ``pptri``, and ``ppcon``. `scipy.linalg.norm` and the ``svd`` family of functions will now use 64-bit integer backends when available. `scipy.ndimage` improvements ------------------------------------------ `scipy.ndimage.convolve`, `scipy.ndimage.correlate` and their 1d counterparts now accept both complex-valued images and/or complex-valued filter kernels. All convolution-based filters also now accept complex-valued inputs (e.g. ``gaussian_filter``, ``uniform_filter``, etc.). Multiple fixes and enhancements to boundary handling were introduced to `scipy.ndimage` interpolation functions (i.e. ``affine_transform``, ``geometric_transform``, ``map_coordinates``, ``rotate``, ``shift``, ``zoom``). A new boundary mode, ``grid-wrap`` was added which wraps images periodically, using a period equal to the shape of the input image grid. This is in contrast to the existing ``wrap`` mode which uses a period that is one sample smaller than the original signal extent along each dimension. A long-standing bug in the ``reflect`` boundary condition has been fixed and the mode ``grid-mirror`` was introduced as a synonym for ``reflect``. A new boundary mode, ``grid-constant`` is now available. This is similar to the existing ndimage ``constant`` mode, but interpolation will still performed at coordinate values outside of the original image extent. This ``grid-constant`` mode is consistent with OpenCV's ``BORDER_CONSTANT`` mode and scikit-image's ``constant`` mode. Spline pre-filtering (used internally by ``ndimage`` interpolation functions when ``order >= 2``), now supports all boundary modes rather than always defaulting to mirror boundary conditions. The standalone functions ``spline_filter`` and ``spline_filter1d`` have analytical boundary conditions that match modes ``mirror``, ``grid-wrap`` and ``reflect``. `scipy.ndimage` interpolation functions now accept complex-valued inputs. In this case, the interpolation is applied independently to the real and imaginary components. The ``ndimage`` tutorials (https://docs.scipy.org/doc/scipy/reference/tutorial/ndimage.html) have been updated with new figures to better clarify the exact behavior of all of the interpolation boundary modes. `scipy.ndimage.zoom` now has a ``grid_mode`` option that changes the coordinate of the center of the first pixel along an axis from 0 to 0.5. This allows resizing in a manner that is consistent with the behavior of scikit-image's ``resize`` and ``rescale`` functions (and OpenCV's ``cv2.resize``). `scipy.optimize` improvements ----------------------------------------- `scipy.optimize.linprog` has fast, new methods for large, sparse problems from the ``HiGHS`` C++ library. ``method='highs-ds'`` uses a high performance dual revised simplex implementation (HSOL), ``method='highs-ipm'`` uses an interior-point method with crossover, and ``method='highs'`` chooses between the two automatically. These methods are typically much faster and often exceed the accuracy of other ``linprog`` methods, so we recommend explicitly specifying one of these three method values when using ``linprog``. `scipy.optimize.quadratic_assignment` has been added for approximate solution of the quadratic assignment problem. `scipy.optimize.linear_sum_assignment` now has a substantially reduced overhead for small cost matrix sizes `scipy.optimize.least_squares` has improved performance when the user provides the jacobian as a sparse jacobian already in ``csr_matrix`` format `scipy.optimize.linprog` now has an ``rr_method`` argument for specification of the method used for redundancy handling, and a new method for this purpose is available based on the interpolative decomposition approach. `scipy.signal` improvements -------------------------------------- `scipy.signal.gammatone` has been added to design FIR or IIR filters that model the human auditory system. `scipy.signal.iircomb` has been added to design IIR peaking/notching comb filters that can boost/attenuate a frequency from a signal. `scipy.signal.sosfilt` performance has been improved to avoid some previously- observed slowdowns `scipy.signal.windows.taylor` has been added--the Taylor window function is commonly used in radar digital signal processing `scipy.signal.gauss_spline` now supports ``list`` type input for consistency with other related SciPy functions `scipy.signal.correlation_lags` has been added to allow calculation of the lag/ displacement indices array for 1D cross-correlation. `scipy.sparse` improvements --------------------------------------- A solver for the minimum weight full matching problem for bipartite graphs, also known as the linear assignment problem, has been added in `scipy.sparse.csgraph.min_weight_full_bipartite_matching`. In particular, this provides functionality analogous to that of `scipy.optimize.linear_sum_assignment`, but with improved performance for sparse inputs, and the ability to handle inputs whose dense representations would not fit in memory. The time complexity of `scipy.sparse.block_diag` has been improved dramatically from quadratic to linear. `scipy.sparse.linalg` improvements ----------------------------------------------- The vendored version of ``SuperLU`` has been updated `scipy.fft` improvements -------------------------------- The vendored ``pocketfft`` library now supports compiling with ARM neon vector extensions and has improved thread pool behavior. `scipy.spatial` improvements --------------------------------------- The python implementation of ``KDTree`` has been dropped and ``KDTree`` is now implemented in terms of ``cKDTree``. You can now expect ``cKDTree``-like performance by default. This also means ``sys.setrecursionlimit`` no longer needs to be increased for querying large trees. ``transform.Rotation`` has been updated with support for Modified Rodrigues Parameters alongside the existing rotation representations (PR gh-12667). `scipy.spatial.transform.Rotation` has been partially cythonized, with some performance improvements observed `scipy.spatial.distance.cdist` has improved performance with the ``minkowski`` metric, especially for p-norm values of 1 or 2. `scipy.stats` improvements ------------------------------------ New distributions have been added to `scipy.stats`: - The asymmetric Laplace continuous distribution has been added as `scipy.stats.laplace_asymmetric`. - The negative hypergeometric distribution has been added as `scipy.stats.nhypergeom`. - The multivariate t distribution has been added as `scipy.stats.multivariate_t`. - The multivariate hypergeometric distribution has been added as `scipy.stats.multivariate_hypergeom`. The ``fit`` method has been overridden for several distributions (``laplace``, ``pareto``, ``rayleigh``, ``invgauss``, ``logistic``, ``gumbel_l``, ``gumbel_r``); they now use analytical, distribution-specific maximum likelihood estimation results for greater speed and accuracy than the generic (numerical optimization) implementation. The one-sample Cram?r-von Mises test has been added as `scipy.stats.cramervonmises`. An option to compute one-sided p-values was added to `scipy.stats.ttest_1samp`, `scipy.stats.ttest_ind_from_stats`, `scipy.stats.ttest_ind` and `scipy.stats.ttest_rel`. The function `scipy.stats.kendalltau` now has an option to compute Kendall's tau-c (also known as Stuart's tau-c), and support has been added for exact p-value calculations for sample sizes ``> 171``. `stats.trapz` was renamed to `stats.trapezoid`, with the former name retained as an alias for backwards compatibility reasons. The function `scipy.stats.linregress` now includes the standard error of the intercept in its return value. The ``_logpdf``, ``_sf``, and ``_isf`` methods have been added to `scipy.stats.nakagami`; ``_sf`` and ``_isf`` methods also added to `scipy.stats.gumbel_r` The ``sf`` method has been added to `scipy.stats.levy` and `scipy.stats.levy_l` for improved precision. `scipy.stats.binned_statistic_dd` performance improvements for the following computed statistics: ``max``, ``min``, ``median``, and ``std``. We gratefully acknowledge the Chan-Zuckerberg Initiative Essential Open Source Software for Science program for supporting many of these improvements to `scipy.stats`. Deprecated features ================ `scipy.spatial` changes ------------------------------- Calling ``KDTree.query`` with ``k=None`` to find all neighbours is deprecated. Use ``KDTree.query_ball_point`` instead. ``distance.wminkowski`` was deprecated; use ``distance.minkowski`` and supply weights with the ``w`` keyword instead. Backwards incompatible changes ========================== `scipy` changes --------------------- Using `scipy.fft` as a function aliasing ``numpy.fft.fft`` was removed after being deprecated in SciPy ``1.4.0``. As a result, the `scipy.fft` submodule must be explicitly imported now, in line with other SciPy subpackages. `scipy.signal` changes ------------------------------- The output of ``decimate``, ``lfilter_zi``, ``lfiltic``, ``sos2tf``, and ``sosfilt_zi`` have been changed to match ``numpy.result_type`` of their inputs. The window function ``slepian`` was removed. It had been deprecated since SciPy ``1.1``. `scipy.spatial` changes ------------------------------- ``cKDTree.query`` now returns 64-bit rather than 32-bit integers on Windows, making behaviour consistent between platforms (PR gh-12673). `scipy.stats` changes ----------------------------- The ``frechet_l`` and ``frechet_r`` distributions were removed. They were deprecated since SciPy ``1.0``. Other changes ============= ``setup_requires`` was removed from ``setup.py``. This means that users invoking ``python setup.py install`` without having numpy already installed will now get an error, rather than having numpy installed for them via ``easy_install``. This install method was always fragile and problematic, users are encouraged to use ``pip`` when installing from source. - Fixed a bug in `scipy.optimize.dual_annealing` ``accept_reject`` calculation that caused uphill jumps to be accepted less frequently. - The time required for (un)pickling of `scipy.stats.rv_continuous`, `scipy.stats.rv_discrete`, and `scipy.stats.rv_frozen` has been significantly reduced (gh12550). Inheriting subclasses should note that ``__setstate__`` no longer calls ``__init__`` upon unpickling. Authors ======= * @endolith * @vkk800 * aditya + * George Bateman + * Christoph Baumgarten * Peter Bell * Tobias Biester + * Keaton J. Burns + * Evgeni Burovski * R?diger Busche + * Matthias Bussonnier * Dominic C + * Corallus Caninus + * CJ Carey * Thomas A Caswell * chapochn + * Luc?a Cheung * Zach Colbert + * Coloquinte + * Yannick Copin + * Devin Crowley + * Terry Davis + * Micha?l Defferrard + * devonwp + * Didier + * divenex + * Thomas Duvernay + * Eoghan O'Connell + * G?k?en Eraslan * Kristian Eschenburg + * Ralf Gommers * Thomas Grainger + * GreatV + * Gregory Gundersen + * h-vetinari + * Matt Haberland * Mark Harfouche + * He He + * Alex Henrie * Chun-Ming Huang + * Martin James McHugh III + * Alex Izvorski + * Joey + * ST John + * Jonas Jonker + * Julius Bier Kirkegaard * Marcin Konowalczyk + * Konrad0 * Sam Van Kooten + * Sergey Koposov + * Peter Mahler Larsen * Eric Larson * Antony Lee * Gregory R. Lee * Lo?c Est?ve * Jean-Luc Margot + * MarkusKoebis + * Nikolay Mayorov * G. D. McBain * Andrew McCluskey + * Nicholas McKibben * Sturla Molden * Denali Molitor + * Eric Moore * Shashaank N + * Prashanth Nadukandi + * nbelakovski + * Andrew Nelson * Nick + * Nikola Forr? + * odidev * ofirr + * Sambit Panda * Dima Pasechnik * Tirth Patel + * Matti Picus * Pawe? Redzy?ski + * Vladimir Philipenko + * Philipp Th?lke + * Ilhan Polat * Eugene Prilepin + * Vladyslav Rachek * Ram Rachum + * Tyler Reddy * Martin Reinecke + * Simon Segerblom Rex + * Lucas Roberts * Benjamin Rowell + * Eli Rykoff + * Atsushi Sakai * Moritz Schulte + * Daniel B. Smith * Steve Smith + * Jan Soedingrekso + * Victor Stinner + * Jose Storopoli + * Diana Sukhoverkhova + * S?ren Fuglede J?rgensen * taoky + * Mike Taves + * Ian Thomas + * Will Tirone + * Frank Torres + * Seth Troisi * Ronald van Elburg + * Hugo van Kemenade * Paul van Mulbregt * Saul Ivan Rivas Vega + * Pauli Virtanen * Jan Vleeshouwers * Samuel Wallan * Warren Weckesser * Ben West + * Eric Wieser * WillTirone + * Levi John Wolf + * Zhiqing Xiao * Rory Yorke + * Yun Wang (Maigo) + * Egor Zemlyanoy + * ZhihuiChen0903 + * Jacob Zhong + A total of 122 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.6.0 ------------------------------- * `#1323 `__: ndimage.shift destroys data from edges (Trac #796) * `#1892 `__: using rptfile= with an existing file causes a Fortran runtime... * `#1903 `__: ndimage.rotate misses some values (Trac #1378) * `#1930 `__: scipy.io.wavfile should be able to read 24 bit signed wave (Trac... * `#3158 `__: Odd casting behaviour of signal.filtfilt * `#3203 `__: interpolation.zoom incorrect output for certain cases * `#3645 `__: BUG: stats: mstats.pearsonr calculation is wrong if the masks... * `#3665 `__: Return Bunch objects from stats functions * `#4922 `__: unexpected zero output values from zoom * `#5202 `__: BUG: stats: Spurious warnings from the pdf method of several... * `#5223 `__: Zoom does not return the same values when resizing a sub-array... * `#5396 `__: scipy.spatial.distance.pdist documention bug * `#5489 `__: ValueError: failed to create intent(cache|hide)|optional array--... * `#6096 `__: loadmat drops dtype of empty arrays when squeeze_me=True * `#6713 `__: sicpy.ndimage.zoom returns artefacts and boundaries in some cases * `#7125 `__: Impossible to know number of dimensions in c function used by... * `#7324 `__: scipy.ndimage.zoom bad interpolation when downsampling (zoom... * `#8131 `__: BUG: geometric_transform wrap mode possible bug * `#8163 `__: LSMR fails on some random values when providing an x0 * `#8210 `__: Why should I choose order > 1 for scipy.ndimage.zoom? * `#8465 `__: Unexpected behavior with reflect mode of ndimage.rotate * `#8776 `__: cdist behavior with Minkowsky and np.inf * `#9168 `__: documentation of pearson3 in scipy.stats unclear * `#9223 `__: Faster implementation of scipy.sparse.block_diag * `#9476 `__: Invalid index in signal.medfilt2d's QUICK_SELECT * `#9857 `__: scipy.odr.Output.sd_beta is not standard error * `#9865 `__: Strange behavior of \`ndimage.shift\` and \`ndimage.affine_transform\` * `#10042 `__: Consider support for multivariate student-t distribution? * `#10134 `__: gausshyper distribution accepts invalid parameters * `#10179 `__: str+bytes concatenation error in test_lapack.py * `#10216 `__: cKDTree.query_ball_point speed regression * `#10463 `__: ENH: vectorize scipy.fft for more CPU architectures * `#10593 `__: Rename \`sum\` ndimage function * `#10595 `__: scipy.stats.ttest_1samp should support alternative hypothesis * `#10610 `__: ndimage.interpolation.spline_filter1d default value of mode * `#10620 `__: ndimage.interpolation.zoom() option to work like skimage.transform.resize() * `#10711 `__: Array Shapes Not Aligned Bug in scipy.optimize._lsq.lsq_linear.py * `#10782 `__: BUG: optimize: methods unknown to \`scipy.optimize.show_options\` * `#10892 `__: Possible typo in an equation of optimize/dual_annealing * `#11020 `__: signal.fftconvolve return a tuple including lag information * `#11093 `__: scipy.interpolate.interp1d can not handle datetime64 * `#11170 `__: Use manylinux2014 to get aarch64/ppc64le support * `#11186 `__: BUG: stats: pearson3 CDF and SF functions incorrect when skew... * `#11366 `__: DeprecationWarning due to invalid escape sequences * `#11403 `__: Optimize raises "ValueError: \`x0\` violates bound constraints"... * `#11558 `__: ENH: IIR comb filter * `#11559 `__: BUG: iirdesign doesn't fail for frequencies above Nyquist * `#11567 `__: scipy.signal.iirdesign doesn't check consistency of wp and ws... * `#11654 `__: ENH: Add Negative Hypergeometric Distribution * `#11720 `__: BUG: stats: wrong shape from median_absolute_deviation for arrays... * `#11746 `__: BUG: stats: pearson3 returns size 1 arrays where other distributions... * `#11756 `__: Improve and fix \*Spline docstrings and code * `#11758 `__: BUG: of scipy.interpolate.CubicSpline when \`bc_type' is set... * `#11925 `__: MAINT: remove character encoding check in CI? * `#11963 `__: Test failures - TestLinprogIPSparseCholmod * `#12102 `__: incorrect first moment of non central t-distribution * `#12113 `__: scipy.stats.poisson docs for rate = 0 * `#12152 `__: ENH: signal.gauss_spline should accept a list * `#12157 `__: BUG: Iteration index initialisation is wrong in scipy.optimize.linesearch.scalar_search_wolfe2 * `#12162 `__: Storing Rotation object in NumPy array returns an array with... * `#12176 `__: cannot modify the slice of an array returned by \`wavfile.read\` * `#12190 `__: retrieve leave colors from dendrogram * `#12196 `__: PERF: scipy.linalg.pinv is very slow compared to numpy.linalg.pinv * `#12222 `__: Interpolating categorical data (interp1d) * `#12231 `__: Is the p-value of the Kruskal-Wallis test two-sided? * `#12249 `__: ENH: least_squares: should not re-instanciate csr_matrix if already... * `#12264 `__: DOC: optimize: linprog method-specific function signature * `#12290 `__: DOC: Convex Hull areas are actually perimeters for 2-dimensional... * `#12308 `__: integrate.solve_ivp with DOP853 method fails when yDot = 0 * `#12326 `__: BUG: stats.exponnorm.pdf returns 0 for small K * `#12337 `__: scipy.sparse.linalg.eigsh documentation is misleading * `#12339 `__: scipy.io.wavfile.write documentation has wrong example * `#12340 `__: sparse.lil_matrix.tocsr() fails silently on matrices with nzn... * `#12350 `__: Create a 2-parameter version of the gamma distribution * `#12369 `__: scipy.signal.correlate has an error in the documentation, examples... * `#12373 `__: interp1d returns incorrect values for step functions * `#12378 `__: interpolate.NearestNDInterpolator.__call__ & LinearNDInterpolator.__call__... * `#12411 `__: scipy.stats.spearmanr mishandles nan variables with "propogate" * `#12413 `__: DOC: Remove the "Basic functions" section from the SciPy tutorial. * `#12415 `__: scipy.stats.dirichlet documentation issue * `#12419 `__: least_squares ValueError with 'lm' method - regression from 1.4.1... * `#12431 `__: Request for Python wrapper for LAPACK's ?pptrf (Cholesky factorization... * `#12458 `__: spearmanr with entire NaN columns produces errors * `#12477 `__: WIP: Addition of MLE for stats.invgauss/wald * `#12483 `__: reading .wav fails when the file is too big on python 3.6.0 * `#12490 `__: BUG: stats: logistic and genlogistic logpdf overflow for large... * `#12499 `__: LinearNDInterpolator raises ValueError when value array has writeable=False... * `#12523 `__: Wrong key in __odrpack.c * `#12547 `__: typo in scipy/cluster/_hierarchy.pyx * `#12549 `__: DOC: least_squares return type is poorly formatted. * `#12578 `__: TST: test_bounds_infeasible_2 failing on wheels repo cron jobs * `#12585 `__: ENH: Add Multivariate Hypergeometric Distribution * `#12604 `__: unintuitive conversion in \`scipy.constants.lambda2nu\` * `#12606 `__: DOC: Invalid syntax in example. * `#12665 `__: List of possible bugs found by automated code analysis * `#12696 `__: scipy.optimize.fminbound, numpy depreciation warning Creating... * `#12699 `__: TestProjections.test_iterative_refinements_dense failure * `#12701 `__: TestDifferentialEvolutionSolver::test_L4 failing * `#12719 `__: Misleading scipy.signal.get_window() docstring with 'exponential'... * `#12740 `__: circstd doesn't handle R = hypot(S, C) > 1 * `#12749 `__: ENH: interp1d Matlab compatibility * `#12773 `__: Meta-issue: ndimage spline boundary handling (NumFOCUS proposal) * `#12813 `__: optimize.root(method="krylov") fails if options["tol_norm"] expects... * `#12815 `__: stats.zscore inconsistent behavior when all values are the same * `#12840 `__: scipy.signal.windows.dpss docstring typo * `#12874 `__: Rotation.random vs stats.special_ortho_group * `#12881 `__: FFT - documentation - examples - linspace construction * `#12904 `__: BUG: parsing in loadarff() * `#12917 `__: GitHub Actions nightly build triggered on forks * `#12919 `__: BUG: numerical precision, use gammaln in nct.mean * `#12924 `__: Rename Sample Based Integration Methods to Comply with Code of... * `#12940 `__: Should the minimum numpy for AIX be bumped to 1.16.5 * `#12951 `__: A possible typo in scipy.stats.weightedtau * `#12952 `__: [Documentation question] Would it be more precise to specify... * `#12970 `__: Documentation presents second order sections as the correct choice... * `#12982 `__: Calculate standard error of the intercept in linregress * `#12985 `__: Possible wrong link in scipy.stats.wilcoxon doc * `#12991 `__: least_squares broken with float32 * `#13001 `__: \`OptimizeResult.message\` from \`L-BFGS-B\` is a bytes, not... * `#13030 `__: BUG: lint_diff.py still fails for backport PRs * `#13077 `__: CI: codecov proper patch diffs * `#13085 `__: Build failing on main branch after HiGHS solver merge * `#13088 `__: BLD, BUG: wheel builds failure with HiGHS/optimize * `#13099 `__: Wrong output format for empty sparse results of kron * `#13108 `__: TST, CI: GitHub Actions MacOS Failures * `#13111 `__: BUG, DOC: refguide check is failing * `#13127 `__: ODR output file writing broken in conda env with system compilers * `#13134 `__: FromTravis migration tracker * `#13140 `__: BUG: signal: \`ss2tf\` incorrectly truncates output to integers. * `#13179 `__: CI: lint is failing because of output to stderr * `#13182 `__: Key appears twice in \`test_optimize.test_show_options\` * `#13191 `__: \`scipy.linalg.lapack.dgesjv\` overwrites original arrays if... * `#13207 `__: TST: Erratic test failure in test_cossin_separate * `#13221 `__: BUG: pavement.py glitch * `#13239 `__: Segmentation fault with \`eigh(..., driver="evx")\` for 10x10... * `#13248 `__: ndimage: improper cval handling for complex-valued inputs Pull requests for 1.6.0 ------------------------------ * `#8032 `__: ENH: Add in taylor window common in Radar processing * `#8779 `__: CI: Run benchmarks * `#9361 `__: ENH: Add Kendall's tau-a and tau-c variants to scipy.stats.kendalltau() * `#11068 `__: ENH: Adds correlation_lags function to scipy.signal * `#11119 `__: ENH: add Cramer-von-Mises (one-sample) test to scipy.stats * `#11249 `__: ENH: optimize: interpolative decomposition redundancy removal... * `#11346 `__: ENH: add fast toeplitz matrix multiplication using FFT * `#11413 `__: ENH: Multivariate t-distribution (stale) * `#11563 `__: ENH: exact p-value in stats.kendalltau() for sample sizes > 171 * `#11691 `__: ENH: add a stack of reversal functions to linprog * `#12043 `__: ENH: optimize: add HiGHS methods to linprog - continued * `#12061 `__: Check parameter consistensy in signal.iirdesign * `#12067 `__: MAINT: Cleanup OLDAPI in ndimage/src/_ctest.c * `#12069 `__: DOC: Add developer guidelines for implementing the nan_policy... * `#12077 `__: MAINT: malloc return value checks for cython * `#12080 `__: MAINT: Remove suppress_warnings * `#12085 `__: ENH: special: support ILP64 Lapack * `#12086 `__: MAINT: Cleanup PyMODINIT_FUNC used during 2to3 * `#12097 `__: ENH: stats: override stats.rayleigh.fit with analytical MLE * `#12112 `__: DOC: Improve integrate.nquad docstring * `#12125 `__: TST: Add a test for stats.gmean with negative input * `#12139 `__: TST: Reduce flakiness in lsmr test * `#12142 `__: DOC: add a note in poisson distribution when mu=0 and k=0 in... * `#12144 `__: DOC: Update ndimage.morphology.distance_transform\* * `#12154 `__: ENH: scipy.signal: allow lists in gauss_spline * `#12170 `__: ENH: scipy.stats: add negative hypergeometric distribution * `#12177 `__: MAINT: Correctly add input line to ValueError * `#12183 `__: ENH: Use fromfile where possible * `#12186 `__: MAINT: generalize tests in SphericalVoronoi * `#12198 `__: TST: Fix str + bytes error * `#12199 `__: ENH: match np.result_type behaviour in some scipy.signal functions * `#12200 `__: ENH: add FIR and IIR gammatone filters to scipy.signal * `#12204 `__: ENH: Add overwrite argument for odr.ODR() and its test. * `#12206 `__: MAINT:lstsq: Switch to tranposed problem if the array is tall * `#12208 `__: wavfile bugfixes and maintenance * `#12214 `__: DOC: fix docstring of "sd_beta" of odr.Output. * `#12234 `__: MAINT: prevent divide by zero warnings in scipy.optimize BFGS... * `#12235 `__: REL: set version to 1.6.0.dev0 * `#12237 `__: BUG: Fix exit condition for QUICK_SELECT pivot * `#12242 `__: ENH: Rename ndimage.sum to ndimage.sum_labels (keep sum as alias) * `#12243 `__: EHN: Update SuperLU * `#12244 `__: MAINT: stats: avoid spurious warnings in ncx2.pdf * `#12245 `__: DOC: Fixed incorrect default for mode in scipy.ndimage.spline_filter1d * `#12248 `__: MAINT: clean up pavement.py * `#12250 `__: ENH: Replaced csr_matrix() by tocsr() and complemented docstring * `#12253 `__: TST, CI: turn on codecov patch diffs * `#12259 `__: MAINT: Remove duplicated test for import cycles * `#12263 `__: ENH: Rename LocalSearchWrapper bounds * `#12265 `__: BUG optimize: Accept np.matrix in lsq_linear * `#12266 `__: BUG: Fix paren error in dual annealing accept_reject calculation * `#12269 `__: MAINT: Included mismatched shapes in error messages. * `#12279 `__: MAINT: \`__array__\` and array protocols cannot be used in sparse. * `#12281 `__: DOC: update wheel DL docs * `#12283 `__: ENH: odr: ILP64 Blas support in ODR * `#12284 `__: ENH: linalg: support for ILP64 BLAS/LAPACK in f2py wrappers * `#12286 `__: ENH: Cythonize scipy.spatial.transform.Rotation * `#12287 `__: ENH: Read arbitrary bit depth (including 24-bit) WAVs * `#12292 `__: BLD: fix musl compilation * `#12293 `__: MAINT: Fix a DeprecationWarning in validate_runtests_log.py. * `#12296 `__: DOC: Clarify area/volume in scipy.spatial.ConvexHull docstrings * `#12302 `__: CI: Run travis builds on master to keep cache up to date * `#12305 `__: TST: Cleanup print statements in tests * `#12323 `__: ENH: Add a Bunch-like class to use as a backwards compatible... * `#12324 `__: BUG: io: Fix an error that occurs when attempting to raise a... * `#12327 `__: DOC: clarify docstrings of \`query_ball_tree\` and \`query_pairs\` * `#12334 `__: PERF: Improve cKDTree.query_ball_point constant time cython overhead * `#12338 `__: DOC: improve consistency and clarity of docs in linalg and sparse/linalg * `#12341 `__: DOC: add Examples for KDTree query_ball_tree and query_pairs * `#12343 `__: DOC: add examples for special.eval_legendre() * `#12349 `__: BUG: avoid overflow in sum() for 32-bit systems * `#12351 `__: DOC: Fix example wavfile to be 16bit * `#12352 `__: [BUG] Consider 0/0 division in DOP853 error estimation * `#12353 `__: Fix exception causes in vq.py * `#12354 `__: MAINT: Cleanup unneeded void\* cast in setlist.pxd * `#12355 `__: TST: Remove hack for old win-amd64 bug * `#12356 `__: ENH: Faster implementation of scipy.sparse.block_diag (#9411... * `#12357 `__: MAINT,TST: update and run scipy/special/utils/convert.py * `#12358 `__: TST: Check mstat.skewtest pvalue * `#12359 `__: TST: Sparse matrix test with int64 indptr and indices * `#12363 `__: DOC: ref. in CloughTocher2DInterpolator * `#12364 `__: DOC: \`sparse_distance_matrix\` and \`count_neighbors\` examples * `#12371 `__: MAINT, CI: bump to latest stable OpenBLAS * `#12372 `__: MAINT: Minor cleanup of (c)KDTree tests * `#12374 `__: DEP: Deprecate \`distance.wminkowski\` * `#12375 `__: ENH: Add fast path for minkowski distance with p=1,2 and support... * `#12376 `__: Fix exception causes in most of the codebase * `#12377 `__: DOC: Quick fix - adds newline to correlation_lags docstring Examples... * `#12381 `__: BENCH: remove obsolete goal_time param * `#12382 `__: ENH: Replace KDTree with a thin wrapper over cKDTree * `#12385 `__: DOC: improve docstrings of interpolate.NearestNDInterpolator.__call__... * `#12387 `__: DOC/STY: add example to scipy.signal.correlate * `#12393 `__: CI: Replace the existing check for non-ASCII characters with... * `#12394 `__: CI: arm64 numpy now available * `#12395 `__: ENH: improve stats.binned_statistic_dd performance * `#12396 `__: DOC, MAINT: forward port 1.5.0 relnotes * `#12398 `__: API: Disable len() and indexing of Rotation instances with single... * `#12399 `__: MAINT: Replace some Unicode dash-like chars with an ASCII hyphen. * `#12402 `__: update .mailmap * `#12404 `__: MAINT: io: Change the encoding comment of test_mio.py to utf-8. * `#12416 `__: CI: cache mingw, azure pipelines * `#12427 `__: BUG: logic error in loop unrolling (cKDTree) * `#12432 `__: DOC: Remove the "Basic functions" section from the SciPy tutorial. * `#12434 `__: ENH:linalg: Add LAPACK wrappers pptrf/pptrs/ppsv/pptri/ppcon * `#12435 `__: DOC: fix simplex math for scipy.stats.dirichlet documentation * `#12439 `__: DOC: add API methods summary for NdPPoly * `#12443 `__: BUG: stats: Improve calculation of exponnorm.pdf * `#12448 `__: DOC: stats: Add "Examples" to the ansari docstring. * `#12450 `__: ENH: add \`leaves_color_list\` for cluster.dendrogram dictionary. * `#12451 `__: MAINT: remove "blacklist" terminology from code base * `#12452 `__: DOC: clarify the meaning of whitening for cluster.vq.whiten() * `#12455 `__: MAINT: clearer error message in setup.py * `#12457 `__: ENH: stats: override stats.pareto.fit with analytical MLE * `#12460 `__: check if column in spearman rho is entirely NaN or Inf * `#12463 `__: DOC: improve and clean up \*Spline docstrings in fitpack2.py * `#12474 `__: ENH: linalg: speedup _sqrtm_triu by moving tight loop to Cython * `#12476 `__: ENH: add IIR comb filter to scipy.signal * `#12484 `__: Fix documentation for minimize * `#12486 `__: DOC: add a note in poisson distribution when mu=0 and k=0 in... * `#12491 `__: MAINT: forward port 1.5.1 release notes * `#12508 `__: Fix exception causes all over the codebase * `#12514 `__: ENH: stats: override stats.invgauss.fit with analytical MLE * `#12519 `__: PERF: Avoid np.zeros when custom initialization is needed anyway * `#12520 `__: DOC: Minor RST section renaming. * `#12521 `__: MAINT: Remove unused imports * `#12522 `__: PERF: Get rid of unnececssary allocation in VarReader5.cread_fieldnames * `#12524 `__: DOC: special: Set Axes3D rect to avoid clipping labels in plot. * `#12525 `__: Fix large sparse nnz * `#12526 `__: DOC: Remove double section and too long header underline. * `#12527 `__: Improve error message for wrong interpolation type * `#12530 `__: Move redundant logic outside loop for conditional speedup in... * `#12532 `__: ENH: Add norm={"forward", "backward"} to \`scipy.fft\` * `#12535 `__: MAINT: Avoid sphinx deprecated aliases for SeeAlso and Only * `#12540 `__: BUG: fix odr.output.work_ind key bug and add its test. * `#12541 `__: ENH: add solver for minimum weight full bipartite matching * `#12550 `__: PERF: pickling speed of rv\* * `#12551 `__: DOC: fix typo in cluster/_hierarchy.pyx * `#12552 `__: CI: Cleanup travis pip installs * `#12556 `__: BUG: Fix problem with Scipy.integrate.solve_bvp for big problems * `#12557 `__: MAINT: Use extern templates to improve sparsetools compile time * `#12558 `__: MAINT: Remove hack to allow scipy.fft to act like a function * `#12563 `__: MAINT: Remove unused mu0 in special/orthogonal.py * `#12564 `__: DOC: fix return type docstring for least_squares * `#12565 `__: DOC: stats: respond to query about Kruskal-Wallis test being... * `#12566 `__: BUG: Interpolate: use stable sort * `#12568 `__: Updated documentation for as_quat * `#12571 `__: DEP: remove deprecated slepian window * `#12573 `__: DEP: remove \`frechet_l\` and \`frechet_r\` * `#12575 `__: BUG: stats: fix multinomial.pmf NaNs when params sum > 1 * `#12576 `__: MAINT: remove warning from LSQSphereBivariateSpline * `#12582 `__: ENH: Multivariate t-distribution * `#12587 `__: ENH: speed up rvs of gengamma in scipy.stats * `#12588 `__: DOC: add Examples add see also sections for LinearNDInterpolator,... * `#12597 `__: ENH: Add single-sided p-values to t-tests * `#12599 `__: Small update to scipy FFT tutorial * `#12600 `__: ENH: disjoint set data structure * `#12602 `__: BUG: add const for Read-only views in interpnd.pyx * `#12605 `__: BUG: correct \`np.asanyarray\` use in \`scipy.constants.lambda2nu\` * `#12610 `__: MAINT: forward port 1.5.2 release notes * `#12612 `__: MAINT: stats: Use explicit keyword parameters instead of \`\*\*kwds\`. * `#12616 `__: DOC: make explicit docstring that interpolate.interp1d only accepts... * `#12618 `__: DOC: Minor doc formatting. * `#12640 `__: MAINT: stats: fix issues with scipy.stats.pearson3 docs, moment,... * `#12647 `__: TST: Add Boost ellipr[cdfgj]_data test data * `#12648 `__: DOC: Update special/utils/README with instructions * `#12649 `__: DOC: simplified pip quickstart guide * `#12650 `__: DOC: stats: Fix boxcox docstring: lambda can be negative. * `#12655 `__: DOC: update Steering Council members listed in governance docs * `#12659 `__: rv_sample expect bug * `#12663 `__: DOC: optimize: try to fix linprog method-specific documentation * `#12664 `__: BUG: stats: Fix logpdf with large negative values for logistic... * `#12666 `__: MAINT: Fixes from static analysis * `#12667 `__: ENH: Adding Modified Rodrigues Parameters to the Rotation class * `#12670 `__: DOC: Update documentation for Gamma distribution * `#12673 `__: API: Unconditionally return np.intp from cKDTree.query * `#12677 `__: MAINT: Add Autogenerated notice to ufuncs.pyi * `#12682 `__: MAINT: Remove _util._valarray * `#12688 `__: MAINT: add f2py-generated scipy.integrate files to .gitignore * `#12689 `__: BENCH: simplify benchmark setup, remove benchmarks/run.py * `#12694 `__: scipy/stats: Add laplace_asymmetric continuous distribution * `#12695 `__: DOC: update Ubuntu quickstart; conda compilers now work! * `#12698 `__: MAINT: Replace np.max with np.maximum * `#12700 `__: TST: bump test precision for constrained trustregion test * `#12702 `__: TST: bump test tolerance for \`DifferentialEvolutionSolver.test_L4\` * `#12703 `__: BUG: Improve input validation for sepfir2d * `#12708 `__: MAINT: fix a typo in scipy.sparse * `#12709 `__: BUG: bvls can fail catastrophically to converge * `#12711 `__: MAINT: Use platform.python_implementation to determine IS_PYPY * `#12713 `__: TST: Fix flaky test_lgmres * `#12716 `__: DOC: add examples and tutorial links for interpolate functions... * `#12717 `__: DOC: Fix Issue #5396 * `#12725 `__: ENH: Support complex-valued images and kernels for many ndimage... * `#12729 `__: DEP: remove setup_requires * `#12732 `__: BENCH: skip benchmarks instead of hiding them when SCIPY_XSLOW=0 * `#12734 `__: CI: Don't ignore line-length in the lint_diff check. * `#12736 `__: DOC: Fix signal.windows.get_window() 'exponential' docstring * `#12737 `__: ENH: stats: override stats.gumbel_r.fit and stats.gumbel_l.fit... * `#12738 `__: ENH: stats: override stats.logistic.fit with system of equations... * `#12743 `__: BUG: Avoid negative variances in circular statistics * `#12744 `__: Prevent build error on GNU/Hurd * `#12746 `__: TST: parameterize the test cases in test_ndimage.py * `#12752 `__: DOC: Add examples for some root finding functions. * `#12754 `__: MAINT, CI: Azure windows deps multiline * `#12756 `__: ENH: stats: Add an sf method to levy for improved precision in... * `#12757 `__: ENH: stats: Add an sf method to levy_l for improved precision. * `#12765 `__: TST, MAINT: infeasible_2 context * `#12767 `__: Fix spline interpolation boundary handling for modes reflect... * `#12769 `__: DOC: syntax error in scipy.interpolate.bspl * `#12770 `__: ENH: add nearest-up rounding to scipy.interpolate.interp1d * `#12771 `__: TST: fix invalid input unit test for scipy.signal.gammatone * `#12775 `__: ENH: Adds quadratic_assignment with two methods * `#12776 `__: ENH: add grid-constant boundary handling in ndimage interpolation... * `#12777 `__: Add Taylor Window function - Common in Radar DSP * `#12779 `__: ENH: Improvements to pocketfft thread pool and ARM neon vectorization * `#12788 `__: API: Rename cKDTree n_jobs argument to workers * `#12792 `__: DOC: remove THANKS.txt file in favor of scipy.org * `#12793 `__: Add new flag to authors tool * `#12802 `__: BENCH: add scipy.ndimage.interpolation benchmarks * `#12803 `__: Do not pin the version of numpy in unsupported python versions * `#12810 `__: CI: fix 32-bit Linux build failure on Azure CI runs * `#12812 `__: ENH: support interpolation of complex-valued images * `#12814 `__: BUG: nonlin_solve shouldn't pass non-vector dx to tol_norm * `#12818 `__: Update ckdtree.pyx * `#12822 `__: MAINT: simplify directed_hausdorff * `#12827 `__: DOC: Fix wrong name w being used instead of worN in docs. * `#12831 `__: DOC: fix typo in sparse/base.py * `#12835 `__: MAINT: stats: Improve vonmises PDF calculation. * `#12839 `__: ENH: scipy.stats: add multivariate hypergeometric distribution * `#12843 `__: changed M to N in windows.dpss * `#12846 `__: MAINT: update minimum NumPy version to 1.16.5 * `#12847 `__: DOC: Unify the formula in docs of scipy.stats.pearsonr() * `#12849 `__: DOC: polish QAP docs for consistency and readability * `#12852 `__: ENH, MAINT: Bring KDTree interface to feature-parity with cKDTree * `#12858 `__: DOC: use :doi: and :arxiv: directives for references * `#12872 `__: lazily import multiprocessing.Pool in MapWrapper * `#12878 `__: DOC: document ScalarFunction * `#12882 `__: MAINT: stats: Change a test to use <= instead of strictly less... * `#12885 `__: numpy.linspace calls edited to ensure correct spacing. * `#12886 `__: DOC: stats: Add 'versionadded' to cramervonmises docstring. * `#12899 `__: TST: make a couple of tests expected to fail on 32-bit architectures * `#12903 `__: DOC: update Windows build guide and move into contributor guide * `#12907 `__: DOC: clarify which array the precenter option applies to * `#12908 `__: MAINT: spatial: Remove two occurrences of unused variables in... * `#12909 `__: ENH: stats: Add methods gumbel_r._sf and gumbel_r._isf * `#12910 `__: CI: travis: Remove some unnecessary code from .travis.yml. * `#12911 `__: Minor fixes to dendrogram plotting * `#12921 `__: CI: don't run GitHub Actions on fork or in cron job * `#12927 `__: MAINT: rename integrate.simps to simpson * `#12934 `__: MAINT: rename trapz and cumtrapz to (cumulative\_)trapezoid * `#12936 `__: MAINT: fix numerical precision in nct.stats * `#12938 `__: MAINT: fix linter on master * `#12941 `__: Update minimum AIX pinnings to match non AIX builds * `#12955 `__: BUG: Fixed wrong NaNs check in scipy.stats.weightedtau * `#12958 `__: ENH: stats: Implement _logpdf, _sf and _isf for nakagami. * `#12962 `__: Correcting that p should be in [0,1] for a variety of discrete... * `#12964 `__: BUG: added line.strip() to split_data_line() * `#12968 `__: ENH: stats: Use only an analytical formula or scalar root-finding... * `#12971 `__: MAINT: Declare support for Python 3.9 * `#12972 `__: MAINT: Remove redundant Python < 3.6 code * `#12980 `__: DOC: Update documentation on optimize.rosen * `#12983 `__: ENH: improvements to stats.linregress * `#12990 `__: DOC: Clarify that using sos as output type for iirdesign can... * `#12992 `__: DOC: capitalization and formatting in lsmr * `#12995 `__: DOC: stats: Several documentation fixes. * `#12996 `__: BUG: Improve error messages for \`range\` arg of binned_statistic_dd * `#12998 `__: MAINT: approx_derivative with FP32 closes #12991 * `#13004 `__: TST: isinstance(OptimizeResult.message, str) closes #13001 * `#13006 `__: Keep correct dtype when loading empty mat arrays. * `#13009 `__: MAINT: clip SLSQP step within bounds * `#13012 `__: DOC: fix bilinear_zpk example labels * `#13013 `__: ENH: Add \`subset\` and \`subsets\` methods to \`DisjointSet\`... * `#13029 `__: MAINT: basinhopping callback for initial mininmisation * `#13032 `__: DOC: fix docstring errors in in stats.wilcoxon * `#13036 `__: BUG: forward port lint_diff shims * `#13041 `__: MAINT: dogbox ensure x is within bounds closes #11403 * `#13042 `__: MAINT: forward port 1.5.4 release notes * `#13046 `__: DOC: Update optimize.least_squares doc for all tolerance must... * `#13052 `__: Typo fix for cluster documentation * `#13054 `__: BUG: fix \`scipy.optimize.show_options\` for unknown methods.... * `#13056 `__: MAINT: fft: Fix a C++ compiler warning. * `#13057 `__: Minor fixes on doc of function csr_tocsc * `#13058 `__: DOC: stats: Replace np.float with np.float64 in a tutorial file. * `#13059 `__: DOC: stats: Update the "Returns" section of the linregress docstring. * `#13060 `__: MAINT: clip_x_for_func should be private * `#13061 `__: DOC: signal.win -> signal.windows.win in Examples * `#13063 `__: MAINT: Add suite-sparse and sksparse installation check * `#13070 `__: MAINT: stats: Remove a couple obsolete comments. * `#13073 `__: BUG: Fix scalar_search_wolfe2 to resolve #12157 * `#13078 `__: CI, MAINT: migrate Lint to Azure * `#13081 `__: BLD: drop Python 3.6 support (NEP 29) * `#13082 `__: MAINT: update minimum NumPy version to 1.16.5 in a couple more... * `#13083 `__: DOC: update toolchain.rst * `#13086 `__: DOC: Update the Parameters section of the correlation docstring * `#13087 `__: ENH:signal: Speed-up Cython implementation of _sosfilt * `#13089 `__: BLD, BUG: add c99 compiler flag to HiGHS basiclu library * `#13091 `__: BUG: Fix GIL handling in _sosfilt * `#13094 `__: DOC: clarify "location" in docstring of cKDTree.query * `#13095 `__: Zoom resize update * `#13097 `__: BUG: fix CubicSpline(..., bc_type="periodic") #11758 * `#13100 `__: BUG: sparse: Correct output format of kron * `#13107 `__: ENH: faster linear_sum_assignment for small cost matrices * `#13110 `__: CI, MAINT: refguide/asv checks to azure * `#13112 `__: CI: fix MacOS CI * `#13113 `__: CI: Install word list package for refguide-check * `#13115 `__: BUG: add value range check for signal.iirdesign() * `#13116 `__: CI: Don't report name errors after an exception in refguide-check * `#13117 `__: CI: move sdist/pre-release test Azure * `#13119 `__: Improve error message on friedmanchisquare function * `#13121 `__: Fix factorial() for NaN on Python 3.10 * `#13123 `__: BLD: Specify file extension for language standard version tests * `#13128 `__: TST: skip Fortran I/O test for ODR * `#13130 `__: TST: skip factorial() float tests on Python 3.10 * `#13136 `__: CI:Add python dbg run to GH Actions * `#13138 `__: CI: Port coverage, 64-bit BLAS, GCC 4.8 build to azure * `#13139 `__: Fix edge case for mode='nearest' in ndimage.interpolation functions * `#13141 `__: BUG: signal: Fix data type of the numerator returned by ss2tf. * `#13144 `__: MAINT: stats: restrict gausshyper z > -1 * `#13146 `__: typo in csr.py * `#13148 `__: BUG: stats: fix typo in stable rvs per gh-12870 * `#13149 `__: DOC: spatial/stats: cross-ref random rotation matrix functions * `#13151 `__: MAINT: stats: Fix a test and a couple PEP-8 issues. * `#13152 `__: MAINT: stats: Use np.take_along_axis in the private function... * `#13154 `__: ENH: stats: Implement defined handling of constant inputs in... * `#13156 `__: DOC: maintain equal display range for ndimage.zoom example * `#13159 `__: CI: Azure: Don't run tests on merge commits, except for coverage * `#13160 `__: DOC: stats: disambiguate location-shifted/noncentral * `#13161 `__: BUG: DifferentialEvolutionSolver.__del__ can fail in garbage... * `#13163 `__: BUG: stats: fix bug in spearmanr nan propagation * `#13167 `__: MAINT: stats: Fix a test. * `#13169 `__: BUG: stats: Fix handling of misaligned masks in mstats.pearsonr. * `#13178 `__: CI: testing.yml --> macos.yml * `#13181 `__: CI: fix lint * `#13190 `__: BUG: optimize: fix a duplicate key bug for \`test_show_options\` * `#13192 `__: BUG:linalg: Add overwrite option to gejsv wrapper * `#13194 `__: BUG: slsqp should be able to use rel_step * `#13199 `__: [skip travis] DOC: 1.6.0 release notes * `#13203 `__: fix typos * `#13209 `__: TST:linalg: set the seed for cossin test * `#13212 `__: [DOC] Backtick and directive consistency. * `#13217 `__: REL: add necessary setuptools and numpy version pins in pyproject.toml... * `#13226 `__: BUG: pavement.py file handle fixes * `#13249 `__: Handle cval correctly for ndimage functions with complex-valued... * `#13253 `__: BUG,MAINT: Ensure all Pool objects are closed * `#13255 `__: BUG:linalg: Fix heevx wrappers and add new tests * `#13260 `__: CI: fix macOS testing * `#13269 `__: CI: github actions: In the linux dbg tests, update apt before... * `#13279 `__: MAINT: 1.6.0 rc2 backports Checksums ========= MD5 ~~~ 8fd605d90e571560d33eecb3c2263abe scipy-1.6.0-cp37-cp37m-macosx_10_9_x86_64.whl ab345b6f61228d8d797083fbe4ecb9df scipy-1.6.0-cp37-cp37m-manylinux1_i686.whl 6df40fd888aacc66c7fa2f905fdc3c8b scipy-1.6.0-cp37-cp37m-manylinux1_x86_64.whl 80adc50489f2d1f3f84c71629a7f0233 scipy-1.6.0-cp37-cp37m-manylinux2014_aarch64.whl 0f60495e80fd143419c594238b1c9f0a scipy-1.6.0-cp37-cp37m-win32.whl b7bcb35ae551a0d78139ed5a8f11fbde scipy-1.6.0-cp37-cp37m-win_amd64.whl fef87b2e8d82638af89c7ff0820033f9 scipy-1.6.0-cp38-cp38-macosx_10_9_x86_64.whl 7e7d04505b315c4cc4567ddca2c9f160 scipy-1.6.0-cp38-cp38-manylinux1_i686.whl 54e934334700a16d84a62b3cd375085c scipy-1.6.0-cp38-cp38-manylinux1_x86_64.whl e03550991ed048d919cbe75b8687dab5 scipy-1.6.0-cp38-cp38-manylinux2014_aarch64.whl 026724f8bec3081520eda2bad41776bb scipy-1.6.0-cp38-cp38-win32.whl 3a1ff95d9ea373196d3594df2303bfff scipy-1.6.0-cp38-cp38-win_amd64.whl 7653cdb2672fc5bd9609947f05a1a9cf scipy-1.6.0-cp39-cp39-macosx_10_9_x86_64.whl ac6f387bcfff36b1909783b9e78173c1 scipy-1.6.0-cp39-cp39-manylinux1_i686.whl d9d9aa57714f58da02b3cb539d3bab7e scipy-1.6.0-cp39-cp39-manylinux1_x86_64.whl ad70a68d2e5190c5bda723fabc877bbe scipy-1.6.0-cp39-cp39-manylinux2014_aarch64.whl 5fcd381d744e9ba272694547750ae9d5 scipy-1.6.0-cp39-cp39-win32.whl fe7ddd09cdc2263f8ecae6001b41f963 scipy-1.6.0-cp39-cp39-win_amd64.whl 550fcf08700e0bf0f32dc5c06c0f793a scipy-1.6.0.tar.gz 38acf883207e044e1ac9393a5806c3b9 scipy-1.6.0.tar.xz b67a16099b676cbbfdb977b45776a822 scipy-1.6.0.zip SHA256 ~~~~~~ 3d4303e3e21d07d9557b26a1707bb9fc065510ee8501c9bf22a0157249a82fd0 scipy-1.6.0-cp37-cp37m-macosx_10_9_x86_64.whl 1bc5b446600c4ff7ab36bade47180673141322f0febaa555f1c433fe04f2a0e3 scipy-1.6.0-cp37-cp37m-manylinux1_i686.whl 8840a9adb4ede3751f49761653d3ebf664f25195fdd42ada394ffea8903dd51d scipy-1.6.0-cp37-cp37m-manylinux1_x86_64.whl 8629135ee00cc2182ac8be8e75643b9f02235942443732c2ed69ab48edcb6614 scipy-1.6.0-cp37-cp37m-manylinux2014_aarch64.whl 58731bbe0103e96b89b2f41516699db9b63066e4317e31b8402891571f6d358f scipy-1.6.0-cp37-cp37m-win32.whl 876badc33eec20709d4e042a09834f5953ebdac4088d45a4f3a1f18b56885718 scipy-1.6.0-cp37-cp37m-win_amd64.whl c0911f3180de343643f369dc5cfedad6ba9f939c2d516bddea4a6871eb000722 scipy-1.6.0-cp38-cp38-macosx_10_9_x86_64.whl b8af26839ae343655f3ca377a5d5e5466f1d3b3ac7432a43449154fe958ae0e0 scipy-1.6.0-cp38-cp38-manylinux1_i686.whl 4f1d9cc977ac6a4a63c124045c1e8bf67ec37098f67c699887a93736961a00ae scipy-1.6.0-cp38-cp38-manylinux1_x86_64.whl eb7928275f3560d47e5538e15e9f32b3d64cd30ea8f85f3e82987425476f53f6 scipy-1.6.0-cp38-cp38-manylinux2014_aarch64.whl 31ab217b5c27ab429d07428a76002b33662f98986095bbce5d55e0788f7e8b15 scipy-1.6.0-cp38-cp38-win32.whl 2f1c2ebca6fd867160e70102200b1bd07b3b2d31a3e6af3c58d688c15d0d07b7 scipy-1.6.0-cp38-cp38-win_amd64.whl 155225621df90fcd151e25d51c50217e412de717475999ebb76e17e310176981 scipy-1.6.0-cp39-cp39-macosx_10_9_x86_64.whl f68d5761a2d2376e2b194c8e9192bbf7c51306ca176f1a0889990a52ef0d551f scipy-1.6.0-cp39-cp39-manylinux1_i686.whl d902d3a5ad7f28874c0a82db95246d24ca07ad932741df668595fe00a4819870 scipy-1.6.0-cp39-cp39-manylinux1_x86_64.whl aef3a2dbc436bbe8f6e0b635f0b5fe5ed024b522eee4637dbbe0b974129ca734 scipy-1.6.0-cp39-cp39-manylinux2014_aarch64.whl cdbc47628184a0ebeb5c08f1892614e1bd4a51f6e0d609c6eed253823a960f5b scipy-1.6.0-cp39-cp39-win32.whl 313785c4dab65060f9648112d025f6d2fec69a8a889c714328882d678a95f053 scipy-1.6.0-cp39-cp39-win_amd64.whl cb6dc9f82dfd95f6b9032a8d7ea70efeeb15d5b5fd6ed4e8537bb3c673580566 scipy-1.6.0.tar.gz dd06d02e8ac6e13e973cbab3da63888daf26a4fec1cd8a8d0804ec872864a7f5 scipy-1.6.0.tar.xz b0d5014d41fed5516d93d0c99a41465061b4d01cf4acd5cdc68c3be9e12c8563 scipy-1.6.0.zip -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Dec 31 11:13:17 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 31 Dec 2020 09:13:17 -0700 Subject: [Numpy-discussion] ANN: SciPy 1.6.0 In-Reply-To: References: Message-ID: On Thu, Dec 31, 2020 at 8:59 AM Tyler Reddy wrote: > Hi all, > > On behalf of the SciPy development team I'm pleased to announce > the release of SciPy 1.6.0. > > Sources and binary wheels can be found at: > https://pypi.org/project/scipy/ > and at: > https://github.com/scipy/scipy/releases/tag/v1.6.0 > > Thanks Tyler. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: