From ralf.gommers at gmail.com Sun Nov 1 17:37:07 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 1 Nov 2020 22:37:07 +0000 Subject: [SciPy-Dev] conda-forge now has 64-bit Windows packages Message-ID: Hi all, I thought this is a big enough deal that it's worth an announcement: at long last we have Windows packages for SciPy on conda-forge, thanks to Isuru Fernando. Previously there were 1.2.1 packages, but those were built with Intel Fortran outside of the conda-forge infrastructure and uploaded by hand (which is painful / not maintainable). Now there's a regular MSVC + mingw-w64 gfortran solution similar to how we build wheels. For those who are interested in the needed changes, see https://github.com/conda-forge/scipy-feedstock/pull/146 for de Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From andyfaff at gmail.com Mon Nov 2 05:41:28 2020 From: andyfaff at gmail.com (Andrew Nelson) Date: Mon, 2 Nov 2020 21:41:28 +1100 Subject: [SciPy-Dev] TravisCI billing changes Message-ID: Some of this might be relevant to us: https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing E.g. macOS wheels. -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Mon Nov 2 11:37:28 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 2 Nov 2020 09:37:28 -0700 Subject: [SciPy-Dev] NumPy 1.19.4 release Message-ID: Hi All, On behalf of the NumPy team I am pleased to announce the release of NumPy 1.19.4. NumPy 1.19.4 is a quick release to revert the OpenBLAS library version. It was hoped that the 0.3.12 OpenBLAS version used in 1.19.3 would work around the Microsoft fmod bug, but problems in some docker environments turned up. Instead, 1.19.4 will use the older library and run a sanity check on import, raising an error if the problem is detected. Microsoft is aware of the problem and has promised a fix, users should upgrade when it becomes available. This release supports Python 3.6-3.9. NumPy Wheels for this release can be downloaded from PyPI , source archives, release notes, and wheel hashes are available on Github . Linux users will need pip >= 0.19.3 in order to install manylinux2010 and manylinux2014 wheels. *Contributors* A total of 1 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Charles Harris *Pull requests merged* A total of 2 pull requests were merged for this release. - #17679: MAINT: Add check for Windows 10 version 2004 bug. - #17680: REV: Revert OpenBLAS to 1.19.2 version for 1.19.4 Cheers, Charles Harris -------------- next part -------------- An HTML attachment was scrubbed... URL: From rivas at snu.ac.kr Mon Nov 2 21:04:40 2020 From: rivas at snu.ac.kr (Jose Carlos RIVAS) Date: Tue, 3 Nov 2020 11:04:40 +0900 Subject: [SciPy-Dev] Chirp Z-transform Message-ID: An HTML attachment was scrubbed... URL: From rivas at snu.ac.kr Tue Nov 3 00:41:03 2020 From: rivas at snu.ac.kr (Jose Carlos RIVAS) Date: Tue, 3 Nov 2020 14:41:03 +0900 Subject: [SciPy-Dev] Chirp Z-transform Message-ID: To SciPy-Dev, Hi all, I've coded the Sukhoy's Chirp Z-transform algorithms, is there any interest in this addition to Scipy? If so, I'd be happy to contribute it. Thanks, J.C. Rivas p.s. sorry for the previous e-mail, I didn't note it was in html format From ralf.gommers at gmail.com Tue Nov 3 04:38:18 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Tue, 3 Nov 2020 09:38:18 +0000 Subject: [SciPy-Dev] Chirp Z-transform In-Reply-To: References: Message-ID: On Tue, Nov 3, 2020 at 2:31 AM Jose Carlos RIVAS wrote: > To SciPy-Dev, > > > Hi all, > > > I've coded the Sukhoy's Chirp Z-transform algorithms, is there any > interest in this addition to Scipy? If so, I'd be happy to contribute it. > Hi Jose Carlos, there's a chirp-Z implementation in progress that's close to ready (after a very long time): https://github.com/scipy/scipy/pull/4607. Could you look at that, and possibly help out there with review, testing, or proposing some improvements if your version turns out to be faster or have an extra feature? Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ihno.schrot at uni-heidelberg.de Tue Nov 3 12:21:16 2020 From: ihno.schrot at uni-heidelberg.de (Ihno Schrot) Date: Tue, 3 Nov 2020 18:21:16 +0100 Subject: [SciPy-Dev] Proposal to add two new linesearch strategies to scipy.optimize Message-ID: Hi all, I would like to propose to add two new linesearch strategies to scipy.optimize(.linesearch.py), namely the "Restrictive Monotonicity Test" (RMT) [1] and the "Backward Step Control" (BSC) [2]. It is well known that Newton-type methods for rootfinding or optimization problems can be forced to converge globally in a domain where the Jacobian or Hessian, respectively, is nonsingular. However, with linesearch strategies which are based on descent arguments with classical merit functions one usually has to pay the price of taking unnecessarily small stepsizes even in the domain where full steps would lead to convergence.?This is also the case for the existing linesearch strategies in SciPy. The suggested linesearch strategies were developed to achieve global convergence of Newton-type methods while avoiding too small step sizes. The RMT already has shown good performance in many practical applications. The BSC is a rather novel strategy but the numerical results available so far are promising and the BSC is proven to enable global convergence under reasonable assumptions. Moreover, I have done extensive numerical tests of both linesearch strategies in my master thesis. Although the tests have been focussed on comparing the two strategies, I also compared them to SciPys linesearch strategy that is based on the Wolfe condition and found that the RMT and BSC can outperform SciPys linesearch strategy in terms of the number of benchmark problems that can be solved. Therefore, I believe that adding the BSC and the RMT to SciPy benefits a wide range of users by providing modern powerful linesearch strategies. Especially, users would have two more linesearch strategies to apply to their problems, which can be helpful given that it is problem dependent which linesearch strategy works the best. If the community thinks that adding those two strategies is an useful enhancement, colleagues and I are happy to improve and eventually add our code to SciPy. Of course, we are also happy to answer any questions from the community regarding this proposal. Greetings from Heidelberg, Ihno [1] https://doi.org/10.1007/978-0-387-35514-6_3 [2] https://doi.org/10.1137/140968586 -- Ihno Schrot Institute for Applied Mathematics Heidelberg University E-Mail: ihno.schrot at uni-heidelberg.de -------------- next part -------------- An HTML attachment was scrubbed... URL: From rivas at snu.ac.kr Tue Nov 3 20:15:09 2020 From: rivas at snu.ac.kr (Jose Carlos RIVAS) Date: Wed, 4 Nov 2020 10:15:09 +0900 Subject: [SciPy-Dev] Chirp Z-transform Message-ID: Clearly I should be more careful and check patent rights in advance. I must withdraw my proposal, sorry. Jose Carlos RIVAS From charlesr.harris at gmail.com Tue Nov 3 21:25:36 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 3 Nov 2020 19:25:36 -0700 Subject: [SciPy-Dev] Chirp Z-transform In-Reply-To: References: Message-ID: On Tue, Nov 3, 2020 at 2:38 AM Ralf Gommers wrote: > > > On Tue, Nov 3, 2020 at 2:31 AM Jose Carlos RIVAS wrote: > >> To SciPy-Dev, >> >> >> Hi all, >> >> >> I've coded the Sukhoy's Chirp Z-transform algorithms, is there any >> interest in this addition to Scipy? If so, I'd be happy to contribute it. >> > > Hi Jose Carlos, there's a chirp-Z implementation in progress that's close > to ready (after a very long time): > https://github.com/scipy/scipy/pull/4607. Could you look at that, and > possibly help out there with review, testing, or proposing some > improvements if your version turns out to be faster or have an extra > feature? > > Cheers, > Ralf > > Hi Jose, Is this the inverse chirp-z that was discovered a year ago? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From rivas at snu.ac.kr Tue Nov 3 21:40:27 2020 From: rivas at snu.ac.kr (Jose Carlos RIVAS) Date: Wed, 4 Nov 2020 11:40:27 +0900 Subject: [SciPy-Dev] Chirp Z-transform Message-ID: An HTML attachment was scrubbed... URL: From martin at MPA-Garching.MPG.DE Wed Nov 4 07:27:48 2020 From: martin at MPA-Garching.MPG.DE (Martin Reinecke) Date: Wed, 4 Nov 2020 13:27:48 +0100 Subject: [SciPy-Dev] Chirp Z-transform Message-ID: (resending this; I sent this yesterday already but the mail was rejected because I was not subscribed.) Hi all, is this implmentation related to the new ICZT algorithm? (https://www.nature.com/articles/s41598-019-50234-9) This is absolutely fascinating stuff, but there also seems to be a patent on the algorithm: https://patentscope.wipo.int/search/en/detail.jsf?docId=WO2019023220 Maybe that's no problem at all (I haven't researched what the licensing terms are), but it might need careful checking. Cheers, Martin From charlesr.harris at gmail.com Wed Nov 4 11:47:23 2020 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 4 Nov 2020 09:47:23 -0700 Subject: [SciPy-Dev] Chirp Z-transform In-Reply-To: References: Message-ID: On Wed, Nov 4, 2020 at 5:28 AM Martin Reinecke wrote: > (resending this; I sent this yesterday already but the mail was rejected > because I was not subscribed.) > > Hi all, > > is this implmentation related to the new ICZT algorithm? > (https://www.nature.com/articles/s41598-019-50234-9) > > This is absolutely fascinating stuff, but there also seems to be a > patent on the algorithm: > https://patentscope.wipo.int/search/en/detail.jsf?docId=WO2019023220 > > Maybe that's no problem at all (I haven't researched what the licensing > terms are), but it might need careful checking. > > Reminds me of the fast Hartley transform that was patented by Stanford in 1987. the patent was dropped in 1995. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at MPA-Garching.MPG.DE Wed Nov 4 13:11:45 2020 From: martin at MPA-Garching.MPG.DE (Martin Reinecke) Date: Wed, 4 Nov 2020 19:11:45 +0100 Subject: [SciPy-Dev] Chirp Z-transform In-Reply-To: References: Message-ID: <1cbba9c1-d76d-d3bb-b590-b91753834aec@mpa-garching.mpg.de> On 11/4/20 5:47 PM, Charles R Harris wrote: > Reminds me of the fast Hartley transform that was patented by Stanford in > 1987. the patent was dropped in 1995. There are cases where a algorithm is patented, but it is still possible to have open-source implementations of it. One example is here: https://www.aanda.org/articles/aa/full_html/2018/08/aa32858-18/aa32858-18.html (ctrl-f "patent"). So maybe it's too early to give up hope. Cheers, Martin From robert.kern at gmail.com Wed Nov 4 13:28:15 2020 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 4 Nov 2020 13:28:15 -0500 Subject: [SciPy-Dev] Chirp Z-transform In-Reply-To: <1cbba9c1-d76d-d3bb-b590-b91753834aec@mpa-garching.mpg.de> References: <1cbba9c1-d76d-d3bb-b590-b91753834aec@mpa-garching.mpg.de> Message-ID: On Wed, Nov 4, 2020 at 1:14 PM Martin Reinecke wrote: > On 11/4/20 5:47 PM, Charles R Harris wrote: > > > Reminds me of the fast Hartley transform that was patented by Stanford in > > 1987. the patent was dropped in 1995. > > There are cases where a algorithm is patented, but it is still possible > to have open-source implementations of it. One example is here: > > > https://www.aanda.org/articles/aa/full_html/2018/08/aa32858-18/aa32858-18.html > (ctrl-f "patent"). > > So maybe it's too early to give up hope. > Sure, it is legal to have open source implementations of patented algorithms. You are just pushing off the responsibility to acquire a patent license onto the user of your software. The copyright license to _redistribute_ your implementation is technically orthogonal to the patent on the _use_ of the algorithm itself. Note that in the case you point to, it is the patent holder that released the open source implementation, which would not be the case here. As a matter of policy, we will not include implementations of patented algorithms in scipy because it makes the licensing story significantly more complicated. We want the rights to use the algorithms in scipy to be as free as the rights to redistribute. The proper place for code that is subject to patents is in a separate project. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Wed Nov 4 22:25:24 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Wed, 4 Nov 2020 20:25:24 -0700 Subject: [SciPy-Dev] ANN: SciPy 1.5.4 Message-ID: Hi all, On behalf of the SciPy development team I'm pleased to announce the release of SciPy 1.5.4, which is a bug fix release that includes Python 3.9 wheels and a more complete fix for build issues on XCode 12. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.5.4 One of a few ways to install this release with pip: pip install scipy==1.5.4 ===================== SciPy 1.5.4 Release Notes ===================== SciPy 1.5.4 is a bug-fix release with no new features compared to 1.5.3. Importantly, wheels are now available for Python 3.9 and a more complete fix has been applied for issues building with XCode 12. Authors ====== * Peter Bell * CJ Carey * Andrew McCluskey + * Andrew Nelson * Tyler Reddy * Eli Rykoff + * Ian Thomas + A total of 7 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.5.4 ------------------------------- * `#12763 `__: ndimage.fourier_ellipsoid segmentation fault * `#12789 `__: TestConvolve2d.test_large_array failing on Windows ILP64 CI job * `#12857 `__: sparse A[0,:] = ndarray is ok, A[:,0] = ndarray ValueError from... * `#12860 `__: BUG: Build failure with Xcode 12 * `#12935 `__: Failure to build with Python 3.9.0 on macOS * `#12966 `__: MAINT: lint_diff.py on some backport PRs * `#12988 `__: BUG: Highly multi-dimensional \`gaussian_kde\` giving \`-inf\`... Pull requests for 1.5.4 ------------------------------ * `#12790 `__: TST: Skip TestConvolve2d.test_large_array if not enough memory * `#12851 `__: BUG: sparse: fix inner indexed assignment of a 1d array * `#12875 `__: BUG: segfault in ndimage.fourier_ellipsoid with length-1 dims * `#12937 `__: CI: macOS3.9 testing * `#12957 `__: MAINT: fixes XCode 12/ python 3.9.0 build for 1.5.x maint branch * `#12959 `__: CI: add Windows Python 3.9 to CI * `#12974 `__: MAINT: Run lint_diff.py against the merge target and only for... * `#12978 `__: DOC: next_fast_len output doesn't match docstring * `#12979 `__: BUG: fft.next_fast_len should accept keyword arguments * `#12989 `__: BUG: improved the stability of kde for highly (1000s) multi-dimension... * `#13017 `__: BUG: Add explicit cast to _tmp sum. * `#13022 `__: TST: xfail test_maxiter_worsening() Checksums ========= MD5 ~~~ 09a446e10033c3132f1f257e3f4d9735 scipy-1.5.4-cp36-cp36m-macosx_10_9_x86_64.whl 25e58fde2fd4eb6c7717719db85e368b scipy-1.5.4-cp36-cp36m-manylinux1_i686.whl 2c9705cd57788ad79ea0c1015208f41f scipy-1.5.4-cp36-cp36m-manylinux1_x86_64.whl d0fb84f3ff45e4149698fbc662ac4d47 scipy-1.5.4-cp36-cp36m-manylinux2014_aarch64.whl f94f0e274cd2960ecb2d8751632e098c scipy-1.5.4-cp36-cp36m-win32.whl f56f4d5b67fccc49fb64331c28bdf7d1 scipy-1.5.4-cp36-cp36m-win_amd64.whl 33e0843f8619b78547866579134a733b scipy-1.5.4-cp37-cp37m-macosx_10_9_x86_64.whl 6720a406d82bd08c4370b665d5eddeb9 scipy-1.5.4-cp37-cp37m-manylinux1_i686.whl eafc3bc8a12d41cb348c73b54ad25ad5 scipy-1.5.4-cp37-cp37m-manylinux1_x86_64.whl 1174418ae0614d621acdb49faeaadcb8 scipy-1.5.4-cp37-cp37m-manylinux2014_aarch64.whl 5ca53c5cd6828498c0a41c3ae747a34b scipy-1.5.4-cp37-cp37m-win32.whl cdb91a7db9cf79b7446680f8d106aabc scipy-1.5.4-cp37-cp37m-win_amd64.whl 02a29a4eec9c61c30aef7439138fe1b3 scipy-1.5.4-cp38-cp38-macosx_10_9_x86_64.whl ce8e02167763493374c4bea807139a1b scipy-1.5.4-cp38-cp38-manylinux1_i686.whl 65ec027bfa6bed805dac62744b45c693 scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl c358b4b332cc9dbcd1eadc229d8b019e scipy-1.5.4-cp38-cp38-manylinux2014_aarch64.whl 492ec3bfe082229076a83d74cfa51d7e scipy-1.5.4-cp38-cp38-win32.whl d5d12211502429f3bc3074b12ca1f541 scipy-1.5.4-cp38-cp38-win_amd64.whl da25e7ac777e8b1b6cd7f117f163e6d2 scipy-1.5.4-cp39-cp39-macosx_10_9_x86_64.whl 12275e3578eb17065081d83d329d18db scipy-1.5.4-cp39-cp39-manylinux1_i686.whl 6778d670f75f536921c3d38e44517280 scipy-1.5.4-cp39-cp39-manylinux1_x86_64.whl efda61c74b29ffe714b6b842ec369a19 scipy-1.5.4-cp39-cp39-manylinux2014_aarch64.whl 107204c14328df879c5fc941e7829389 scipy-1.5.4-cp39-cp39-win32.whl ed6970f7538d38dd91a42950bd6843b7 scipy-1.5.4-cp39-cp39-win_amd64.whl 293401ee7ac354a2f2313373b497f40e scipy-1.5.4.tar.gz d446ec7a6b0bc44484389ab7589eccf5 scipy-1.5.4.tar.xz 47d0dabdc684475bc2aac7e8db9eea6f scipy-1.5.4.zip SHA256 ~~~~~~ 4f12d13ffbc16e988fa40809cbbd7a8b45bc05ff6ea0ba8e3e41f6f4db3a9e47 scipy-1.5.4-cp36-cp36m-macosx_10_9_x86_64.whl a254b98dbcc744c723a838c03b74a8a34c0558c9ac5c86d5561703362231107d scipy-1.5.4-cp36-cp36m-manylinux1_i686.whl 368c0f69f93186309e1b4beb8e26d51dd6f5010b79264c0f1e9ca00cd92ea8c9 scipy-1.5.4-cp36-cp36m-manylinux1_x86_64.whl 4598cf03136067000855d6b44d7a1f4f46994164bcd450fb2c3d481afc25dd06 scipy-1.5.4-cp36-cp36m-manylinux2014_aarch64.whl e98d49a5717369d8241d6cf33ecb0ca72deee392414118198a8e5b4c35c56340 scipy-1.5.4-cp36-cp36m-win32.whl 65923bc3809524e46fb7eb4d6346552cbb6a1ffc41be748535aa502a2e3d3389 scipy-1.5.4-cp36-cp36m-win_amd64.whl 9ad4fcddcbf5dc67619379782e6aeef41218a79e17979aaed01ed099876c0e62 scipy-1.5.4-cp37-cp37m-macosx_10_9_x86_64.whl f87b39f4d69cf7d7529d7b1098cb712033b17ea7714aed831b95628f483fd012 scipy-1.5.4-cp37-cp37m-manylinux1_i686.whl 25b241034215247481f53355e05f9e25462682b13bd9191359075682adcd9554 scipy-1.5.4-cp37-cp37m-manylinux1_x86_64.whl fa789583fc94a7689b45834453fec095245c7e69c58561dc159b5d5277057e4c scipy-1.5.4-cp37-cp37m-manylinux2014_aarch64.whl d6d25c41a009e3c6b7e757338948d0076ee1dd1770d1c09ec131f11946883c54 scipy-1.5.4-cp37-cp37m-win32.whl 2c872de0c69ed20fb1a9b9cf6f77298b04a26f0b8720a5457be08be254366c6e scipy-1.5.4-cp37-cp37m-win_amd64.whl e360cb2299028d0b0d0f65a5c5e51fc16a335f1603aa2357c25766c8dab56938 scipy-1.5.4-cp38-cp38-macosx_10_9_x86_64.whl 3397c129b479846d7eaa18f999369a24322d008fac0782e7828fa567358c36ce scipy-1.5.4-cp38-cp38-manylinux1_i686.whl 168c45c0c32e23f613db7c9e4e780bc61982d71dcd406ead746c7c7c2f2004ce scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl 213bc59191da2f479984ad4ec39406bf949a99aba70e9237b916ce7547b6ef42 scipy-1.5.4-cp38-cp38-manylinux2014_aarch64.whl 634568a3018bc16a83cda28d4f7aed0d803dd5618facb36e977e53b2df868443 scipy-1.5.4-cp38-cp38-win32.whl b03c4338d6d3d299e8ca494194c0ae4f611548da59e3c038813f1a43976cb437 scipy-1.5.4-cp38-cp38-win_amd64.whl 3d5db5d815370c28d938cf9b0809dade4acf7aba57eaf7ef733bfedc9b2474c4 scipy-1.5.4-cp39-cp39-macosx_10_9_x86_64.whl 6b0ceb23560f46dd236a8ad4378fc40bad1783e997604ba845e131d6c680963e scipy-1.5.4-cp39-cp39-manylinux1_i686.whl ed572470af2438b526ea574ff8f05e7f39b44ac37f712105e57fc4d53a6fb660 scipy-1.5.4-cp39-cp39-manylinux1_x86_64.whl 8c8d6ca19c8497344b810b0b0344f8375af5f6bb9c98bd42e33f747417ab3f57 scipy-1.5.4-cp39-cp39-manylinux2014_aarch64.whl d84cadd7d7998433334c99fa55bcba0d8b4aeff0edb123b2a1dfcface538e474 scipy-1.5.4-cp39-cp39-win32.whl cc1f78ebc982cd0602c9a7615d878396bec94908db67d4ecddca864d049112f2 scipy-1.5.4-cp39-cp39-win_amd64.whl 4a453d5e5689de62e5d38edf40af3f17560bfd63c9c5bd228c18c1f99afa155b scipy-1.5.4.tar.gz 5c87347bfe2db6e23d391aa226584f6b280248c0ca71e08f26f1faf9d7a76bc9 scipy-1.5.4.tar.xz e0bcc10c133a151937550bb42301c56439d34098b1b8f9dd18c5919d604edd37 scipy-1.5.4.zip -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Sat Nov 7 17:48:32 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Sat, 7 Nov 2020 15:48:32 -0700 Subject: [SciPy-Dev] Proposed 1.6.0 Release Schedule Message-ID: Hi, SciPy 1.5.0 was released June 21 (~5 months ago), and I think we'd like to keep a roughly biannual release cadence. I'd like to propose the following schedule for 1.6.0: - November 17: branch 1.6.x - November 20: rc1 - December 1: rc2 (if needed) - December 10: final release As always, it is a good idea to start tagging things that should be in 1.6.0 & please do help with reviewing PRs/issues that are tagged--current counts are: - PRs: 45 open with 1.6.0 milestone - issues: 25 open with 1.6.0 milestone While helping with that, also great if the release notes wiki is updated for appropriate changes: https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.6.0 Thoughts/objections for the schedule? Best wishes, Tyler -------------- next part -------------- An HTML attachment was scrubbed... URL: From omar.younis at hotmail.it Sun Nov 8 09:58:21 2020 From: omar.younis at hotmail.it (Omar Younis) Date: Sun, 8 Nov 2020 14:58:21 +0000 Subject: [SciPy-Dev] Add remove method to scipy.spatial.cKDTree Message-ID: I've sent a pull request to add remove method to scipy.spatial.cKDTree. You can find it here: https://github.com/scipy/scipy/pull/13050 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sun Nov 8 16:53:17 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 8 Nov 2020 21:53:17 +0000 Subject: [SciPy-Dev] Proposed 1.6.0 Release Schedule In-Reply-To: References: Message-ID: On Sat, Nov 7, 2020 at 10:48 PM Tyler Reddy wrote: > Hi, > > SciPy 1.5.0 was released June 21 (~5 months ago), and I think we'd like to > keep a roughly biannual release cadence. > > I'd like to propose the following schedule for 1.6.0: > - November 17: branch 1.6.x > - November 20: rc1 > - December 1: rc2 (if needed) > - December 10: final release > > As always, it is a good idea to start tagging things that should be in > 1.6.0 & please do help with reviewing PRs/issues that are tagged--current > counts are: > > - PRs: 45 open with 1.6.0 milestone > - issues: 25 open with 1.6.0 milestone > > While helping with that, also great if the release notes wiki is updated > for appropriate changes: > https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.6.0 > > Thoughts/objections for the schedule? > It does seem like there's a lot of PRs open, having only one weekend left seems optimistic. The majority can be bumped to 1.7.0, but there's a bit of a backlog of nice PRs that have been ready for a while and would be nice to get in. For example: - KDTree/cKDTree feature parity: https://github.com/scipy/scipy/pull/12852 - HiGHS solver as linprog method: https://github.com/scipy/scipy/pull/12043 - Balanced cut tree: https://github.com/scipy/scipy/pull/10730 - Andrew's set of optimize PRs that he brought up on the mailing list recently - Greg's set of ndimage PRs, complex kernels and the boundary handling ones (#12725, #12767, #12776) If it's possible, given your time constraints, to bump the schedule by one week then that may be useful. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Sun Nov 8 17:43:21 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Sun, 8 Nov 2020 15:43:21 -0700 Subject: [SciPy-Dev] Proposed 1.6.0 Release Schedule In-Reply-To: References: Message-ID: Ok, I'll bump it by a week then. - November 24: branch maintenance/1.6.x - November 27: rc1 - December 8: rc2 (if needed) - December 17: final release I was trying to avoid the overlap with US Thanksgiving and a final release in late December near Winter Break, but I'll manage. Tyler On Sun, 8 Nov 2020 at 14:53, Ralf Gommers wrote: > > > On Sat, Nov 7, 2020 at 10:48 PM Tyler Reddy > wrote: > >> Hi, >> >> SciPy 1.5.0 was released June 21 (~5 months ago), and I think we'd like >> to keep a roughly biannual release cadence. >> >> I'd like to propose the following schedule for 1.6.0: >> - November 17: branch 1.6.x >> - November 20: rc1 >> - December 1: rc2 (if needed) >> - December 10: final release >> >> As always, it is a good idea to start tagging things that should be in >> 1.6.0 & please do help with reviewing PRs/issues that are tagged--current >> counts are: >> >> - PRs: 45 open with 1.6.0 milestone >> - issues: 25 open with 1.6.0 milestone >> >> While helping with that, also great if the release notes wiki is updated >> for appropriate changes: >> https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.6.0 >> >> Thoughts/objections for the schedule? >> > > > It does seem like there's a lot of PRs open, having only one weekend left > seems optimistic. The majority can be bumped to 1.7.0, but there's a bit of > a backlog of nice PRs that have been ready for a while and would be nice to > get in. For example: > > - KDTree/cKDTree feature parity: https://github.com/scipy/scipy/pull/12852 > - HiGHS solver as linprog method: > https://github.com/scipy/scipy/pull/12043 > - Balanced cut tree: https://github.com/scipy/scipy/pull/10730 > - Andrew's set of optimize PRs that he brought up on the mailing list > recently > - Greg's set of ndimage PRs, complex kernels and the boundary handling > ones (#12725, #12767, #12776) > > If it's possible, given your time constraints, to bump the schedule by one > week then that may be useful. > > Cheers, > Ralf > > > > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Mon Nov 9 02:11:40 2020 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Mon, 9 Nov 2020 10:11:40 +0300 Subject: [SciPy-Dev] Proposed 1.6.0 Release Schedule In-Reply-To: References: Message-ID: How about shifting the release towards January then? ??, 9 ????. 2020 ?., 1:43 Tyler Reddy : > Ok, I'll bump it by a week then. > > - November 24: branch maintenance/1.6.x > - November 27: rc1 > - December 8: rc2 (if needed) > - December 17: final release > > > I was trying to avoid the overlap with US Thanksgiving and a final release > in late December near Winter Break, but I'll manage. > > Tyler > > On Sun, 8 Nov 2020 at 14:53, Ralf Gommers wrote: > >> >> >> On Sat, Nov 7, 2020 at 10:48 PM Tyler Reddy >> wrote: >> >>> Hi, >>> >>> SciPy 1.5.0 was released June 21 (~5 months ago), and I think we'd like >>> to keep a roughly biannual release cadence. >>> >>> I'd like to propose the following schedule for 1.6.0: >>> - November 17: branch 1.6.x >>> - November 20: rc1 >>> - December 1: rc2 (if needed) >>> - December 10: final release >>> >>> As always, it is a good idea to start tagging things that should be in >>> 1.6.0 & please do help with reviewing PRs/issues that are tagged--current >>> counts are: >>> >>> - PRs: 45 open with 1.6.0 milestone >>> - issues: 25 open with 1.6.0 milestone >>> >>> While helping with that, also great if the release notes wiki is updated >>> for appropriate changes: >>> https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.6.0 >>> >>> >>> Thoughts/objections for the schedule? >>> >> >> >> It does seem like there's a lot of PRs open, having only one weekend left >> seems optimistic. The majority can be bumped to 1.7.0, but there's a bit of >> a backlog of nice PRs that have been ready for a while and would be nice to >> get in. For example: >> >> - KDTree/cKDTree feature parity: >> https://github.com/scipy/scipy/pull/12852 >> - HiGHS solver as linprog method: >> https://github.com/scipy/scipy/pull/12043 >> - Balanced cut tree: https://github.com/scipy/scipy/pull/10730 >> - Andrew's set of optimize PRs that he brought up on the mailing list >> recently >> - Greg's set of ndimage PRs, complex kernels and the boundary handling >> ones (#12725, #12767, #12776) >> >> If it's possible, given your time constraints, to bump the schedule by >> one week then that may be useful. >> >> Cheers, >> Ralf >> >> >> >> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at python.org >> https://mail.python.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Wed Nov 11 17:16:52 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Wed, 11 Nov 2020 15:16:52 -0700 Subject: [SciPy-Dev] proposal to drop Python 3.6 and NumPy 1.14 / 1.15 In-Reply-To: References: Message-ID: Time to follow up here I think. At the moment, our wheels repo weekly cron job for POSIX systems includes multiple Python 3.6 matrix entries and they all build/pass tests: https://travis-ci.com/github/MacPython/scipy-wheels/builds/199857636 This means a few things: 1) Obviously, we have not yet turned off support for Python 3.6 in the master branch yet 2) At the moment, Python 3.6 does not appear to be causing problems Of course, just because there are no problems with 3.6 now does not mean that there won't be problems when backporting later as master branch evolves. Is the final decision here to turn off 3.6 support before we branch for 1.6.x series? Best wishes, Tyler On Fri, 11 Sep 2020 at 15:42, Ralf Gommers wrote: > > > On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < > warren.weckesser at gmail.com> wrote: > >> Sorry for the top post, but this question doesn't follow immediately >> from any previous comment. >> >> Is the conclusion of this thread that with SciPy 1.6 we will >> definitely drop Python 3.6 support? > > > I think so. Given that there were a few hesitations at least, I'll split > it in separate PRs - will do the NumPy one first. > > It might be useful to use >> dataclasses (https://docs.python.org/3/library/dataclasses.html), and >> they are new in Python 3.7. >> > > You want to start using them now, or is it more a "nice to have the > option"? > > Cheers, > Ralf > > >> Warren >> >> >> On 8/20/20, Charles R Harris wrote: >> > On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat >> wrote: >> > >> >> I have too many personas in my head arguing about this. And many of >> them >> >> are in conflict. >> >> >> >> 1- I think I had enough arguments online to be known as buying none of >> >> that scientific stability views. That doesn't mean I don't get their >> >> point, >> >> I really really do, but still. It's in the job description. Software >> is a >> >> big part of science just like the papers. (What Matti said) >> >> 2- Even a two-year old code is not safe to let it collect dust and >> >> constantly requires attention and becomes a liability rather than a >> tool >> >> (what Bennet said). >> >> 3- Matlab breaks things all the time with 0 user consideration. None. >> But >> >> admittedly they break in style. We shouldn't be like them (not even >> >> close) >> >> but the point is when it is a commercial product, we don't hear the >> >> uproar >> >> we receive. People silently fix things. >> >> 4- Just because we update the python version, a lot of packages stop >> >> working. This is seriously disheartening. Happens to me all the time >> >> (protobuf, influxdb etc). And really annoying if one package has not >> >> released the new version yet. >> >> 5- Python is releasing too quick. I know Python is not Fortran >> >> (thankfully) but this is the other end of the spectrum. With every >> >> version >> >> one or two of us spent at least 2 weeks of intensive "What did they >> >> change >> >> on Windows again?" bug hunting. Hence our platform is not reliable and >> >> now >> >> they want 12 months. (What Ralf said) >> >> 6- There are always new users, downloading Python for the first time >> and >> >> it is expected that SciPy stack should work off-the-shelf. Hence we >> don't >> >> have the luxury to wait and see (see 4th item). >> >> 7- If there are limited resources for software support, then why people >> >> take the risk and update their stuff is something that has been >> discussed >> >> for decades now. And no one seems to converge to a point. (What Matti >> >> said) >> >> 8- what Evgeni suggested >> >> >> >> I'm not sure what to do about this but I am also more worried about the >> >> Python version than the NumPy version, my limited anectodal evidence >> >> around >> >> me suggests that people update NumPy + SciPy together mainly when they >> >> use >> >> pip with -U (--upgrade) on some package that has SciPy as a dependency. >> >> So >> >> I feel that it is safe to bump. >> >> >> >> >> > Just for the NumPy perspective, there is nothing in the forthcoming >> NumPy >> > 1.20 that doesn't work with Python 3.6, it is just a question of what >> > wheels to provide. OTOH, as Python advances I don't want to worry about >> > someone using newer features, whatever they are. I go back and forth, >> but >> > feel that 48 months is about the right support window and that argues >> for >> > dropping Python 3.6. With the faster pace of Python development we will >> > probably want to support the last four versions going forward. Note that >> > SciPy doesn't need to support all the NumPy versions that support a >> > particular version of Python, just the latest one. The main concern >> doesn't >> > seem to be compatibility as much as availability. >> > >> > >> > >> > Chuck >> > >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at python.org >> https://mail.python.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andyfaff at gmail.com Wed Nov 11 17:51:52 2020 From: andyfaff at gmail.com (Andrew Nelson) Date: Thu, 12 Nov 2020 09:51:52 +1100 Subject: [SciPy-Dev] proposal to drop Python 3.6 and NumPy 1.14 / 1.15 In-Reply-To: References: Message-ID: My thoughts would be to drop 3.6 for 1.6, but obviously we need to retain for 1.5.X. At the moment we're supporting 4 Python releases. It would make CI burden a bit less. -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivetti at fbk.eu Wed Nov 11 18:45:23 2020 From: olivetti at fbk.eu (Emanuele Olivetti) Date: Thu, 12 Nov 2020 00:45:23 +0100 Subject: [SciPy-Dev] Cannot generate very large very sparse random matrix Message-ID: Hi, I've just noticed that it is not possible to generate very large very sparse random matrices with scipy.sparse.random(). For example: scipy.sparse.random(1_000_000, 1_000_000, density = 1e-11) should create a sparse matrix with only 10 non-zero values... but instead triggers a MemoryError: ---- MemoryError Traceback (most recent call last) in ----> 1 scipy.sparse.random(1_000_000, 1_000_000, density = 1e-11) ~/miniconda3/envs/lap/lib/python3.8/site-packages/scipy/sparse/construct.py in random(m, n, density, format, dtype, random_state, data_rvs) 787 data_rvs = partial(random_state.uniform, 0., 1.) 788 --> 789 ind = random_state.choice(mn, size=k, replace=False) 790 791 j = np.floor(ind * 1. / m).astype(tp, copy=False) mtrand.pyx in numpy.random.mtrand.RandomState.choice() mtrand.pyx in numpy.random.mtrand.RandomState.permutation() MemoryError: Unable to allocate 7.28 TiB for an array with shape (1000000000000,) and data type int64 ---- Here is the problematic line in current master branch of SciPy: https://github.com/scipy/scipy/blob/master/scipy/sparse/construct.py#L806 In short, the issue is due to random_state.choice(... replace=False) which needs to allocate the humongous array in order to pick the ten random numbers... I understand the technical difficulty of generating random numbers without replacement, but it is quite counterintuitive that in order to generate a sparse random matrix it is necessary to create an equally large but *dense* vector first. Is there a solution to this problem? Thanks in advance, Emanuele -- -- Le informazioni contenute nella presente comunicazione sono di natura? privata e come tali sono da considerarsi riservate ed indirizzate? esclusivamente ai destinatari indicati e per le finalit? strettamente? legate al relativo contenuto. Se avete ricevuto questo messaggio per? errore, vi preghiamo di eliminarlo e di inviare una comunicazione? all?indirizzo e-mail del mittente. -- The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you received this in error, please contact the sender and delete the material. -------------- next part -------------- An HTML attachment was scrubbed... URL: From perimosocordiae at gmail.com Fri Nov 13 10:22:42 2020 From: perimosocordiae at gmail.com (CJ Carey) Date: Fri, 13 Nov 2020 10:22:42 -0500 Subject: [SciPy-Dev] Cannot generate very large very sparse random matrix In-Reply-To: References: Message-ID: This is a known issue, see https://github.com/scipy/scipy/issues/9699. I haven't checked on the status of numpy.random.Generator.choice() in a while, so maybe the issue can be resolved now. On Wed, Nov 11, 2020 at 6:46 PM Emanuele Olivetti wrote: > Hi, > > I've just noticed that it is not possible to generate very large very > sparse random matrices with scipy.sparse.random(). For example: > scipy.sparse.random(1_000_000, 1_000_000, density = 1e-11) > should create a sparse matrix with only 10 non-zero values... but instead > triggers a MemoryError: > ---- > MemoryError Traceback (most recent call last) > in > ----> 1 scipy.sparse.random(1_000_000, 1_000_000, density = 1e-11) > > ~/miniconda3/envs/lap/lib/python3.8/site-packages/scipy/sparse/construct.py > in random(m, n, density, format, dtype, random_state, data_rvs) > 787 data_rvs = partial(random_state.uniform, 0., 1.) > 788 > --> 789 ind = random_state.choice(mn, size=k, replace=False) > 790 > 791 j = np.floor(ind * 1. / m).astype(tp, copy=False) > > mtrand.pyx in numpy.random.mtrand.RandomState.choice() > > mtrand.pyx in numpy.random.mtrand.RandomState.permutation() > > MemoryError: Unable to allocate 7.28 TiB for an array with shape > (1000000000000,) and data type int64 > ---- > > Here is the problematic line in current master branch of SciPy: > > https://github.com/scipy/scipy/blob/master/scipy/sparse/construct.py#L806 > > In short, the issue is due to random_state.choice(... replace=False) which > needs to allocate the humongous array in order to pick the ten random > numbers... > > I understand the technical difficulty of generating random numbers without > replacement, but it is quite counterintuitive that in order to generate a > sparse random matrix it is necessary to create an equally large but *dense* > vector first. > > Is there a solution to this problem? > > Thanks in advance, > > Emanuele > > > > > -- > Le informazioni contenute nella presente comunicazione sono di natura privata > e come tali sono da considerarsi riservate ed indirizzate esclusivamente > ai destinatari indicati e per le finalit? strettamente legate al relativo > contenuto. Se avete ricevuto questo messaggio per errore, vi preghiamo di > eliminarlo e di inviare una comunicazione all?indirizzo e-mail del > mittente. > -- > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and/or privileged > material. If you received this in error, please contact the sender and > delete the material. > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivetti at fbk.eu Fri Nov 13 18:28:29 2020 From: olivetti at fbk.eu (Emanuele Olivetti) Date: Sat, 14 Nov 2020 00:28:29 +0100 Subject: [SciPy-Dev] Cannot generate very large very sparse random matrix In-Reply-To: References: Message-ID: Thank you for your response. Indeed numpy.random.Generator.choice() solves the problem: ---- rng = np.random.default_rng() rng.choice(1_000_0000_000_000_000, size=10, replace=False) array([7363643319410659, 1001129358099623, 7384908776761990, 3610742892883208, 9484192959193500, 6273686405826185, 1550972534180773, 1845765940909299, 144504113475750, 7853188631204629]) ---- while: ---- np.random.choice(1_000_0000_000_000_000, size=10, replace=False) --------------------------------------------------------------------------- MemoryError Traceback (most recent call last) in ----> 1 np.random.choice(1_000_0000_000_000_000, size=10, replace=False) mtrand.pyx in numpy.random.mtrand.RandomState.choice() mtrand.pyx in numpy.random.mtrand.RandomState.permutation() MemoryError: Unable to allocate 71.1 PiB for an array with shape (10000000000000000,) and data type int64 ---- According to the latest comment on the github issue you mentioned: "It looks like np.random.Generator should be available from numpy 1.17 on, and the current minimum numpy version is 1.16.5."... So this may require a little while... As a quick fix but also meaningful new feature, would it be possible to extend the API of scipy.sparse.random() and to add the option "replace=False" (then piped to np.random.choice()) which, if set to True, would give the liberty to the user to solve the issue for very large very sparse matrices at the cost of some (rare) collisions? I would gladly accept it - and that's also my current fix on my local SciPy. Best, Emanuele On Fri, Nov 13, 2020 at 4:23 PM CJ Carey wrote: > This is a known issue, see https://github.com/scipy/scipy/issues/9699. > > I haven't checked on the status of numpy.random.Generator.choice() in a > while, so maybe the issue can be resolved now. > > On Wed, Nov 11, 2020 at 6:46 PM Emanuele Olivetti wrote: > >> Hi, >> >> I've just noticed that it is not possible to generate very large very >> sparse random matrices with scipy.sparse.random(). For example: >> scipy.sparse.random(1_000_000, 1_000_000, density = 1e-11) >> should create a sparse matrix with only 10 non-zero values... but instead >> triggers a MemoryError: >> ---- >> MemoryError Traceback (most recent call >> last) >> in >> ----> 1 scipy.sparse.random(1_000_000, 1_000_000, density = 1e-11) >> >> ~/miniconda3/envs/lap/lib/python3.8/site-packages/scipy/sparse/construct.py >> in random(m, n, density, format, dtype, random_state, data_rvs) >> 787 data_rvs = partial(random_state.uniform, 0., 1.) >> 788 >> --> 789 ind = random_state.choice(mn, size=k, replace=False) >> 790 >> 791 j = np.floor(ind * 1. / m).astype(tp, copy=False) >> >> mtrand.pyx in numpy.random.mtrand.RandomState.choice() >> >> mtrand.pyx in numpy.random.mtrand.RandomState.permutation() >> >> MemoryError: Unable to allocate 7.28 TiB for an array with shape >> (1000000000000,) and data type int64 >> ---- >> >> Here is the problematic line in current master branch of SciPy: >> >> https://github.com/scipy/scipy/blob/master/scipy/sparse/construct.py#L806 >> >> In short, the issue is due to random_state.choice(... replace=False) >> which needs to allocate the humongous array in order to pick the ten random >> numbers... >> >> I understand the technical difficulty of generating random numbers >> without replacement, but it is quite counterintuitive that in order to >> generate a sparse random matrix it is necessary to create an equally large >> but *dense* vector first. >> >> Is there a solution to this problem? >> >> Thanks in advance, >> >> Emanuele >> >> >> >> >> -- >> Le informazioni contenute nella presente comunicazione sono di natura privata >> e come tali sono da considerarsi riservate ed indirizzate esclusivamente >> ai destinatari indicati e per le finalit? strettamente legate al >> relativo contenuto. Se avete ricevuto questo messaggio per errore, vi >> preghiamo di eliminarlo e di inviare una comunicazione all?indirizzo >> e-mail del mittente. >> -- >> The information transmitted is intended only for the person or entity to >> which it is addressed and may contain confidential and/or privileged >> material. If you received this in error, please contact the sender and >> delete the material. >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at python.org >> https://mail.python.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -- -- Le informazioni contenute nella presente comunicazione sono di natura? privata e come tali sono da considerarsi riservate ed indirizzate? esclusivamente ai destinatari indicati e per le finalit? strettamente? legate al relativo contenuto. Se avete ricevuto questo messaggio per? errore, vi preghiamo di eliminarlo e di inviare una comunicazione? all?indirizzo e-mail del mittente. -- The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you received this in error, please contact the sender and delete the material. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sat Nov 14 17:39:00 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sat, 14 Nov 2020 22:39:00 +0000 Subject: [SciPy-Dev] proposal to drop Python 3.6 and NumPy 1.14 / 1.15 In-Reply-To: References: Message-ID: On Wed, Nov 11, 2020 at 10:17 PM Tyler Reddy wrote: > Time to follow up here I think. At the moment, our wheels repo weekly cron > job for POSIX systems includes multiple Python 3.6 matrix entries and they > all build/pass tests: > https://travis-ci.com/github/MacPython/scipy-wheels/builds/199857636 > > This means a few things: > 1) Obviously, we have not yet turned off support for Python 3.6 in the > master branch yet > 2) At the moment, Python 3.6 does not appear to be causing problems > > Of course, just because there are no problems with 3.6 now does not mean > that there won't be problems when backporting later as master branch > evolves. > > Is the final decision here to turn off 3.6 support before we branch for > 1.6.x series? > Yes, we should. Timing of this conversation was a bit unfortunate for me, so I didn't get around to opening a PR in September. I can do so tomorrow. Cheers, Ralf > Best wishes, > Tyler > > On Fri, 11 Sep 2020 at 15:42, Ralf Gommers wrote: > >> >> >> On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < >> warren.weckesser at gmail.com> wrote: >> >>> Sorry for the top post, but this question doesn't follow immediately >>> from any previous comment. >>> >>> Is the conclusion of this thread that with SciPy 1.6 we will >>> definitely drop Python 3.6 support? >> >> >> I think so. Given that there were a few hesitations at least, I'll split >> it in separate PRs - will do the NumPy one first. >> >> It might be useful to use >>> dataclasses (https://docs.python.org/3/library/dataclasses.html), and >>> they are new in Python 3.7. >>> >> >> You want to start using them now, or is it more a "nice to have the >> option"? >> >> Cheers, >> Ralf >> >> >>> Warren >>> >>> >>> On 8/20/20, Charles R Harris wrote: >>> > On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat >>> wrote: >>> > >>> >> I have too many personas in my head arguing about this. And many of >>> them >>> >> are in conflict. >>> >> >>> >> 1- I think I had enough arguments online to be known as buying none of >>> >> that scientific stability views. That doesn't mean I don't get their >>> >> point, >>> >> I really really do, but still. It's in the job description. Software >>> is a >>> >> big part of science just like the papers. (What Matti said) >>> >> 2- Even a two-year old code is not safe to let it collect dust and >>> >> constantly requires attention and becomes a liability rather than a >>> tool >>> >> (what Bennet said). >>> >> 3- Matlab breaks things all the time with 0 user consideration. None. >>> But >>> >> admittedly they break in style. We shouldn't be like them (not even >>> >> close) >>> >> but the point is when it is a commercial product, we don't hear the >>> >> uproar >>> >> we receive. People silently fix things. >>> >> 4- Just because we update the python version, a lot of packages stop >>> >> working. This is seriously disheartening. Happens to me all the time >>> >> (protobuf, influxdb etc). And really annoying if one package has not >>> >> released the new version yet. >>> >> 5- Python is releasing too quick. I know Python is not Fortran >>> >> (thankfully) but this is the other end of the spectrum. With every >>> >> version >>> >> one or two of us spent at least 2 weeks of intensive "What did they >>> >> change >>> >> on Windows again?" bug hunting. Hence our platform is not reliable and >>> >> now >>> >> they want 12 months. (What Ralf said) >>> >> 6- There are always new users, downloading Python for the first time >>> and >>> >> it is expected that SciPy stack should work off-the-shelf. Hence we >>> don't >>> >> have the luxury to wait and see (see 4th item). >>> >> 7- If there are limited resources for software support, then why >>> people >>> >> take the risk and update their stuff is something that has been >>> discussed >>> >> for decades now. And no one seems to converge to a point. (What Matti >>> >> said) >>> >> 8- what Evgeni suggested >>> >> >>> >> I'm not sure what to do about this but I am also more worried about >>> the >>> >> Python version than the NumPy version, my limited anectodal evidence >>> >> around >>> >> me suggests that people update NumPy + SciPy together mainly when they >>> >> use >>> >> pip with -U (--upgrade) on some package that has SciPy as a >>> dependency. >>> >> So >>> >> I feel that it is safe to bump. >>> >> >>> >> >>> > Just for the NumPy perspective, there is nothing in the forthcoming >>> NumPy >>> > 1.20 that doesn't work with Python 3.6, it is just a question of what >>> > wheels to provide. OTOH, as Python advances I don't want to worry about >>> > someone using newer features, whatever they are. I go back and forth, >>> but >>> > feel that 48 months is about the right support window and that argues >>> for >>> > dropping Python 3.6. With the faster pace of Python development we will >>> > probably want to support the last four versions going forward. Note >>> that >>> > SciPy doesn't need to support all the NumPy versions that support a >>> > particular version of Python, just the latest one. The main concern >>> doesn't >>> > seem to be compatibility as much as availability. >>> > >>> > >>> > >>> > Chuck >>> > >>> _______________________________________________ >>> SciPy-Dev mailing list >>> SciPy-Dev at python.org >>> https://mail.python.org/mailman/listinfo/scipy-dev >>> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at python.org >> https://mail.python.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Sat Nov 14 17:47:02 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Sat, 14 Nov 2020 15:47:02 -0700 Subject: [SciPy-Dev] proposal to drop Python 3.6 and NumPy 1.14 / 1.15 In-Reply-To: References: Message-ID: Hi Ralf, I opened one yesterday: https://github.com/scipy/scipy/pull/13081 Feel free to replace it though. It was based almost purely on the commit where you did the same thing for 3.5 last time. Tyler On Sat, 14 Nov 2020 at 15:39, Ralf Gommers wrote: > > > On Wed, Nov 11, 2020 at 10:17 PM Tyler Reddy > wrote: > >> Time to follow up here I think. At the moment, our wheels repo weekly >> cron job for POSIX systems includes multiple Python 3.6 matrix entries and >> they all build/pass tests: >> https://travis-ci.com/github/MacPython/scipy-wheels/builds/199857636 >> >> This means a few things: >> 1) Obviously, we have not yet turned off support for Python 3.6 in the >> master branch yet >> 2) At the moment, Python 3.6 does not appear to be causing problems >> >> Of course, just because there are no problems with 3.6 now does not mean >> that there won't be problems when backporting later as master branch >> evolves. >> >> Is the final decision here to turn off 3.6 support before we branch for >> 1.6.x series? >> > > Yes, we should. Timing of this conversation was a bit unfortunate for me, > so I didn't get around to opening a PR in September. I can do so tomorrow. > > Cheers, > Ralf > > > >> Best wishes, >> Tyler >> >> On Fri, 11 Sep 2020 at 15:42, Ralf Gommers >> wrote: >> >>> >>> >>> On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < >>> warren.weckesser at gmail.com> wrote: >>> >>>> Sorry for the top post, but this question doesn't follow immediately >>>> from any previous comment. >>>> >>>> Is the conclusion of this thread that with SciPy 1.6 we will >>>> definitely drop Python 3.6 support? >>> >>> >>> I think so. Given that there were a few hesitations at least, I'll split >>> it in separate PRs - will do the NumPy one first. >>> >>> It might be useful to use >>>> dataclasses (https://docs.python.org/3/library/dataclasses.html), and >>>> they are new in Python 3.7. >>>> >>> >>> You want to start using them now, or is it more a "nice to have the >>> option"? >>> >>> Cheers, >>> Ralf >>> >>> >>>> Warren >>>> >>>> >>>> On 8/20/20, Charles R Harris wrote: >>>> > On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat >>>> wrote: >>>> > >>>> >> I have too many personas in my head arguing about this. And many of >>>> them >>>> >> are in conflict. >>>> >> >>>> >> 1- I think I had enough arguments online to be known as buying none >>>> of >>>> >> that scientific stability views. That doesn't mean I don't get their >>>> >> point, >>>> >> I really really do, but still. It's in the job description. Software >>>> is a >>>> >> big part of science just like the papers. (What Matti said) >>>> >> 2- Even a two-year old code is not safe to let it collect dust and >>>> >> constantly requires attention and becomes a liability rather than a >>>> tool >>>> >> (what Bennet said). >>>> >> 3- Matlab breaks things all the time with 0 user consideration. >>>> None. But >>>> >> admittedly they break in style. We shouldn't be like them (not even >>>> >> close) >>>> >> but the point is when it is a commercial product, we don't hear the >>>> >> uproar >>>> >> we receive. People silently fix things. >>>> >> 4- Just because we update the python version, a lot of packages stop >>>> >> working. This is seriously disheartening. Happens to me all the time >>>> >> (protobuf, influxdb etc). And really annoying if one package has not >>>> >> released the new version yet. >>>> >> 5- Python is releasing too quick. I know Python is not Fortran >>>> >> (thankfully) but this is the other end of the spectrum. With every >>>> >> version >>>> >> one or two of us spent at least 2 weeks of intensive "What did they >>>> >> change >>>> >> on Windows again?" bug hunting. Hence our platform is not reliable >>>> and >>>> >> now >>>> >> they want 12 months. (What Ralf said) >>>> >> 6- There are always new users, downloading Python for the first time >>>> and >>>> >> it is expected that SciPy stack should work off-the-shelf. Hence we >>>> don't >>>> >> have the luxury to wait and see (see 4th item). >>>> >> 7- If there are limited resources for software support, then why >>>> people >>>> >> take the risk and update their stuff is something that has been >>>> discussed >>>> >> for decades now. And no one seems to converge to a point. (What Matti >>>> >> said) >>>> >> 8- what Evgeni suggested >>>> >> >>>> >> I'm not sure what to do about this but I am also more worried about >>>> the >>>> >> Python version than the NumPy version, my limited anectodal evidence >>>> >> around >>>> >> me suggests that people update NumPy + SciPy together mainly when >>>> they >>>> >> use >>>> >> pip with -U (--upgrade) on some package that has SciPy as a >>>> dependency. >>>> >> So >>>> >> I feel that it is safe to bump. >>>> >> >>>> >> >>>> > Just for the NumPy perspective, there is nothing in the forthcoming >>>> NumPy >>>> > 1.20 that doesn't work with Python 3.6, it is just a question of what >>>> > wheels to provide. OTOH, as Python advances I don't want to worry >>>> about >>>> > someone using newer features, whatever they are. I go back and >>>> forth, but >>>> > feel that 48 months is about the right support window and that argues >>>> for >>>> > dropping Python 3.6. With the faster pace of Python development we >>>> will >>>> > probably want to support the last four versions going forward. Note >>>> that >>>> > SciPy doesn't need to support all the NumPy versions that support a >>>> > particular version of Python, just the latest one. The main concern >>>> doesn't >>>> > seem to be compatibility as much as availability. >>>> > >>>> > >>>> > >>>> > Chuck >>>> > >>>> _______________________________________________ >>>> SciPy-Dev mailing list >>>> SciPy-Dev at python.org >>>> https://mail.python.org/mailman/listinfo/scipy-dev >>>> >>> _______________________________________________ >>> SciPy-Dev mailing list >>> SciPy-Dev at python.org >>> https://mail.python.org/mailman/listinfo/scipy-dev >>> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at python.org >> https://mail.python.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sat Nov 14 18:06:25 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sat, 14 Nov 2020 23:06:25 +0000 Subject: [SciPy-Dev] proposal to drop Python 3.6 and NumPy 1.14 / 1.15 In-Reply-To: References: Message-ID: On Sat, Nov 14, 2020 at 10:47 PM Tyler Reddy wrote: > Hi Ralf, > > I opened one yesterday: https://github.com/scipy/scipy/pull/13081 > > Feel free to replace it though. It was based almost purely on the commit > where you did the same thing for 3.5 last time. > Sorry, I've got my notifications disabled. Ignore the noise, your PR looks good! Cheers, Ralf > Tyler > > On Sat, 14 Nov 2020 at 15:39, Ralf Gommers wrote: > >> >> >> On Wed, Nov 11, 2020 at 10:17 PM Tyler Reddy >> wrote: >> >>> Time to follow up here I think. At the moment, our wheels repo weekly >>> cron job for POSIX systems includes multiple Python 3.6 matrix entries and >>> they all build/pass tests: >>> https://travis-ci.com/github/MacPython/scipy-wheels/builds/199857636 >>> >>> This means a few things: >>> 1) Obviously, we have not yet turned off support for Python 3.6 in the >>> master branch yet >>> 2) At the moment, Python 3.6 does not appear to be causing problems >>> >>> Of course, just because there are no problems with 3.6 now does not mean >>> that there won't be problems when backporting later as master branch >>> evolves. >>> >>> Is the final decision here to turn off 3.6 support before we branch for >>> 1.6.x series? >>> >> >> Yes, we should. Timing of this conversation was a bit unfortunate for me, >> so I didn't get around to opening a PR in September. I can do so tomorrow. >> >> Cheers, >> Ralf >> >> >> >>> Best wishes, >>> Tyler >>> >>> On Fri, 11 Sep 2020 at 15:42, Ralf Gommers >>> wrote: >>> >>>> >>>> >>>> On Mon, Aug 24, 2020 at 10:47 PM Warren Weckesser < >>>> warren.weckesser at gmail.com> wrote: >>>> >>>>> Sorry for the top post, but this question doesn't follow immediately >>>>> from any previous comment. >>>>> >>>>> Is the conclusion of this thread that with SciPy 1.6 we will >>>>> definitely drop Python 3.6 support? >>>> >>>> >>>> I think so. Given that there were a few hesitations at least, I'll >>>> split it in separate PRs - will do the NumPy one first. >>>> >>>> It might be useful to use >>>>> dataclasses (https://docs.python.org/3/library/dataclasses.html), and >>>>> they are new in Python 3.7. >>>>> >>>> >>>> You want to start using them now, or is it more a "nice to have the >>>> option"? >>>> >>>> Cheers, >>>> Ralf >>>> >>>> >>>>> Warren >>>>> >>>>> >>>>> On 8/20/20, Charles R Harris wrote: >>>>> > On Sun, Aug 16, 2020 at 5:17 PM Ilhan Polat >>>>> wrote: >>>>> > >>>>> >> I have too many personas in my head arguing about this. And many of >>>>> them >>>>> >> are in conflict. >>>>> >> >>>>> >> 1- I think I had enough arguments online to be known as buying none >>>>> of >>>>> >> that scientific stability views. That doesn't mean I don't get their >>>>> >> point, >>>>> >> I really really do, but still. It's in the job description. >>>>> Software is a >>>>> >> big part of science just like the papers. (What Matti said) >>>>> >> 2- Even a two-year old code is not safe to let it collect dust and >>>>> >> constantly requires attention and becomes a liability rather than a >>>>> tool >>>>> >> (what Bennet said). >>>>> >> 3- Matlab breaks things all the time with 0 user consideration. >>>>> None. But >>>>> >> admittedly they break in style. We shouldn't be like them (not even >>>>> >> close) >>>>> >> but the point is when it is a commercial product, we don't hear the >>>>> >> uproar >>>>> >> we receive. People silently fix things. >>>>> >> 4- Just because we update the python version, a lot of packages stop >>>>> >> working. This is seriously disheartening. Happens to me all the time >>>>> >> (protobuf, influxdb etc). And really annoying if one package has not >>>>> >> released the new version yet. >>>>> >> 5- Python is releasing too quick. I know Python is not Fortran >>>>> >> (thankfully) but this is the other end of the spectrum. With every >>>>> >> version >>>>> >> one or two of us spent at least 2 weeks of intensive "What did they >>>>> >> change >>>>> >> on Windows again?" bug hunting. Hence our platform is not reliable >>>>> and >>>>> >> now >>>>> >> they want 12 months. (What Ralf said) >>>>> >> 6- There are always new users, downloading Python for the first >>>>> time and >>>>> >> it is expected that SciPy stack should work off-the-shelf. Hence we >>>>> don't >>>>> >> have the luxury to wait and see (see 4th item). >>>>> >> 7- If there are limited resources for software support, then why >>>>> people >>>>> >> take the risk and update their stuff is something that has been >>>>> discussed >>>>> >> for decades now. And no one seems to converge to a point. (What >>>>> Matti >>>>> >> said) >>>>> >> 8- what Evgeni suggested >>>>> >> >>>>> >> I'm not sure what to do about this but I am also more worried about >>>>> the >>>>> >> Python version than the NumPy version, my limited anectodal evidence >>>>> >> around >>>>> >> me suggests that people update NumPy + SciPy together mainly when >>>>> they >>>>> >> use >>>>> >> pip with -U (--upgrade) on some package that has SciPy as a >>>>> dependency. >>>>> >> So >>>>> >> I feel that it is safe to bump. >>>>> >> >>>>> >> >>>>> > Just for the NumPy perspective, there is nothing in the forthcoming >>>>> NumPy >>>>> > 1.20 that doesn't work with Python 3.6, it is just a question of what >>>>> > wheels to provide. OTOH, as Python advances I don't want to worry >>>>> about >>>>> > someone using newer features, whatever they are. I go back and >>>>> forth, but >>>>> > feel that 48 months is about the right support window and that >>>>> argues for >>>>> > dropping Python 3.6. With the faster pace of Python development we >>>>> will >>>>> > probably want to support the last four versions going forward. Note >>>>> that >>>>> > SciPy doesn't need to support all the NumPy versions that support a >>>>> > particular version of Python, just the latest one. The main concern >>>>> doesn't >>>>> > seem to be compatibility as much as availability. >>>>> > >>>>> > >>>>> > >>>>> > Chuck >>>>> > >>>>> _______________________________________________ >>>>> SciPy-Dev mailing list >>>>> SciPy-Dev at python.org >>>>> https://mail.python.org/mailman/listinfo/scipy-dev >>>>> >>>> _______________________________________________ >>>> SciPy-Dev mailing list >>>> SciPy-Dev at python.org >>>> https://mail.python.org/mailman/listinfo/scipy-dev >>>> >>> _______________________________________________ >>> SciPy-Dev mailing list >>> SciPy-Dev at python.org >>> https://mail.python.org/mailman/listinfo/scipy-dev >>> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at python.org >> https://mail.python.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Sat Nov 21 16:09:54 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Sat, 21 Nov 2020 14:09:54 -0700 Subject: [SciPy-Dev] Travis CI credits exceeded Message-ID: Hi, Note that Travis CI is no longer running in our test matrix because we're out of open source credits: https://github.com/scipy/scipy/issues/13109 So, probably good to use some caution in merging MRs as it will just be silently absent. We'll need to continue migrating matrix entries to other services. Best wishes, Tyler -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyler.je.reddy at gmail.com Sun Nov 22 14:12:59 2020 From: tyler.je.reddy at gmail.com (Tyler Reddy) Date: Sun, 22 Nov 2020 12:12:59 -0700 Subject: [SciPy-Dev] Proposed 1.6.0 Release Schedule In-Reply-To: References: Message-ID: We may need to delay the release a bit. Mostly because of the disruption to the Travis CI service (not running/credit limit), which would probably affect the wheels repo before a final release too. I've tried to move a few of the simpler jobs to Azure, but probably running out of steam for that effort this weekend. I've reached out to Travis CI support and asked NumFOCUS informally what we might do here. A few kind folks have offered to chip in to keep the CI running short-term. Let's see what happens with the former inquiries first? Open PR count is currently 26 for the 1.6.0 milestone, though probably a few more I can bump to next milestone. Best wishes, Tyler On Mon, 9 Nov 2020 at 00:12, Evgeni Burovski wrote: > How about shifting the release towards January then? > > ??, 9 ????. 2020 ?., 1:43 Tyler Reddy : > >> Ok, I'll bump it by a week then. >> >> - November 24: branch maintenance/1.6.x >> - November 27: rc1 >> - December 8: rc2 (if needed) >> - December 17: final release >> >> >> I was trying to avoid the overlap with US Thanksgiving and a final >> release in late December near Winter Break, but I'll manage. >> >> Tyler >> >> On Sun, 8 Nov 2020 at 14:53, Ralf Gommers wrote: >> >>> >>> >>> On Sat, Nov 7, 2020 at 10:48 PM Tyler Reddy >>> wrote: >>> >>>> Hi, >>>> >>>> SciPy 1.5.0 was released June 21 (~5 months ago), and I think we'd like >>>> to keep a roughly biannual release cadence. >>>> >>>> I'd like to propose the following schedule for 1.6.0: >>>> - November 17: branch 1.6.x >>>> - November 20: rc1 >>>> - December 1: rc2 (if needed) >>>> - December 10: final release >>>> >>>> As always, it is a good idea to start tagging things that should be in >>>> 1.6.0 & please do help with reviewing PRs/issues that are tagged--current >>>> counts are: >>>> >>>> - PRs: 45 open with 1.6.0 milestone >>>> - issues: 25 open with 1.6.0 milestone >>>> >>>> While helping with that, also great if the release notes wiki is >>>> updated for appropriate changes: >>>> https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.6.0 >>>> >>>> >>>> Thoughts/objections for the schedule? >>>> >>> >>> >>> It does seem like there's a lot of PRs open, having only one weekend >>> left seems optimistic. The majority can be bumped to 1.7.0, but there's a >>> bit of a backlog of nice PRs that have been ready for a while and would be >>> nice to get in. For example: >>> >>> - KDTree/cKDTree feature parity: >>> https://github.com/scipy/scipy/pull/12852 >>> - HiGHS solver as linprog method: >>> https://github.com/scipy/scipy/pull/12043 >>> - Balanced cut tree: https://github.com/scipy/scipy/pull/10730 >>> - Andrew's set of optimize PRs that he brought up on the mailing list >>> recently >>> - Greg's set of ndimage PRs, complex kernels and the boundary handling >>> ones (#12725, #12767, #12776) >>> >>> If it's possible, given your time constraints, to bump the schedule by >>> one week then that may be useful. >>> >>> Cheers, >>> Ralf >>> >>> >>> >>> >>> _______________________________________________ >>> SciPy-Dev mailing list >>> SciPy-Dev at python.org >>> https://mail.python.org/mailman/listinfo/scipy-dev >>> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at python.org >> https://mail.python.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilhanpolat at gmail.com Sun Nov 22 15:23:25 2020 From: ilhanpolat at gmail.com (Ilhan Polat) Date: Sun, 22 Nov 2020 21:23:25 +0100 Subject: [SciPy-Dev] Proposed 1.6.0 Release Schedule In-Reply-To: References: Message-ID: I don't mean to disrupt your flow but please feel free to assign some dummy work to us if need be. Given the pandemic and all, it's much easier to get overwhelmed with stuff these days. On Sun, Nov 22, 2020 at 8:14 PM Tyler Reddy wrote: > We may need to delay the release a bit. Mostly because of the disruption > to the Travis CI service (not running/credit limit), which would probably > affect the wheels repo before a final release too. I've tried to move a few > of the simpler jobs to Azure, but probably running out of steam for that > effort this weekend. > > I've reached out to Travis CI support and asked NumFOCUS informally what > we might do here. A few kind folks have offered to chip in to keep the CI > running short-term. Let's see what happens with the former inquiries first? > > Open PR count is currently 26 for the 1.6.0 milestone, though probably a > few more I can bump to next milestone. > > Best wishes, > Tyler > > > > On Mon, 9 Nov 2020 at 00:12, Evgeni Burovski > wrote: > >> How about shifting the release towards January then? >> >> ??, 9 ????. 2020 ?., 1:43 Tyler Reddy : >> >>> Ok, I'll bump it by a week then. >>> >>> - November 24: branch maintenance/1.6.x >>> - November 27: rc1 >>> - December 8: rc2 (if needed) >>> - December 17: final release >>> >>> >>> I was trying to avoid the overlap with US Thanksgiving and a final >>> release in late December near Winter Break, but I'll manage. >>> >>> Tyler >>> >>> On Sun, 8 Nov 2020 at 14:53, Ralf Gommers >>> wrote: >>> >>>> >>>> >>>> On Sat, Nov 7, 2020 at 10:48 PM Tyler Reddy >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> SciPy 1.5.0 was released June 21 (~5 months ago), and I think we'd >>>>> like to keep a roughly biannual release cadence. >>>>> >>>>> I'd like to propose the following schedule for 1.6.0: >>>>> - November 17: branch 1.6.x >>>>> - November 20: rc1 >>>>> - December 1: rc2 (if needed) >>>>> - December 10: final release >>>>> >>>>> As always, it is a good idea to start tagging things that should be in >>>>> 1.6.0 & please do help with reviewing PRs/issues that are tagged--current >>>>> counts are: >>>>> >>>>> - PRs: 45 open with 1.6.0 milestone >>>>> - issues: 25 open with 1.6.0 milestone >>>>> >>>>> While helping with that, also great if the release notes wiki is >>>>> updated for appropriate changes: >>>>> https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.6.0 >>>>> >>>>> >>>>> Thoughts/objections for the schedule? >>>>> >>>> >>>> >>>> It does seem like there's a lot of PRs open, having only one weekend >>>> left seems optimistic. The majority can be bumped to 1.7.0, but there's a >>>> bit of a backlog of nice PRs that have been ready for a while and would be >>>> nice to get in. For example: >>>> >>>> - KDTree/cKDTree feature parity: >>>> https://github.com/scipy/scipy/pull/12852 >>>> - HiGHS solver as linprog method: >>>> https://github.com/scipy/scipy/pull/12043 >>>> - Balanced cut tree: https://github.com/scipy/scipy/pull/10730 >>>> - Andrew's set of optimize PRs that he brought up on the mailing list >>>> recently >>>> - Greg's set of ndimage PRs, complex kernels and the boundary handling >>>> ones (#12725, #12767, #12776) >>>> >>>> If it's possible, given your time constraints, to bump the schedule by >>>> one week then that may be useful. >>>> >>>> Cheers, >>>> Ralf >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> SciPy-Dev mailing list >>>> SciPy-Dev at python.org >>>> https://mail.python.org/mailman/listinfo/scipy-dev >>>> >>> _______________________________________________ >>> SciPy-Dev mailing list >>> SciPy-Dev at python.org >>> https://mail.python.org/mailman/listinfo/scipy-dev >>> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at python.org >> https://mail.python.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.home at sbcglobal.net Sun Nov 22 18:38:56 2020 From: frank.home at sbcglobal.net (frank.home at sbcglobal.net) Date: Sun, 22 Nov 2020 23:38:56 +0000 (UTC) Subject: [SciPy-Dev] SciPy-Dev Digest, Vol 205, Issue 14 In-Reply-To: References: Message-ID: <1596041383.486594.1606088336758@mail.yahoo.com> It's worth some effort to see what happens with the inquiries, even if just to learn more about how others want to handle these types of situations. But I feel there are enough of us willing to chip in to keep it running short term, and time is better spent on scipy code quality, so if it gets to the point that the inquiries/discussions get frustrating or time consuming, maybe just let us know how to chip in? On Sunday, November 22, 2020, 12:24:33 PM PST, scipy-dev-request at python.org wrote: Send SciPy-Dev mailing list submissions to ??? scipy-dev at python.org To subscribe or unsubscribe via the World Wide Web, visit ??? https://mail.python.org/mailman/listinfo/scipy-dev or, via email, send a message with subject or body 'help' to ??? scipy-dev-request at python.org You can reach the person managing the list at ??? scipy-dev-owner at python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of SciPy-Dev digest..." Today's Topics: ? 1. Re: Proposed 1.6.0 Release Schedule (Tyler Reddy) ? 2. Re: Proposed 1.6.0 Release Schedule (Ilhan Polat) We may need to delay the release a bit. Mostly because of the disruption to the Travis CI service (not running/credit limit), which would probably affect the wheels repo before a final release too. I've tried to move a few of the simpler jobs to Azure, but probably running out of steam for that effort this weekend. I've reached out to Travis CI support and asked NumFOCUS informally what we might do here. A few kind folks have offered to chip in to keep the CI running short-term. Let's see what happens with the former inquiries first? Open PR count is currently 26 for the 1.6.0 milestone, though probably a few more I can bump to next milestone. Best wishes,Tyler On Mon, 9 Nov 2020 at 00:12, Evgeni Burovski wrote: How about shifting the release towards January then?? ??, 9 ????. 2020 ?., 1:43 Tyler Reddy : Ok, I'll bump it by a week then. - November 24: branch maintenance/1.6.x - November 27: rc1 - December 8: rc2 (if needed) - December 17: final release I was trying to avoid the overlap with US Thanksgiving and a final release in late December near Winter Break, but I'll manage. Tyler On Sun, 8 Nov 2020 at 14:53, Ralf Gommers wrote: On Sat, Nov 7, 2020 at 10:48 PM Tyler Reddy wrote: Hi, SciPy 1.5.0 was released June 21 (~5 months ago), and I think we'd like to keep a roughly biannual release cadence. I'd like to propose the following schedule for 1.6.0:- November 17: branch 1.6.x - November 20: rc1 - December 1: rc2 (if needed) - December 10: final release As always, it is a good idea to start tagging things that should be in 1.6.0 & please do help with reviewing PRs/issues that are tagged--current counts are: - PRs: 45 open with 1.6.0 milestone- issues: 25 open with 1.6.0 milestone While helping with that, also great if the release notes wiki is updated for appropriate changes:?https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.6.0? Thoughts/objections for the schedule? It does seem like there's a lot of PRs open, having only one weekend left seems optimistic. The majority can be bumped to 1.7.0, but there's a bit of a backlog of nice PRs that have been ready for a while and would be nice to get in. For example: - KDTree/cKDTree feature parity: https://github.com/scipy/scipy/pull/12852- HiGHS solver as linprog method: https://github.com/scipy/scipy/pull/12043- Balanced cut tree: https://github.com/scipy/scipy/pull/10730- Andrew's set of optimize PRs that he brought up on the mailing list recently- Greg's set of ndimage PRs, complex kernels and the boundary handling ones (#12725, #12767, #12776) If it's possible, given your time constraints, to bump the schedule by one week then that may be useful. Cheers, Ralf _______________________________________________ SciPy-Dev mailing list SciPy-Dev at python.org https://mail.python.org/mailman/listinfo/scipy-dev _______________________________________________ SciPy-Dev mailing list SciPy-Dev at python.org https://mail.python.org/mailman/listinfo/scipy-dev _______________________________________________ SciPy-Dev mailing list SciPy-Dev at python.org https://mail.python.org/mailman/listinfo/scipy-dev I don't mean to disrupt your flow but please feel free to assign some dummy work to us if need be. Given the pandemic and all, it's much easier to get overwhelmed with stuff these days. On Sun, Nov 22, 2020 at 8:14 PM Tyler Reddy wrote: We may need to delay the release a bit. Mostly because of the disruption to the Travis CI service (not running/credit limit), which would probably affect the wheels repo before a final release too. I've tried to move a few of the simpler jobs to Azure, but probably running out of steam for that effort this weekend. I've reached out to Travis CI support and asked NumFOCUS informally what we might do here. A few kind folks have offered to chip in to keep the CI running short-term. Let's see what happens with the former inquiries first? Open PR count is currently 26 for the 1.6.0 milestone, though probably a few more I can bump to next milestone. Best wishes,Tyler On Mon, 9 Nov 2020 at 00:12, Evgeni Burovski wrote: How about shifting the release towards January then?? ??, 9 ????. 2020 ?., 1:43 Tyler Reddy : Ok, I'll bump it by a week then. - November 24: branch maintenance/1.6.x - November 27: rc1 - December 8: rc2 (if needed) - December 17: final release I was trying to avoid the overlap with US Thanksgiving and a final release in late December near Winter Break, but I'll manage. Tyler On Sun, 8 Nov 2020 at 14:53, Ralf Gommers wrote: On Sat, Nov 7, 2020 at 10:48 PM Tyler Reddy wrote: Hi, SciPy 1.5.0 was released June 21 (~5 months ago), and I think we'd like to keep a roughly biannual release cadence. I'd like to propose the following schedule for 1.6.0:- November 17: branch 1.6.x - November 20: rc1 - December 1: rc2 (if needed) - December 10: final release As always, it is a good idea to start tagging things that should be in 1.6.0 & please do help with reviewing PRs/issues that are tagged--current counts are: - PRs: 45 open with 1.6.0 milestone- issues: 25 open with 1.6.0 milestone While helping with that, also great if the release notes wiki is updated for appropriate changes:?https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.6.0? Thoughts/objections for the schedule? It does seem like there's a lot of PRs open, having only one weekend left seems optimistic. The majority can be bumped to 1.7.0, but there's a bit of a backlog of nice PRs that have been ready for a while and would be nice to get in. For example: - KDTree/cKDTree feature parity: https://github.com/scipy/scipy/pull/12852- HiGHS solver as linprog method: https://github.com/scipy/scipy/pull/12043- Balanced cut tree: https://github.com/scipy/scipy/pull/10730- Andrew's set of optimize PRs that he brought up on the mailing list recently- Greg's set of ndimage PRs, complex kernels and the boundary handling ones (#12725, #12767, #12776) If it's possible, given your time constraints, to bump the schedule by one week then that may be useful. Cheers, Ralf _______________________________________________ SciPy-Dev mailing list SciPy-Dev at python.org https://mail.python.org/mailman/listinfo/scipy-dev _______________________________________________ SciPy-Dev mailing list SciPy-Dev at python.org https://mail.python.org/mailman/listinfo/scipy-dev _______________________________________________ SciPy-Dev mailing list SciPy-Dev at python.org https://mail.python.org/mailman/listinfo/scipy-dev _______________________________________________ SciPy-Dev mailing list SciPy-Dev at python.org https://mail.python.org/mailman/listinfo/scipy-dev _______________________________________________ SciPy-Dev mailing list SciPy-Dev at python.org https://mail.python.org/mailman/listinfo/scipy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Thu Nov 26 09:28:50 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Thu, 26 Nov 2020 14:28:50 +0000 Subject: [SciPy-Dev] Proposal to add two new linesearch strategies to scipy.optimize In-Reply-To: References: Message-ID: Hi Ihno, Thanks for this suggestion! On Tue, Nov 3, 2020 at 5:38 PM Ihno Schrot wrote: > Hi all, > > I would like to propose to add two new linesearch strategies to > scipy.optimize(.linesearch.py), namely the "Restrictive Monotonicity Test" > (RMT) [1] and the "Backward Step Control" (BSC) [2]. > > It is well known that Newton-type methods for rootfinding or optimization > problems can be forced to converge globally in a domain where the Jacobian > or Hessian, respectively, is nonsingular. However, with linesearch > strategies which are based on descent arguments with classical merit > functions one usually has to pay the price of taking unnecessarily small > stepsizes even in the domain where full steps would lead to > convergence. This is also the case for the existing linesearch strategies > in SciPy. > > The suggested linesearch strategies were developed to achieve global > convergence of Newton-type methods while avoiding too small step sizes. The > RMT already has shown good performance in many practical applications. The > BSC is a rather novel strategy but the numerical results available so far > are promising and the BSC is proven to enable global convergence under > reasonable assumptions. Moreover, I have done extensive numerical tests of > both linesearch strategies in my master thesis. Although the tests have > been focussed on comparing the two strategies, I also compared them to > SciPys linesearch strategy that is based on the Wolfe condition and found > that the RMT and BSC can outperform SciPys linesearch strategy in terms of > the number of benchmark problems that can be solved. > For new optimization methods, this is usually our main criterion - if it does better on benchmarks, then we'd like to add the method. I'm not sure we actually have accuracy benchmarks that use line search methods under the hood. Adding those could be very helpful, analogous to the ones we have for global optimization functions. Cheers, Ralf Therefore, I believe that adding the BSC and the RMT to SciPy benefits a > wide range of users by providing modern powerful linesearch strategies. > Especially, users would have two more linesearch strategies to apply to > their problems, which can be helpful given that it is problem dependent > which linesearch strategy works the best. > > If the community thinks that adding those two strategies is an useful > enhancement, colleagues and I are happy to improve and eventually add our > code to SciPy. Of course, we are also happy to answer any questions from > the community regarding this proposal. > > Greetings from Heidelberg, > Ihno > > [1] https://doi.org/10.1007/978-0-387-35514-6_3 > [2] https://doi.org/10.1137/140968586 > > -- > Ihno Schrot > Institute for Applied Mathematics > Heidelberg University > E-Mail: ihno.schrot at uni-heidelberg.de > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at python.org > https://mail.python.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michelle.teplitski at gmail.com Sun Nov 29 12:02:32 2020 From: michelle.teplitski at gmail.com (Michelle Teplitski) Date: Sun, 29 Nov 2020 12:02:32 -0500 Subject: [SciPy-Dev] Shapelet Functions Message-ID: Hi scipy-dev community, I am working on a library to generate shapelet functions for use in image processing. Applications range from analyzing images of galaxies to finding defects in nano-structured materials. Would the community be interested in this as a contribution to scipy? Thanks, Michelle Teplitski -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sun Nov 29 12:23:26 2020 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 29 Nov 2020 17:23:26 +0000 Subject: [SciPy-Dev] Shapelet Functions In-Reply-To: References: Message-ID: On Sun, Nov 29, 2020 at 5:02 PM Michelle Teplitski < michelle.teplitski at gmail.com> wrote: > Hi scipy-dev community, > > I am working on a library to generate shapelet functions for use in image > processing. Applications range from analyzing images of galaxies to finding > defects in nano-structured materials. Would the community be interested in > this as a contribution to scipy? > Hi Michelle, thanks for asking. This sounds interesting, but *lets are too specialized for SciPy. Even the bit of wavelets functionality we have isn't great. Depending on what your library looks like, it could fit in PyWavelets (see https://github.com/PyWavelets/pywt/issues/58#issuecomment-100665496). Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: