From jtaylor.debian at googlemail.com Tue Jun 3 10:13:19 2014 From: jtaylor.debian at googlemail.com (Julian Taylor) Date: Tue, 3 Jun 2014 16:13:19 +0200 Subject: [SciPy-Dev] scipy/io/idl.py usage permission by ITT Visual Information Message-ID: hi, scipy/io/idl.py contains code to read Save/Restore files written by IDL. These files contain following header (e.g. scipy/io/tests/ data/*.sav): NOTICE: IDL Save/Restore files embody unpublished proprietary information about the IDL program. Reverse engineering of this file is therefore forbidden under the terms of the IDL End User License Agreement (IDL EULA). All IDL users are required to read and agree to the terms of the IDL EULA at the time that they install IDL. Software that reads or writes files in the IDL Save/Restore format must have a license from ITT Visual Information Solutions explicitly granting the right to do so. In this case, the license will be included with the software for your inspection. Please report software that does not have such a license to ITT Visual Information Solutions (info at ittvis.com). So without a license from ITT using this code is illegal (within jurisdiction where this notice is valid). Though the idl.py file implementing the io contains following lines: # This code was developed by with permission from ITT Visual Information # Systems. IDL(r) is a registered trademark of ITT Visual Information Systems, # Inc. for their Interactive Data Language software. This is a little vague but I couldn't find any more information in the source and the addition predates github move. Is there more details on this permission available? Does the permission grant all users of scipy the full unencumbered right to read and write Save/Restore files? Does this permission extend to third parties redistributing scipy, e.g. via linux distributions? As is the IDL code does most likely not comply with the Debian free software guidelines and all code related to it will have to be removed from the Debian package. Cheers, Julian Taylor From njs at pobox.com Tue Jun 3 12:15:18 2014 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 3 Jun 2014 17:15:18 +0100 Subject: [SciPy-Dev] scipy/io/idl.py usage permission by ITT Visual Information In-Reply-To: References: Message-ID: On Tue, Jun 3, 2014 at 3:13 PM, Julian Taylor wrote: > hi, > scipy/io/idl.py contains code to read Save/Restore files written by IDL. > These files contain following header (e.g. scipy/io/tests/ > data/*.sav): > > NOTICE: > > IDL Save/Restore files embody unpublished proprietary information > about the IDL program. Reverse engineering of this file is therefore > forbidden under the terms of the IDL End User License Agreement > (IDL EULA). All IDL users are required to read and agree to the > terms of the IDL EULA at the time that they install IDL. > Software that reads or writes files in the IDL Save/Restore format > must have a license from ITT Visual Information Solutions > explicitly granting the right to do so. In this case, the license > will be included with the software for your inspection. Please > report software that does not have such a license to > ITT Visual Information Solutions (info at ittvis.com). > > So without a license from ITT using this code is illegal (within > jurisdiction where this notice is valid). Do any such jurisdictions exist? Even if the EULA's reverse-engineering restrictions would hold up in court, they certainly wouldn't apply to anyone who hadn't accepted that license (e.g., me). And the details of this format have been publicly available for ~15 years: http://www.physics.wisc.edu/~craigm/idl/savefmt/ If I used that document to implement an IDL file reader, then I'm not aware of any legal doctrine which could restrict usage or distribution of my hypothetical code in any way, even in theory... > This is a little vague but I couldn't find any more information in the > source and the addition predates github move. The history appears to be here: https://astrofrog.github.io/idlsave/ https://github.com/astrofrog/idlsave -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org From kyle.mandli at gmail.com Tue Jun 3 19:11:00 2014 From: kyle.mandli at gmail.com (Kyle Mandli) Date: Tue, 3 Jun 2014 18:11:00 -0500 Subject: [SciPy-Dev] SciPy 2014 BoF Panel Message-ID: Hello everyone, As one of the co-chairs in charge of organizing the birds-of-a-feather sesssions at the SciPy conference this year, I wanted to solicit through the SciPy list to start a discussion on what kind of BoF or set of BoF sessions might be worthwhile at SciPy this year. Since SciPy is particularly broad (especially if you include the sci-kits) I was thinking it might be worthwhile to cast a wide net and see what interest there was. I can help facilitate organization and ideas on what kind of format for each particular BoF that is proposed might have. Thanks! Kyle Manldi (and via proxy Matt McCormick) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jtaylor.debian at googlemail.com Sun Jun 8 16:34:51 2014 From: jtaylor.debian at googlemail.com (Julian Taylor) Date: Sun, 08 Jun 2014 22:34:51 +0200 Subject: [SciPy-Dev] ANN: NumPy 1.9.0 beta release Message-ID: <5394C8EB.3020404@googlemail.com> Hello, I'm happy to announce the fist beta release of Numpy 1.9.0. 1.9.0 will be a new feature release supporting Python 2.6 - 2.7 and 3.2 - 3.4. Due to low demand windows binaries for the beta are only available for Python 2.7, 3.3 and 3.4. Please try it and report any issues to the numpy-discussion mailing list or on github. The 1.9 release will consists of mainly of many small improvements and bugfixes. The highlights are: * Addition of __numpy_ufunc__ to allow overriding ufuncs in ndarray subclasses. Please note that there are still some known issues with this mechanism which we hope to resolve before the final release (e.g. #4753) * Numerous performance improvements in various areas, most notably indexing and operations on small arrays are significantly faster. Indexing operations now also release the GIL. * Addition of nanmedian and nanpercentile rounds out the nanfunction set. The changes involve a lot of small changes that might affect some applications, please read the release notes for the full details on all changes: https://github.com/numpy/numpy/blob/maintenance/1.9.x/doc/release/1.9.0-notes.rst Please also take special note of the future changes section which will apply to the following release 1.10.0 and make sure to check if your applications would be affected by them. Source tarballs, windows installers and release notes can be found at https://sourceforge.net/projects/numpy/files/NumPy/1.9.0b1 Cheers, Julian Taylor -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From charlesr.harris at gmail.com Mon Jun 9 19:29:30 2014 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Jun 2014 17:29:30 -0600 Subject: [SciPy-Dev] Reporting a bug to Apple. Message-ID: Hi All, Julian has tracked down a bug in the Accelerate library in Maverick, details here . Is there a registered Apple Developer here who can report the bug to Apple? TIA, Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From nouiz at nouiz.org Mon Jun 9 20:27:43 2014 From: nouiz at nouiz.org (=?UTF-8?B?RnLDqWTDqXJpYyBCYXN0aWVu?=) Date: Mon, 9 Jun 2014 20:27:43 -0400 Subject: [SciPy-Dev] [Numpy-discussion] Reporting a bug to Apple. In-Reply-To: References: Message-ID: Hi, We already did a bug report to Apple, but they didn't acted on this yet. A process call the kernel and it loop infinitly in the kernel. The only way to "kill" the process is to reboot. Arnaud, how did you report it? Good luck, and if they act on this, I would be happy to know how you did. Fr?d?ric Bastien On Mon, Jun 9, 2014 at 7:29 PM, Charles R Harris wrote: > Hi All, > > Julian has tracked down a bug in the Accelerate library in Maverick, > details here > . Is > there a registered Apple Developer here who can report the bug to Apple? > > TIA, > > Chuck > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Tue Jun 10 08:57:54 2014 From: matthew.brett at gmail.com (Matthew Brett) Date: Tue, 10 Jun 2014 13:57:54 +0100 Subject: [SciPy-Dev] [Numpy-discussion] Reporting a bug to Apple. In-Reply-To: <840115269424053607.630244sturla.molden-gmail.com@news.gmane.org> References: <840115269424053607.630244sturla.molden-gmail.com@news.gmane.org> Message-ID: Hi, On Tue, Jun 10, 2014 at 1:52 AM, Sturla Molden wrote: > Fr?d?ric Bastien wrote: > OSes >> Good luck, and if they act on this, I would be happy to know how you did. >> > > If we report a segfault Apple will probably think we are to blame. 99.9 % > of bug reports come from idiots, and those who screen bug reports are > basically payed to hit the ignore button. Indeed, good luck on reporting > it. > > I think in the short run we will be better off re-implementing cblas_sgemv > with cblas_sgemm on Apple OSes. Luckily the error in cblas_sgemv does not > silently produce garbage data but segfaults the process, so anyone affected > will see it. Would you consider doing a PR for that? I believe I can build a 32 / 64 bit numpy binary against ATLAS, but it's certainly much less straightforward. Cheers, Matthew From aron at ahmadia.net Tue Jun 10 09:46:38 2014 From: aron at ahmadia.net (Aron Ahmadia) Date: Tue, 10 Jun 2014 09:46:38 -0400 Subject: [SciPy-Dev] [Numpy-discussion] Reporting a bug to Apple. In-Reply-To: <840115269424053607.630244sturla.molden-gmail.com@news.gmane.org> References: <840115269424053607.630244sturla.molden-gmail.com@news.gmane.org> Message-ID: On Mon, Jun 9, 2014 at 8:52 PM, Sturla Molden wrote: > > If we report a segfault Apple will probably think we are to blame. 99.9 % > of bug reports come from idiots, and those who screen bug reports are > basically payed to hit the ignore button. Indeed, good luck on reporting > it. I've reported bugs to Apple and seen them fixed in major releases of the operating system (though not necessarily minor releases). Unless things have changed considerably since my last bug report, I don't think it's fair to characterize their quality assurance team this way, particularly without evidence to back up your point. A -------------- next part -------------- An HTML attachment was scrubbed... URL: From aizvorski at gmail.com Tue Jun 17 04:56:05 2014 From: aizvorski at gmail.com (Alexander Izvorski) Date: Tue, 17 Jun 2014 01:56:05 -0700 Subject: [SciPy-Dev] Announce: scikit-video Message-ID: Hello, This is to announce the very early version of scikit-video, a collection of algorithms for all your video-processing needs :) Code: https://github.com/aizvorski/scikit-video Status: There is functional video reading code, which is close to a replacement for cv2.VideoCapture. Otherwise, mostly a placeholder. Motivation: This scikit is intended as a companion to scikit-image: the two are complementary and not competitive. What code is better in video than in image? Roughly, if the code *only* makes sense when applied to a sequence of images, then it may be better placed in scikit-video. An example of this would be temporal denoise. Other code which would make sense to have in here is code which is much more often seen or used in a video context, even though it may in principle be used for both images or video. An example of this would be the SSIM quality metric. It is a goal that scikit-image and scikit-video would "just work" together, for example you might read a video file using scikit-video, use a scikit-image filter on each frame and write out the filtered frames to another file. Roadmap: - Skeleton project from scikit-example - DONE - Video IO by wrapping ffmpeg/avconv - reading DONE, writing not yet - Video metrics - Unit tests and CI (beyond this point are just possible future directions) - (?) Pyramid motion estimation - (?) TLD algorithm (http://personal.ee.surrey.ac.uk/Personal/Z.Kalal/) - (?) 3D denoise - (?) OpenGL display and a mini video player - (?) Any existing off-the-shelf projects that would be good to merge into this I expect that if this gathers enough interest it will transition to a group-maintained project similar to skimage or sklearn. Your suggestions, ideas, and code are most welcome. Regards, Alex Izvorski -------------- next part -------------- An HTML attachment was scrubbed... URL: From william.ratcliff at gmail.com Tue Jun 17 09:32:44 2014 From: william.ratcliff at gmail.com (william ratcliff) Date: Tue, 17 Jun 2014 09:32:44 -0400 Subject: [SciPy-Dev] Levenberg-Marquardt Implementation In-Reply-To: References: Message-ID: This issue does seem to come up regularly. There is a BSD licensed version: https://github.com/newville/lmfit-py For mpfit.py, at one point, I got license permissions for scipy from the original authors, but no one wanted to include it. Has that changed? I haven't followed the issue since 2012. Best, William On Tue, Apr 1, 2014 at 7:57 AM, Johann cohen-tanugi < johann.cohentanugi at gmail.com> wrote: > FWIW I came across a python LM implementation of the minpack LM routine in > https://code.google.com/p/astrolibpy/source/browse/mpfit/mpfit.py > The license is GPLv3 though. > Johann > > > On Tue, Feb 25, 2014 at 7:25 PM, Pauli Virtanen wrote: > >> 25.02.2014 17:45, Benny Malengier kirjoitti: >> > This is present already, See >> > >> http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.leastsq.html >> > >> > which wraps: http://www.math.utah.edu/software/minpack/minpack >> > /lmder.html >> >> The current leastsq implementation does not support sparse Jacobians >> or constraints. >> >> MINPACK does dense QR factorizations, and this approach doesn't work >> well for problems where the number of variables is too big. This was one >> of our the GSoC topic ideas [1] --- if you have suggestions on how to >> improve these, please speak up. >> >> [1] https://github.com/scipy/scipy/wiki/GSoC-project-ideas >> >> -- >> Pauli Virtanen >> >> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla.molden at gmail.com Tue Jun 17 10:28:46 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Tue, 17 Jun 2014 14:28:46 +0000 (UTC) Subject: [SciPy-Dev] Levenberg-Marquardt Implementation References: Message-ID: <285817149424708026.514167sturla.molden-gmail.com@news.gmane.org> william ratcliff wrote: > This issue does seem to come up regularly. There is a BSD licensed > version: > https://github.com/newville/lmfit-py > > For mpfit.py, at one point, I got license permissions for scipy from the > original authors, but no one wanted to include it. Has that changed? I > haven't followed the issue since 2012. I just looked briefly at the code. It seems to use scipy.optimize.leastsq for Levenberg-Marquardt, so what is the extra benefit? Sturla From sturla.molden at gmail.com Tue Jun 17 10:32:45 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Tue, 17 Jun 2014 14:32:45 +0000 (UTC) Subject: [SciPy-Dev] Levenberg-Marquardt Implementation References: <285817149424708026.514167sturla.molden-gmail.com@news.gmane.org> Message-ID: <1737417469424708185.255175sturla.molden-gmail.com@news.gmane.org> Sturla Molden wrote: > william ratcliff wrote: >> This issue does seem to come up regularly. There is a BSD licensed >> version: >> https://github.com/newville/lmfit-py >> >> For mpfit.py, at one point, I got license permissions for scipy from the >> original authors, but no one wanted to include it. Has that changed? I >> haven't followed the issue since 2012. > > I just looked briefly at the code. It seems to use scipy.optimize.leastsq > for Levenberg-Marquardt, so what is the extra benefit? For the record: scipy.optimize.leastsq and scipy.optimize.curve_fit are Levenberg-Marquardt solvers. Sturla From josef.pktd at gmail.com Tue Jun 17 10:45:00 2014 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Tue, 17 Jun 2014 10:45:00 -0400 Subject: [SciPy-Dev] Levenberg-Marquardt Implementation In-Reply-To: <1737417469424708185.255175sturla.molden-gmail.com@news.gmane.org> References: <285817149424708026.514167sturla.molden-gmail.com@news.gmane.org> <1737417469424708185.255175sturla.molden-gmail.com@news.gmane.org> Message-ID: On Tue, Jun 17, 2014 at 10:32 AM, Sturla Molden wrote: > Sturla Molden wrote: >> william ratcliff wrote: >>> This issue does seem to come up regularly. There is a BSD licensed >>> version: >>> https://github.com/newville/lmfit-py >>> >>> For mpfit.py, at one point, I got license permissions for scipy from the >>> original authors, but no one wanted to include it. Has that changed? I >>> haven't followed the issue since 2012. >> >> I just looked briefly at the code. It seems to use scipy.optimize.leastsq >> for Levenberg-Marquardt, so what is the extra benefit? > > For the record: scipy.optimize.leastsq and scipy.optimize.curve_fit are > Levenberg-Marquardt solvers. curve_fit and lmfit are wrappers for Levenberg-Marquardt. What scipy needs, among other things, are more pure optimizers. lmfit allows fancier parameter definitions, imposes constraints through reparameterization and produces more results statistics. The optimizer that it uses is just Levenberg-Marquardt, although IIRC it also wraps other optimizers. Josef > > Sturla > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev From newville at cars.uchicago.edu Tue Jun 17 13:57:23 2014 From: newville at cars.uchicago.edu (Matt Newville) Date: Tue, 17 Jun 2014 12:57:23 -0500 Subject: [SciPy-Dev] SciPy-Dev Digest, Vol 128, Issue 5 In-Reply-To: References: Message-ID: Hi William, Sturla, Josef, All Message: 2 > Date: Tue, 17 Jun 2014 09:32:44 -0400 > From: william ratcliff > Subject: Re: [SciPy-Dev] Levenberg-Marquardt Implementation > To: SciPy Developers List > Message-ID: > < > CAFt3ydvxgC3KVofi0v4U8-pCmfgdo3bBhJfhrkne7ZLkWcTVfQ at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > This issue does seem to come up regularly. There is a BSD licensed > version: > https://github.com/newville/lmfit-py > > For mpfit.py, at one point, I got license permissions for scipy from the > original authors, but no one wanted to include it. Has that changed? I > haven't followed the issue since 2012. > > Best, > William > > > On Tue, Apr 1, 2014 at 7:57 AM, Johann cohen-tanugi < > johann.cohentanugi at gmail.com> wrote: > > > FWIW I came across a python LM implementation of the minpack LM routine > in > > https://code.google.com/p/astrolibpy/source/browse/mpfit/mpfit.py > > The license is GPLv3 though. > > Johann > > > > > > On Tue, Feb 25, 2014 at 7:25 PM, Pauli Virtanen wrote: > > > >> 25.02.2014 17:45, Benny Malengier kirjoitti: > >> > This is present already, See > >> > > >> > http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.leastsq.html > >> > > >> > which wraps: http://www.math.utah.edu/software/minpack/minpack > >> > /lmder.html > >> > >> The current leastsq implementation does not support sparse Jacobians > >> or constraints. > >> > >> MINPACK does dense QR factorizations, and this approach doesn't work > >> well for problems where the number of variables is too big. This was one > >> of our the GSoC topic ideas [1] --- if you have suggestions on how to > >> improve these, please speak up. > >> > >> [1] https://github.com/scipy/scipy/wiki/GSoC-project-ideas > >> > >> -- > >> Pauli Virtanen > >> > >> > >> _______________________________________________ > >> SciPy-Dev mailing list > >> SciPy-Dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/scipy-dev > >> > > > > > > _______________________________________________ > > SciPy-Dev mailing list > > SciPy-Dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mail.scipy.org/pipermail/scipy-dev/attachments/20140617/c7252b45/attachment-0001.html > > ------------------------------ > > Message: 3 > Date: Tue, 17 Jun 2014 14:28:46 +0000 (UTC) > From: Sturla Molden > Subject: Re: [SciPy-Dev] Levenberg-Marquardt Implementation > To: scipy-dev at scipy.org > Message-ID: > <285817149424708026.514167sturla.molden-gmail.com at news.gmane.org> > Content-Type: text/plain; charset=UTF-8 > > william ratcliff wrote: > > This issue does seem to come up regularly. There is a BSD licensed > > version: > > > https://github.com/newville/lmfit-py > > > > For mpfit.py, at one point, I got license permissions for scipy from the > > original authors, but no one wanted to include it. Has that changed? I > > haven't followed the issue since 2012. > > I just looked briefly at the code. It seems to use scipy.optimize.leastsq > for Levenberg-Marquardt, so what is the extra benefit? > > Sturla > > > > ------------------------------ > > Message: 4 > Date: Tue, 17 Jun 2014 14:32:45 +0000 (UTC) > From: Sturla Molden > Subject: Re: [SciPy-Dev] Levenberg-Marquardt Implementation > To: scipy-dev at scipy.org > Message-ID: > <1737417469424708185.255175sturla.molden-gmail.com at news.gmane.org> > Content-Type: text/plain; charset=UTF-8 > > Sturla Molden wrote: > > william ratcliff wrote: > >> This issue does seem to come up regularly. There is a BSD licensed > >> version: > >> > https://github.com/newville/lmfit-py > >> > >> For mpfit.py, at one point, I got license permissions for scipy from the > >> original authors, but no one wanted to include it. Has that changed? > I > >> haven't followed the issue since 2012. > > > > I just looked briefly at the code. It seems to use scipy.optimize.leastsq > > for Levenberg-Marquardt, so what is the extra benefit? > > For the record: scipy.optimize.leastsq and scipy.optimize.curve_fit are > Levenberg-Marquardt solvers. > > Sturla > > > > ------------------------------ > > Message: 5 > Date: Tue, 17 Jun 2014 10:45:00 -0400 > From: josef.pktd at gmail.com > Subject: Re: [SciPy-Dev] Levenberg-Marquardt Implementation > To: SciPy Developers List > Message-ID: > deMWvueVPKCywY5_PAwQVpw at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Tue, Jun 17, 2014 at 10:32 AM, Sturla Molden > wrote: > > Sturla Molden wrote: > >> william ratcliff wrote: > >>> This issue does seem to come up regularly. There is a BSD licensed > >>> version: > >>> > https://github.com/newville/lmfit-py > >>> > >>> For mpfit.py, at one point, I got license permissions for scipy from > the > >>> original authors, but no one wanted to include it. Has that changed? > I > >>> haven't followed the issue since 2012. > >> > >> I just looked briefly at the code. It seems to use > scipy.optimize.leastsq > >> for Levenberg-Marquardt, so what is the extra benefit? > > > > For the record: scipy.optimize.leastsq and scipy.optimize.curve_fit are > > Levenberg-Marquardt solvers. > > curve_fit and lmfit are wrappers for Levenberg-Marquardt. > > What scipy needs, among other things, are more pure optimizers. > > lmfit allows fancier parameter definitions, imposes constraints > through reparameterization and produces more results statistics. The > optimizer that it uses is just Levenberg-Marquardt, although IIRC it > also wraps other optimizers. > > Josef > That's all correct. LMFIT is essentially a wrapper around scipy.optimize.leastsq, as is scipy.optimize.curve_fit, only the two wrapping have different goals. LMFIT can also wrap other minimizers from scipy.optimize, and makes it easy to switch between minimizers. MPFIT.py is a translation by my colleague and co-worker Mark Rivers from IDL to Python of Craig Markwardt's re-implementation in IDL of the original Fortran MINPACK-1 code of Garbow, Hillstrom, and More (the same code that scipy.optimize.leastsq wraps). Aside from a fairly clunky addition of bounds and tied parameters, the interesting part of MPFIT.py is that it is in pure Python, not using scipy or relying on compiled code other than numpy (and Numeric before that). In the meantime, Craig's IDL implementation has been translated yet again to C ( http://www.physics.wisc.edu/~craigm/idl/cmpfit.html). For clarity, MPFIT in IDL, Python, and C are not explicitly licensed with GPL v3 (as suggested above). The licenses are not explicitly stated, but I am sure that Mark and Craig would be amenable to making these codes MIT-licensed if the codes were of interest. In many ways, the original motivation for LMFIT was to replace MPFIT using scipy.optimize.leastsq and MINPACK-1 directly, and to have something more like Python and less like Fortran or IDL. I'm biased, but would recommend that anyone thinking about using MPFIT for a new project would do better to use LMFIT. I don't disagree that scipy could use more pure optimizers, but I also think that striving for a more consistent and elegant interface to these would be very helpful. With the notable exception of the relatively recent unification of the scaler minimizers with minimize(), it seems that many of the existing methods are fairly bare-bones wrappings of underlying C or Fortran code. Of course, having such wrapping is critically important, but I think there is a need for a higher level interface as well. --Matt Newville -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Wed Jun 18 18:10:20 2014 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Thu, 19 Jun 2014 00:10:20 +0200 Subject: [SciPy-Dev] SciPy 2014 BoF Panel In-Reply-To: References: Message-ID: On Wed, Jun 4, 2014 at 1:11 AM, Kyle Mandli wrote: > Hello everyone, > > As one of the co-chairs in charge of organizing the birds-of-a-feather > sesssions at the SciPy conference this year, I wanted to solicit through > the SciPy list to start a discussion on what kind of BoF or set of BoF > sessions might be worthwhile at SciPy this year. Since SciPy is > particularly broad (especially if you include the sci-kits) I was thinking > it might be worthwhile to cast a wide net and see what interest there was. > I can help facilitate organization and ideas on what kind of format for > each particular BoF that is proposed might have. > Hi Kyle, looks like there won't be a lot of scipy devs at the SciPy conference. Most of us are based in Europe. In principle I like the idea of such a BoF though, thanks for bringing it up. We'll actually have a sprint at EuroSciPy (Aug 31) where I think holding a discussion on the scipy roadmap would make sense. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla.molden at gmail.com Wed Jun 18 19:48:08 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Wed, 18 Jun 2014 23:48:08 +0000 (UTC) Subject: [SciPy-Dev] Levenberg-Marquardt Implementation References: <285817149424708026.514167sturla.molden-gmail.com@news.gmane.org> <1737417469424708185.255175sturla.molden-gmail.com@news.gmane.org> Message-ID: <1371265320424827154.411560sturla.molden-gmail.com@news.gmane.org> wrote: > lmfit allows fancier parameter definitions, imposes constraints > through reparameterization and produces more results statistics. The > optimizer that it uses is just Levenberg-Marquardt, although IIRC it > also wraps other optimizers. Yes, and since we were talking about Levenberg-Marquardt, trust-region optimizers are among the ones I miss... Among the one-dimensional minimizers Newton-Raphson is a basic method which is missing, though we have it for root-finding. While it is possible to search for the root of the first derivative, but it would be better to have a convergence check for function minimization instead. Multidimensional Newton-Raphson is also missing. Though multidimensional root-finding is dubious business, it is sometimes useful as a minimizer when the Hessian is known. We could probably also use some form of the EM algorithm in scipy.optimize. (Yes I know what to do with it, just post a PR...) Sturla From kyle.mandli at gmail.com Thu Jun 19 12:51:12 2014 From: kyle.mandli at gmail.com (Kyle Mandli) Date: Thu, 19 Jun 2014 11:51:12 -0500 Subject: [SciPy-Dev] SciPy 2014 BoF Panel In-Reply-To: References: Message-ID: Thank Ralf, I figured this might be a hard one to organize. There are a couple of more general BoFs in the works where SciPy the stack will be discussed. If there's interest we could try and get a tele-conferencing type of thing going (google hangouts, skype, Mozilla's etherpad, etc.). Kyle On Wed, Jun 18, 2014 at 5:10 PM, Ralf Gommers wrote: > > > > On Wed, Jun 4, 2014 at 1:11 AM, Kyle Mandli wrote: > >> Hello everyone, >> >> As one of the co-chairs in charge of organizing the birds-of-a-feather >> sesssions at the SciPy conference this year, I wanted to solicit through >> the SciPy list to start a discussion on what kind of BoF or set of BoF >> sessions might be worthwhile at SciPy this year. Since SciPy is >> particularly broad (especially if you include the sci-kits) I was thinking >> it might be worthwhile to cast a wide net and see what interest there was. >> I can help facilitate organization and ideas on what kind of format for >> each particular BoF that is proposed might have. >> > > Hi Kyle, looks like there won't be a lot of scipy devs at the SciPy > conference. Most of us are based in Europe. In principle I like the idea of > such a BoF though, thanks for bringing it up. > > We'll actually have a sprint at EuroSciPy (Aug 31) where I think holding a > discussion on the scipy roadmap would make sense. > > Cheers, > Ralf > > > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaime.frio at gmail.com Thu Jun 19 13:46:25 2014 From: jaime.frio at gmail.com (=?UTF-8?Q?Jaime_Fern=C3=A1ndez_del_R=C3=ADo?=) Date: Thu, 19 Jun 2014 10:46:25 -0700 Subject: [SciPy-Dev] Use of `input` in ndimage Message-ID: I submitted a PR ( https://github.com/scipy/scipy/pull/3745#issuecomment-46590839) to remove use of the built-in Python function name `input` from the ndimage module, where it is used as the argument name for the input data in almost every single function. A full list of functions involved has been pulled by Warren on the discussion of that PR. Warren wisely pointed out that changing the name of an argument changes the API, and should therefore not be done lightly, as it could break existing user code. One could argue. or at least wish, that, since `input` is typically the first argument of the functions involved, and is non-optional, calls using it as a keyword argument should be rare. But the potential harm is undeniable. It is ugly and a clear violation of PEP8, but although this kind of things are usually an accident waiting to happen, it has also proven harmless so far. So for now I have simply closed the PR, letting practicality beat purity. And so it will remain, unless the community thinks this special case is not special enough to break the rules. Inputs on how to handle the API change to minimize impact to users are also welcome. Thanks, Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes de dominaci?n mundial. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Fri Jun 20 19:00:27 2014 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 21 Jun 2014 01:00:27 +0200 Subject: [SciPy-Dev] Use of `input` in ndimage In-Reply-To: References: Message-ID: On Thu, Jun 19, 2014 at 7:46 PM, Jaime Fern?ndez del R?o wrote: > I submitted a PR > (https://github.com/scipy/scipy/pull/3745#issuecomment-46590839) to remove > use of the built-in Python function name `input` from the ndimage module, > where it is used as the argument name for the input data in almost every > single function. A full list of functions involved has been pulled by Warren > on the discussion of that PR. > > Warren wisely pointed out that changing the name of an argument changes the > API, and should therefore not be done lightly, as it could break existing > user code. One could argue. or at least wish, that, since `input` is > typically the first argument of the functions involved, and is non-optional, > calls using it as a keyword argument should be rare. But the potential harm > is undeniable. > > It is ugly and a clear violation of PEP8, but although this kind of things > are usually an accident waiting to happen, it has also proven harmless so > far. So for now I have simply closed the PR, letting practicality beat > purity. And so it will remain, unless the community thinks this special case > is not special enough to break the rules. Inputs on how to handle the API > change to minimize impact to users are also welcome. I don't find the current situation particularly problematic. `input()` is not a function that is used anywhere in ndimage (indeed, using it would be a much worse problem than shadowing a builtin), and ndimage is not so frequently modified that anything like this is actually likely to cause a problem in the future. I do see that `>>> input = ...` is used in some of the docstring examples; cleaning those up *would* be worth doing. -- Robert Kern From Simon.Tournier at alumni.enseeiht.fr Fri Jun 20 20:54:44 2014 From: Simon.Tournier at alumni.enseeiht.fr (simon tournier) Date: Sat, 21 Jun 2014 00:54:44 +0000 Subject: [SciPy-Dev] Reordering methods for Sparse matrices Message-ID: Hi, >> I have written two sparse matrix reordering functions in Cython and >> I >> am wondering if they would be worthy additions to the scipy.sparse >> module? >> These are currently being used in our QuTiP library (qutip.org). >> >> The first function finds the Reverse Cuthill-Mckee (RCM) ordering >> [1] >> of a symmetric sparse matrix, similar to the Matlab ?symrcm' >> function. >> This is useful for reducing the bandwidth of the underlying matrix >> to >> help eliminate fill-in when using direct or iterative solvers. >> If the matrix is not symmetric, it looks at A+trans(A). Since the >> matrix is symmetric, this works for both CSR and CSC matrices. >> >> The second, is a maximal traversal algorithm based on a >> breadth-first-search >> method [2]. This returns a set of row permutations that makes >> the diagonal zero-free. Since the permutation is not symmetric, >> this works only on CSC matrices. Like RCM this is useful >> for preordering in both direct and iterative solvers. > Sparse matrix permutation algorithms would certainly be a welcome > addition. > > As the code is already written, integrating it in scipy.sparse is > probably fairly simple. If you have questions, Are these functions merged and now included in scipy.sparse or scipy.sparse.csgraph ? If yes, I have not seen the documentation and I have missed some points... so I'm sorry for the question. If no, is it planned ? Theses orderings would be a nice feature for the sparse linear algebra, in particular for both direct and iterative solvers. And a common version of RCM should be useful as explained in this thread: http://mail.scipy.org/pipermail/scipy-dev/2011-December/016786.html Moreover, to the question: "how to draw a good line between what belongs in scipy.sparse and what doesn't" [jakevdp] asked here: https://github.com/scipy/scipy/pull/119 and answered [rgommers] in the same thread by: -- should have potential uses on more than one project that depends on scipy -- should appear in an introductory algorithm book such as "Introduction to Algorithms", Cormen et al. from my experience of playing with sparse linear solvers, the useful algorithms are: Random, column, minimum degree, Dulmage-Mendelsohn and reverse Cuthill-McKee permutations. In other words, those included in matlab (http://www.mathworks.com/help/matlab/reordering-algorithms.html). I have maybe wrong and I am just not able to find them in scipy (or numpy). If not, is it planned to integrate them in scipy (or numpy) ? Are they already implemented in scikit-learn or sfepy or qutip or ... ? cheers, simon From nonhermitian at gmail.com Fri Jun 20 23:24:59 2014 From: nonhermitian at gmail.com (Paul Nation) Date: Sat, 21 Jun 2014 12:24:59 +0900 Subject: [SciPy-Dev] Reordering methods for Sparse matrices In-Reply-To: References: Message-ID: I have yet to do a pull request for the RCM routine due to a lack of time (new baby, lectures, and conference organization). However, the code itself is already written https://github.com/qutip/qutip/blob/master/qutip/cy/graph_utils.pyx and just needs to be reworked to match the SciPy format. Since you reminded me, perhaps I can find time this weekend to push this out so at least the pull request is done. Regards, Paul On Jun 21, 2014, at 9:54 AM, simon tournier wrote: > Hi, > >>> I have written two sparse matrix reordering functions in Cython and >>> I >>> am wondering if they would be worthy additions to the scipy.sparse >>> module? >>> These are currently being used in our QuTiP library (qutip.org). >>> >>> The first function finds the Reverse Cuthill-Mckee (RCM) ordering >>> [1] >>> of a symmetric sparse matrix, similar to the Matlab ?symrcm' >>> function. >>> This is useful for reducing the bandwidth of the underlying matrix >>> to >>> help eliminate fill-in when using direct or iterative solvers. >>> If the matrix is not symmetric, it looks at A+trans(A). Since the >>> matrix is symmetric, this works for both CSR and CSC matrices. >>> >>> The second, is a maximal traversal algorithm based on a >>> breadth-first-search >>> method [2]. This returns a set of row permutations that makes >>> the diagonal zero-free. Since the permutation is not symmetric, >>> this works only on CSC matrices. Like RCM this is useful >>> for preordering in both direct and iterative solvers. > >> Sparse matrix permutation algorithms would certainly be a welcome >> addition. >> >> As the code is already written, integrating it in scipy.sparse is >> probably fairly simple. If you have questions, > > Are these functions merged and now included in scipy.sparse or > scipy.sparse.csgraph ? > If yes, I have not seen the documentation and I have missed some > points... so I'm sorry for the question. > If no, is it planned ? > > Theses orderings would be a nice feature for the sparse linear algebra, > in particular for both direct and iterative solvers. > And a common version of RCM should be useful as explained in this > thread: > http://mail.scipy.org/pipermail/scipy-dev/2011-December/016786.html > > Moreover, to the question: "how to draw a good line between what > belongs in scipy.sparse and what doesn't" [jakevdp] asked here: > https://github.com/scipy/scipy/pull/119 > and answered [rgommers] in the same thread by: > -- should have potential uses on more than one project that depends > on scipy > -- should appear in an introductory algorithm book such as > "Introduction to Algorithms", Cormen et al. > from my experience of playing with sparse linear solvers, the useful > algorithms are: Random, column, minimum degree, Dulmage-Mendelsohn and > reverse Cuthill-McKee permutations. In other words, those included in > matlab > (http://www.mathworks.com/help/matlab/reordering-algorithms.html). > > I have maybe wrong and I am just not able to find them in scipy (or > numpy). > If not, is it planned to integrate them in scipy (or numpy) ? > Are they already implemented in scikit-learn or sfepy or qutip or ... ? > > > cheers, > simon > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev From ralf.gommers at gmail.com Thu Jun 26 03:14:08 2014 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Thu, 26 Jun 2014 09:14:08 +0200 Subject: [SciPy-Dev] SciPy 2014 BoF Panel In-Reply-To: References: Message-ID: On Thu, Jun 19, 2014 at 6:51 PM, Kyle Mandli wrote: > Thank Ralf, I figured this might be a hard one to organize. There are a > couple of more general BoFs in the works where SciPy the stack will be > discussed. If there's interest we could try and get a tele-conferencing > type of thing going (google hangouts, skype, Mozilla's etherpad, etc.). > I'd rather not do the teleconf thing unless there's something very specific I'd need to weigh in on. My experience is that usually calling in slows things down a lot for people that are physically present. Ralf > > Kyle > > > On Wed, Jun 18, 2014 at 5:10 PM, Ralf Gommers > wrote: > >> >> >> >> On Wed, Jun 4, 2014 at 1:11 AM, Kyle Mandli >> wrote: >> >>> Hello everyone, >>> >>> As one of the co-chairs in charge of organizing the birds-of-a-feather >>> sesssions at the SciPy conference this year, I wanted to solicit through >>> the SciPy list to start a discussion on what kind of BoF or set of BoF >>> sessions might be worthwhile at SciPy this year. Since SciPy is >>> particularly broad (especially if you include the sci-kits) I was thinking >>> it might be worthwhile to cast a wide net and see what interest there was. >>> I can help facilitate organization and ideas on what kind of format for >>> each particular BoF that is proposed might have. >>> >> >> Hi Kyle, looks like there won't be a lot of scipy devs at the SciPy >> conference. Most of us are based in Europe. In principle I like the idea of >> such a BoF though, thanks for bringing it up. >> >> We'll actually have a sprint at EuroSciPy (Aug 31) where I think holding >> a discussion on the scipy roadmap would make sense. >> >> Cheers, >> Ralf >> >> >> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla.molden at gmail.com Sat Jun 28 05:10:38 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Sat, 28 Jun 2014 09:10:38 +0000 (UTC) Subject: [SciPy-Dev] Use *trsm for scipy.linalg.solve_triangular? Message-ID: <1066708020425637952.223110sturla.molden-gmail.com@news.gmane.org> scipy.linalg.solve_triangular uses *trtrs from LAPACK. I must take a great deal of the responsibility for this, since I might have suggested it... In retrospect I think this was a mistake. *trtrs can solve op(A) X = B where op(A) is A or A.T *trsm from BLAS is more general. It can solve op(A) X = alpha * B or X op(A) = alpha * B. The latter might e.g. be useful for constructing an efficient scipy.linalg.solve for C ordered arrays. *trtrs is just a shallow wrapper over *trsm. It optionally checks A for singularity and then calls *trsm. The optional singularity check just amounts to checking that none of the diagonal elements are zero. I think we should make scipy.linalg.blas expose *trsm and then use that method for solve_triangular. Later we can use *trsm to optimize scipy.linalg.solve for C ordered arrays (i.e. avoid generating transposed copies). Regards, Sturla From fperez.net at gmail.com Sat Jun 28 22:31:46 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 28 Jun 2014 19:31:46 -0700 Subject: [SciPy-Dev] BoF at Scipy'14 about Numpy's future. Message-ID: Hi folks, [ Please excuse the repost, this is just to try to get as many good ideas as possible in the agenda] I've just created a page on the numpy wiki: https://github.com/numpy/numpy/wiki/Numpy-BoF-at-Scipy-2014 I'd appreciate it if people could put down on that page the title of topics, plus a brief summary (with links to this or other relevant threads), they'd like to see discussed. BoFs can be very useful but they can also easily devolve into random digressions. I'll do my best to fulfill my role as discussion facilitator by pushing for an agenda for the discussion that can fit realistically in the time we have, and trying to keep us on track. It would help a lot if people put their ideas in advance, so we can sort/refine/prioritize a little bit. Hopefully this will lead to a more productive conversation. Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyle.mandli at gmail.com Sun Jun 29 15:49:21 2014 From: kyle.mandli at gmail.com (Kyle Mandli) Date: Sun, 29 Jun 2014 14:49:21 -0500 Subject: [SciPy-Dev] SciPy 2014 BoF Panel In-Reply-To: References: Message-ID: Let's just table this for now, I honestly think talking about the entirety of the SciPy package would be difficult in any case. Next year or at the EeroSciPy perhaps we can aim for having more specific discussions about some of the SciPy packages and maybe an overarching discussion on future directions for all the packages? Kyle On Thu, Jun 26, 2014 at 2:14 AM, Ralf Gommers wrote: > > > > On Thu, Jun 19, 2014 at 6:51 PM, Kyle Mandli > wrote: > >> Thank Ralf, I figured this might be a hard one to organize. There are a >> couple of more general BoFs in the works where SciPy the stack will be >> discussed. If there's interest we could try and get a tele-conferencing >> type of thing going (google hangouts, skype, Mozilla's etherpad, etc.). >> > > I'd rather not do the teleconf thing unless there's something very > specific I'd need to weigh in on. My experience is that usually calling in > slows things down a lot for people that are physically present. > > Ralf > > >> >> Kyle >> >> >> On Wed, Jun 18, 2014 at 5:10 PM, Ralf Gommers >> wrote: >> >>> >>> >>> >>> On Wed, Jun 4, 2014 at 1:11 AM, Kyle Mandli >>> wrote: >>> >>>> Hello everyone, >>>> >>>> As one of the co-chairs in charge of organizing the birds-of-a-feather >>>> sesssions at the SciPy conference this year, I wanted to solicit through >>>> the SciPy list to start a discussion on what kind of BoF or set of BoF >>>> sessions might be worthwhile at SciPy this year. Since SciPy is >>>> particularly broad (especially if you include the sci-kits) I was thinking >>>> it might be worthwhile to cast a wide net and see what interest there was. >>>> I can help facilitate organization and ideas on what kind of format for >>>> each particular BoF that is proposed might have. >>>> >>> >>> Hi Kyle, looks like there won't be a lot of scipy devs at the SciPy >>> conference. Most of us are based in Europe. In principle I like the idea of >>> such a BoF though, thanks for bringing it up. >>> >>> We'll actually have a sprint at EuroSciPy (Aug 31) where I think holding >>> a discussion on the scipy roadmap would make sense. >>> >>> Cheers, >>> Ralf >>> >>> >>> >>> _______________________________________________ >>> SciPy-Dev mailing list >>> SciPy-Dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>> >>> >> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla.molden at gmail.com Mon Jun 30 20:05:43 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Tue, 1 Jul 2014 00:05:43 +0000 (UTC) Subject: [SciPy-Dev] SciPy-Dev Digest, Vol 128, Issue 5 References: Message-ID: <1816260967425865047.889482sturla.molden-gmail.com@news.gmane.org> Matt Newville wrote: > I don't disagree that scipy could use more pure optimizers, but I also > think that striving for a more consistent and elegant interface to these > would be very helpful. With the notable exception of the relatively recent > unification of the scaler minimizers with minimize(), it seems that many of > the existing methods are fairly bare-bones wrappings of underlying C or > Fortran code. Of course, having such wrapping is critically important, > but I think there is a need for a higher level interface as well. The raison d'etre for SciPy is "nice to use". So clearly simple and intuitive high-level interfaces are needed. If we only cared about speed we should all be coding in Fortran 77. Personally I am willing to scrifice a lot of speed for a nice high-level interface. Currently my main interest in SciPy's LM is the underlying solver, though. It's a very old Fortran code that even supplies its own linear algebra solvers because it was written before LAPACK. It's not very nice on modern computers, for various reasons. Sturla