From enzomich at gmail.com Thu Oct 1 00:07:18 2009 From: enzomich at gmail.com (Enzo Michelangeli) Date: Thu, 1 Oct 2009 12:07:18 +0800 Subject: [SciPy-dev] Fast (O(n log(n)) ) implementation of Kendall Tau References: <8752305D07F74413BC569D657AD65551@EMLT> <4AC3D5C5.8030107@molden.no> Message-ID: <571F15D6C3AC4F848A8F2243C163B23B@EMLT> From: "Sturla Molden" Sent: Thursday, October 01, 2009 6:03 AM > Enzo Michelangeli skrev: >> Dear all, >> >> A few days ago I posted to http://projects.scipy.org/scipy/ticket/999 a >> drop-in replacement for scipy.stats.kendalltau. My code implements the >> algorithm with complexity O(n log()) described by William R. Knight in a >> paper of 1966 archived at http://www.jstor.org/pss/2282833 , whereas the >> function currently in SciPy has complexity O(n^2), which makes it >> unusable >> > There is also: > > http://projects.scipy.org/scipy/ticket/893 > > It has a contigency table version that would be fast for large data > sets, in theory O(n). Yes, but that requires a native module. Enzo From enzomich at gmail.com Thu Oct 1 00:07:42 2009 From: enzomich at gmail.com (Enzo Michelangeli) Date: Thu, 1 Oct 2009 12:07:42 +0800 Subject: [SciPy-dev] Fast (O(n log(n)) ) implementation of Kendall Tau References: <8752305D07F74413BC569D657AD65551@EMLT> <4AC3D710.8090505@molden.no> Message-ID: From: "Sturla Molden" Sent: Thursday, October 01, 2009 6:09 AM > Enzo Michelangeli skrev: >> Dear all, >> >> A few days ago I posted to http://projects.scipy.org/scipy/ticket/999 a >> drop-in replacement for scipy.stats.kendalltau. > Why do you re-implement mergesort in pure Python? > > ndarrays have a sort method that can use mergesort. > > Python lists has the same (timsort is mergesort on steroids). Because Knight's algorithm needs to count the number of swaps (or, to be more precise, the number of swaps that would be performed by an equivalent bubblesort). In the code, that's the purpose of the variable exchcnt . An alternative algorithm for the Kendall Tau developed by David Christensen, and described at http://www.springerlink.com/content/p33qu44058984082/ (with a Delphi implementation), replaces the mergesort step with one based on balanced binary trees (AVL in his case, but I guess RBT would also work). Unfortunately, neither the standard Python library nor NumPy/SciPy appear to implement such useful data structures, and what is available either doesn't allow O(log(n)) random access (heapq) or lacks a O(log(n)) insert (sorted lists accessed through bisect). Enzo From charlesr.harris at gmail.com Thu Oct 1 02:22:38 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 1 Oct 2009 00:22:38 -0600 Subject: [SciPy-dev] Fast (O(n log(n)) ) implementation of Kendall Tau In-Reply-To: References: <8752305D07F74413BC569D657AD65551@EMLT> <4AC3D710.8090505@molden.no> Message-ID: On Wed, Sep 30, 2009 at 10:07 PM, Enzo Michelangeli wrote: > From: "Sturla Molden" > Sent: Thursday, October 01, 2009 6:09 AM > > > Enzo Michelangeli skrev: > >> Dear all, > >> > >> A few days ago I posted to http://projects.scipy.org/scipy/ticket/999 a > >> drop-in replacement for scipy.stats.kendalltau. > > Why do you re-implement mergesort in pure Python? > > > > ndarrays have a sort method that can use mergesort. > > > > Python lists has the same (timsort is mergesort on steroids). > > Because Knight's algorithm needs to count the number of swaps (or, to be > more precise, the number of swaps that would be performed by an equivalent > bubblesort). In the code, that's the purpose of the variable exchcnt . > > An alternative algorithm for the Kendall Tau developed by David > Christensen, > and described at http://www.springerlink.com/content/p33qu44058984082/(with > a Delphi implementation), replaces the mergesort step with one based on > balanced binary trees (AVL in his case, but I guess RBT would also work). > Unfortunately, neither the standard Python library nor NumPy/SciPy appear > to > implement such useful data structures, and what is available either doesn't > Yep, we could use some more of those useful data structures. A "computer science" library would be useful. The scipy.spacial library is step in that direction but it would be nice if there were more such. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott.sinclair.za at gmail.com Thu Oct 1 04:28:55 2009 From: scott.sinclair.za at gmail.com (Scott Sinclair) Date: Thu, 1 Oct 2009 10:28:55 +0200 Subject: [SciPy-dev] Statsmodels documentation buglet Message-ID: <6a17e9ee0910010128v55ad0048ka15e76689b0e9e17@mail.gmail.com> Hi, I'm having a look at the statsmodels scikit and have noticed a minor bug in the docs at: http://statsmodels.sourceforge.net/generated/scikits.statsmodels.regression.OLS.html The definition Y = [1,3,4,5,2,3,4], in the example contains a trailing comma that causes the example to fail when used with IPython's cpaste mode. Cheers, Scott From jsseabold at gmail.com Thu Oct 1 11:01:05 2009 From: jsseabold at gmail.com (Skipper Seabold) Date: Thu, 1 Oct 2009 11:01:05 -0400 Subject: [SciPy-dev] Statsmodels documentation buglet In-Reply-To: <6a17e9ee0910010128v55ad0048ka15e76689b0e9e17@mail.gmail.com> References: <6a17e9ee0910010128v55ad0048ka15e76689b0e9e17@mail.gmail.com> Message-ID: On Thu, Oct 1, 2009 at 4:28 AM, Scott Sinclair wrote: > Hi, > > I'm having a look at the statsmodels scikit and have noticed a minor > bug in the docs at: > > http://statsmodels.sourceforge.net/generated/scikits.statsmodels.regression.OLS.html > > The definition Y = [1,3,4,5,2,3,4], in the example contains a trailing > comma that causes the example to fail when used with IPython's cpaste > mode. > Thanks, should be fixed shortly. Skipper From josef.pktd at gmail.com Thu Oct 1 11:59:13 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Thu, 1 Oct 2009 11:59:13 -0400 Subject: [SciPy-dev] Statsmodels documentation buglet In-Reply-To: References: <6a17e9ee0910010128v55ad0048ka15e76689b0e9e17@mail.gmail.com> Message-ID: <1cd32cbb0910010859h76b7ee20x91556e6850f79285@mail.gmail.com> On Thu, Oct 1, 2009 at 11:01 AM, Skipper Seabold wrote: > On Thu, Oct 1, 2009 at 4:28 AM, Scott Sinclair > wrote: >> Hi, >> >> I'm having a look at the statsmodels scikit and have noticed a minor >> bug in the docs at: >> >> http://statsmodels.sourceforge.net/generated/scikits.statsmodels.regression.OLS.html >> >> The definition Y = [1,3,4,5,2,3,4], in the example contains a trailing >> comma that causes the example to fail when used with IPython's cpaste >> mode. >> > > Thanks, should be fixed shortly. I did it already in trunk, but got stuck with some examples. I moved the tutorial files from the sandbox to the examples folder and will push to launchpad soon. WLS with one of the regressors squared as weights still looks "weird" (example_wls.py). We haven't run the doctests in a while. However the scripts in the example folder are always checked whether they run (not whether they produce correct results). Scott, any comments about the usage of statsmodels are very welcome. (And I'm still waiting for missing bug reports.) Josef > > Skipper > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From prabhu at aero.iitb.ac.in Thu Oct 1 16:48:57 2009 From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran) Date: Fri, 02 Oct 2009 02:18:57 +0530 Subject: [SciPy-dev] [ANN] SciPy India conference in Dec. 2009 Message-ID: <4AC515B9.4070604@aero.iitb.ac.in> Greetings, The first "Scientific Computing with Python" conference in India (http://scipy.in) will be held from December 12th to 17th, 2009 at the Technopark in Trivandrum, Kerala, India (http://www.technopark.org/). The theme of the conference will be "Scientific Python in Action" with respect to application and teaching. We are pleased to have Travis Oliphant, the creator and lead developer of numpy (http://numpy.scipy.org) as the keynote speaker. Here is a rough schedule of the conference: Sat. Dec. 12 (conference) Sun. Dec. 13 (conference) Mon. Dec. 14 (tutorials) Tues. Dec. 15 (tutorials) Wed. Dec. 16 (sprint) Thu. Dec. 17 (sprint) The tutorial sessions will have two tracks, one specifically for teachers and one for the general public. There are no registration fees. Please register at: http://scipy.in The call for papers will be announced soon. This conference is organized by the FOSSEE project (http://fossee.in) funded by the Ministry of Human Resources and Development's National Mission on Education (NME) through Information and Communication Technology (ICT) jointly with SPACE-Kerala (http://www.space-kerala.org). Regards, Prabhu Ramachandran and Jarrod Millman From scott.sinclair.za at gmail.com Fri Oct 2 02:04:49 2009 From: scott.sinclair.za at gmail.com (Scott Sinclair) Date: Fri, 2 Oct 2009 08:04:49 +0200 Subject: [SciPy-dev] Statsmodels documentation buglet In-Reply-To: <1cd32cbb0910010859h76b7ee20x91556e6850f79285@mail.gmail.com> References: <6a17e9ee0910010128v55ad0048ka15e76689b0e9e17@mail.gmail.com> <1cd32cbb0910010859h76b7ee20x91556e6850f79285@mail.gmail.com> Message-ID: <6a17e9ee0910012304g59234aafqfecc7d69b036ea15@mail.gmail.com> >2009/10/1 : > Scott, any comments about the usage of statsmodels are very welcome. > (And I'm still waiting for missing bug reports.) Just poking around and finding it pretty useable so far. Happy to report any bugs if I find them. Cheers, Scott From arkapravobhaumik at gmail.com Fri Oct 2 08:24:50 2009 From: arkapravobhaumik at gmail.com (Arkapravo Bhaumik) Date: Fri, 2 Oct 2009 13:24:50 +0100 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: Message-ID: > Dear Sir/Ma'am > > I was in e-mail contact with one of your colleagues , Travis ; I am very > interested in contributing to scipy project as a volunteer. I believe that I > can contribute in > > (1) Documentation > (2) Suggesting and developing newer functionality > (3) Study similar software as Matlab, Mathematica, Maple etc trying to look > for inspiration for possible improvements in scipy > > Some of my dabbling python are discussed in my blogspot, > http://programming-unlimited.blogspot.com/ and I am eager to be a part of > scipy : a revolution in the making. > > I look forward to your reply. > > Kind regards > > Arkapravo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arkapravobhaumik at gmail.com Fri Oct 2 08:32:36 2009 From: arkapravobhaumik at gmail.com (Arkapravo Bhaumik) Date: Fri, 2 Oct 2009 13:32:36 +0100 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: Message-ID: > Dear Sir/Ma'am > > I was in e-mail contact with one of your colleagues , Travis ; I am very > interested in contributing to scipy project as a volunteer. I believe that I > can contribute in > > (1) Documentation > (2) Suggesting and developing newer functionality > (3) Study similar software as Matlab, Mathematica, Maple etc trying to look > for inspiration for possible improvements in scipy > > Some of my dabbling python are discussed in my blogspot, > http://programming-unlimited.blogspot.com/ and I am eager to be a part of > scipy : a revolution in the making. > > I look forward to your reply. > > Kind regards > > Arkapravo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Fri Oct 2 16:14:48 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Fri, 2 Oct 2009 13:14:48 -0700 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: Message-ID: <45d1ab480910021314j41998d8bi5162d3668845460@mail.gmail.com> On Fri, Oct 2, 2009 at 5:24 AM, Arkapravo Bhaumik < arkapravobhaumik at gmail.com> wrote: > > Dear Sir/Ma'am >> >> I was in e-mail contact with one of your colleagues , Travis ; I am very >> interested in contributing to scipy project as a volunteer. I believe that I >> can contribute in >> >> (1) Documentation >> > Arkapravo, Great! As far as helping with the documentation is concerned, here's what you do: 0) Visit and read docs.scipy.org/numpy 1) As it says there, register a username at http://docs.scipy.org/numpy/accounts/register 2) Subscribe to scipy-dev at scipy.org at http://mail.scipy.org/mailman/listinfo/scipy-dev and email there the username created in Step 1) along with an explanation that you would like docstring editing privileges. (Subscribing is not strictly necessary - an alternative is described at 0) - but all numpy and scipy development and documentation issues are discussed there, so it really is the most convenient way to communicate with the development community.) 3) Once you receive confirmation of your editing privileges, you can login and visit http://docs.scipy.org/numpy/Milestones and http://docs.scipy.org/numpy/docs/ to see numpy docstrings that need editing (relatively few now) and http://docs.scipy.org/numpy/Milestones for scipy docstrings that need editing (still plentiful). (Before you edit, be sure to have read, at a minimum, http://projects.scipy.org/scipy/numpy/wiki/CodingStyleGuidelines#docstring-standard .) 4) Questions can be posted here, in the Discussion section following each individual docstring editing page, or at the Q+A page http://docs.scipy.org/nump/Questions+Answers/. Thanks again, David Goldsmith Technical Editor Olympia, Washington, USA -------------- next part -------------- An HTML attachment was scrubbed... URL: From pav at iki.fi Sat Oct 3 08:22:25 2009 From: pav at iki.fi (Pauli Virtanen) Date: Sat, 03 Oct 2009 15:22:25 +0300 Subject: [SciPy-dev] Doc edit rights In-Reply-To: <4AC374AD.3090201@stsci.edu> References: <4AC368E6.9080705@stsci.edu> <4AC374AD.3090201@stsci.edu> Message-ID: <1254572545.5855.0.camel@idol> ke, 2009-09-30 kello 11:09 -0400, Michael Droettboom kirjoitti: > My username is mdroe. You have them now. Cheers, Pauli From prabhu at aero.iitb.ac.in Sat Oct 3 23:53:35 2009 From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran) Date: Sun, 04 Oct 2009 09:23:35 +0530 Subject: [SciPy-dev] numpy.distutils: passing config_fc options in setup.py Message-ID: <4AC81C3F.5010700@aero.iitb.ac.in> Hi, I've been using f2py to generate an extension module for someone's free-form f90 code. I need to pass extra arguments to config_fc and right now to get everything working I do this: python setup.py config_fc --f90flags='-ffree-form -fno-underscoring' build_ext --inplace Is there an easy way to set the --f90flags set by default to what I want in the setup.py script itself? I.e. I'd like users to be able to do: python setup.py build_ext --inplace This doesn't seem documented anywhere, is there some place I should look? This is how my setup.py looks currently: #---- from numpy.distutils.core import Extension, setup setup(ext_modules = [Extension("varsha", ["varsha.pyf", "s2p.f", "fft.c", "invLeg.c"], define_macros=[('NO_APPEND_FORTRAN', None)], ) ], ) #---- passing config_fc={...} to either Extension or setup does not work. The numpy version is 1.4.0.dev6976. Thanks! cheers, prabhu From david at ar.media.kyoto-u.ac.jp Mon Oct 5 03:25:51 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 05 Oct 2009 16:25:51 +0900 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: Message-ID: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> Hi Arkapravo, Arkapravo Bhaumik wrote: > > Dear Sir/Ma'am > > I was in e-mail contact with one of your colleagues , Travis ; I > am very interested in contributing to scipy project as a > volunteer. I believe that I can contribute in > > (1) Documentation > (2) Suggesting and developing newer functionality > (3) Study similar software as Matlab, Mathematica, Maple etc > trying to look for inspiration for possible improvements in scipy > Thanks for your help. As in most open source projects, the best way to contribute is to work on something that you actually need for yourself, be it code enhancement, bug fixes, documentation, etc... Depending on your interests and proficiency in python vs. C vs. Fortran, there are several things that could be worked on: the ones which need the most work ATM are scipy.special and scipy.interpolate. Please note also that you should be careful when 'studying' proprietary software - reimplementing from scratch a similar functionality is OK, re-using the implementation is almost never ok. cheers, David From peridot.faceted at gmail.com Mon Oct 5 16:40:46 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Mon, 5 Oct 2009 16:40:46 -0400 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> Message-ID: 2009/10/5 David Cournapeau : > Hi Arkapravo, > > Arkapravo Bhaumik wrote: >> >> ? ? Dear Sir/Ma'am >> >> ? ? I was in e-mail contact with one of your colleagues , Travis ; ?I >> ? ? am very interested in contributing to scipy project as a >> ? ? volunteer. I believe that I can contribute in >> >> ? ? (1) Documentation >> ? ? (2) Suggesting and developing newer functionality >> ? ? (3) Study similar software as Matlab, Mathematica, Maple etc >> ? ? trying to look for inspiration for possible improvements in scipy >> > > Thanks for your help. As in most open source projects, the best way to > contribute is to work on something that you actually need for yourself, > be it code enhancement, bug fixes, documentation, etc... > > Depending on your interests and proficiency in python vs. C vs. Fortran, > there are several things that could be worked on: the ones which need > the most work ATM are scipy.special and scipy.interpolate. > > Please note also that you should be careful when 'studying' proprietary > software - reimplementing from scratch a similar functionality is OK, > re-using the implementation is almost never ok. I would add a mode of working that is definitely okay: find a reference to a journal article on how to compute the thing you're interested in (possibly by finding the reference in matlab/r/maple/mathematica/whatever source ordoumentation) and implementing what is described in the research paper. This is a great way to fill gaps in scipy with good, well-thought-out algorithms. Anne From d.l.goldsmith at gmail.com Mon Oct 5 16:54:11 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Mon, 5 Oct 2009 13:54:11 -0700 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> Message-ID: <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Just curious, Anne: have you anything in particular in mind (i.e., are there some small - or gaping - holes in scipy (IYO, of course) which you know could be filled by a careful implementation of something(s) extant in the literature)? DG On Mon, Oct 5, 2009 at 1:40 PM, Anne Archibald wrote: > 2009/10/5 David Cournapeau : > > Hi Arkapravo, > > > > Arkapravo Bhaumik wrote: > >> > >> Dear Sir/Ma'am > >> > >> I was in e-mail contact with one of your colleagues , Travis ; I > >> am very interested in contributing to scipy project as a > >> volunteer. I believe that I can contribute in > >> > >> (1) Documentation > >> (2) Suggesting and developing newer functionality > >> (3) Study similar software as Matlab, Mathematica, Maple etc > >> trying to look for inspiration for possible improvements in scipy > >> > > > > Thanks for your help. As in most open source projects, the best way to > > contribute is to work on something that you actually need for yourself, > > be it code enhancement, bug fixes, documentation, etc... > > > > Depending on your interests and proficiency in python vs. C vs. Fortran, > > there are several things that could be worked on: the ones which need > > the most work ATM are scipy.special and scipy.interpolate. > > > > Please note also that you should be careful when 'studying' proprietary > > software - reimplementing from scratch a similar functionality is OK, > > re-using the implementation is almost never ok. > > I would add a mode of working that is definitely okay: find a > reference to a journal article on how to compute the thing you're > interested in (possibly by finding the reference in > matlab/r/maple/mathematica/whatever source ordoumentation) and > implementing what is described in the research paper. This is a great > way to fill gaps in scipy with good, well-thought-out algorithms. > > Anne > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Mon Oct 5 17:20:53 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Mon, 5 Oct 2009 17:20:53 -0400 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Message-ID: 2009/10/5 David Goldsmith : > Just curious, Anne: have you anything in particular in mind (i.e., are there > some small - or gaping - holes in scipy (IYO, of course) which you know > could be filled by a careful implementation of something(s) extant in the > literature)? Well, not exactly - the examples I had in mind were minor and/or in the past. I ran into problems with scipy's hyp2f1, for example, so I went and looked up the best algorithm I could find for it (and I think I contributed that code). I wanted the Kuiper test as an alternative to the Kolmogorov-Smirnov test (it's invariant under cyclic permutations, and is sensitive to different features of the distribution) so I looked up the test and the special function needed to interpret its results. (I haven't contributed this to scipy yet, mostly because I chose an interface that's not particularly compatible with that for scipy's K-S test.) And on a larger scale, that's what scipy.spatial's kdtree implementation is. For examples where I think a bit of lit review plus implementation work might help, I'd say that the orthogonal polynomials could use some work - the generic implementation in scipy.special falls apart rapidly as you go to higher orders. I always implement my own Chebyshev polynomials using the cos(n*arccos(x)) expression, for example, and special implementations for the others might be very useful. Anne From charlesr.harris at gmail.com Mon Oct 5 20:16:50 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 5 Oct 2009 18:16:50 -0600 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Message-ID: On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald wrote: > 2009/10/5 David Goldsmith : > > Just curious, Anne: have you anything in particular in mind (i.e., are > there > > some small - or gaping - holes in scipy (IYO, of course) which you know > > could be filled by a careful implementation of something(s) extant in the > > literature)? > > Well, not exactly - the examples I had in mind were minor and/or in > the past. I ran into problems with scipy's hyp2f1, for example, so I > went and looked up the best algorithm I could find for it (and I think > I contributed that code). I wanted the Kuiper test as an alternative > to the Kolmogorov-Smirnov test (it's invariant under cyclic > permutations, and is sensitive to different features of the > distribution) so I looked up the test and the special function needed > to interpret its results. (I haven't contributed this to scipy yet, > mostly because I chose an interface that's not particularly compatible > with that for scipy's K-S test.) And on a larger scale, that's what > scipy.spatial's kdtree implementation is. > > For examples where I think a bit of lit review plus implementation > work might help, I'd say that the orthogonal polynomials could use > some work - the generic implementation in scipy.special falls apart > rapidly as you go to higher orders. I always implement my own > Chebyshev polynomials using the cos(n*arccos(x)) expression, for > example, and special implementations for the others might be very > useful. > > At what order does the scipy implementation of the Chebyshev polynomials fall apart? I looked briefly at that package a long time ago, but never used it. I ask so I can check the chebyshev module that is going into numpy. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Mon Oct 5 21:29:52 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Mon, 5 Oct 2009 21:29:52 -0400 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Message-ID: 2009/10/5 Charles R Harris : > > > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald > wrote: >> >> 2009/10/5 David Goldsmith : >> > Just curious, Anne: have you anything in particular in mind (i.e., are >> > there >> > some small - or gaping - holes in scipy (IYO, of course) which you know >> > could be filled by a careful implementation of something(s) extant in >> > the >> > literature)? >> >> Well, not exactly - the examples I had in mind were minor and/or in >> the past. I ran into problems with scipy's hyp2f1, for example, so I >> went and looked up the best algorithm I could find for it (and I think >> I contributed that code). I wanted the Kuiper test as an alternative >> to the Kolmogorov-Smirnov test (it's invariant under cyclic >> permutations, and is sensitive to different features of the >> distribution) so I looked up the test and the special function needed >> to interpret its results. (I haven't contributed this to scipy yet, >> mostly because I chose an interface that's not particularly compatible >> with that for scipy's K-S test.) And on a larger scale, that's what >> scipy.spatial's kdtree implementation is. >> >> For examples where I think a bit of lit review plus implementation >> work might help, I'd say that the orthogonal polynomials could use >> some work - the generic implementation in scipy.special falls apart >> rapidly as you go to higher orders. I always implement my own >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for >> example, and special implementations for the others might be very >> useful. >> > > At what order does the scipy implementation of the Chebyshev polynomials > fall apart? I looked briefly at that package a long time ago, but never used > it. I ask so I can check the chebyshev module that is going into numpy. By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off by 0.1, and by n=35 they are off by four - not great for values that should be in the interval [-1,1]. This may seem like an outrageously high degree for a polynomial, but there's no reason they need be this bad, and one could quite reasonably want to use an order this high, say for function approximation. I think this inaccuracy is probably inevitable in a scheme that computes values using a recurrence relation, and something like it probably occurs for all the orthogonal polynomials that don't have special-purpose evaluators. Anne From charlesr.harris at gmail.com Mon Oct 5 23:12:49 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 5 Oct 2009 21:12:49 -0600 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Message-ID: On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald wrote: > 2009/10/5 Charles R Harris : > > > > > > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/5 David Goldsmith : > >> > Just curious, Anne: have you anything in particular in mind (i.e., are > >> > there > >> > some small - or gaping - holes in scipy (IYO, of course) which you > know > >> > could be filled by a careful implementation of something(s) extant in > >> > the > >> > literature)? > >> > >> Well, not exactly - the examples I had in mind were minor and/or in > >> the past. I ran into problems with scipy's hyp2f1, for example, so I > >> went and looked up the best algorithm I could find for it (and I think > >> I contributed that code). I wanted the Kuiper test as an alternative > >> to the Kolmogorov-Smirnov test (it's invariant under cyclic > >> permutations, and is sensitive to different features of the > >> distribution) so I looked up the test and the special function needed > >> to interpret its results. (I haven't contributed this to scipy yet, > >> mostly because I chose an interface that's not particularly compatible > >> with that for scipy's K-S test.) And on a larger scale, that's what > >> scipy.spatial's kdtree implementation is. > >> > >> For examples where I think a bit of lit review plus implementation > >> work might help, I'd say that the orthogonal polynomials could use > >> some work - the generic implementation in scipy.special falls apart > >> rapidly as you go to higher orders. I always implement my own > >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for > >> example, and special implementations for the others might be very > >> useful. > >> > > > > At what order does the scipy implementation of the Chebyshev polynomials > > fall apart? I looked briefly at that package a long time ago, but never > used > > it. I ask so I can check the chebyshev module that is going into numpy. > > By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off > by 0.1, and by n=35 they are off by four - not great for values that > should be in the interval [-1,1]. This may seem like an outrageously > high degree for a polynomial, but there's no reason they need be this > bad, and one could quite reasonably want to use an order this high, > say for function approximation. > > Hmm, I get an maximum error of about 1e-13 for n=100 using my routine, which isn't great and can be improved a bit with a few tricks, but is probably good enough. That's using simple Clenshaw recursion for the evaluation. There must be something seriously wrong with the scipy version. I routinely use fits with n > 50 because truncating such a series gives a good approximation to the minmax fit and it's also nice to see how the coefficients fall off to estimate the size of the truncation error. I think this inaccuracy is probably inevitable in a scheme that > computes values using a recurrence relation Not so, using the Cheybshev recurrence in either direction should be stable for |x| <= 1. It's like multiplying with a complex number of modulus 1, i.e., a unitary matrix. > and something like it probably occurs for all the orthogonal polynomials > that don't have > special-purpose evaluators. > Depends on the recursion, the direction of the recursion, and the domain. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From arkapravobhaumik at gmail.com Mon Oct 5 23:32:29 2009 From: arkapravobhaumik at gmail.com (Arkapravo Bhaumik) Date: Tue, 6 Oct 2009 04:32:29 +0100 Subject: [SciPy-dev] Scipy-dev Digest, Vol 72, Issue 6 In-Reply-To: References: Message-ID: Hi My immediate plan was to try out scipy for some of those 'exotic' integrals which can only be sought through the route of contour integration ( http://programming-unlimited.blogspot.com/2009/10/exotic-integrals-in-scipy.html). My immediate observation has been the comments made in scipy along with the output is not very consistent. I will soon post my detailed observations. Comparing with Matlab etc , I am sure we cannot 'copy' a module (and I doubt that may help). My plan was to used Matlab as a motivation , I guess that in about 3-5 years scipy will take over all the numerical/scientific software. I was wondering that employing some Pythonic-GUI (wx, tk,turtle, frog etc) is it possible to develop software as Simulink ? Best regards Arkapravo 2009/10/6 > Send Scipy-dev mailing list submissions to > scipy-dev at scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request at scipy.org > > You can reach the person managing the list at > scipy-dev-owner at scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Scipy-dev digest..." > > > Today's Topics: > > 1. Re: Volunteer for Scipy Project (Anne Archibald) > 2. Re: Volunteer for Scipy Project (David Goldsmith) > 3. Re: Volunteer for Scipy Project (Anne Archibald) > 4. Re: Volunteer for Scipy Project (Charles R Harris) > 5. Re: Volunteer for Scipy Project (Anne Archibald) > 6. Re: Volunteer for Scipy Project (Charles R Harris) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 5 Oct 2009 16:40:46 -0400 > From: Anne Archibald > Subject: Re: [SciPy-dev] Volunteer for Scipy Project > To: SciPy Developers List > Message-ID: > > Content-Type: text/plain; charset=UTF-8 > > 2009/10/5 David Cournapeau : > > Hi Arkapravo, > > > > Arkapravo Bhaumik wrote: > >> > >> ? ? Dear Sir/Ma'am > >> > >> ? ? I was in e-mail contact with one of your colleagues , Travis ; ?I > >> ? ? am very interested in contributing to scipy project as a > >> ? ? volunteer. I believe that I can contribute in > >> > >> ? ? (1) Documentation > >> ? ? (2) Suggesting and developing newer functionality > >> ? ? (3) Study similar software as Matlab, Mathematica, Maple etc > >> ? ? trying to look for inspiration for possible improvements in scipy > >> > > > > Thanks for your help. As in most open source projects, the best way to > > contribute is to work on something that you actually need for yourself, > > be it code enhancement, bug fixes, documentation, etc... > > > > Depending on your interests and proficiency in python vs. C vs. Fortran, > > there are several things that could be worked on: the ones which need > > the most work ATM are scipy.special and scipy.interpolate. > > > > Please note also that you should be careful when 'studying' proprietary > > software - reimplementing from scratch a similar functionality is OK, > > re-using the implementation is almost never ok. > > I would add a mode of working that is definitely okay: find a > reference to a journal article on how to compute the thing you're > interested in (possibly by finding the reference in > matlab/r/maple/mathematica/whatever source ordoumentation) and > implementing what is described in the research paper. This is a great > way to fill gaps in scipy with good, well-thought-out algorithms. > > Anne > > > ------------------------------ > > Message: 2 > Date: Mon, 5 Oct 2009 13:54:11 -0700 > From: David Goldsmith > Subject: Re: [SciPy-dev] Volunteer for Scipy Project > To: SciPy Developers List > Message-ID: > <45d1ab480910051354m3cc0f210hdb49b5a281948807 at mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Just curious, Anne: have you anything in particular in mind (i.e., are > there > some small - or gaping - holes in scipy (IYO, of course) which you know > could be filled by a careful implementation of something(s) extant in the > literature)? > > DG > > On Mon, Oct 5, 2009 at 1:40 PM, Anne Archibald >wrote: > > > 2009/10/5 David Cournapeau : > > > Hi Arkapravo, > > > > > > Arkapravo Bhaumik wrote: > > >> > > >> Dear Sir/Ma'am > > >> > > >> I was in e-mail contact with one of your colleagues , Travis ; I > > >> am very interested in contributing to scipy project as a > > >> volunteer. I believe that I can contribute in > > >> > > >> (1) Documentation > > >> (2) Suggesting and developing newer functionality > > >> (3) Study similar software as Matlab, Mathematica, Maple etc > > >> trying to look for inspiration for possible improvements in scipy > > >> > > > > > > Thanks for your help. As in most open source projects, the best way to > > > contribute is to work on something that you actually need for yourself, > > > be it code enhancement, bug fixes, documentation, etc... > > > > > > Depending on your interests and proficiency in python vs. C vs. > Fortran, > > > there are several things that could be worked on: the ones which need > > > the most work ATM are scipy.special and scipy.interpolate. > > > > > > Please note also that you should be careful when 'studying' proprietary > > > software - reimplementing from scratch a similar functionality is OK, > > > re-using the implementation is almost never ok. > > > > I would add a mode of working that is definitely okay: find a > > reference to a journal article on how to compute the thing you're > > interested in (possibly by finding the reference in > > matlab/r/maple/mathematica/whatever source ordoumentation) and > > implementing what is described in the research paper. This is a great > > way to fill gaps in scipy with good, well-thought-out algorithms. > > > > Anne > > _______________________________________________ > > Scipy-dev mailing list > > Scipy-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mail.scipy.org/pipermail/scipy-dev/attachments/20091005/d95b297e/attachment-0001.html > > ------------------------------ > > Message: 3 > Date: Mon, 5 Oct 2009 17:20:53 -0400 > From: Anne Archibald > Subject: Re: [SciPy-dev] Volunteer for Scipy Project > To: SciPy Developers List > Message-ID: > > Content-Type: text/plain; charset=UTF-8 > > 2009/10/5 David Goldsmith : > > Just curious, Anne: have you anything in particular in mind (i.e., are > there > > some small - or gaping - holes in scipy (IYO, of course) which you know > > could be filled by a careful implementation of something(s) extant in the > > literature)? > > Well, not exactly - the examples I had in mind were minor and/or in > the past. I ran into problems with scipy's hyp2f1, for example, so I > went and looked up the best algorithm I could find for it (and I think > I contributed that code). I wanted the Kuiper test as an alternative > to the Kolmogorov-Smirnov test (it's invariant under cyclic > permutations, and is sensitive to different features of the > distribution) so I looked up the test and the special function needed > to interpret its results. (I haven't contributed this to scipy yet, > mostly because I chose an interface that's not particularly compatible > with that for scipy's K-S test.) And on a larger scale, that's what > scipy.spatial's kdtree implementation is. > > For examples where I think a bit of lit review plus implementation > work might help, I'd say that the orthogonal polynomials could use > some work - the generic implementation in scipy.special falls apart > rapidly as you go to higher orders. I always implement my own > Chebyshev polynomials using the cos(n*arccos(x)) expression, for > example, and special implementations for the others might be very > useful. > > Anne > > > ------------------------------ > > Message: 4 > Date: Mon, 5 Oct 2009 18:16:50 -0600 > From: Charles R Harris > Subject: Re: [SciPy-dev] Volunteer for Scipy Project > To: SciPy Developers List > Message-ID: > > Content-Type: text/plain; charset="iso-8859-1" > > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald >wrote: > > > 2009/10/5 David Goldsmith : > > > Just curious, Anne: have you anything in particular in mind (i.e., are > > there > > > some small - or gaping - holes in scipy (IYO, of course) which you know > > > could be filled by a careful implementation of something(s) extant in > the > > > literature)? > > > > Well, not exactly - the examples I had in mind were minor and/or in > > the past. I ran into problems with scipy's hyp2f1, for example, so I > > went and looked up the best algorithm I could find for it (and I think > > I contributed that code). I wanted the Kuiper test as an alternative > > to the Kolmogorov-Smirnov test (it's invariant under cyclic > > permutations, and is sensitive to different features of the > > distribution) so I looked up the test and the special function needed > > to interpret its results. (I haven't contributed this to scipy yet, > > mostly because I chose an interface that's not particularly compatible > > with that for scipy's K-S test.) And on a larger scale, that's what > > scipy.spatial's kdtree implementation is. > > > > For examples where I think a bit of lit review plus implementation > > work might help, I'd say that the orthogonal polynomials could use > > some work - the generic implementation in scipy.special falls apart > > rapidly as you go to higher orders. I always implement my own > > Chebyshev polynomials using the cos(n*arccos(x)) expression, for > > example, and special implementations for the others might be very > > useful. > > > > > At what order does the scipy implementation of the Chebyshev polynomials > fall apart? I looked briefly at that package a long time ago, but never > used > it. I ask so I can check the chebyshev module that is going into numpy. > > Chuck > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mail.scipy.org/pipermail/scipy-dev/attachments/20091005/41c05d5e/attachment-0001.html > > ------------------------------ > > Message: 5 > Date: Mon, 5 Oct 2009 21:29:52 -0400 > From: Anne Archibald > Subject: Re: [SciPy-dev] Volunteer for Scipy Project > To: SciPy Developers List > Message-ID: > > Content-Type: text/plain; charset=UTF-8 > > 2009/10/5 Charles R Harris : > > > > > > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/5 David Goldsmith : > >> > Just curious, Anne: have you anything in particular in mind (i.e., are > >> > there > >> > some small - or gaping - holes in scipy (IYO, of course) which you > know > >> > could be filled by a careful implementation of something(s) extant in > >> > the > >> > literature)? > >> > >> Well, not exactly - the examples I had in mind were minor and/or in > >> the past. I ran into problems with scipy's hyp2f1, for example, so I > >> went and looked up the best algorithm I could find for it (and I think > >> I contributed that code). I wanted the Kuiper test as an alternative > >> to the Kolmogorov-Smirnov test (it's invariant under cyclic > >> permutations, and is sensitive to different features of the > >> distribution) so I looked up the test and the special function needed > >> to interpret its results. (I haven't contributed this to scipy yet, > >> mostly because I chose an interface that's not particularly compatible > >> with that for scipy's K-S test.) And on a larger scale, that's what > >> scipy.spatial's kdtree implementation is. > >> > >> For examples where I think a bit of lit review plus implementation > >> work might help, I'd say that the orthogonal polynomials could use > >> some work - the generic implementation in scipy.special falls apart > >> rapidly as you go to higher orders. I always implement my own > >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for > >> example, and special implementations for the others might be very > >> useful. > >> > > > > At what order does the scipy implementation of the Chebyshev polynomials > > fall apart? I looked briefly at that package a long time ago, but never > used > > it. I ask so I can check the chebyshev module that is going into numpy. > > By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off > by 0.1, and by n=35 they are off by four - not great for values that > should be in the interval [-1,1]. This may seem like an outrageously > high degree for a polynomial, but there's no reason they need be this > bad, and one could quite reasonably want to use an order this high, > say for function approximation. > > I think this inaccuracy is probably inevitable in a scheme that > computes values using a recurrence relation, and something like it > probably occurs for all the orthogonal polynomials that don't have > special-purpose evaluators. > > Anne > > > ------------------------------ > > Message: 6 > Date: Mon, 5 Oct 2009 21:12:49 -0600 > From: Charles R Harris > Subject: Re: [SciPy-dev] Volunteer for Scipy Project > To: SciPy Developers List > Message-ID: > > Content-Type: text/plain; charset="iso-8859-1" > > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald >wrote: > > > 2009/10/5 Charles R Harris : > > > > > > > > > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald < > > peridot.faceted at gmail.com> > > > wrote: > > >> > > >> 2009/10/5 David Goldsmith : > > >> > Just curious, Anne: have you anything in particular in mind (i.e., > are > > >> > there > > >> > some small - or gaping - holes in scipy (IYO, of course) which you > > know > > >> > could be filled by a careful implementation of something(s) extant > in > > >> > the > > >> > literature)? > > >> > > >> Well, not exactly - the examples I had in mind were minor and/or in > > >> the past. I ran into problems with scipy's hyp2f1, for example, so I > > >> went and looked up the best algorithm I could find for it (and I think > > >> I contributed that code). I wanted the Kuiper test as an alternative > > >> to the Kolmogorov-Smirnov test (it's invariant under cyclic > > >> permutations, and is sensitive to different features of the > > >> distribution) so I looked up the test and the special function needed > > >> to interpret its results. (I haven't contributed this to scipy yet, > > >> mostly because I chose an interface that's not particularly compatible > > >> with that for scipy's K-S test.) And on a larger scale, that's what > > >> scipy.spatial's kdtree implementation is. > > >> > > >> For examples where I think a bit of lit review plus implementation > > >> work might help, I'd say that the orthogonal polynomials could use > > >> some work - the generic implementation in scipy.special falls apart > > >> rapidly as you go to higher orders. I always implement my own > > >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for > > >> example, and special implementations for the others might be very > > >> useful. > > >> > > > > > > At what order does the scipy implementation of the Chebyshev > polynomials > > > fall apart? I looked briefly at that package a long time ago, but never > > used > > > it. I ask so I can check the chebyshev module that is going into numpy. > > > > By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off > > by 0.1, and by n=35 they are off by four - not great for values that > > should be in the interval [-1,1]. This may seem like an outrageously > > high degree for a polynomial, but there's no reason they need be this > > bad, and one could quite reasonably want to use an order this high, > > say for function approximation. > > > > > Hmm, I get an maximum error of about 1e-13 for n=100 using my routine, > which > isn't great and can be improved a bit with a few tricks, but is probably > good enough. That's using simple Clenshaw recursion for the evaluation. > There must be something seriously wrong with the scipy version. I routinely > use fits with n > 50 because truncating such a series gives a good > approximation to the minmax fit and it's also nice to see how the > coefficients fall off to estimate the size of the truncation error. > > I think this inaccuracy is probably inevitable in a scheme that > > computes values using a recurrence relation > > > Not so, using the Cheybshev recurrence in either direction should be stable > for |x| <= 1. It's like multiplying with a complex number of modulus 1, > i.e., a unitary matrix. > > > > and something like it probably occurs for all the orthogonal polynomials > > that don't have > > special-purpose evaluators. > > > > Depends on the recursion, the direction of the recursion, and the domain. > > Chuck > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mail.scipy.org/pipermail/scipy-dev/attachments/20091005/f991a2ef/attachment.html > > ------------------------------ > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > End of Scipy-dev Digest, Vol 72, Issue 6 > **************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Tue Oct 6 00:11:59 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Mon, 5 Oct 2009 21:11:59 -0700 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Message-ID: <45d1ab480910052111r7b0d34ep5f846e67b0d00d7c@mail.gmail.com> Beside the obvious one of eventually requiring people to change their code (hopefully only very trivially), would there be any other negative ramifications to steering users (present and would-be) away from the scipy Cheb to the numpy Cheb (and eventually deprecating the former)? Or is there a good reason to maintain the same functionality in two separate namespaces? DG On Mon, Oct 5, 2009 at 8:12 PM, Charles R Harris wrote: > > > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald wrote: > >> 2009/10/5 Charles R Harris : >> > >> > >> > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald < >> peridot.faceted at gmail.com> >> > wrote: >> >> >> >> 2009/10/5 David Goldsmith : >> >> > Just curious, Anne: have you anything in particular in mind (i.e., >> are >> >> > there >> >> > some small - or gaping - holes in scipy (IYO, of course) which you >> know >> >> > could be filled by a careful implementation of something(s) extant in >> >> > the >> >> > literature)? >> >> >> >> Well, not exactly - the examples I had in mind were minor and/or in >> >> the past. I ran into problems with scipy's hyp2f1, for example, so I >> >> went and looked up the best algorithm I could find for it (and I think >> >> I contributed that code). I wanted the Kuiper test as an alternative >> >> to the Kolmogorov-Smirnov test (it's invariant under cyclic >> >> permutations, and is sensitive to different features of the >> >> distribution) so I looked up the test and the special function needed >> >> to interpret its results. (I haven't contributed this to scipy yet, >> >> mostly because I chose an interface that's not particularly compatible >> >> with that for scipy's K-S test.) And on a larger scale, that's what >> >> scipy.spatial's kdtree implementation is. >> >> >> >> For examples where I think a bit of lit review plus implementation >> >> work might help, I'd say that the orthogonal polynomials could use >> >> some work - the generic implementation in scipy.special falls apart >> >> rapidly as you go to higher orders. I always implement my own >> >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for >> >> example, and special implementations for the others might be very >> >> useful. >> >> >> > >> > At what order does the scipy implementation of the Chebyshev polynomials >> > fall apart? I looked briefly at that package a long time ago, but never >> used >> > it. I ask so I can check the chebyshev module that is going into numpy. >> >> By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off >> by 0.1, and by n=35 they are off by four - not great for values that >> should be in the interval [-1,1]. This may seem like an outrageously >> high degree for a polynomial, but there's no reason they need be this >> bad, and one could quite reasonably want to use an order this high, >> say for function approximation. >> >> > Hmm, I get an maximum error of about 1e-13 for n=100 using my routine, > which isn't great and can be improved a bit with a few tricks, but is > probably good enough. That's using simple Clenshaw recursion for the > evaluation. There must be something seriously wrong with the scipy version. > I routinely use fits with n > 50 because truncating such a series gives a > good approximation to the minmax fit and it's also nice to see how the > coefficients fall off to estimate the size of the truncation error. > > I think this inaccuracy is probably inevitable in a scheme that >> computes values using a recurrence relation > > > Not so, using the Cheybshev recurrence in either direction should be stable > for |x| <= 1. It's like multiplying with a complex number of modulus 1, > i.e., a unitary matrix. > > >> and something like it probably occurs for all the orthogonal polynomials >> that don't have >> special-purpose evaluators. >> > > Depends on the recursion, the direction of the recursion, and the domain. > > Chuck > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Tue Oct 6 00:14:36 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 6 Oct 2009 00:14:36 -0400 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Message-ID: 2009/10/5 Charles R Harris : > > > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald > wrote: >> >> 2009/10/5 Charles R Harris : >> > >> > >> > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald >> > >> > wrote: >> >> >> >> 2009/10/5 David Goldsmith : >> >> > Just curious, Anne: have you anything in particular in mind (i.e., >> >> > are >> >> > there >> >> > some small - or gaping - holes in scipy (IYO, of course) which you >> >> > know >> >> > could be filled by a careful implementation of something(s) extant in >> >> > the >> >> > literature)? >> >> >> >> Well, not exactly - the examples I had in mind were minor and/or in >> >> the past. I ran into problems with scipy's hyp2f1, for example, so I >> >> went and looked up the best algorithm I could find for it (and I think >> >> I contributed that code). I wanted the Kuiper test as an alternative >> >> to the Kolmogorov-Smirnov test (it's invariant under cyclic >> >> permutations, and is sensitive to different features of the >> >> distribution) so I looked up the test and the special function needed >> >> to interpret its results. (I haven't contributed this to scipy yet, >> >> mostly because I chose an interface that's not particularly compatible >> >> with that for scipy's K-S test.) And on a larger scale, that's what >> >> scipy.spatial's kdtree implementation is. >> >> >> >> For examples where I think a bit of lit review plus implementation >> >> work might help, I'd say that the orthogonal polynomials could use >> >> some work - the generic implementation in scipy.special falls apart >> >> rapidly as you go to higher orders. I always implement my own >> >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for >> >> example, and special implementations for the others might be very >> >> useful. >> >> >> > >> > At what order does the scipy implementation of the Chebyshev polynomials >> > fall apart? I looked briefly at that package a long time ago, but never >> > used >> > it. I ask so I can check the chebyshev module that is going into numpy. >> >> By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off >> by 0.1, and by n=35 they are off by four - not great for values that >> should be in the interval [-1,1]. This may seem like an outrageously >> high degree for a polynomial, but there's no reason they need be this >> bad, and one could quite reasonably want to use an order this high, >> say for function approximation. >> > > Hmm, I get an maximum error of about 1e-13 for n=100 using my routine, which > isn't great and can be improved a bit with a few tricks, but is probably > good enough.? That's using simple Clenshaw recursion for the evaluation. > There must be something seriously wrong with the scipy version. I routinely > use fits with n > 50 because truncating such a series gives a good > approximation to the minmax fit and it's also nice to see how the > coefficients fall off to estimate the size of the truncation error. Upon closer inspection, it seems that scipy starts from the recurrence relation but then uses it only to extract roots, which it then uses to construct a poly1d instance. (Which presumably then goes on to convert it into the 1, x, x**2, ... basis.) This is numerically disastrous. The reason it does this is because scipy doesn't (didn't) have a general-purpose polynomial class that can keep the polynomial in other representations. What do you think of the idea of a polynomial class that supports several different bases for the set of polynomials? (At least, the power basis and Chebyshev polynomials, but possibly also the other orthogonal polynomials. Translated and scaled bases might come in handy too.) Too general? Not general enough (barycentric representations, representation in terms of roots)? I think it could be done without too much difficulty. Anne From charlesr.harris at gmail.com Tue Oct 6 00:36:10 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 5 Oct 2009 22:36:10 -0600 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Message-ID: On Mon, Oct 5, 2009 at 10:14 PM, Anne Archibald wrote: > 2009/10/5 Charles R Harris : > > > > > > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/5 Charles R Harris : > >> > > >> > > >> > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald > >> > > >> > wrote: > >> >> > >> >> 2009/10/5 David Goldsmith : > >> >> > Just curious, Anne: have you anything in particular in mind (i.e., > >> >> > are > >> >> > there > >> >> > some small - or gaping - holes in scipy (IYO, of course) which you > >> >> > know > >> >> > could be filled by a careful implementation of something(s) extant > in > >> >> > the > >> >> > literature)? > >> >> > >> >> Well, not exactly - the examples I had in mind were minor and/or in > >> >> the past. I ran into problems with scipy's hyp2f1, for example, so I > >> >> went and looked up the best algorithm I could find for it (and I > think > >> >> I contributed that code). I wanted the Kuiper test as an alternative > >> >> to the Kolmogorov-Smirnov test (it's invariant under cyclic > >> >> permutations, and is sensitive to different features of the > >> >> distribution) so I looked up the test and the special function needed > >> >> to interpret its results. (I haven't contributed this to scipy yet, > >> >> mostly because I chose an interface that's not particularly > compatible > >> >> with that for scipy's K-S test.) And on a larger scale, that's what > >> >> scipy.spatial's kdtree implementation is. > >> >> > >> >> For examples where I think a bit of lit review plus implementation > >> >> work might help, I'd say that the orthogonal polynomials could use > >> >> some work - the generic implementation in scipy.special falls apart > >> >> rapidly as you go to higher orders. I always implement my own > >> >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for > >> >> example, and special implementations for the others might be very > >> >> useful. > >> >> > >> > > >> > At what order does the scipy implementation of the Chebyshev > polynomials > >> > fall apart? I looked briefly at that package a long time ago, but > never > >> > used > >> > it. I ask so I can check the chebyshev module that is going into > numpy. > >> > >> By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off > >> by 0.1, and by n=35 they are off by four - not great for values that > >> should be in the interval [-1,1]. This may seem like an outrageously > >> high degree for a polynomial, but there's no reason they need be this > >> bad, and one could quite reasonably want to use an order this high, > >> say for function approximation. > >> > > > > Hmm, I get an maximum error of about 1e-13 for n=100 using my routine, > which > > isn't great and can be improved a bit with a few tricks, but is probably > > good enough. That's using simple Clenshaw recursion for the evaluation. > > There must be something seriously wrong with the scipy version. I > routinely > > use fits with n > 50 because truncating such a series gives a good > > approximation to the minmax fit and it's also nice to see how the > > coefficients fall off to estimate the size of the truncation error. > > Upon closer inspection, it seems that scipy starts from the recurrence > relation but then uses it only to extract roots, which it then uses to > construct a poly1d instance. (Which presumably then goes on to convert > it into the 1, x, x**2, ... basis.) This is numerically disastrous. > > The reason it does this is because scipy doesn't (didn't) have a > general-purpose polynomial class that can keep the polynomial in other > representations. What do you think of the idea of a polynomial class > that supports several different bases for the set of polynomials? (At > least, the power basis and Chebyshev polynomials, but possibly also > the other orthogonal polynomials. Translated and scaled bases might > come in handy too.) The new Chebyshev class has a domain [beg, end] that defines the translation and scaling. The polynomial class needs one also. In fact, I would like to define a Polynomial class to replace poly1d. > Too general? Not general enough (barycentric > representations, representation in terms of roots)? I think it could > be done without too much difficulty. > The difficulty is in between. I've been thinking of a base series class that would handle the domains and access to the coefficients, as well as addition/subtraction, scalar multiplication/division, and pickling. Derived classes could then overload the division/multiplication/integration/differentiation. Roots would come directly from the recursion relation. I don't know if the stability of the Clenshaw recursion (essentially synthetic division with the value as the remainder) is guaranteed for the polynomials/domains of interest, but we could make a list and look into it. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Oct 6 00:47:07 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 5 Oct 2009 22:47:07 -0600 Subject: [SciPy-dev] Volunteer for Scipy Project In-Reply-To: References: <4AC99F7F.3060000@ar.media.kyoto-u.ac.jp> <45d1ab480910051354m3cc0f210hdb49b5a281948807@mail.gmail.com> Message-ID: On Mon, Oct 5, 2009 at 10:14 PM, Anne Archibald wrote: > 2009/10/5 Charles R Harris : > > > > > > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/5 Charles R Harris : > >> > > >> > > >> > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald > >> > > >> > wrote: > >> >> > >> >> 2009/10/5 David Goldsmith : > >> >> > Just curious, Anne: have you anything in particular in mind (i.e., > >> >> > are > >> >> > there > >> >> > some small - or gaping - holes in scipy (IYO, of course) which you > >> >> > know > >> >> > could be filled by a careful implementation of something(s) extant > in > >> >> > the > >> >> > literature)? > >> >> > >> >> Well, not exactly - the examples I had in mind were minor and/or in > >> >> the past. I ran into problems with scipy's hyp2f1, for example, so I > >> >> went and looked up the best algorithm I could find for it (and I > think > >> >> I contributed that code). I wanted the Kuiper test as an alternative > >> >> to the Kolmogorov-Smirnov test (it's invariant under cyclic > >> >> permutations, and is sensitive to different features of the > >> >> distribution) so I looked up the test and the special function needed > >> >> to interpret its results. (I haven't contributed this to scipy yet, > >> >> mostly because I chose an interface that's not particularly > compatible > >> >> with that for scipy's K-S test.) And on a larger scale, that's what > >> >> scipy.spatial's kdtree implementation is. > >> >> > >> >> For examples where I think a bit of lit review plus implementation > >> >> work might help, I'd say that the orthogonal polynomials could use > >> >> some work - the generic implementation in scipy.special falls apart > >> >> rapidly as you go to higher orders. I always implement my own > >> >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for > >> >> example, and special implementations for the others might be very > >> >> useful. > >> >> > >> > > >> > At what order does the scipy implementation of the Chebyshev > polynomials > >> > fall apart? I looked briefly at that package a long time ago, but > never > >> > used > >> > it. I ask so I can check the chebyshev module that is going into > numpy. > >> > >> By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off > >> by 0.1, and by n=35 they are off by four - not great for values that > >> should be in the interval [-1,1]. This may seem like an outrageously > >> high degree for a polynomial, but there's no reason they need be this > >> bad, and one could quite reasonably want to use an order this high, > >> say for function approximation. > >> > > > > Hmm, I get an maximum error of about 1e-13 for n=100 using my routine, > which > > isn't great and can be improved a bit with a few tricks, but is probably > > good enough. That's using simple Clenshaw recursion for the evaluation. > > There must be something seriously wrong with the scipy version. I > routinely > > use fits with n > 50 because truncating such a series gives a good > > approximation to the minmax fit and it's also nice to see how the > > coefficients fall off to estimate the size of the truncation error. > > Upon closer inspection, it seems that scipy starts from the recurrence > relation but then uses it only to extract roots, which it then uses to > construct a poly1d instance. (Which presumably then goes on to convert > it into the 1, x, x**2, ... basis.) This is numerically disastrous. > > The reason it does this is because scipy doesn't (didn't) have a > general-purpose polynomial class that can keep the polynomial in other > representations. What do you think of the idea of a polynomial class > that supports several different bases for the set of polynomials? (At > least, the power basis and Chebyshev polynomials, but possibly also > the other orthogonal polynomials. Translated and scaled bases might > come in handy too.) Too general? Not general enough (barycentric > representations, representation in terms of roots)? Barycentric representation using the type II Chebyshev points would be the way to go for *all* the functions without singularities at the ends, except I couldn't figure out how to get the remainder during division, and the remainder is needed for converting to the coefficients of the various types of series -- it's just like converting a decimal representation to binary by repeated division by two and keeping the remainders. That doesn't mean it can't be done, however... Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Tue Oct 6 11:02:17 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 6 Oct 2009 11:02:17 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) Message-ID: 2009/10/6 Charles R Harris : > > > On Mon, Oct 5, 2009 at 10:14 PM, Anne Archibald > wrote: >> >> 2009/10/5 Charles R Harris : >> > >> > >> > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald >> > >> > wrote: >> >> >> >> 2009/10/5 Charles R Harris : >> >> > >> >> > >> >> > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald >> >> > >> >> > wrote: >> >> >> For examples where I think a bit of lit review plus implementation >> >> >> work might help, I'd say that the orthogonal polynomials could use >> >> >> some work - the generic implementation in scipy.special falls apart >> >> >> rapidly as you go to higher orders. I always implement my own >> >> >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for >> >> >> example, and special implementations for the others might be very >> >> >> useful. >> >> >> >> >> > >> >> > At what order does the scipy implementation of the Chebyshev >> >> > polynomials >> >> > fall apart? I looked briefly at that package a long time ago, but >> >> > never >> >> > used >> >> > it. I ask so I can check the chebyshev module that is going into >> >> > numpy. >> >> >> >> By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are off >> >> by 0.1, and by n=35 they are off by four - not great for values that >> >> should be in the interval [-1,1]. This may seem like an outrageously >> >> high degree for a polynomial, but there's no reason they need be this >> >> bad, and one could quite reasonably want to use an order this high, >> >> say for function approximation. >> >> >> > >> > Hmm, I get an maximum error of about 1e-13 for n=100 using my routine, >> > which >> > isn't great and can be improved a bit with a few tricks, but is probably >> > good enough.? That's using simple Clenshaw recursion for the evaluation. >> > There must be something seriously wrong with the scipy version. I >> > routinely >> > use fits with n > 50 because truncating such a series gives a good >> > approximation to the minmax fit and it's also nice to see how the >> > coefficients fall off to estimate the size of the truncation error. >> >> Upon closer inspection, it seems that scipy starts from the recurrence >> relation but then uses it only to extract roots, which it then uses to >> construct a poly1d instance. (Which presumably then goes on to convert >> it into the 1, x, x**2, ... basis.) This is numerically disastrous. >> >> The reason it does this is because scipy doesn't (didn't) have a >> general-purpose polynomial class that can keep the polynomial in other >> representations. What do you think of the idea of a polynomial class >> that supports several different bases for the set of polynomials? (At >> least, the power basis and Chebyshev polynomials, but possibly also >> the other orthogonal polynomials. Translated and scaled bases might >> come in handy too.) Too general? Not general enough (barycentric >> representations, representation in terms of roots)? > > Barycentric representation using the type II Chebyshev points would be the > way to go for *all* the functions without singularities at the ends, except > I couldn't figure out how to get the remainder during division, and the > remainder is needed for converting to the coefficients of the various types > of series -- it's just like converting a decimal representation to binary by > repeated division by two and keeping the remainders. That doesn't mean it > can't be done, however... We even already have compiled code for evaluation. But there are several polynomial operations I can't see an easy way to handle using this. It seems to me that with a polynomial class one should be able to easily: * evaluate * add/subtract * multiply by another polynomial * divide with remainder * extract derivatives and antiderivatives, both at a point and as polynomials * express in another polynomial basis * measure degree (Have I missed any?) With barycentric interpolation, it's not clear how to divide or take antiderivatives (and hence converting bases becomes awkward). On the other hand, multiplication is O(n) rather than O(n^2). And I think antidifferentiation could be worked out with some cleverness. Nevertheless I think one would want to be able to work with orthogonal polynomial bases, for example when dealing with an infinite (or unspecified) interval. Plus one needs to record all the various extra information about each family of orthogonal polynomials (roots, weights for integration, etc.) Anne From charlesr.harris at gmail.com Tue Oct 6 11:42:48 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 09:42:48 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 9:02 AM, Anne Archibald wrote: > 2009/10/6 Charles R Harris : > > > > > > On Mon, Oct 5, 2009 at 10:14 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/5 Charles R Harris : > >> > > >> > > >> > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald > >> > > >> > wrote: > >> >> > >> >> 2009/10/5 Charles R Harris : > >> >> > > >> >> > > >> >> > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald > >> >> > > >> >> > wrote: > >> >> >> For examples where I think a bit of lit review plus implementation > >> >> >> work might help, I'd say that the orthogonal polynomials could use > >> >> >> some work - the generic implementation in scipy.special falls > apart > >> >> >> rapidly as you go to higher orders. I always implement my own > >> >> >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for > >> >> >> example, and special implementations for the others might be very > >> >> >> useful. > >> >> >> > >> >> > > >> >> > At what order does the scipy implementation of the Chebyshev > >> >> > polynomials > >> >> > fall apart? I looked briefly at that package a long time ago, but > >> >> > never > >> >> > used > >> >> > it. I ask so I can check the chebyshev module that is going into > >> >> > numpy. > >> >> > >> >> By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are > off > >> >> by 0.1, and by n=35 they are off by four - not great for values that > >> >> should be in the interval [-1,1]. This may seem like an outrageously > >> >> high degree for a polynomial, but there's no reason they need be this > >> >> bad, and one could quite reasonably want to use an order this high, > >> >> say for function approximation. > >> >> > >> > > >> > Hmm, I get an maximum error of about 1e-13 for n=100 using my routine, > >> > which > >> > isn't great and can be improved a bit with a few tricks, but is > probably > >> > good enough. That's using simple Clenshaw recursion for the > evaluation. > >> > There must be something seriously wrong with the scipy version. I > >> > routinely > >> > use fits with n > 50 because truncating such a series gives a good > >> > approximation to the minmax fit and it's also nice to see how the > >> > coefficients fall off to estimate the size of the truncation error. > >> > >> Upon closer inspection, it seems that scipy starts from the recurrence > >> relation but then uses it only to extract roots, which it then uses to > >> construct a poly1d instance. (Which presumably then goes on to convert > >> it into the 1, x, x**2, ... basis.) This is numerically disastrous. > >> > >> The reason it does this is because scipy doesn't (didn't) have a > >> general-purpose polynomial class that can keep the polynomial in other > >> representations. What do you think of the idea of a polynomial class > >> that supports several different bases for the set of polynomials? (At > >> least, the power basis and Chebyshev polynomials, but possibly also > >> the other orthogonal polynomials. Translated and scaled bases might > >> come in handy too.) Too general? Not general enough (barycentric > >> representations, representation in terms of roots)? > > > > Barycentric representation using the type II Chebyshev points would be > the > > way to go for *all* the functions without singularities at the ends, > except > > I couldn't figure out how to get the remainder during division, and the > > remainder is needed for converting to the coefficients of the various > types > > of series -- it's just like converting a decimal representation to binary > by > > repeated division by two and keeping the remainders. That doesn't mean it > > can't be done, however... > > We even already have compiled code for evaluation. But there are > several polynomial operations I can't see an easy way to handle using > this. It seems to me that with a polynomial class one should be able > to easily: > > > * evaluate > check > * add/subtract > check > * multiply by another polynomial > of the same family, check * divide with remainder > check * extract derivatives and antiderivatives, both at a point and as > polynomials > I'd split this. into derivatives/antiderivatives of the series, then use evaluation at a point (or array of points. > * express in another polynomial basis > see below, but it may be difficult to guarantee stability. * measure degree > len() returns the length of the coefficient array. We could throw a deg method in there. > (Have I missed any?) > > With barycentric interpolation, it's not clear how to divide or take > antiderivatives (and hence converting bases becomes awkward). Yeah, the barycentric form would be a natural for rational functions. Polynomials are more like integers. Hmm, it strikes me that the terms y_j/(x - x_j) in the barycentric form are just remainders. Maybe something *can* be done here... > On the > other hand, multiplication is O(n) rather than O(n^2). And I think > antidifferentiation could be worked out with some cleverness. > > The conversion between polynomial types can also be done by function composition. For instance, conversion of a polynomial to a Chebyshev series would be mypoly.val(mycheb), where mycheb is a Chebyshev object representing 'x'. It goes the other way also. This wouldn't work well for the Barycentric form because there are still divisions. Or so it seems at first thought... > Nevertheless I think one would want to be able to work with orthogonal > polynomial bases, for example when dealing with an infinite (or > unspecified) interval. > Plus one needs to record all the various extra > information about each family of orthogonal polynomials (roots, > weights for integration, etc.) > > Ah, now that becomes more specialized. The weights/zeros for integration should probably come from the Gaussian integration package. I feel that the packages in numpy should cover the common uses and scipy should have the parts for which specialized routines are available. I'd still like to settle the coefficient access order for the Chebyshev class. As is, it is like the poly1d class. The constructor and coef attribute contain the coefficients in high to low order, while indexing like mypoly[i] goes the other way. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From Emmanuel.Lambert at intec.ugent.be Tue Oct 6 11:48:27 2009 From: Emmanuel.Lambert at intec.ugent.be (Emmanuel Lambert) Date: Tue, 06 Oct 2009 17:48:27 +0200 Subject: [SciPy-dev] [SciPy-Dec] scipy.weave crashes for pure inline In-Reply-To: <1253686418.3851.125.camel@emmanuel-ubuntu> References: <25530916.post@talk.nabble.com> <1253686418.3851.125.camel@emmanuel-ubuntu> Message-ID: <1254844107.4560.3768.camel@emmanuel-ubuntu> For the record of the archives/mailinglist, I just want to mention that there is a patch avaialble for the problem described below. It can be found at : http://projects.scipy.org/scipy/ticket/855 > > > On Tue, 2009-09-22 at 14:47 -0700, Thomas Robitaille wrote: > > Hi, > > > > I'm using a recent svn revision of scipy (5925). After installing it I went > > to scipy/weave/examples and ran 'python array3d.py'. I get the following > > error message (below). Can other people reproduce this problem? If not, > > maybe it's some local installation issue. > > > > Thanks, > > > > Thomas > > > > --- > > > > numpy: > > [[[ 0 1 2 3] > > [ 4 5 6 7] > > [ 8 9 10 11]] > > > > [[12 13 14 15] > > [16 17 18 19] > > [20 21 22 23]]] > > Pure Inline: > > Traceback (most recent call last): > > File "array3d.py", line 105, in > > main() > > File "array3d.py", line 98, in main > > pure_inline(arr) > > File "array3d.py", line 57, in pure_inline > > weave.inline(code, ['arr']) > > File > > "/Users/tom/Library/Python/2.6/site-packages/scipy/weave/inline_tools.py", > > line 324, in inline > > results = attempt_function_call(code,local_dict,global_dict) > > File > > "/Users/tom/Library/Python/2.6/site-packages/scipy/weave/inline_tools.py", > > line 392, in attempt_function_call > > function_list = function_catalog.get_functions(code,module_dir) > > File "/Users/tom/Library/Python/2.6/site-packages/scipy/weave/catalog.py", > > line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/Users/tom/Library/Python/2.6/site-packages/scipy/weave/catalog.py", > > line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File > > "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/shelve.py", > > line 110, in __contains__ > > return key in self.dict > > File > > "/Users/tom/Library/Python/2.6/site-packages/scipy/io/dumbdbm_patched.py", > > line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > ------------------------------------------------------------------- > > On Mon, 2009-09-21 at 16:46 +0200, Emmanuel Lambert wrote: > > Hi, > > > > I compiled SciPy and Numpy on a machine with Scientific Linux. > > > > We detected a problem with Weave and after > > investigation, it turns out that some of the unit tests delivered > > with Scipy-Weave also fail ! Below is a list of tests that fail in for > > example the "test_c_spec" file. They all raise a KeyError. > > > > This is with SciPy 0.7.1 on Python 2.6. I also downloaded the latest > > Weave code again from the SVN repository, but the problem is not > > resolved. > > > > Any idea on how to tackle this problem? There are no posts that help > > me further. I don't have this problem with the same standard scipy > > package that was is available for Ubuntu 9.04 (apparently the weave > > version number is the same). > > > > It looks like the compilation works fine, see sample stdout also > > below. > > > > What could cause this? > > > > thanks for any help. > > Emmanuel > > > > ******************* SAMPLE OF STDOUT ****************** > > > > -------------------- >> begin captured stdout << --------------------- > > > > running build_ext > > running build_src > > building extension "sc_d133102ab45193e072f8dbb5a1f6848513" sources > > customize UnixCCompiler > > customize UnixCCompiler using build_ext > > customize UnixCCompiler > > customize UnixCCompiler using build_ext > > building 'sc_d133102ab45193e072f8dbb5a1f6848513' extension > > compiling C++ sources > > C compiler: g++ -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 - > > Wall -fPIC > > > > compile options: '-I/user/home/gent/vsc401/vsc40157/scipy-runtime/ > > scipy/weave -I/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/ > > weave/scxx -I/user/home/gent/vsc401/vsc40157/numpy-runtime/numpy/core/ > > include -I/apps/gent/gengar/harpertown/software/Python/2.6.2- > > gimkl-0.5.0/include/python2.6 -c' > > g++: /user/home/gent/vsc401/vsc40157/.python26_compiled/ > > sc_d133102ab45193e072f8dbb5a1f6848513.cpp > > g++ -pthread -shared /tmp/vsc40157/python26_intermediate/ > > compiler_c1b5f1b73f1ce7d0c836cdad4c7c5ded/user/home/gent/vsc401/ > > vsc40157/.python26_compiled/sc_d133102ab45193e072f8dbb5a1f6848513.o / > > tmp/vsc40157/python26_intermediate/ > > compiler_c1b5f1b73f1ce7d0c836cdad4c7c5ded/user/home/gent/vsc401/ > > vsc40157/scipy-runtime/scipy/weave/scxx/weave_imp.o -o /user/home/gent/ > > vsc401/vsc40157/.python26_compiled/ > > sc_d133102ab45193e072f8dbb5a1f6848513.so > > running scons > > > > --------------------- >> end captured stdout << ---------------------- > > > > > > ********************** TESTS THAT FAIL *********************** > > > > -bash-3.2$ python ./test_c_spec.py > > E..........EE.................EEEE......E..........EE.................EEEE.............. > > ====================================================================== > > ERROR: test_call_function (test_c_spec.CallableConverter) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 296, in test_call_function > > compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_file_to_py (test_c_spec.FileConverter) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 262, in test_file_to_py > > force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_py_to_file (test_c_spec.FileConverter) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 246, in test_py_to_file > > inline_tools.inline(code,['file'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_convert_to_dict (test_c_spec.SequenceConverter) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 305, in test_convert_to_dict > > inline_tools.inline("",['d'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_convert_to_list (test_c_spec.SequenceConverter) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 309, in test_convert_to_list > > inline_tools.inline("",['l'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_convert_to_string (test_c_spec.SequenceConverter) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 313, in test_convert_to_string > > inline_tools.inline("",['s'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_convert_to_tuple (test_c_spec.SequenceConverter) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 317, in test_convert_to_tuple > > inline_tools.inline("",['t'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_call_function (test_c_spec.TestCallableConverterUnix) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 296, in test_call_function > > compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_file_to_py (test_c_spec.TestFileConverterUnix) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 262, in test_file_to_py > > force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_py_to_file (test_c_spec.TestFileConverterUnix) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 246, in test_py_to_file > > inline_tools.inline(code,['file'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_convert_to_dict (test_c_spec.TestSequenceConverterUnix) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 305, in test_convert_to_dict > > inline_tools.inline("",['d'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_convert_to_list (test_c_spec.TestSequenceConverterUnix) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 309, in test_convert_to_list > > inline_tools.inline("",['l'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_convert_to_string (test_c_spec.TestSequenceConverterUnix) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 313, in test_convert_to_string > > inline_tools.inline("",['s'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ====================================================================== > > ERROR: test_convert_to_tuple (test_c_spec.TestSequenceConverterUnix) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > tests/test_c_spec.py", line 317, in test_convert_to_tuple > > inline_tools.inline("",['t'],compiler=self.compiler,force=1) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > inline_tools.py", line 301, in inline > > function_catalog.add_function(code,func,module_dir) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 648, in add_function > > self.cache[code] = self.get_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 615, in get_functions > > function_list = self.get_cataloged_functions(code) > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/weave/ > > catalog.py", line 529, in get_cataloged_functions > > if cat is not None and code in cat: > > File "/apps/gent/gengar/harpertown/software/Python/2.6.2-gimkl-0.5.0/ > > lib/python2.6/shelve.py", line 110, in __contains__ > > return key in self.dict > > File "/user/home/gent/vsc401/vsc40157/scipy-runtime/scipy/io/ > > dumbdbm_patched.py", line 73, in __getitem__ > > pos, siz = self._index[key] # may raise KeyError > > KeyError: 0 > > > > ---------------------------------------------------------------------- > > Ran 88 tests in 32.581s > > > > FAILED (errors=14) From peridot.faceted at gmail.com Tue Oct 6 12:03:20 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 6 Oct 2009 12:03:20 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/6 Charles R Harris : > > > On Tue, Oct 6, 2009 at 9:02 AM, Anne Archibald > wrote: >> >> 2009/10/6 Charles R Harris : >> > >> > >> > On Mon, Oct 5, 2009 at 10:14 PM, Anne Archibald >> > >> > wrote: >> >> >> >> 2009/10/5 Charles R Harris : >> >> > >> >> > >> >> > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald >> >> > >> >> > wrote: >> >> >> >> >> >> 2009/10/5 Charles R Harris : >> >> >> > >> >> >> > >> >> >> > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald >> >> >> > >> >> >> > wrote: >> >> >> >> For examples where I think a bit of lit review plus >> >> >> >> implementation >> >> >> >> work might help, I'd say that the orthogonal polynomials could >> >> >> >> use >> >> >> >> some work - the generic implementation in scipy.special falls >> >> >> >> apart >> >> >> >> rapidly as you go to higher orders. I always implement my own >> >> >> >> Chebyshev polynomials using the cos(n*arccos(x)) expression, for >> >> >> >> example, and special implementations for the others might be very >> >> >> >> useful. >> >> >> >> >> >> >> > >> >> >> > At what order does the scipy implementation of the Chebyshev >> >> >> > polynomials >> >> >> > fall apart? I looked briefly at that package a long time ago, but >> >> >> > never >> >> >> > used >> >> >> > it. I ask so I can check the chebyshev module that is going into >> >> >> > numpy. >> >> >> >> >> >> By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are >> >> >> off >> >> >> by 0.1, and by n=35 they are off by four - not great for values that >> >> >> should be in the interval [-1,1]. This may seem like an outrageously >> >> >> high degree for a polynomial, but there's no reason they need be >> >> >> this >> >> >> bad, and one could quite reasonably want to use an order this high, >> >> >> say for function approximation. >> >> >> >> >> > >> >> > Hmm, I get an maximum error of about 1e-13 for n=100 using my >> >> > routine, >> >> > which >> >> > isn't great and can be improved a bit with a few tricks, but is >> >> > probably >> >> > good enough. That's using simple Clenshaw recursion for the >> >> > evaluation. >> >> > There must be something seriously wrong with the scipy version. I >> >> > routinely >> >> > use fits with n > 50 because truncating such a series gives a good >> >> > approximation to the minmax fit and it's also nice to see how the >> >> > coefficients fall off to estimate the size of the truncation error. >> >> >> >> Upon closer inspection, it seems that scipy starts from the recurrence >> >> relation but then uses it only to extract roots, which it then uses to >> >> construct a poly1d instance. (Which presumably then goes on to convert >> >> it into the 1, x, x**2, ... basis.) This is numerically disastrous. >> >> >> >> The reason it does this is because scipy doesn't (didn't) have a >> >> general-purpose polynomial class that can keep the polynomial in other >> >> representations. What do you think of the idea of a polynomial class >> >> that supports several different bases for the set of polynomials? (At >> >> least, the power basis and Chebyshev polynomials, but possibly also >> >> the other orthogonal polynomials. Translated and scaled bases might >> >> come in handy too.) Too general? Not general enough (barycentric >> >> representations, representation in terms of roots)? >> > >> > Barycentric representation using the type II Chebyshev points would be >> > the >> > way to go for *all* the functions without singularities at the ends, >> > except >> > I couldn't figure out how to get the remainder during division, and the >> > remainder is needed for converting to the coefficients of the various >> > types >> > of series -- it's just like converting a decimal representation to >> > binary by >> > repeated division by two and keeping the remainders. That doesn't mean >> > it >> > can't be done, however... >> >> We even already have compiled code for evaluation. But there are >> several polynomial operations I can't see an easy way to handle using >> this. It seems to me that with a polynomial class one should be able >> to easily: >> > >> >> * evaluate > > check > >> >> * add/subtract > > > check > >> >> * multiply by another polynomial > > of the same family, check > >> * divide with remainder > > check > >> * extract derivatives and antiderivatives, both at a point and as >> polynomials > > I'd split this. into derivatives/antiderivatives of the series, then use > evaluation at a point (or array of points. I put them together because I thought that on the one hand one would want convenience functions, and on the other hand in some representations it might be more efficient to evaluate the derivative/antiderivative at a point than it is ton construct the polynomial representation. Though come to think of it, that's probably only true if the derivative/antiderivative evaluation code is written in C. And in any case I doubt it will be a big win. >> >> * express in another polynomial basis > > > see below, but it may be difficult to guarantee stability. Sure, some polynomial bases are going to give trouble. But I think that it shouldn't be too hard to convert without introducing much additional instability beyond what the representations themselves imply. >> * measure degree > > len() returns the length of the coefficient array. We could throw a deg > method in there. Sure. I was imagining an implementation (for example) that uses barycentric interpolation at a power-of-two number of points, so that raising the number of interpolating points is a rare operation and you can always reuse the values you have. But then getting the degree becomes a numerically-unstable operation, which isn't very nice. >> >> (Have I missed any?) >> >> With barycentric interpolation, it's not clear how to divide or take >> antiderivatives (and hence converting bases becomes awkward). > > Yeah, the barycentric form would be a natural for rational functions. > Polynomials are more like integers. Hmm, it strikes me that the terms y_j/(x > - x_j) in the barycentric form are just remainders. Maybe something *can* be > done here... >> >> On the >> other hand, multiplication is O(n) rather than O(n^2). And I think >> antidifferentiation could be worked out with some cleverness. >> > > The conversion between polynomial types can also be done by function > composition. For instance, conversion of a polynomial to a Chebyshev series > would be mypoly.val(mycheb), where mycheb is a Chebyshev object > representing 'x'. It goes the other way also. This wouldn't work well for > the Barycentric form because there are still divisions. Or so it seems at > first thought... I don't know that it's such a common operation that we need special syntax for it. But if we had classes for polynomial representations (ChebyshevBasis, PowerBasis, Barycentric) it would indeed be natural to use call syntax on these classes to transform polynomials. >> Nevertheless I think one would want to be able to work with orthogonal >> polynomial bases, for example when dealing with an infinite (or >> unspecified) interval. >> >> Plus one needs to record all the various extra >> information about each family of orthogonal polynomials (roots, >> weights for integration, etc.) >> > > Ah, now that becomes more specialized. The weights/zeros for integration > should probably come from the Gaussian integration package. I feel that the > packages in numpy should cover the common uses and scipy should have the > parts for which specialized routines are available. Well, it's reasonable, if one has Chebyshev polynomials as basis functions, to want to know where their zeros and maxima/minima are. It's also not a difficult thing to do (i.e. no special functions needed), since they can be expressed in terms of cosines. Programmatically it's a little awkward, since they are associated to Chebyshev polynomials in general, not to any particular Chebyshev polynomial. > I'd still like to settle the coefficient access order for the Chebyshev > class. As is, it is like the poly1d class. The constructor and coef > attribute contain the coefficients in high to low order, while indexing like > mypoly[i] goes the other way. Personally I think that the high-to-low order in poly1d is a mistake, but at this point I think it may be one we're stuck with. But since chebyshev polynomials are all new, they do offer the chance to start fresh. Low-to-high order also has the advantage that you could implement access to higher coefficients returning zero, if you wanted, so that they could be treated something like their mathematical notion of series that happen to terminate. That said, I haven't used poly1d much at all, so I'm not necessarily the best person to ask. Anne From charlesr.harris at gmail.com Tue Oct 6 12:24:08 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 10:24:08 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 10:03 AM, Anne Archibald wrote: > 2009/10/6 Charles R Harris : > > > > > > On Tue, Oct 6, 2009 at 9:02 AM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/6 Charles R Harris : > >> > > >> > > >> > On Mon, Oct 5, 2009 at 10:14 PM, Anne Archibald > >> > > >> > wrote: > >> >> > >> >> 2009/10/5 Charles R Harris : > >> >> > > >> >> > > >> >> > On Mon, Oct 5, 2009 at 7:29 PM, Anne Archibald > >> >> > > >> >> > wrote: > >> >> >> > >> >> >> 2009/10/5 Charles R Harris : > >> >> >> > > >> >> >> > > >> >> >> > On Mon, Oct 5, 2009 at 3:20 PM, Anne Archibald > >> >> >> > > >> >> >> > wrote: > >> >> >> >> For examples where I think a bit of lit review plus > >> >> >> >> implementation > >> >> >> >> work might help, I'd say that the orthogonal polynomials could > >> >> >> >> use > >> >> >> >> some work - the generic implementation in scipy.special falls > >> >> >> >> apart > >> >> >> >> rapidly as you go to higher orders. I always implement my own > >> >> >> >> Chebyshev polynomials using the cos(n*arccos(x)) expression, > for > >> >> >> >> example, and special implementations for the others might be > very > >> >> >> >> useful. > >> >> >> >> > >> >> >> > > >> >> >> > At what order does the scipy implementation of the Chebyshev > >> >> >> > polynomials > >> >> >> > fall apart? I looked briefly at that package a long time ago, > but > >> >> >> > never > >> >> >> > used > >> >> >> > it. I ask so I can check the chebyshev module that is going into > >> >> >> > numpy. > >> >> >> > >> >> >> By n=30 they are off by as much as 0.0018 on [-1,1]; n=31 they are > >> >> >> off > >> >> >> by 0.1, and by n=35 they are off by four - not great for values > that > >> >> >> should be in the interval [-1,1]. This may seem like an > outrageously > >> >> >> high degree for a polynomial, but there's no reason they need be > >> >> >> this > >> >> >> bad, and one could quite reasonably want to use an order this > high, > >> >> >> say for function approximation. > >> >> >> > >> >> > > >> >> > Hmm, I get an maximum error of about 1e-13 for n=100 using my > >> >> > routine, > >> >> > which > >> >> > isn't great and can be improved a bit with a few tricks, but is > >> >> > probably > >> >> > good enough. That's using simple Clenshaw recursion for the > >> >> > evaluation. > >> >> > There must be something seriously wrong with the scipy version. I > >> >> > routinely > >> >> > use fits with n > 50 because truncating such a series gives a good > >> >> > approximation to the minmax fit and it's also nice to see how the > >> >> > coefficients fall off to estimate the size of the truncation error. > >> >> > >> >> Upon closer inspection, it seems that scipy starts from the > recurrence > >> >> relation but then uses it only to extract roots, which it then uses > to > >> >> construct a poly1d instance. (Which presumably then goes on to > convert > >> >> it into the 1, x, x**2, ... basis.) This is numerically disastrous. > >> >> > >> >> The reason it does this is because scipy doesn't (didn't) have a > >> >> general-purpose polynomial class that can keep the polynomial in > other > >> >> representations. What do you think of the idea of a polynomial class > >> >> that supports several different bases for the set of polynomials? (At > >> >> least, the power basis and Chebyshev polynomials, but possibly also > >> >> the other orthogonal polynomials. Translated and scaled bases might > >> >> come in handy too.) Too general? Not general enough (barycentric > >> >> representations, representation in terms of roots)? > >> > > >> > Barycentric representation using the type II Chebyshev points would be > >> > the > >> > way to go for *all* the functions without singularities at the ends, > >> > except > >> > I couldn't figure out how to get the remainder during division, and > the > >> > remainder is needed for converting to the coefficients of the various > >> > types > >> > of series -- it's just like converting a decimal representation to > >> > binary by > >> > repeated division by two and keeping the remainders. That doesn't mean > >> > it > >> > can't be done, however... > >> > >> We even already have compiled code for evaluation. But there are > >> several polynomial operations I can't see an easy way to handle using > >> this. It seems to me that with a polynomial class one should be able > >> to easily: > >> > > > >> > >> * evaluate > > > > check > > > >> > >> * add/subtract > > > > > > check > > > >> > >> * multiply by another polynomial > > > > of the same family, check > > > >> * divide with remainder > > > > check > > > >> * extract derivatives and antiderivatives, both at a point and as > >> polynomials > > > > I'd split this. into derivatives/antiderivatives of the series, then use > > evaluation at a point (or array of points. > > I put them together because I thought that on the one hand one would > want convenience functions, and on the other hand in some > representations it might be more efficient to evaluate the > derivative/antiderivative at a point than it is ton construct the > polynomial representation. Though come to think of it, that's probably > only true if the derivative/antiderivative evaluation code is written > in C. And in any case I doubt it will be a big win. > > >> > >> * express in another polynomial basis > > > > > > see below, but it may be difficult to guarantee stability. > > Sure, some polynomial bases are going to give trouble. But I think > that it shouldn't be too hard to convert without introducing much > additional instability beyond what the representations themselves > imply. > > >> * measure degree > > > > len() returns the length of the coefficient array. We could throw a deg > > method in there. > > Sure. I was imagining an implementation (for example) that uses > barycentric interpolation at a power-of-two number of points, so that > raising the number of interpolating points is a rare operation and you > can always reuse the values you have. But then getting the degree > becomes a numerically-unstable operation, which isn't very nice. > > >> > >> (Have I missed any?) > >> > >> With barycentric interpolation, it's not clear how to divide or take > >> antiderivatives (and hence converting bases becomes awkward). > > > > Yeah, the barycentric form would be a natural for rational functions. > > Polynomials are more like integers. Hmm, it strikes me that the terms > y_j/(x > > - x_j) in the barycentric form are just remainders. Maybe something *can* > be > > done here... > > >> > >> On the > >> other hand, multiplication is O(n) rather than O(n^2). And I think > >> antidifferentiation could be worked out with some cleverness. > >> > > > > The conversion between polynomial types can also be done by function > > composition. For instance, conversion of a polynomial to a Chebyshev > series > > would be mypoly.val(mycheb), where mycheb is a Chebyshev object > > representing 'x'. It goes the other way also. This wouldn't work well > for > > the Barycentric form because there are still divisions. Or so it seems at > > first thought... > > I don't know that it's such a common operation that we need special > syntax for it. But if we had classes for polynomial representations > (ChebyshevBasis, PowerBasis, Barycentric) it would indeed be natural > to use call syntax on these classes to transform polynomials. > > It's not a special syntax, it's just using a class instead of a number as the argument, the same (python) code works for both. Well, it doesn't work for the poly1d class because poly1d exports an array interface and behaves badly. But it works for a properly constructed class. > >> Nevertheless I think one would want to be able to work with orthogonal > >> polynomial bases, for example when dealing with an infinite (or > >> unspecified) interval. > >> > >> Plus one needs to record all the various extra > >> information about each family of orthogonal polynomials (roots, > >> weights for integration, etc.) > >> > > > > Ah, now that becomes more specialized. The weights/zeros for integration > > should probably come from the Gaussian integration package. I feel that > the > > packages in numpy should cover the common uses and scipy should have the > > parts for which specialized routines are available. > > Well, it's reasonable, if one has Chebyshev polynomials as basis > functions, to want to know where their zeros and maxima/minima are. > It's also not a difficult thing to do (i.e. no special functions > needed), since they can be expressed in terms of cosines. > You can always call the roots method, which works for Chebyshev series as well as for isolated terms. It doesn't provide the weights, though. > Programmatically it's a little awkward, since they are associated to > Chebyshev polynomials in general, not to any particular Chebyshev > polynomial. > > > I'd still like to settle the coefficient access order for the Chebyshev > > class. As is, it is like the poly1d class. The constructor and coef > > attribute contain the coefficients in high to low order, while indexing > like > > mypoly[i] goes the other way. > > Personally I think that the high-to-low order in poly1d is a mistake, > but at this point I think it may be one we're stuck with. But since > chebyshev polynomials are all new, they do offer the chance to start > fresh. Low-to-high order also has the advantage that you could > implement access to higher coefficients returning zero, Interesting thought. > if you wanted, > so that they could be treated something like their mathematical notion > of series that happen to terminate. That said, I haven't used poly1d > much at all, so I'm not necessarily the best person to ask. > > I would like to make the change, but I would like some sort of consensus first. Of course, I will need to rewrite all the code/documentation if it happens ;) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From pav at iki.fi Tue Oct 6 13:48:01 2009 From: pav at iki.fi (Pauli Virtanen) Date: Tue, 06 Oct 2009 20:48:01 +0300 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: <1254851280.5496.19.camel@idol> ti, 2009-10-06 kello 10:24 -0600, Charles R Harris kirjoitti: [clip] > I would like to make the change, but I would like some sort of > consensus first. Of course, I will need to rewrite all the > code/documentation if it happens ;) One question to ask is what goes where. To me, functionality dealing with polynomials in different bases (including Chebyshev) sounds definitely like a part of Scipy, and not a part of a basic numerical array package that we might want to keep Numpy as. Personally, I'd rather leave poly1d alone, and effectively replace it by a proper polynomial handling package in Scipy, as it sounds like the amount of features is ballooning. Thoughts? A second question is how much of this has to do with scipy.special. Quite often, one just needs values of orthogonal polynomials at certain points, and it could be useful to have separate fast ufuncs for computing these, rather than having to construct a bulky class instance first. Pauli From charlesr.harris at gmail.com Tue Oct 6 14:11:43 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 12:11:43 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <1254851280.5496.19.camel@idol> References: <1254851280.5496.19.camel@idol> Message-ID: On Tue, Oct 6, 2009 at 11:48 AM, Pauli Virtanen wrote: > ti, 2009-10-06 kello 10:24 -0600, Charles R Harris kirjoitti: > > > [clip] > > I would like to make the change, but I would like some sort of > > consensus first. Of course, I will need to rewrite all the > > code/documentation if it happens ;) > > One question to ask is what goes where. > > To me, functionality dealing with polynomials in different bases > (including Chebyshev) sounds definitely like a part of Scipy, and not a > part of a basic numerical array package that we might want to keep Numpy > as. Personally, I'd rather leave poly1d alone, and effectively replace > it by a proper polynomial handling package in Scipy, as it sounds like > the amount of features is ballooning. Thoughts? > > The features in chebyshev are pretty much the same as poly1d. You can look, I posted the source on the numpy list. I want the class in numpy because chebyshev support is pretty much essential to numerical work, right up there with power series, and some of the existing/proposed functions in scipy could make use of it. As Anne pointed out, the existing interface to the scipy chebyshev functions is useless because the result is returned as a polynomial. > A second question is how much of this has to do with scipy.special. > Quite often, one just needs values of orthogonal polynomials at certain > points, and it could be useful to have separate fast ufuncs for > computing these, rather than having to construct a bulky class instance > first. > The class isn't bulky. The module is ~1400 lines, but most of that is documentation. If you want millions of points using standard c types, then a fast routine is essential. But if you just want to do ordinary work, possibly with more obscure numeric types such as rationals or polynomials, then the class is more convenient and easier to maintain. But I don't intend to duplicate all the orthogonal functions in special. What I was proposing was an abstract base class that could be used to build standard interfaces to the functions. Given just polynomials and chebyshev series it wasn't worth the hassle, but if someone wants to go ahead and interface a bunch more polynomial types it starts to look useful. That base class probably belongs in scipy. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From pav at iki.fi Tue Oct 6 14:29:50 2009 From: pav at iki.fi (Pauli Virtanen) Date: Tue, 06 Oct 2009 21:29:50 +0300 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1254851280.5496.19.camel@idol> Message-ID: <1254853789.5496.35.camel@idol> ti, 2009-10-06 kello 12:11 -0600, Charles R Harris kirjoitti: [clip] > The features in chebyshev are pretty much the same as poly1d. You can > look, I posted the source on the numpy list. I want the class in numpy > because chebyshev support is pretty much essential to numerical work, > right up there with power series, and some of the existing/proposed > functions in scipy could make use of it. This would also work if it was in Scipy, IMHO. But I guess that if we have financial functions in Numpy, maybe there's room for more... > As Anne pointed out, the existing interface to the scipy chebyshev > functions is useless because the result is returned as a polynomial. Yes, also our documentation warns that the current interface is useless. > A second question is how much of this has to do with > scipy.special. > Quite often, one just needs values of orthogonal polynomials > at certain > points, and it could be useful to have separate fast ufuncs > for > computing these, rather than having to construct a bulky class > instance > first. > > The class isn't bulky. The module is ~1400 lines, but most of that is > documentation. If you want millions of points using standard c types, > then a fast routine is essential. Yeah, Python overhead is the bulky thing, which is why we might want to have fast routines in addition to the ones returning class instances. A suitable base class or a more robust polynomial representation could be useful for the class-based interface, and yes, is a good idea in general. Cheers, Pauli From d.l.goldsmith at gmail.com Tue Oct 6 14:59:00 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Tue, 6 Oct 2009 11:59:00 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <1254853789.5496.35.camel@idol> References: <1254851280.5496.19.camel@idol> <1254853789.5496.35.camel@idol> Message-ID: <45d1ab480910061159r63c1b535v2d465625556e1508@mail.gmail.com> Personally, I'd like to see NumPy "go back to" being the namespace for ndarray and broadcasting, plus such objects' "most immediately necessary and/or useful" accoutrement; of course, reaching an objective - or even a consensus - definition of "most immediately necessary and/or useful" is probably impossible, but IMO neither financial functions nor a polynomial class qualify. Certainly, it would be optimal if one didn't need to install the whole SciPy package to use a polynomial object (or _any_ of SciPy's distinct modules, for that matter) but even if/while that's not possible, I'm still "+1" for putting these things in SciPy, not NumPy. DG On Tue, Oct 6, 2009 at 11:29 AM, Pauli Virtanen wrote: > ti, 2009-10-06 kello 12:11 -0600, Charles R Harris kirjoitti: > > [clip] > > The features in chebyshev are pretty much the same as poly1d. You can > > look, I posted the source on the numpy list. I want the class in numpy > > because chebyshev support is pretty much essential to numerical work, > > right up there with power series, and some of the existing/proposed > > functions in scipy could make use of it. > > This would also work if it was in Scipy, IMHO. But I guess that if we > have financial functions in Numpy, maybe there's room for more... > > > As Anne pointed out, the existing interface to the scipy chebyshev > > functions is useless because the result is returned as a polynomial. > > Yes, also our documentation warns that the current interface is useless. > > > A second question is how much of this has to do with > > scipy.special. > > Quite often, one just needs values of orthogonal polynomials > > at certain > > points, and it could be useful to have separate fast ufuncs > > for > > computing these, rather than having to construct a bulky class > > instance > > first. > > > > The class isn't bulky. The module is ~1400 lines, but most of that is > > documentation. If you want millions of points using standard c types, > > then a fast routine is essential. > > Yeah, Python overhead is the bulky thing, which is why we might want to > have fast routines in addition to the ones returning class instances. > > A suitable base class or a more robust polynomial representation could > be useful for the class-based interface, and yes, is a good idea in > general. > > Cheers, > Pauli > > > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Oct 6 15:14:13 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 13:14:13 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <45d1ab480910061159r63c1b535v2d465625556e1508@mail.gmail.com> References: <1254851280.5496.19.camel@idol> <1254853789.5496.35.camel@idol> <45d1ab480910061159r63c1b535v2d465625556e1508@mail.gmail.com> Message-ID: On Tue, Oct 6, 2009 at 12:59 PM, David Goldsmith wrote: > Personally, I'd like to see NumPy "go back to" being the namespace for > ndarray and broadcasting, plus such objects' "most immediately necessary > and/or useful" accoutrement; of course, reaching an objective - or even a > consensus - definition of "most immediately necessary and/or useful" is > probably impossible, but IMO neither financial functions nor a polynomial > class qualify. Certainly, it would be optimal if one didn't need to install > the whole SciPy package to use a polynomial object (or _any_ of SciPy's > distinct modules, for that matter) but even if/while that's not possible, > I'm still "+1" for putting these things in SciPy, not NumPy. > > Well, I'd go that way if there were an "extras" package that was easy to get hold of and that consisted solely of python/cython modules for easy building/installation. Call it scipy-lite. Most of numpy/lib could go into it and I'd be happy with that. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Tue Oct 6 15:53:03 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Tue, 6 Oct 2009 12:53:03 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1254851280.5496.19.camel@idol> <1254853789.5496.35.camel@idol> <45d1ab480910061159r63c1b535v2d465625556e1508@mail.gmail.com> Message-ID: <45d1ab480910061253m24b83d6fh970edeef5587be4d@mail.gmail.com> Noted! If we ever get a ticketing system dedicated to packaging issues, let's try to remember to turn this into a ticket. :-) DG On Tue, Oct 6, 2009 at 12:14 PM, Charles R Harris wrote: > > > On Tue, Oct 6, 2009 at 12:59 PM, David Goldsmith wrote: > >> Personally, I'd like to see NumPy "go back to" being the namespace for >> ndarray and broadcasting, plus such objects' "most immediately necessary >> and/or useful" accoutrement; of course, reaching an objective - or even a >> consensus - definition of "most immediately necessary and/or useful" is >> probably impossible, but IMO neither financial functions nor a polynomial >> class qualify. Certainly, it would be optimal if one didn't need to install >> the whole SciPy package to use a polynomial object (or _any_ of SciPy's >> distinct modules, for that matter) but even if/while that's not possible, >> I'm still "+1" for putting these things in SciPy, not NumPy. >> >> > Well, I'd go that way if there were an "extras" package that was easy to > get hold of and that consisted solely of python/cython modules for easy > building/installation. Call it scipy-lite. Most of numpy/lib could go into > it and I'd be happy with that. > > > > Chuck > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Oct 6 15:54:33 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 13:54:33 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 10:03 AM, Anne Archibald wrote: > > Personally I think that the high-to-low order in poly1d is a mistake, > but at this point I think it may be one we're stuck with. But since > chebyshev polynomials are all new, they do offer the chance to start > fresh. Low-to-high order also has the advantage that you could > implement access to higher coefficients returning zero, if you wanted, > so that they could be treated something like their mathematical notion > of series that happen to terminate. That said, I haven't used poly1d > much at all, so I'm not necessarily the best person to ask. > > The more I think about it, the more I am tempted to change the coefficient order to low to high and remove the indexing altogether so the object looks more like an immutable type. Folks can still assign directly to the coef attribute, but that makes it look more like the abuse it is, so maybe it won't be used that much. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Oct 6 15:57:40 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 13:57:40 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <45d1ab480910061253m24b83d6fh970edeef5587be4d@mail.gmail.com> References: <1254851280.5496.19.camel@idol> <1254853789.5496.35.camel@idol> <45d1ab480910061159r63c1b535v2d465625556e1508@mail.gmail.com> <45d1ab480910061253m24b83d6fh970edeef5587be4d@mail.gmail.com> Message-ID: On Tue, Oct 6, 2009 at 1:53 PM, David Goldsmith wrote: > Noted! If we ever get a ticketing system dedicated to packaging issues, > let's try to remember to turn this into a ticket. :-) > > I'm thinking the name numpy-extras would make its relationship to numpy a bit clearer. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Oct 6 16:10:01 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 6 Oct 2009 13:10:01 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 9:03 AM, Anne Archibald wrote: > Personally I think that the high-to-low order in poly1d is a mistake, > but at this point I think it may be one we're stuck with. Sometimes I wonder if we shouldn't push a new poly class with the other order. I can't stand that API, and I make actual mistakes pretty much every time I use it. I know I could make my own personal wrapper, but I've taught with this code and I always have to explain things about it, students make mistakes that puzzle them, etc. Perhaps a new one with a cleaned-up API (and obviously new names), leaving the old one for backwards compatibility with a Pending, then real, DeprecationWarning would be worthwhile? If we're the only two people unhappy about this, so be it. But if it's a general feeling, now that Chuck is working on the Chebyshev polys, it might be an opportunity to overhaul orthogonal polynomial support in Scipy and do it right. Just a thought from the peanut gallery though, because I'm unlikely to have any time to do this myself... Cheers, f From charlesr.harris at gmail.com Tue Oct 6 16:17:38 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 14:17:38 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 2:10 PM, Fernando Perez wrote: > On Tue, Oct 6, 2009 at 9:03 AM, Anne Archibald > wrote: > > Personally I think that the high-to-low order in poly1d is a mistake, > > but at this point I think it may be one we're stuck with. > > Sometimes I wonder if we shouldn't push a new poly class with the > other order. I can't stand that API, and I make actual mistakes > pretty much every time I use it. I know I could make my own personal > wrapper, but I've taught with this code and I always have to explain > things about it, students make mistakes that puzzle them, etc. > > What else about the API do you like/dislike? I do want to push a new polynomial class. Unfortunately, the numpy namespace is already cluttered with the old one. > Perhaps a new one with a cleaned-up API (and obviously new names), > leaving the old one for backwards compatibility with a Pending, then > real, DeprecationWarning would be worthwhile? > > If we're the only two people unhappy about this, so be it. But if > it's a general feeling, now that Chuck is working on the Chebyshev > polys, it might be an opportunity to overhaul orthogonal polynomial > support in Scipy and do it right. > > Just a thought from the peanut gallery though, because I'm unlikely to > have any time to do this myself... > > That makes four people, including myself, who have weighed in and all prefer the low to high order. Sounds like the way to go. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Tue Oct 6 17:57:40 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 6 Oct 2009 17:57:40 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/6 Fernando Perez : > On Tue, Oct 6, 2009 at 9:03 AM, Anne Archibald > wrote: >> Personally I think that the high-to-low order in poly1d is a mistake, >> but at this point I think it may be one we're stuck with. > > Sometimes I wonder if we shouldn't push a new poly class with the > other order. I can't stand that API, and I make actual mistakes > pretty much every time I use it. I know I could make my own personal > wrapper, but I've taught with this code and I always have to explain > things about it, students make mistakes that puzzle them, etc. > > Perhaps a new one with a cleaned-up API (and obviously new names), > leaving the old one for backwards compatibility with a Pending, then > real, DeprecationWarning would be worthwhile? I think poly1d needs to die. Even apart from the coefficient order, it tempts people to write numerically disastrous code - as we see with the orthogonal polynomials in scipy.special. Even quite ordinary polynomial fitting tends to go very badly when applied with power-basis polynomials. On the other hand, it does seem valuable to have a polynomial object of some sort, and I very much don't want my ballooning ideas for features to impede Charles Harris' actual implementation. > If we're the only two people unhappy about this, so be it. But if > it's a general feeling, now that Chuck is working on the Chebyshev > polys, it might be an opportunity to overhaul orthogonal polynomial > support in Scipy and do it right. I think my vote would be to remove polynomial support in numpy entirely (deprecating along the way as usual). Scipy should definitely have a full-featured polynomial class (exactly what features those should be is open for debate). It is probably also worth having ufunc versions for direct evaluation of those orthogonal polynomials for which a special implementation is appropriate (e.g. Chebyshev cos(n*arccos(x)), or Legendre polynomials using the cephes code). Anne From charlesr.harris at gmail.com Tue Oct 6 18:29:32 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 16:29:32 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 3:57 PM, Anne Archibald wrote: > 2009/10/6 Fernando Perez : > > On Tue, Oct 6, 2009 at 9:03 AM, Anne Archibald > > wrote: > >> Personally I think that the high-to-low order in poly1d is a mistake, > >> but at this point I think it may be one we're stuck with. > > > > Sometimes I wonder if we shouldn't push a new poly class with the > > other order. I can't stand that API, and I make actual mistakes > > pretty much every time I use it. I know I could make my own personal > > wrapper, but I've taught with this code and I always have to explain > > things about it, students make mistakes that puzzle them, etc. > > > > Perhaps a new one with a cleaned-up API (and obviously new names), > > leaving the old one for backwards compatibility with a Pending, then > > real, DeprecationWarning would be worthwhile? > > I think poly1d needs to die. Even apart from the coefficient order, it > tempts people to write numerically disastrous code - as we see with > the orthogonal polynomials in scipy.special. Even quite ordinary > polynomial fitting tends to go very badly when applied with > power-basis polynomials. > > On the other hand, it does seem valuable to have a polynomial object > of some sort, and I very much don't want my ballooning ideas for > features to impede Charles Harris' actual implementation. > > > If we're the only two people unhappy about this, so be it. But if > > it's a general feeling, now that Chuck is working on the Chebyshev > > polys, it might be an opportunity to overhaul orthogonal polynomial > > support in Scipy and do it right. > > I think my vote would be to remove polynomial support in numpy > entirely (deprecating along the way as usual). Scipy should definitely > have a full-featured polynomial class (exactly what features those > should be is open for debate). It is probably also worth having ufunc > versions for direct evaluation of those orthogonal polynomials for > which a special implementation is appropriate (e.g. Chebyshev > cos(n*arccos(x)), or Legendre polynomials using the cephes code). > > I can board that train, but where should those classes go? Note that both the polynomial and chebyshev modules come with extra functions for manipulating series independently of the classes and Robert wants those functions. So, should those functions be left in? I also went to some trouble to set things up so the chebyshev functions would also work with object coefficients, although there are a few places I haven't modified for that yet. But perhaps that is an over general target and intrude on areas that would be best left to other projects like sympy. Having extended precision coefficients can be useful, though. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Tue Oct 6 18:35:20 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Tue, 6 Oct 2009 15:35:20 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: <45d1ab480910061535t32b9228ek9875b43e48eaaaf@mail.gmail.com> On Tue, Oct 6, 2009 at 1:17 PM, Charles R Harris wrote: > > > That makes four people, including myself, who have weighed in and all > prefer the low to high order. Sounds like the way to go. > I'm neutral, but whichever order is decided upon for internal representation, I think it'd be "nice and useful" to include a flag specifying user's preferred order, along with a "private" method to automatically convert to internal rep. if the user has set the flag oppositely. DG > > Chuck > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Tue Oct 6 18:39:00 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Tue, 6 Oct 2009 15:39:00 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> IMO this thread has matured to the point where someone(s) should formally propose a feature spec/object structure for community vetting. :-) DG -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Tue Oct 6 18:41:24 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Tue, 6 Oct 2009 15:41:24 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> Message-ID: <45d1ab480910061541t1b4ee7cek5321a13112f28c48@mail.gmail.com> PS: once we've settled on a spec, I'm interested in being on the "development team" (polys are something I have an independent interest in and, I think, enough knowledge of that I can hopefully contribute meaningfully). DG On Tue, Oct 6, 2009 at 3:39 PM, David Goldsmith wrote: > IMO this thread has matured to the point where someone(s) should formally > propose a feature spec/object structure for community vetting. :-) > > DG > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Oct 6 18:43:18 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 16:43:18 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <45d1ab480910061535t32b9228ek9875b43e48eaaaf@mail.gmail.com> References: <45d1ab480910061535t32b9228ek9875b43e48eaaaf@mail.gmail.com> Message-ID: On Tue, Oct 6, 2009 at 4:35 PM, David Goldsmith wrote: > On Tue, Oct 6, 2009 at 1:17 PM, Charles R Harris < > charlesr.harris at gmail.com> wrote: > >> >> >> That makes four people, including myself, who have weighed in and all >> prefer the low to high order. Sounds like the way to go. >> > > I'm neutral, but whichever order is decided upon for internal > representation, I think it'd be "nice and useful" to include a flag > specifying user's preferred order, along with a "private" method to > automatically convert to internal rep. if the user has set the flag > oppositely. > > Nah, you don't want variety in these things, it makes for confusing code 'cause you have to pay careful attention to the value of the flag. It is the KISS principal, pick one that folks feel comfortable with and stick to it. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Oct 6 20:32:10 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 6 Oct 2009 17:32:10 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: Hi Chuck, On Tue, Oct 6, 2009 at 1:17 PM, Charles R Harris wrote: > > What else about the API do you like/dislike? I do want to push a new > polynomial class. Unfortunately, the numpy namespace is already cluttered > with the old one. No, mostly just that coefficient order issue, which trips me up every time. I've only used them for teaching small examples, never 'in anger', so I'm not the most qualified to talk. When I needed polynomial evaluation codes in the past, speed was critical and I needed them evaluated over arrays of points (quadrature nodes) for all orders at all nodes, so I just rolled my own code in C by hand (well, using weave.inline to fill pre-allocated empty numpy arrays). But having poly code in scipy with: - the correct choice of coefficient order - that is stable and fast both for point-wise and for array-wise evaluations would be great, as I wouldn't (for example) have had to roll my own. Others with more extensive needs and experience on this, like Anne, can pitch in with more informed ideas for the API than I can come up with right now. Cheers, f From charlesr.harris at gmail.com Tue Oct 6 20:46:27 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 18:46:27 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 6:32 PM, Fernando Perez wrote: > Hi Chuck, > > On Tue, Oct 6, 2009 at 1:17 PM, Charles R Harris > wrote: > > > > What else about the API do you like/dislike? I do want to push a new > > polynomial class. Unfortunately, the numpy namespace is already cluttered > > with the old one. > > No, mostly just that coefficient order issue, which trips me up every > time. I've only used them for teaching small examples, never 'in > anger', so I'm not the most qualified to talk. When I needed > polynomial evaluation codes in the past, speed was critical and I > needed them evaluated over arrays of points (quadrature nodes) for all > orders at all nodes, so I just rolled my own code in C by hand (well, > using weave.inline to fill pre-allocated empty numpy arrays). > > But having poly code in scipy with: > > - the correct choice of coefficient order > OK, I'm changing that now. > - that is stable and fast both for point-wise and for array-wise > evaluations > > Just to be clear, you mean 1 polynomial in 1 variable evaluated at many points ? The current code evaluates at an arbitrary ndarray of points, but is a bit wasteful of memory: there are essentially three active copies of the data points at any one time. > would be great, as I wouldn't (for example) have had to roll my own. > Others with more extensive needs and experience on this, like Anne, > can pitch in with more informed ideas for the API than I can come up > with right now. > > Do you ever use the indexing in poly1d? I'm thinking of dropping that. After all, the coefficient array is available if one wants to cheat. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Oct 6 20:57:10 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 6 Oct 2009 17:57:10 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 5:46 PM, Charles R Harris wrote: > > > On Tue, Oct 6, 2009 at 6:32 PM, Fernando Perez wrote: >> - the correct choice of coefficient order > > OK, I'm changing that now. Thank you! >> - that is stable and fast both for point-wise and for array-wise >> evaluations >> > > Just to be clear, you mean 1 polynomial in 1 variable evaluated at many > points ? The current code evaluates at an arbitrary ndarray of points, but > is a bit wasteful of memory: there are essentially three active copies of > the data points at any one time. I had a peculiar need: I needed to evaluate a family of all Legendre polynomials of order up to n-1, at an array with n points. This was fairly performance-critical, so I rolled my own code, though I could have done the loop over polynomial index. But ideally, we'd have the evaluation code (fast, stable) for whole arrays accessible in C(ython), so that a funky use case like mine could be implemented both easily and efficiently with a Cython loop that would do pure C calls without jumping up into the python layer. Dreaming here :) >> would be great, as I wouldn't (for example) have had to roll my own. >> Others with more extensive needs and experience on this, like Anne, >> can pitch in with more informed ideas for the API than I can come up >> with right now. >> > > Do you ever use the indexing in poly1d? I'm thinking of dropping that. After > all, the coefficient array is available if one wants to cheat. I didn't, but this is a sample of 1, so take it for what it's worth :) Cheers, f From charlesr.harris at gmail.com Tue Oct 6 21:25:41 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 19:25:41 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 6:57 PM, Fernando Perez wrote: > On Tue, Oct 6, 2009 at 5:46 PM, Charles R Harris > wrote: > > > > > > On Tue, Oct 6, 2009 at 6:32 PM, Fernando Perez > wrote: > > >> - the correct choice of coefficient order > > > > OK, I'm changing that now. > > Thank you! > > >> - that is stable and fast both for point-wise and for array-wise > >> evaluations > >> > > > > Just to be clear, you mean 1 polynomial in 1 variable evaluated at many > > points ? The current code evaluates at an arbitrary ndarray of points, > but > > is a bit wasteful of memory: there are essentially three active copies of > > the data points at any one time. > > I had a peculiar need: I needed to evaluate a family of all Legendre > polynomials of order up to n-1, at an array with n points. This was > fairly performance-critical, so I rolled my own code, though I could > have done the loop over polynomial index. But ideally, we'd have the > evaluation code (fast, stable) for whole arrays accessible in > C(ython), That wouldn't make it much faster for large arrays of data points, the code is just multiply and add applied repeatedly to whole arrays. Well, OK, maybe the ndarray multiply and add overhead might slow things down. > so that a funky use case like mine could be implemented both > easily and efficiently with a Cython loop that would do pure C calls > without jumping up into the python layer. > > Dreaming here :) > > > >> would be great, as I wouldn't (for example) have had to roll my own. > >> Others with more extensive needs and experience on this, like Anne, > >> can pitch in with more informed ideas for the API than I can come up > >> with right now. > If you happen to have an example still lying around I could try running a benchmark. > >> > > > > Do you ever use the indexing in poly1d? I'm thinking of dropping that. > After > > all, the coefficient array is available if one wants to cheat. > > I didn't, but this is a sample of 1, so take it for what it's worth :) > > Can't decide on that one. It's easy enough, but one can just change out the coefficient array or, better I think, create new objects. Unless object creation was a bottle neck. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Tue Oct 6 21:37:17 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 6 Oct 2009 21:37:17 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> Message-ID: 2009/10/6 David Goldsmith : > IMO this thread has matured to the point where someone(s) should formally > propose a feature spec/object structure for community vetting. :-) http://www.scipy.org/NewPolynomials I don't propose an object structure, because I'm not sure how that should look, and I'm also not sure how reasonable my various requirements and restrictions are. On the other hand I did list at the end some references I found in a quick literature search; it appears the wisdom is that for general polynomial manipulation it's best to use the Bernstein basis (the same used in Bezier splines) or the Lagrange basis (representing polynomials by (x,y) pairs). In either case papers are available describing algorithms for all the basic polynomial operations. Anne From charlesr.harris at gmail.com Tue Oct 6 21:38:16 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 19:38:16 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 7:25 PM, Charles R Harris wrote: > > > On Tue, Oct 6, 2009 at 6:57 PM, Fernando Perez wrote: > >> On Tue, Oct 6, 2009 at 5:46 PM, Charles R Harris >> wrote: >> > >> > >> > On Tue, Oct 6, 2009 at 6:32 PM, Fernando Perez >> wrote: >> >> >> - the correct choice of coefficient order >> > >> > OK, I'm changing that now. >> >> Thank you! >> >> >> - that is stable and fast both for point-wise and for array-wise >> >> evaluations >> >> >> > >> > Just to be clear, you mean 1 polynomial in 1 variable evaluated at many >> > points ? The current code evaluates at an arbitrary ndarray of points, >> but >> > is a bit wasteful of memory: there are essentially three active copies >> of >> > the data points at any one time. >> >> I had a peculiar need: I needed to evaluate a family of all Legendre >> polynomials of order up to n-1, at an array with n points. This was >> fairly performance-critical, so I rolled my own code, though I could >> have done the loop over polynomial index. But ideally, we'd have the >> evaluation code (fast, stable) for whole arrays accessible in >> C(ython), > > > That wouldn't make it much faster for large arrays of data points, the code > is just multiply and add applied repeatedly to whole arrays. Well, OK, maybe > the ndarray multiply and add overhead might slow things down. > > Well, good cache/register usage could make a *big* difference. But that would require getting down to the c level. Hmm, that could be left as an implementation optimization to be done later while keeping the same interface. Cython would probably do that job quite nicely, although object type arrays wouldn't benefit much from that. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Oct 6 22:00:19 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 20:00:19 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> Message-ID: On Tue, Oct 6, 2009 at 7:37 PM, Anne Archibald wrote: > 2009/10/6 David Goldsmith : > > IMO this thread has matured to the point where someone(s) should formally > > propose a feature spec/object structure for community vetting. :-) > > http://www.scipy.org/NewPolynomials > > I don't propose an object structure, because I'm not sure how that > should look, and I'm also not sure how reasonable my various > requirements and restrictions are. On the other hand I did list at the > end some references I found in a quick literature search; it appears > the wisdom is that for general polynomial manipulation it's best to > use the Bernstein basis (the same used in Bezier splines) or the > Lagrange basis (representing polynomials by (x,y) pairs). In either > case papers are available describing algorithms for all the basic > polynomial operations. > > Bernstein and Lagrange would both be defined on interval domains, [0,1] and [-1,1] respectively, and that would define both center and scaling if arbitrary intervals are mapped to those domains, just like for the Chebyshev polynomials. Hmm... Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Tue Oct 6 23:34:39 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 6 Oct 2009 23:34:39 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> Message-ID: 2009/10/6 Charles R Harris : > > > On Tue, Oct 6, 2009 at 7:37 PM, Anne Archibald > wrote: >> >> 2009/10/6 David Goldsmith : >> > IMO this thread has matured to the point where someone(s) should >> > formally >> > propose a feature spec/object structure for community vetting. :-) >> >> http://www.scipy.org/NewPolynomials >> >> I don't propose an object structure, because I'm not sure how that >> should look, and I'm also not sure how reasonable my various >> requirements and restrictions are. On the other hand I did list at the >> end some references I found in a quick literature search; it appears >> the wisdom is that for general polynomial manipulation it's best to >> use the Bernstein basis (the same used in Bezier splines) or the >> Lagrange basis (representing polynomials by (x,y) pairs). In either >> case papers are available describing algorithms for all the basic >> polynomial operations. >> > > Bernstein and Lagrange would both be defined on interval domains, [0,1] and > [-1,1] respectively, and that would define both center and scaling if > arbitrary intervals are mapped to those domains, just like for the Chebyshev > polynomials. Hmm... Yes, I haven't found any discussion of polynomial bases for unbounded domains. It's not even clear to me what the right criterion for error is, since any polynomial basis is going to become large far enough away from the origin. If what one cared about were some sort of weighted mean squared error, then some of the classical orthogonal polynomials would be the right choice, but that has little relevance for questions like root-finding or max-difference error. I suppose some sort of weighted max-difference error might be usable - perhaps weighted by exp(-x**2) or some such thing - but without some serious numerical analysis it's not clear what to do there. I'd guess that for most practical applications a specific interval can be chosen. That does mean one needs to have parameterized bases, but is otherwise inoffensive. There's a comment in one of those references (I forget which just now) to the effect that the Chebyshev basis is a poor choice if one wants to find the roots of the resulting polynomial; on the other hand it's an excellent basis for finding approximating polynomials. This is the sort of thing that makes me want multiple representations. Anne From charlesr.harris at gmail.com Wed Oct 7 00:00:28 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 6 Oct 2009 22:00:28 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> Message-ID: On Tue, Oct 6, 2009 at 9:34 PM, Anne Archibald wrote: > 2009/10/6 Charles R Harris : > > > > > > On Tue, Oct 6, 2009 at 7:37 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/6 David Goldsmith : > >> > IMO this thread has matured to the point where someone(s) should > >> > formally > >> > propose a feature spec/object structure for community vetting. :-) > >> > >> http://www.scipy.org/NewPolynomials > >> > >> I don't propose an object structure, because I'm not sure how that > >> should look, and I'm also not sure how reasonable my various > >> requirements and restrictions are. On the other hand I did list at the > >> end some references I found in a quick literature search; it appears > >> the wisdom is that for general polynomial manipulation it's best to > >> use the Bernstein basis (the same used in Bezier splines) or the > >> Lagrange basis (representing polynomials by (x,y) pairs). In either > >> case papers are available describing algorithms for all the basic > >> polynomial operations. > >> > > > > Bernstein and Lagrange would both be defined on interval domains, [0,1] > and > > [-1,1] respectively, and that would define both center and scaling if > > arbitrary intervals are mapped to those domains, just like for the > Chebyshev > > polynomials. Hmm... > > Yes, I haven't found any discussion of polynomial bases for unbounded > domains. It's not even clear to me what the right criterion for error > is, since any polynomial basis is going to become large far enough > away from the origin. If what one cared about were some sort of > weighted mean squared error, then some of the classical orthogonal > polynomials would be the right choice, but that has little relevance > for questions like root-finding or max-difference error. I suppose > some sort of weighted max-difference error might be usable - perhaps > weighted by exp(-x**2) or some such thing - but without some serious > numerical analysis it's not clear what to do there. > > I'd guess that for most practical applications a specific interval can > be chosen. That does mean one needs to have parameterized bases, but > is otherwise inoffensive. > > There's a comment in one of those references (I forget which just now) > to the effect that the Chebyshev basis is a poor choice if one wants > to find the roots of the resulting polynomial; on the other hand it's > an excellent basis for finding approximating polynomials. This is the > sort of thing that makes me want multiple representations. > > There was something like that for the second Wilkinson polynomial, and understandable as the wiggles got very small. All the zero finding algorithms seemed to depend on some form of the companion matrix, but of course the exact form of the companion matrix depended on the basis. The Lagrange basis were parameterized by the sample points, so there are actually lots, and lots, well, and lots of those basis. If the samples are at the Chebyshev points they are just a well behaved transform away from Chebyshev coefficients. I wasn't able to get to the paper on division, which was one that really roused my curiosity. Maybe the library will have it. One of the papers didn't regard degree as important, which is probably because 1) the Bernstein polynomials aren't graded by degree and 2) neither are the Lagrange functions. And because it is difficult to define degree in those contexts. At some point I think we will have to decide whether we want "polynomial approximation", "orthogonal polynomials", or both. Both seem to have their virtues. Thanks for putting that page together, that helps get things organized. It also grows what started as a simple Chebyshev class into something more extensive ;) Oh well... Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Wed Oct 7 01:09:01 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Wed, 7 Oct 2009 01:09:01 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> Message-ID: 2009/10/7 Charles R Harris : > I wasn't able to get to the paper on division, which was one that really > roused my curiosity. Maybe the library will have it. It's less interesting than it looks; his abstract algorithm is really just grade-school long division with different operations to extract the head term and form the one-term-less quotients. It may also have poorer numerical stability than previously known matrix methods. > One of the papers didn't regard degree as important, which is probably > because 1) the Bernstein polynomials aren't graded by degree and 2) neither > are the Lagrange functions. And because it is difficult to define degree in > those contexts. At some point I think we will have to decide whether we want > "polynomial approximation", "orthogonal polynomials", or both. Both seem to > have their virtues. Well, degree is an issue even for monomial-basis polynomials once you start adding and subtracting - the highest coefficient could be nearly zero and you'd have to decide whether it was close enough to zero to discard. Whether that's true depends on the x values you're interested in... At least for Chebyshev-basis polynomials there's a clear criterion for when it can be treated as zero > Thanks for putting that page together, that helps get things organized. It > also grows what started as a simple Chebyshev class into something more > extensive ;) Oh well... Heh. Well, I don't think you should delay releasing your Chebyshev class at all. Putting code that works aside in favour of code that doesn't exist yet seems like a mistake. Anne From d.l.goldsmith at gmail.com Wed Oct 7 02:33:09 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Tue, 6 Oct 2009 23:33:09 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> Message-ID: <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Thanks for doing that, Anne! DG On Tue, Oct 6, 2009 at 6:37 PM, Anne Archibald wrote: > 2009/10/6 David Goldsmith : > > IMO this thread has matured to the point where someone(s) should formally > > propose a feature spec/object structure for community vetting. :-) > > http://www.scipy.org/NewPolynomials > > I don't propose an object structure, because I'm not sure how that > should look, and I'm also not sure how reasonable my various > requirements and restrictions are. On the other hand I did list at the > end some references I found in a quick literature search; it appears > the wisdom is that for general polynomial manipulation it's best to > use the Bernstein basis (the same used in Bezier splines) or the > Lagrange basis (representing polynomials by (x,y) pairs). In either > case papers are available describing algorithms for all the basic > polynomial operations. > > Anne > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan at sun.ac.za Wed Oct 7 07:46:19 2009 From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=) Date: Wed, 7 Oct 2009 13:46:19 +0200 Subject: [SciPy-dev] ANN: SciPy October Sprint Message-ID: <9457e7c80910070446k2c1ae895u4011d242abc224b@mail.gmail.com> Hi all, Our patch queue keeps getting longer and longer, so here is an opportunity to do some spring cleaning (it's spring in South Africa, at least)! Please join us for an October SciPy sprint: * Date: 24/25 October 2009 (Sat/Sun) * More information: http://projects.scipy.org/scipy/wiki/SciPySprint200910 We are looking for volunteers to write documentation, review code, fix bugs or design marketing material. New contributors are most welcome, and mentoring will be available. See you there, Regards St?fan From pearu.peterson at gmail.com Wed Oct 7 07:57:25 2009 From: pearu.peterson at gmail.com (Pearu Peterson) Date: Wed, 07 Oct 2009 14:57:25 +0300 Subject: [SciPy-dev] Note on changeset 5878: Add old_behavior arg for convolve2d and correlate2d Message-ID: <4ACC8225.7030209@cens.ioc.ee> Hi, Currently scipy.signal.convolve gives the following warning by default: """ Current default behavior of convolve and correlate functions is deprecated. Convolve and corelate currently swap their arguments if the second argument has dimensions larger than the first one, and the mode is relative to the input with the largest dimension. The new behavior is to never swap the inputs, which is what most people expects, and is how correlation is usually defined. You can control the behavior with the old_behavior flag - the flag will disappear in scipy 0.9.0, and the functions will then implement the new behavior only. """ The problem right now is that it is difficult to write a stable application that would not give this warning. Using old_behavior=False argument means that when upgrading to scipy 0.9.0, the application will become broken because the kw argument will be removed. In my application the old and new behaviors are identical. My suggestion is to raise the deprecation warning only when the old and the new behavior are in conflict. For example, consider the diff below. What do you think? Pearu $ svn diff signaltools.py Index: signaltools.py =================================================================== --- signaltools.py (revision 5953) +++ signaltools.py (working copy) @@ -100,10 +100,11 @@ val = _valfrommode(mode) if old_behavior: - warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) if np.iscomplexobj(in2): + # hmm, how is this related to deprecation warning? in2 = in2.conjugate() if in1.size < in2.size: + warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) swp = in2 in2 = in1 in1 = swp @@ -210,8 +211,8 @@ slice_obj = [slice(None,None,-1)]*len(kernel.shape) if old_behavior: - warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) if (product(kernel.shape,axis=0) > product(volume.shape,axis=0)): + warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) temp = kernel kernel = volume volume = temp @@ -383,11 +384,8 @@ """ if old_behavior: - warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) - - if old_behavior: - warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) if (product(np.shape(in2),axis=0) > product(np.shape(in1),axis=0)): + warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) temp = in1 in1 = in2 in2 = temp @@ -436,8 +434,8 @@ cross-correlation of in1 with in2. """ - if old_behavior: - warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) + #if old_behavior: + # warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) val = _valfrommode(mode) bval = _bvalfromboundary(boundary) From stefan at sun.ac.za Wed Oct 7 08:49:14 2009 From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=) Date: Wed, 7 Oct 2009 14:49:14 +0200 Subject: [SciPy-dev] Note on changeset 5878: Add old_behavior arg for convolve2d and correlate2d In-Reply-To: <4ACC8225.7030209@cens.ioc.ee> References: <4ACC8225.7030209@cens.ioc.ee> Message-ID: <9457e7c80910070549j74413649s9301b4813416f814@mail.gmail.com> 2009/10/7 Pearu Peterson : > The problem right now is that it is difficult to write a stable > application that would not give this warning. > Using old_behavior=False argument means that when upgrading to scipy > 0.9.0, the application will become broken because the kw argument will > be removed. In my application the old and new behaviors are identical. If the old and new behaviours are identical, you could just suppress the warning: http://docs.python.org/library/warnings.html#temporarily-suppressing-warnings Cheers St?fan From pearu.peterson at gmail.com Wed Oct 7 10:00:06 2009 From: pearu.peterson at gmail.com (Pearu Peterson) Date: Wed, 07 Oct 2009 17:00:06 +0300 Subject: [SciPy-dev] Note on changeset 5878: Add old_behavior arg for convolve2d and correlate2d In-Reply-To: <9457e7c80910070549j74413649s9301b4813416f814@mail.gmail.com> References: <4ACC8225.7030209@cens.ioc.ee> <9457e7c80910070549j74413649s9301b4813416f814@mail.gmail.com> Message-ID: <4ACC9EE6.7000405@cens.ioc.ee> St?fan van der Walt wrote: > 2009/10/7 Pearu Peterson : >> The problem right now is that it is difficult to write a stable >> application that would not give this warning. >> Using old_behavior=False argument means that when upgrading to scipy >> 0.9.0, the application will become broken because the kw argument will >> be removed. In my application the old and new behaviors are identical. > > If the old and new behaviours are identical, you could just suppress > the warning: > > http://docs.python.org/library/warnings.html#temporarily-suppressing-warnings Sure. But suppressing warnings is a temporary solution and pollutes an application code. In fact, an easier temporary solution would be just to use old_behavior=False. I was looking for a more permanent solution, that is, fixing the source of the problem rather than forcing workarounds to (possible many) applications. Thanks, Pearu From smattacus at gmail.com Wed Oct 7 20:35:34 2009 From: smattacus at gmail.com (Sean Mattingly) Date: Wed, 7 Oct 2009 17:35:34 -0700 Subject: [SciPy-dev] ANN: SciPy October Sprint In-Reply-To: <9457e7c80910070446k2c1ae895u4011d242abc224b@mail.gmail.com> References: <9457e7c80910070446k2c1ae895u4011d242abc224b@mail.gmail.com> Message-ID: <856175f80910071735s6a071627p39a13a931b38d192@mail.gmail.com> Hi Stefan, Do I sign up somehow, or should I just show up in the Scipy irc channel? Cheers, - Sean 2009/10/7 St?fan van der Walt > Hi all, > > Our patch queue keeps getting longer and longer, so here is an > opportunity to do some spring cleaning (it's spring in South Africa, > at least)! > > Please join us for an October SciPy sprint: > > * Date: 24/25 October 2009 (Sat/Sun) > * More information: > http://projects.scipy.org/scipy/wiki/SciPySprint200910 > > We are looking for volunteers to write documentation, review code, fix > bugs or design marketing material. New contributors are most welcome, > and mentoring will be available. > > See you there, > > Regards > St?fan > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Wed Oct 7 21:12:31 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 7 Oct 2009 18:12:31 -0700 Subject: [SciPy-dev] ANN: SciPy October Sprint In-Reply-To: <9457e7c80910070446k2c1ae895u4011d242abc224b@mail.gmail.com> References: <9457e7c80910070446k2c1ae895u4011d242abc224b@mail.gmail.com> Message-ID: <45d1ab480910071812r5e5de24v6c2ea409570d71a5@mail.gmail.com> I suggest that prospective contributors add their name in the wiki below, under the topic (e.g., documentation, closing tickets, etc.) they plan on contributing to. Unless someone objects, I'll add this suggestion to the wiki shortly. (Not doing so, however, won't mean that your participation will be shunned.) :-) DG 2009/10/7 St?fan van der Walt > Hi all, > > Our patch queue keeps getting longer and longer, so here is an > opportunity to do some spring cleaning (it's spring in South Africa, > at least)! > > Please join us for an October SciPy sprint: > > * Date: 24/25 October 2009 (Sat/Sun) > * More information: > http://projects.scipy.org/scipy/wiki/SciPySprint200910 > > We are looking for volunteers to write documentation, review code, fix > bugs or design marketing material. New contributors are most welcome, > and mentoring will be available. > > See you there, > > Regards > St?fan > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at ar.media.kyoto-u.ac.jp Wed Oct 7 23:11:42 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 08 Oct 2009 12:11:42 +0900 Subject: [SciPy-dev] Note on changeset 5878: Add old_behavior arg for convolve2d and correlate2d In-Reply-To: <4ACC8225.7030209@cens.ioc.ee> References: <4ACC8225.7030209@cens.ioc.ee> Message-ID: <4ACD586E.3060907@ar.media.kyoto-u.ac.jp> Pearu Peterson wrote: > Hi, > > Currently scipy.signal.convolve gives the following warning by default: > > """ > Current default behavior of convolve and correlate functions is deprecated. > > Convolve and corelate currently swap their arguments if the second argument > has dimensions larger than the first one, and the mode is relative to > the input > with the largest dimension. The new behavior is to never swap the > inputs, which > is what most people expects, and is how correlation is usually defined. > > You can control the behavior with the old_behavior flag - the flag will > disappear in scipy 0.9.0, and the functions will then implement the new > behavior only. > """ > > The problem right now is that it is difficult to write a stable > application that would not give this warning. > Using old_behavior=False argument means that when upgrading to scipy > 0.9.0, the application will become broken because the kw argument will > be removed. In my application the old and new behaviors are identical. > > My suggestion is to raise the deprecation warning only when the old > and the new behavior are in conflict. Yes, that's a good idea. I will try to fix this today, ping me if I forgot > For example, consider the diff below. > > What do you think? > > Pearu > > $ svn diff signaltools.py > Index: signaltools.py > =================================================================== > --- signaltools.py (revision 5953) > +++ signaltools.py (working copy) > @@ -100,10 +100,11 @@ > val = _valfrommode(mode) > > if old_behavior: > - warnings.warn(DeprecationWarning(_SWAP_INPUTS_DEPRECATION_MSG)) > if np.iscomplexobj(in2): > + # hmm, how is this related to deprecation warning? > in2 = in2.conjugate() There is a doc problem here: the new behavior is not only to never swap inputs, but also to take the conjugate of the second argument (that is the usual definition of convolution). cheers, David From stefan at sun.ac.za Thu Oct 8 02:42:37 2009 From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=) Date: Thu, 8 Oct 2009 08:42:37 +0200 Subject: [SciPy-dev] ANN: SciPy October Sprint In-Reply-To: <856175f80910071735s6a071627p39a13a931b38d192@mail.gmail.com> References: <9457e7c80910070446k2c1ae895u4011d242abc224b@mail.gmail.com> <856175f80910071735s6a071627p39a13a931b38d192@mail.gmail.com> Message-ID: <9457e7c80910072342h51fe078at97ea6b283ac43df2@mail.gmail.com> Hey Sean 2009/10/8 Sean Mattingly : > Do I sign up somehow, or should I just show up in the Scipy irc channel? Glad to have you on board! We'll hang out on IRC, and hopefully some other groups will meet in person as well. If there are enough interested people in your area, feel free to start your own group. There is a time-zone difference, so here in South Africa we'll probably be at it from around 0800 UTC until 1700 UTC each day. Cheers St?fan From prabhu at aero.iitb.ac.in Thu Oct 8 05:31:02 2009 From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran) Date: Thu, 08 Oct 2009 15:01:02 +0530 Subject: [SciPy-dev] numpy.distutils: passing config_fc options in setup.py In-Reply-To: <4AC9C888.90002@cens.ioc.ee> References: <4AC81C3F.5010700@aero.iitb.ac.in> <4AC9C888.90002@cens.ioc.ee> Message-ID: <4ACDB156.6040902@aero.iitb.ac.in> On Monday 05 October 2009 03:50 PM, Pearu Peterson wrote: >> passing config_fc={...} to either Extension or setup does not work. >> The numpy version is 1.4.0.dev6976. > > A hackish solution would be to insert the following code to the > beginning of the setup.py file: > > import sys > sys.argv[:] = sys.argv[:1] + ['config_fc'. "--f90flags='-ffree-form > -fno-underscoring'"] + sys.argv[1:] > ) Thanks for the suggestion. I was hoping for a cleaner solution but this will work too. cheers, prabhu From peridot.faceted at gmail.com Thu Oct 8 11:37:08 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Thu, 8 Oct 2009 11:37:08 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: 2009/10/7 David Goldsmith : > Thanks for doing that, Anne! There is now a rough prototype on github: http://github.com/aarchiba/scikits.polynomial It certainly needs more tests and features, but it does support both the power basis and the Lagrange basis (polynomials represented by values at Chebyshev points). Anne From charlesr.harris at gmail.com Thu Oct 8 12:29:00 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 8 Oct 2009 10:29:00 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: Hi Anne, On Thu, Oct 8, 2009 at 9:37 AM, Anne Archibald wrote: > 2009/10/7 David Goldsmith : > > Thanks for doing that, Anne! > > There is now a rough prototype on github: > http://github.com/aarchiba/scikits.polynomial > It certainly needs more tests and features, but it does support both > the power basis and the Lagrange basis (polynomials represented by > values at Chebyshev points). > > Just took a quick look, which is probably all I'll get to for a few days as I'm going out of town tomorrow. Anyway, the Chebyshev points there are type II, which should probably be distinguished from type I (and III & IV). I also had the impression that the base class could have a few more functions and NotImplemented bits. The Polynomial class is implemented as a wrapper, it might even make sense to use multiple inheritance (horrors) to get specific polynomial types, but anyway it caught my attention and that part of the design might be worth spending some time thinking about. It also might be worth distinguishing series as a separate base because series do admit the division operators //, %, and divmod. Scalar multiplication/division (__truedivision__) should also be built in. I've also been using "from __future__ import division" up at the top to be py3k ready. For a series basis I was thinking of using what I've got for Chebyshev but with a bunch of the __foo__ functions raising the NotImplementedError. I've also got a single function for importing the coefficient arrays and doing the type conversions/checking. It's worth doing that one way for all the implementations as it makes it easier to fix/extend things. I've attached the low->high version of the chebyshev.py file just for further reference. The Chebyshev class is at the end. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: chebyshev.zip Type: application/zip Size: 11972 bytes Desc: not available URL: From charlesr.harris at gmail.com Thu Oct 8 12:45:20 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 8 Oct 2009 10:45:20 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: On Thu, Oct 8, 2009 at 10:29 AM, Charles R Harris wrote: > Hi Anne, > > On Thu, Oct 8, 2009 at 9:37 AM, Anne Archibald wrote: > >> 2009/10/7 David Goldsmith : >> > Thanks for doing that, Anne! >> >> There is now a rough prototype on github: >> http://github.com/aarchiba/scikits.polynomial >> It certainly needs more tests and features, but it does support both >> the power basis and the Lagrange basis (polynomials represented by >> values at Chebyshev points). >> >> > Just took a quick look, which is probably all I'll get to for a few days as > I'm going out of town tomorrow. Anyway, the Chebyshev points there are type > II, which should probably be distinguished from type I (and III & IV). I > also had the impression that the base class could have a few more functions > and NotImplemented bits. The Polynomial class is implemented as a wrapper, > it might even make sense to use multiple inheritance (horrors) to get > specific polynomial types, but anyway it caught my attention and that part > of the design might be worth spending some time thinking about. It also > might be worth distinguishing series as a separate base because series do > admit the division operators //, %, and divmod. Scalar > multiplication/division (__truedivision__) should also be built in. I've > also been using "from __future__ import division" up at the top to be py3k > ready. For a series basis I was thinking of using what I've got for > Chebyshev but with a bunch of the __foo__ functions raising the > NotImplementedError. I've also got a single function for importing the > coefficient arrays and doing the type conversions/checking. It's worth doing > that one way for all the implementations as it makes it easier to fix/extend > things. > > I've attached the low->high version of the chebyshev.py file just for > further reference. The Chebyshev class is at the end. > > Oh, and one other thing. I'm thinking that there should be *no* true inplace functions. That is, every operation returns a new object. That way the various classes look like basic, immutable, types which makes life simpler and less error prone. No views with side effects to worry about, and the coefficient arrays are likely to be small enough that the overhead to make copies will be small. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From nwagner at iam.uni-stuttgart.de Thu Oct 8 14:10:36 2009 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Thu, 08 Oct 2009 20:10:36 +0200 Subject: [SciPy-dev] scipy.test() segfaults Message-ID: Hi, Is this a known issue ? >>> import scipy >>> scipy.test() Running unit tests for scipy NumPy version 1.4.0.dev7507 NumPy is installed in /home/nwagner/local/lib64/python2.6/site-packages/numpy SciPy version 0.8.0.dev5964 SciPy is installed in /home/nwagner/local/lib64/python2.6/site-packages/scipy Python version 2.6 (r26:66714, Feb 3 2009, 20:49:49) [GCC 4.3.2 [gcc-4_3-branch revision 141291]] nose version 0.10.4 Program received signal SIGSEGV, Segmentation fault. PyArray_NeighborhoodIterNew (x=0x0, bounds=0x7ffff7ac2330, mode=-161985568, fill=0x7ffff7ac2330) at numpy/core/src/multiarray/iterators.c:1935 1935 array_iter_base_init((PyArrayIterObject*)ret, x->ao); (gdb) bt #0 PyArray_NeighborhoodIterNew (x=0x0, bounds=0x7ffff7ac2330, mode=-161985568, fill=0x7ffff7ac2330) at numpy/core/src/multiarray/iterators.c:1935 #1 0x00007fffee5d7e98 in init_path () at /home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/__multiarray_api.h:1206 Nils From charlesr.harris at gmail.com Thu Oct 8 14:29:48 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 8 Oct 2009 12:29:48 -0600 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: References: Message-ID: On Thu, Oct 8, 2009 at 12:10 PM, Nils Wagner wrote: > Hi, > > Is this a known issue ? > > Maybe. What release of numpy was scipy compiled against? If it was after the datetime commit but before David's fix, then it is the API ordering problem. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From gokhansever at gmail.com Thu Oct 8 14:44:02 2009 From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=) Date: Thu, 8 Oct 2009 13:44:02 -0500 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: References: Message-ID: <49d6b3500910081144y13ae6faak29a09257b29005b4@mail.gmail.com> On Thu, Oct 8, 2009 at 1:10 PM, Nils Wagner wrote: > Hi, > > Is this a known issue ? > > >>> import scipy > >>> scipy.test() > Running unit tests for scipy > NumPy version 1.4.0.dev7507 > NumPy is installed in > /home/nwagner/local/lib64/python2.6/site-packages/numpy > SciPy version 0.8.0.dev5964 > Do you use svn to check-out the code? For some reason I never get dev revisions appear in the listing NumPy version 1.4.0.dev NumPy is installed in /home/gsever/Desktop/python-repo/numpy/numpy SciPy version 0.8.0.dev Could this be due to my installation method? I do python setupegg.py develop > SciPy is installed in > /home/nwagner/local/lib64/python2.6/site-packages/scipy > Python version 2.6 (r26:66714, Feb 3 2009, 20:49:49) [GCC > 4.3.2 [gcc-4_3-branch revision 141291]] > nose version 0.10.4 > > Program received signal SIGSEGV, Segmentation fault. > PyArray_NeighborhoodIterNew (x=0x0, bounds=0x7ffff7ac2330, > mode=-161985568, fill=0x7ffff7ac2330) > at numpy/core/src/multiarray/iterators.c:1935 > 1935 array_iter_base_init((PyArrayIterObject*)ret, > x->ao); > (gdb) bt > #0 PyArray_NeighborhoodIterNew (x=0x0, > bounds=0x7ffff7ac2330, mode=-161985568, > fill=0x7ffff7ac2330) > at numpy/core/src/multiarray/iterators.c:1935 > #1 0x00007fffee5d7e98 in init_path () > at > > /home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/__multiarray_api.h:1206 > > > Nils > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -- G?khan -------------- next part -------------- An HTML attachment was scrubbed... URL: From nwagner at iam.uni-stuttgart.de Thu Oct 8 14:48:59 2009 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Thu, 08 Oct 2009 20:48:59 +0200 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: References: Message-ID: On Thu, 8 Oct 2009 12:29:48 -0600 Charles R Harris wrote: > On Thu, Oct 8, 2009 at 12:10 PM, Nils Wagner > wrote: > >> Hi, >> >> Is this a known issue ? >> >> > Maybe. What release of numpy was scipy compiled against? I am using the latest svn version of numpy and scipy. >>> import numpy >>> numpy.__version__ '1.4.0.dev7507' >>> import scipy >>> scipy.__version__ '0.8.0.dev5964' Nils >If it was after the > datetime commit but before David's fix, then it is the >API ordering problem. > > > > Chuck From peridot.faceted at gmail.com Thu Oct 8 15:03:08 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Thu, 8 Oct 2009 15:03:08 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: 2009/10/8 Charles R Harris : > Hi Anne, > > On Thu, Oct 8, 2009 at 9:37 AM, Anne Archibald > wrote: >> >> 2009/10/7 David Goldsmith : >> > Thanks for doing that, Anne! >> >> There is now a rough prototype on github: >> http://github.com/aarchiba/scikits.polynomial >> It certainly needs more tests and features, but it does support both >> the power basis and the Lagrange basis (polynomials represented by >> values at Chebyshev points). >> > > Just took a quick look, which is probably all I'll get to for a few days as > I'm going out of town tomorrow. Anyway, the Chebyshev points there are type > II, which should probably be distinguished from type I (and III & IV). I > also had the impression that the base class could have a few more functions > and NotImplemented bits. The Polynomial class is implemented as a wrapper, > it might even make sense to use multiple inheritance (horrors) to get > specific polynomial types, but anyway it caught my attention and that part > of the design might be worth spending some time thinking about. It also > might be worth distinguishing series as a separate base because series do > admit the division operators //, %, and divmod. Scalar > multiplication/division (__truedivision__) should also be built in. I've > also been using "from __future__ import division" up at the top to be py3k > ready. For a series basis I was thinking of using what I've got for > Chebyshev but with a bunch of the __foo__? functions raising the > NotImplementedError. I've also got a single function for importing the > coefficient arrays and doing the type conversions/checking. It's worth doing > that one way for all the implementations as it makes it easier to fix/extend > things. The polynomial class as a wrapper was a design decision. My reasoning was that certain data - roots, integration schemes, weights for barycentric interpolation, and so on - are associated with the basis rather than any particular polynomial. The various algorithms are also associated with the basis, of course (or rather the family of bases). So that leaves little in the way of code to be attached to the polynomials themselves; basically just adapter code, as you noted. This also allows users to stick to working with plain arrays of coefficients, as with chebint/chebder/etc. if they prefer. But the design is very much open for discussion. I agree, there are some good reasons to implement a class for graded polynomial bases in which the ith polynomial has degree i. One would presumably implement a further class for polynomial bases based on orthogonal families specified in terms of recurrence relations. Division operators make sense to implement, yes; there are sensible notions of division even for polynomials in the Lagrange or Bernstein bases. I just hadn't included those functions yet. > I've attached the low->high version of the chebyshev.py file just for > further reference. The Chebyshev class is at the end. Thanks, I'll take a look at it. Anne From charlesr.harris at gmail.com Thu Oct 8 15:03:50 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 8 Oct 2009 13:03:50 -0600 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: References: Message-ID: On Thu, Oct 8, 2009 at 12:48 PM, Nils Wagner wrote: > On Thu, 8 Oct 2009 12:29:48 -0600 > Charles R Harris wrote: > > On Thu, Oct 8, 2009 at 12:10 PM, Nils Wagner > > wrote: > > > >> Hi, > >> > >> Is this a known issue ? > >> > >> > > Maybe. What release of numpy was scipy compiled against? > > I am using the latest svn version of numpy and scipy. > >>> import numpy > >>> numpy.__version__ > '1.4.0.dev7507' > >>> import scipy > >>> scipy.__version__ > '0.8.0.dev5964' > > Did you delete the build directories? Did you compile numpy first, then scipy? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From nwagner at iam.uni-stuttgart.de Thu Oct 8 15:04:54 2009 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Thu, 08 Oct 2009 21:04:54 +0200 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: References: Message-ID: On Thu, 8 Oct 2009 13:03:50 -0600 Charles R Harris wrote: > On Thu, Oct 8, 2009 at 12:48 PM, Nils Wagner > wrote: > >> On Thu, 8 Oct 2009 12:29:48 -0600 >> Charles R Harris wrote: >> > On Thu, Oct 8, 2009 at 12:10 PM, Nils Wagner >> > wrote: >> > >> >> Hi, >> >> >> >> Is this a known issue ? >> >> >> >> >> > Maybe. What release of numpy was scipy compiled >>against? >> >> I am using the latest svn version of numpy and scipy. >> >>> import numpy >> >>> numpy.__version__ >> '1.4.0.dev7507' >> >>> import scipy >> >>> scipy.__version__ >> '0.8.0.dev5964' >> >> > Did you delete the build directories? Did you compile >numpy first, then > scipy? > Yes and yes. Nils > Chuck From peridot.faceted at gmail.com Thu Oct 8 15:06:12 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Thu, 8 Oct 2009 15:06:12 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: 2009/10/8 Charles R Harris : > > > On Thu, Oct 8, 2009 at 10:29 AM, Charles R Harris > wrote: >> >> Hi Anne, >> >> On Thu, Oct 8, 2009 at 9:37 AM, Anne Archibald >> wrote: >>> >>> 2009/10/7 David Goldsmith : >>> > Thanks for doing that, Anne! >>> >>> There is now a rough prototype on github: >>> http://github.com/aarchiba/scikits.polynomial >>> It certainly needs more tests and features, but it does support both >>> the power basis and the Lagrange basis (polynomials represented by >>> values at Chebyshev points). >>> >> >> Just took a quick look, which is probably all I'll get to for a few days >> as I'm going out of town tomorrow. Anyway, the Chebyshev points there are >> type II, which should probably be distinguished from type I (and III & IV). >> I also had the impression that the base class could have a few more >> functions and NotImplemented bits. The Polynomial class is implemented as a >> wrapper, it might even make sense to use multiple inheritance (horrors) to >> get specific polynomial types, but anyway it caught my attention and that >> part of the design might be worth spending some time thinking about. It also >> might be worth distinguishing series as a separate base because series do >> admit the division operators //, %, and divmod. Scalar >> multiplication/division (__truedivision__) should also be built in. I've >> also been using "from __future__ import division" up at the top to be py3k >> ready. For a series basis I was thinking of using what I've got for >> Chebyshev but with a bunch of the __foo__? functions raising the >> NotImplementedError. I've also got a single function for importing the >> coefficient arrays and doing the type conversions/checking. It's worth doing >> that one way for all the implementations as it makes it easier to fix/extend >> things. >> >> I've attached the low->high version of the chebyshev.py file just for >> further reference. The Chebyshev class is at the end. >> > > Oh, and one other thing. I'm thinking that there should be *no* true inplace > functions. That is, every operation returns a new object. That way the > various classes look like basic, immutable, types which makes life simpler > and less error prone. No views with side effects to worry about, and the > coefficient arrays are likely to be small enough that the overhead to make > copies will be small. I agree; both polynomials and bases should be apparently immutable. I say apparently because a certain amount of caching/precalculation can speed some things up; I've already got this in the LagrangeBasis class, where it caches the Chebyshev points it uses. Anne From charlesr.harris at gmail.com Thu Oct 8 15:18:49 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 8 Oct 2009 13:18:49 -0600 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: References: Message-ID: On Thu, Oct 8, 2009 at 1:04 PM, Nils Wagner wrote: > On Thu, 8 Oct 2009 13:03:50 -0600 > Charles R Harris wrote: > > On Thu, Oct 8, 2009 at 12:48 PM, Nils Wagner > > wrote: > > > >> On Thu, 8 Oct 2009 12:29:48 -0600 > >> Charles R Harris wrote: > >> > On Thu, Oct 8, 2009 at 12:10 PM, Nils Wagner > >> > wrote: > >> > > >> >> Hi, > >> >> > >> >> Is this a known issue ? > >> >> > >> >> > >> > Maybe. What release of numpy was scipy compiled > >>against? > >> > >> I am using the latest svn version of numpy and scipy. > >> >>> import numpy > >> >>> numpy.__version__ > >> '1.4.0.dev7507' > >> >>> import scipy > >> >>> scipy.__version__ > >> '0.8.0.dev5964' > >> > >> > > Did you delete the build directories? Did you compile > >numpy first, then > > scipy? > > > Yes and yes. > > OK, I'm seeing this also. It may be related to David's change to NeighborhoodIter in numpy r7452. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From nwagner at iam.uni-stuttgart.de Thu Oct 8 15:27:05 2009 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Thu, 08 Oct 2009 21:27:05 +0200 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: References: Message-ID: On Thu, 8 Oct 2009 13:18:49 -0600 Charles R Harris wrote: > On Thu, Oct 8, 2009 at 1:04 PM, Nils Wagner >wrote: > >> On Thu, 8 Oct 2009 13:03:50 -0600 >> Charles R Harris wrote: >> > On Thu, Oct 8, 2009 at 12:48 PM, Nils Wagner >> > wrote: >> > >> >> On Thu, 8 Oct 2009 12:29:48 -0600 >> >> Charles R Harris wrote: >> >> > On Thu, Oct 8, 2009 at 12:10 PM, Nils Wagner >> >> > wrote: >> >> > >> >> >> Hi, >> >> >> >> >> >> Is this a known issue ? >> >> >> >> >> >> >> >> > Maybe. What release of numpy was scipy compiled >> >>against? >> >> >> >> I am using the latest svn version of numpy and scipy. >> >> >>> import numpy >> >> >>> numpy.__version__ >> >> '1.4.0.dev7507' >> >> >>> import scipy >> >> >>> scipy.__version__ >> >> '0.8.0.dev5964' >> >> >> >> >> > Did you delete the build directories? Did you compile >> >numpy first, then >> > scipy? >> > >> Yes and yes. >> >> > OK, I'm seeing this also. It may be related to David's >change to > NeighborhoodIter in numpy r7452. > > Chuck I have also removed everything under /home/nwagner/local/lib64/python2.6/site-packages/numpy /home/nwagner/local/lib64/python2.6/site-packages/scipy Now scipy.test() passed. Ran 3866 tests in 85.541s FAILED (KNOWNFAIL=6, SKIP=17, failures=7) Nils From charlesr.harris at gmail.com Thu Oct 8 15:33:07 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 8 Oct 2009 13:33:07 -0600 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: References: Message-ID: On Thu, Oct 8, 2009 at 1:27 PM, Nils Wagner wrote: > On Thu, 8 Oct 2009 13:18:49 -0600 > Charles R Harris wrote: > > On Thu, Oct 8, 2009 at 1:04 PM, Nils Wagner > >wrote: > > > >> On Thu, 8 Oct 2009 13:03:50 -0600 > >> Charles R Harris wrote: > >> > On Thu, Oct 8, 2009 at 12:48 PM, Nils Wagner > >> > wrote: > >> > > >> >> On Thu, 8 Oct 2009 12:29:48 -0600 > >> >> Charles R Harris wrote: > >> >> > On Thu, Oct 8, 2009 at 12:10 PM, Nils Wagner > >> >> > wrote: > >> >> > > >> >> >> Hi, > >> >> >> > >> >> >> Is this a known issue ? > >> >> >> > >> >> >> > >> >> > Maybe. What release of numpy was scipy compiled > >> >>against? > >> >> > >> >> I am using the latest svn version of numpy and scipy. > >> >> >>> import numpy > >> >> >>> numpy.__version__ > >> >> '1.4.0.dev7507' > >> >> >>> import scipy > >> >> >>> scipy.__version__ > >> >> '0.8.0.dev5964' > >> >> > >> >> > >> > Did you delete the build directories? Did you compile > >> >numpy first, then > >> > scipy? > >> > > >> Yes and yes. > >> > >> > > OK, I'm seeing this also. It may be related to David's > >change to > > NeighborhoodIter in numpy r7452. > > > > Chuck > > I have also removed everything under > > /home/nwagner/local/lib64/python2.6/site-packages/numpy > /home/nwagner/local/lib64/python2.6/site-packages/scipy > > So did I, but I forgot to sudo the removals and nothing actually happened O_o . Things now work here too. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From gokhansever at gmail.com Thu Oct 8 21:03:06 2009 From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=) Date: Thu, 8 Oct 2009 20:03:06 -0500 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: <49d6b3500910081144y13ae6faak29a09257b29005b4@mail.gmail.com> References: <49d6b3500910081144y13ae6faak29a09257b29005b4@mail.gmail.com> Message-ID: <49d6b3500910081803h42d14d6ctde105a487613870a@mail.gmail.com> Sorry for interruption. svn info lists the information that I seek which learnt from setup.py file in numpy main directory. [gsever at ccn numpy]$ svn info Path: . URL: http://svn.scipy.org/svn/numpy/trunk Repository Root: http://svn.scipy.org/svn/numpy Repository UUID: 94b884b6-d6fd-0310-90d3-974f1d3f35e1 Revision: 7452 Node Kind: directory Schedule: normal Last Changed Author: cdavid Last Changed Rev: 7452 Last Changed Date: 2009-10-06 01:44:35 -0500 (Tue, 06 Oct 2009) which "python setupegg.py develop" build-installation doesn't propagate revision information. Probably not a big deal, comparing to segfaults, but just noting :) On Thu, Oct 8, 2009 at 1:44 PM, G?khan Sever wrote: > > > On Thu, Oct 8, 2009 at 1:10 PM, Nils Wagner wrote: > >> Hi, >> >> Is this a known issue ? >> >> >>> import scipy >> >>> scipy.test() >> Running unit tests for scipy >> NumPy version 1.4.0.dev7507 >> NumPy is installed in >> /home/nwagner/local/lib64/python2.6/site-packages/numpy >> SciPy version 0.8.0.dev5964 >> > > Do you use svn to check-out the code? For some reason I never get dev > revisions appear in the listing > > NumPy version 1.4.0.dev > NumPy is installed in /home/gsever/Desktop/python-repo/numpy/numpy > SciPy version 0.8.0.dev > > Could this be due to my installation method? > > I do python setupegg.py develop > > > >> SciPy is installed in >> /home/nwagner/local/lib64/python2.6/site-packages/scipy >> Python version 2.6 (r26:66714, Feb 3 2009, 20:49:49) [GCC >> 4.3.2 [gcc-4_3-branch revision 141291]] >> nose version 0.10.4 >> >> Program received signal SIGSEGV, Segmentation fault. >> PyArray_NeighborhoodIterNew (x=0x0, bounds=0x7ffff7ac2330, >> mode=-161985568, fill=0x7ffff7ac2330) >> at numpy/core/src/multiarray/iterators.c:1935 >> 1935 array_iter_base_init((PyArrayIterObject*)ret, >> x->ao); >> (gdb) bt >> #0 PyArray_NeighborhoodIterNew (x=0x0, >> bounds=0x7ffff7ac2330, mode=-161985568, >> fill=0x7ffff7ac2330) >> at numpy/core/src/multiarray/iterators.c:1935 >> #1 0x00007fffee5d7e98 in init_path () >> at >> >> /home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/__multiarray_api.h:1206 >> >> >> Nils >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > > > > -- > G?khan > -- G?khan -------------- next part -------------- An HTML attachment was scrubbed... URL: From gokhansever at gmail.com Thu Oct 8 21:24:20 2009 From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=) Date: Thu, 8 Oct 2009 20:24:20 -0500 Subject: [SciPy-dev] Numpy_Example_List_With_Doc page Message-ID: <49d6b3500910081824g181f20dcwf6fd1e62dca787f8@mail.gmail.com> Hello, http://www.scipy.org/Numpy_Example_List_With_Doc This page doesn't tell anything different than the original numpy documentation. Also causes confusion in my opinion because of this statement: "To update this page just follow the instructions. " Has it left intentionally or forgotten? -- G?khan -------------- next part -------------- An HTML attachment was scrubbed... URL: From cournape at gmail.com Thu Oct 8 22:42:05 2009 From: cournape at gmail.com (David Cournapeau) Date: Fri, 9 Oct 2009 11:42:05 +0900 Subject: [SciPy-dev] scipy.test() segfaults In-Reply-To: <49d6b3500910081803h42d14d6ctde105a487613870a@mail.gmail.com> References: <49d6b3500910081144y13ae6faak29a09257b29005b4@mail.gmail.com> <49d6b3500910081803h42d14d6ctde105a487613870a@mail.gmail.com> Message-ID: <5b8d13220910081942v48993d32je31b16b2f4a7f6b8@mail.gmail.com> On Fri, Oct 9, 2009 at 10:03 AM, G?khan Sever wrote: > which "python setupegg.py develop" build-installation doesn't propagate > revision information. did you rebuild numpy after the update ? David From pearu.peterson at gmail.com Fri Oct 9 02:49:02 2009 From: pearu.peterson at gmail.com (Pearu Peterson) Date: Fri, 09 Oct 2009 09:49:02 +0300 Subject: [SciPy-dev] numpy.distutils: passing config_fc options in setup.py In-Reply-To: <4ACDB156.6040902@aero.iitb.ac.in> References: <4AC81C3F.5010700@aero.iitb.ac.in> <4AC9C888.90002@cens.ioc.ee> <4ACDB156.6040902@aero.iitb.ac.in> Message-ID: <4ACEDCDE.6020502@cens.ioc.ee> Prabhu Ramachandran wrote: > On Monday 05 October 2009 03:50 PM, Pearu Peterson wrote: >>> passing config_fc={...} to either Extension or setup does not work. >>> The numpy version is 1.4.0.dev6976. >> >> A hackish solution would be to insert the following code to the >> beginning of the setup.py file: >> >> import sys >> sys.argv[:] = sys.argv[:1] + ['config_fc'. "--f90flags='-ffree-form >> -fno-underscoring'"] + sys.argv[1:] >> ) > > Thanks for the suggestion. I was hoping for a cleaner solution but this > will work too. I just recalled that there is another solution. See http://projects.scipy.org/numpy/wiki/DistutilsDoc#specifing-config-fc-options-for-libraries-in-setup-py-script HTH, Pearu From scott.sinclair.za at gmail.com Fri Oct 9 02:57:41 2009 From: scott.sinclair.za at gmail.com (Scott Sinclair) Date: Fri, 9 Oct 2009 08:57:41 +0200 Subject: [SciPy-dev] Numpy_Example_List_With_Doc page In-Reply-To: <49d6b3500910081824g181f20dcwf6fd1e62dca787f8@mail.gmail.com> References: <49d6b3500910081824g181f20dcwf6fd1e62dca787f8@mail.gmail.com> Message-ID: <6a17e9ee0910082357m24e9b53duccf93e6e1a270c1c@mail.gmail.com> >2009/10/9 G?khan Sever : > Hello, > > http://www.scipy.org/Numpy_Example_List_With_Doc > > This page doesn't tell anything different than the original numpy > documentation. Also causes confusion in my opinion because of this > statement: "To update this page just follow the instructions. " > > Has it left intentionally or forgotten? This page is from a time before the current NumPy docs (and the doc editor) existed. I think that some people still find it useful, so perhaps the correct answer to your question is "intentionally forgotten" ? Of course, the first place to look for docs should definitely be http://docs.scipy.org. Cheers, Scott From charlesr.harris at gmail.com Fri Oct 9 04:01:39 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 9 Oct 2009 02:01:39 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: On Thu, Oct 8, 2009 at 1:03 PM, Anne Archibald wrote: > 2009/10/8 Charles R Harris : > > Hi Anne, > > > > On Thu, Oct 8, 2009 at 9:37 AM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/7 David Goldsmith : > >> > Thanks for doing that, Anne! > >> > >> There is now a rough prototype on github: > >> http://github.com/aarchiba/scikits.polynomial > >> It certainly needs more tests and features, but it does support both > >> the power basis and the Lagrange basis (polynomials represented by > >> values at Chebyshev points). > >> > > > > Just took a quick look, which is probably all I'll get to for a few days > as > > I'm going out of town tomorrow. Anyway, the Chebyshev points there are > type > > II, which should probably be distinguished from type I (and III & IV). I > > also had the impression that the base class could have a few more > functions > > and NotImplemented bits. The Polynomial class is implemented as a > wrapper, > > it might even make sense to use multiple inheritance (horrors) to get > > specific polynomial types, but anyway it caught my attention and that > part > > of the design might be worth spending some time thinking about. It also > > might be worth distinguishing series as a separate base because series do > > admit the division operators //, %, and divmod. Scalar > > multiplication/division (__truedivision__) should also be built in. I've > > also been using "from __future__ import division" up at the top to be > py3k > > ready. For a series basis I was thinking of using what I've got for > > Chebyshev but with a bunch of the __foo__ functions raising the > > NotImplementedError. I've also got a single function for importing the > > coefficient arrays and doing the type conversions/checking. It's worth > doing > > that one way for all the implementations as it makes it easier to > fix/extend > > things. > > The polynomial class as a wrapper was a design decision. My reasoning > was that certain data - roots, integration schemes, weights for > barycentric interpolation, and so on - are associated with the basis > rather than any particular polynomial. The various algorithms are also > associated with the basis, of course (or rather the family of bases). > So that leaves little in the way of code to be attached to the > polynomials themselves; basically just adapter code, as you noted. > This also allows users to stick to working with plain arrays of > coefficients, as with chebint/chebder/etc. if they prefer. But the > design is very much open for discussion. > > I agree, there are some good reasons to implement a class for graded > polynomial bases in which the ith polynomial has degree i. One would > presumably implement a further class for polynomial bases based on > orthogonal families specified in terms of recurrence relations. > > Division operators make sense to implement, yes; there are sensible > notions of division even for polynomials in the Lagrange or Bernstein > bases. I just hadn't included those functions yet. > > > I've attached the low->high version of the chebyshev.py file just for > > further reference. The Chebyshev class is at the end. > > Thanks, I'll take a look at it. > > I'm thinking that instead of a wrapper class what we want is essentially a class template replicated with different values for, i.e., multiplication, division, etc. One way to do that in python is with a class factory. For example In [3]: def cat() : print 'meow' ...: In [4]: def dog() : print 'woof' ...: In [31]: def pet_factory(f) : ....: class pet : ....: def talk(self) : ....: print f() ....: return pet ....: In [32]: DogClass = pet_factory(dog) In [33]: mydog = DogClass() In [34]: mydog.talk() woof None In [35]: CatClass = pet_factory(cat) In [36]: mycat = CatClass() In [37]: mycat.talk() meow None I'm not sure why the None return is printing, but that is the general idea. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.walter at gmail.com Fri Oct 9 04:28:37 2009 From: sebastian.walter at gmail.com (Sebastian Walter) Date: Fri, 9 Oct 2009 10:28:37 +0200 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: def cat() : print 'meow' should be def cat(): return 'meow' that should get rid of the None On Fri, Oct 9, 2009 at 10:01 AM, Charles R Harris wrote: > > > On Thu, Oct 8, 2009 at 1:03 PM, Anne Archibald > wrote: >> >> 2009/10/8 Charles R Harris : >> > Hi Anne, >> > >> > On Thu, Oct 8, 2009 at 9:37 AM, Anne Archibald >> > >> > wrote: >> >> >> >> 2009/10/7 David Goldsmith : >> >> > Thanks for doing that, Anne! >> >> >> >> There is now a rough prototype on github: >> >> http://github.com/aarchiba/scikits.polynomial >> >> It certainly needs more tests and features, but it does support both >> >> the power basis and the Lagrange basis (polynomials represented by >> >> values at Chebyshev points). >> >> >> > >> > Just took a quick look, which is probably all I'll get to for a few days >> > as >> > I'm going out of town tomorrow. Anyway, the Chebyshev points there are >> > type >> > II, which should probably be distinguished from type I (and III & IV). I >> > also had the impression that the base class could have a few more >> > functions >> > and NotImplemented bits. The Polynomial class is implemented as a >> > wrapper, >> > it might even make sense to use multiple inheritance (horrors) to get >> > specific polynomial types, but anyway it caught my attention and that >> > part >> > of the design might be worth spending some time thinking about. It also >> > might be worth distinguishing series as a separate base because series >> > do >> > admit the division operators //, %, and divmod. Scalar >> > multiplication/division (__truedivision__) should also be built in. I've >> > also been using "from __future__ import division" up at the top to be >> > py3k >> > ready. For a series basis I was thinking of using what I've got for >> > Chebyshev but with a bunch of the __foo__ functions raising the >> > NotImplementedError. I've also got a single function for importing the >> > coefficient arrays and doing the type conversions/checking. It's worth >> > doing >> > that one way for all the implementations as it makes it easier to >> > fix/extend >> > things. >> >> The polynomial class as a wrapper was a design decision. My reasoning >> was that certain data - roots, integration schemes, weights for >> barycentric interpolation, and so on - are associated with the basis >> rather than any particular polynomial. The various algorithms are also >> associated with the basis, of course (or rather the family of bases). >> So that leaves little in the way of code to be attached to the >> polynomials themselves; basically just adapter code, as you noted. >> This also allows users to stick to working with plain arrays of >> coefficients, as with chebint/chebder/etc. if they prefer. But the >> design is very much open for discussion. >> >> I agree, there are some good reasons to implement a class for graded >> polynomial bases in which the ith polynomial has degree i. One would >> presumably implement a further class for polynomial bases based on >> orthogonal families specified in terms of recurrence relations. >> >> Division operators make sense to implement, yes; there are sensible >> notions of division even for polynomials in the Lagrange or Bernstein >> bases. I just hadn't included those functions yet. >> >> > I've attached the low->high version of the chebyshev.py file just for >> > further reference. The Chebyshev class is at the end. >> >> Thanks, I'll take a look at it. >> > > I'm thinking that instead of a wrapper class what we want is essentially a > class template replicated with different values for, i.e., multiplication, > division, etc. One way to do that in python is with a class factory. For > example > > In [3]: def cat() : print 'meow' > ...: > > In [4]: def dog() : print 'woof' > ...: > > In [31]: def pet_factory(f) : > ....: class pet : > ....: def talk(self) : > ....: print f() > ....: return pet > ....: > > In [32]: DogClass = pet_factory(dog) > > In [33]: mydog = DogClass() > > In [34]: mydog.talk() > woof > None > > In [35]: CatClass = pet_factory(cat) > > In [36]: mycat = CatClass() > > In [37]: mycat.talk() > meow > None > > I'm not sure why the None return is printing, but that is the general idea. > > Chuck > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From d.l.goldsmith at gmail.com Fri Oct 9 12:41:50 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Fri, 9 Oct 2009 09:41:50 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: <45d1ab480910090941l72f432b4n8dafe39c99ddd975@mail.gmail.com> On Fri, Oct 9, 2009 at 1:28 AM, Sebastian Walter wrote: > def cat() : print 'meow' > should be > def cat(): return 'meow' > > that should get rid of the None > And return a really loud meow (as the cat() is def, it can't hear how loud it is and overcompensates). ;-) DG > > > On Fri, Oct 9, 2009 at 10:01 AM, Charles R Harris > wrote: > > > > > > On Thu, Oct 8, 2009 at 1:03 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/8 Charles R Harris : > >> > Hi Anne, > >> > > >> > On Thu, Oct 8, 2009 at 9:37 AM, Anne Archibald > >> > > >> > wrote: > >> >> > >> >> 2009/10/7 David Goldsmith : > >> >> > Thanks for doing that, Anne! > >> >> > >> >> There is now a rough prototype on github: > >> >> http://github.com/aarchiba/scikits.polynomial > >> >> It certainly needs more tests and features, but it does support both > >> >> the power basis and the Lagrange basis (polynomials represented by > >> >> values at Chebyshev points). > >> >> > >> > > >> > Just took a quick look, which is probably all I'll get to for a few > days > >> > as > >> > I'm going out of town tomorrow. Anyway, the Chebyshev points there are > >> > type > >> > II, which should probably be distinguished from type I (and III & IV). > I > >> > also had the impression that the base class could have a few more > >> > functions > >> > and NotImplemented bits. The Polynomial class is implemented as a > >> > wrapper, > >> > it might even make sense to use multiple inheritance (horrors) to get > >> > specific polynomial types, but anyway it caught my attention and that > >> > part > >> > of the design might be worth spending some time thinking about. It > also > >> > might be worth distinguishing series as a separate base because series > >> > do > >> > admit the division operators //, %, and divmod. Scalar > >> > multiplication/division (__truedivision__) should also be built in. > I've > >> > also been using "from __future__ import division" up at the top to be > >> > py3k > >> > ready. For a series basis I was thinking of using what I've got for > >> > Chebyshev but with a bunch of the __foo__ functions raising the > >> > NotImplementedError. I've also got a single function for importing the > >> > coefficient arrays and doing the type conversions/checking. It's worth > >> > doing > >> > that one way for all the implementations as it makes it easier to > >> > fix/extend > >> > things. > >> > >> The polynomial class as a wrapper was a design decision. My reasoning > >> was that certain data - roots, integration schemes, weights for > >> barycentric interpolation, and so on - are associated with the basis > >> rather than any particular polynomial. The various algorithms are also > >> associated with the basis, of course (or rather the family of bases). > >> So that leaves little in the way of code to be attached to the > >> polynomials themselves; basically just adapter code, as you noted. > >> This also allows users to stick to working with plain arrays of > >> coefficients, as with chebint/chebder/etc. if they prefer. But the > >> design is very much open for discussion. > >> > >> I agree, there are some good reasons to implement a class for graded > >> polynomial bases in which the ith polynomial has degree i. One would > >> presumably implement a further class for polynomial bases based on > >> orthogonal families specified in terms of recurrence relations. > >> > >> Division operators make sense to implement, yes; there are sensible > >> notions of division even for polynomials in the Lagrange or Bernstein > >> bases. I just hadn't included those functions yet. > >> > >> > I've attached the low->high version of the chebyshev.py file just for > >> > further reference. The Chebyshev class is at the end. > >> > >> Thanks, I'll take a look at it. > >> > > > > I'm thinking that instead of a wrapper class what we want is essentially > a > > class template replicated with different values for, i.e., > multiplication, > > division, etc. One way to do that in python is with a class factory. For > > example > > > > In [3]: def cat() : print 'meow' > > ...: > > > > In [4]: def dog() : print 'woof' > > ...: > > > > In [31]: def pet_factory(f) : > > ....: class pet : > > ....: def talk(self) : > > ....: print f() > > ....: return pet > > ....: > > > > In [32]: DogClass = pet_factory(dog) > > > > In [33]: mydog = DogClass() > > > > In [34]: mydog.talk() > > woof > > None > > > > In [35]: CatClass = pet_factory(cat) > > > > In [36]: mycat = CatClass() > > > > In [37]: mycat.talk() > > meow > > None > > > > I'm not sure why the None return is printing, but that is the general > idea. > > > > Chuck > > > > > > _______________________________________________ > > Scipy-dev mailing list > > Scipy-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Fri Oct 9 13:08:40 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Fri, 9 Oct 2009 13:08:40 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: 2009/10/9 Charles R Harris : > > I'm thinking that instead of a wrapper class what we want is essentially a > class template replicated with different values for, i.e., multiplication, > division, etc. One way to do that in python is with a class factory. For > example > I see that this is possible, but I'm not sure I see why it's a good idea. We need basis objects, since even apart from any code, bases can contain substantial amounts of information. For example, if someone wants to work with the Lagrange interpolating polynomial on a set of points, the natural basis is the Lagrange basis on those points, so the polynomial must remember the list of points. Storing that in every polynomial is inefficient, not just in terms of space but also because basis comparisons - necessary for type-safety on every operation - would have to actually go out and compare all the points. So every polynomial needs to contain (a reference to) the basis it's defined on, and there must be a type hierarchy of bases. But if that type hierarchy exists, why have a hierarchy of polynomials? After all, the algorithms defined depend on the basis, rather than any feature of the individual polynomial, and a polynomial is completely defined by its coefficients and the basis. If you're concerned that the polynomial class exists without any real code inside it, well, that's a bit awkward, I agree. But if we have basis objects and polynomial objects, the code has to go in one or the other, and it seems more natural to put it in the basis objects. In a way this is similar to the design of your Chebyshev class - you have a collection of functions that act on bare coefficient arrays (in my design they're in a class, in yours they're module functions) and a wrapper class whose code is only there to provide an easy-to-use object-oriented interface. And the wrapper class is not completely trivial - it handles checking basis equality (in principle it could check compatibility, if that turns out to be useful), scalar multiplication, and an operator-friendly interface. In object-oriented buzzword-speak, rather than using (possibly multiple) inheritance to provide features using an is-a relationship, I'm using inclusion to provide a has-a relationship. Code reuse through inheritance will apply at the level of bases, rather than at the level of individual polynomials. Anne From charlesr.harris at gmail.com Fri Oct 9 14:24:56 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 9 Oct 2009 12:24:56 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: On Fri, Oct 9, 2009 at 11:08 AM, Anne Archibald wrote: > 2009/10/9 Charles R Harris : > > > > I'm thinking that instead of a wrapper class what we want is essentially > a > > class template replicated with different values for, i.e., > multiplication, > > division, etc. One way to do that in python is with a class factory. For > > example > > > > I see that this is possible, but I'm not sure I see why it's a good idea. > > We need basis objects, since even apart from any code, bases can > contain substantial amounts of information. For example, if someone > wants to work with the Lagrange interpolating polynomial on a set of > points, the natural basis is the Lagrange basis on those points, so > the polynomial must remember the list of points. Storing that in every > polynomial is inefficient, not just in terms of space but also because > basis comparisons - necessary for type-safety on every operation - > would have to actually go out and compare all the points. > > So every polynomial needs to contain (a reference to) the basis it's > defined on, and there must be a type hierarchy of bases. But if that > type hierarchy exists, why have a hierarchy of polynomials? After all, > the algorithms defined depend on the basis, rather than any feature of > the individual polynomial, and a polynomial is completely defined by > its coefficients and the basis. > > If you're concerned that the polynomial class exists without any real > code inside it, well, that's a bit awkward, I agree. But if we have > basis objects and polynomial objects, the code has to go in one or the > other, and it seems more natural to put it in the basis objects. In a > way this is similar to the design of your Chebyshev class - you have a > collection of functions that act on bare coefficient arrays (in my > design they're in a class, in yours they're module functions) and a > wrapper class whose code is only there to provide an easy-to-use > object-oriented interface. And the wrapper class is not completely > trivial - it handles checking basis equality (in principle it could > check compatibility, if that turns out to be useful), scalar > multiplication, and an operator-friendly interface. > > In object-oriented buzzword-speak, rather than using (possibly > multiple) inheritance to provide features using an is-a relationship, > I'm using inclusion to provide a has-a relationship. Code reuse > through inheritance will apply at the level of bases, rather than at > the level of individual polynomials. > brcuase > It's not inheritence, it's a class templaate. Because classes are objects in python you can return them from functions.with a defined enviroment. Another way to look at it is completions. A basis would just be another passed parameter. and the class code would still only heed to be written once for all the different types of polynnomial basis.. I think it is a cleaner way to go. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Fri Oct 9 15:35:00 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Fri, 9 Oct 2009 15:35:00 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: 2009/10/9 Charles R Harris : > > > On Fri, Oct 9, 2009 at 11:08 AM, Anne Archibald > wrote: >> >> 2009/10/9 Charles R Harris : >> > >> > I'm thinking that instead of a wrapper class what we want is essentially >> > a >> > class template replicated with different values for, i.e., >> > multiplication, >> > division, etc. One way to do that in python is with a class factory. For >> > example >> > >> >> I see that this is possible, but I'm not sure I see why it's a good idea. >> >> We need basis objects, since even apart from any code, bases can >> contain substantial amounts of information. For example, if someone >> wants to work with the Lagrange interpolating polynomial on a set of >> points, the natural basis is the Lagrange basis on those points, so >> the polynomial must remember the list of points. Storing that in every >> polynomial is inefficient, not just in terms of space but also because >> basis comparisons - necessary for type-safety on every operation - >> would have to actually go out and compare all the points. >> >> So every polynomial needs to contain (a reference to) the basis it's >> defined on, and there must be a type hierarchy of bases. But if that >> type hierarchy exists, why have a hierarchy of polynomials? After all, >> the algorithms defined depend on the basis, rather than any feature of >> the individual polynomial, and a polynomial is completely defined by >> its coefficients and the basis. >> >> If you're concerned that the polynomial class exists without any real >> code inside it, well, that's a bit awkward, I agree. But if we have >> basis objects and polynomial objects, the code has to go in one or the >> other, and it seems more natural to put it in the basis objects. In a >> way this is similar to the design of your Chebyshev class - you have a >> collection of functions that act on bare coefficient arrays (in my >> design they're in a class, in yours they're module functions) and a >> wrapper class whose code is only there to provide an easy-to-use >> object-oriented interface. And the wrapper class is not completely >> trivial - it handles checking basis equality (in principle it could >> check compatibility, if that turns out to be useful), scalar >> multiplication, and an operator-friendly interface. >> >> In object-oriented buzzword-speak, rather than using (possibly >> multiple) inheritance to provide features using an is-a relationship, >> I'm using inclusion to provide a has-a relationship. Code reuse >> through inheritance will apply at the level of bases, rather than at >> the level of individual polynomials. >> brcuase > > > It's not inheritence, it's a class templaate.? Because classes are objects > in python you can return them from functions.with a defined enviroment. > Another way to look at it is completions. A basis would just be another > passed parameter. and the class code would still only heed to be written > once for all the different types of polynnomial basis.. I think it is a > cleaner way to go. I don't think I understand what you're getting at, then. In your design, where would the code to implement, say, evaluation of polynomials in the Lagrange basis go? In the Lagrange basis object? In the polynomial factory function? In separate toplevel functions? Maybe it would help to write a mockup, with just one or two functions, based on the scikits.polynomial code? If you'd like I can add you as a collaborator on github. (I'm a little new to git, so please bear with me as I figure out how I'm supposed to do things.) Anne From mdroe at stsci.edu Mon Oct 12 11:40:47 2009 From: mdroe at stsci.edu (Michael Droettboom) Date: Mon, 12 Oct 2009 11:40:47 -0400 Subject: [SciPy-dev] chararray docstrings Message-ID: <4AD34DFF.6000809@stsci.edu> I was able to make my big chararray commit today. If I understand correctly, I need to wait 24 hours for the doc editor to sync with SVN, and then I should mark all the chararray-related docstrings as "needs review". The primary change to the docstrings is that all of the methods of the chararray class are now free functions. These free functions represent the "primary" entry points, and thus have detailed documentation, and the chararray methods now have short "pointer" docstrings to the free functions. Where the docstring content itself has been updated, it is mainly to bring them closer to the Python standard library descriptions of these functions, which in most cases was more precise (since we are, in fact, calling the stdlib function under the hood) and concise (because the stdlib docs have been through a number of revisions and really get it right by now). I do have a concern about one phrase that was used in a number of places that probably deserves some discussion: "The chararray module exists for backwards compatibility with Numarray, it is not recommended for new development. If one needs arrays of strings, use arrays of dtype object." There are many use cases (such as handling a binary structured format like FITS) where a dtype of 'string_' is more appropriate than a dtype of 'object_', and we shouldn't imply that all uses of chararray should now use object arrays. Additionally, fast vectorized string operations will perform best on arrays of type 'string_' and 'unicode_', though 'object_' will work, it requires casting all objects to strings along the way, and could fail thousands of items in to an operation. It's a "best tool for the job" judgment call, not a "one tool fits all". Perhaps the above should read: "If one needs arrays of strings, use arrays of dtype string_ or unicode_. If one needs arrays of variable-length strings, use arrays of dtype object_." Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA From ralf.gommers at googlemail.com Mon Oct 12 13:09:18 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Mon, 12 Oct 2009 19:09:18 +0200 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <4AD34DFF.6000809@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> Message-ID: On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom wrote: > I was able to make my big chararray commit today. If I understand > correctly, I need to wait 24 hours for the doc editor to sync with SVN, > and then I should mark all the chararray-related docstrings as "needs > review". > Great! Thanks for all the work. > > The primary change to the docstrings is that all of the methods of the > chararray class are now free functions. These free functions represent > the "primary" entry points, and thus have detailed documentation, and > the chararray methods now have short "pointer" docstrings to the free > functions. > > Where the docstring content itself has been updated, it is mainly to > bring them closer to the Python standard library descriptions of these > functions, which in most cases was more precise (since we are, in fact, > calling the stdlib function under the hood) and concise (because the > stdlib docs have been through a number of revisions and really get it > right by now). > > All sounds very sensible. > I do have a concern about one phrase that was used in a number of places > that probably deserves some discussion: > > "The chararray module exists for backwards compatibility with Numarray, > it is not recommended for new development. If one needs arrays of > strings, use arrays of dtype > object." > > There are many use cases (such as handling a binary structured format > like FITS) where a dtype of 'string_' is more appropriate than a dtype > of 'object_', and we shouldn't imply that all uses of chararray should > now use object arrays. Additionally, fast vectorized string operations > will perform best on arrays of type 'string_' and 'unicode_', though > 'object_' will work, it requires casting all objects to strings along > the way, and could fail thousands of items in to an operation. It's a > "best tool for the job" judgment call, not a "one tool fits all". > I recently added that note (and mentioned that on the list) in three places based on the email discussions some weeks ago. Now that a lot of issues are solved and we are not considering deprecation anymore, this note can be removed again. Perhaps the above should read: > > "If one needs arrays of strings, use arrays of dtype > string_ or > unicode_. If one needs arrays of variable-length strings, use arrays of > dtype object_." > Sounds good. It would still be nice to have a slightly longer (tutorial-style) discussion somewhere in the docs. Or otherwise maybe a link to FITS docs that are illustrative? I still have a hard time understanding why operating on astronomical images with Python string methods is a good idea. Cheers, Ralf > Mike > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Mon Oct 12 13:11:18 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Mon, 12 Oct 2009 10:11:18 -0700 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <4AD34DFF.6000809@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> Message-ID: <45d1ab480910121011y7c5a23bek760e1c938acc672f@mail.gmail.com> All sounds good to me (except, if you're gonna say "if you need arrays of variable length strings, use such and such," be explicit in the previous sentence about the recommendation being for fixed length strings, IMO). DG On Mon, Oct 12, 2009 at 8:40 AM, Michael Droettboom wrote: > I was able to make my big chararray commit today. If I understand > correctly, I need to wait 24 hours for the doc editor to sync with SVN, > and then I should mark all the chararray-related docstrings as "needs > review". > > The primary change to the docstrings is that all of the methods of the > chararray class are now free functions. These free functions represent > the "primary" entry points, and thus have detailed documentation, and > the chararray methods now have short "pointer" docstrings to the free > functions. > > Where the docstring content itself has been updated, it is mainly to > bring them closer to the Python standard library descriptions of these > functions, which in most cases was more precise (since we are, in fact, > calling the stdlib function under the hood) and concise (because the > stdlib docs have been through a number of revisions and really get it > right by now). > > I do have a concern about one phrase that was used in a number of places > that probably deserves some discussion: > > "The chararray module exists for backwards compatibility with Numarray, > it is not recommended for new development. If one needs arrays of > strings, use arrays of dtype > object." > > There are many use cases (such as handling a binary structured format > like FITS) where a dtype of 'string_' is more appropriate than a dtype > of 'object_', and we shouldn't imply that all uses of chararray should > now use object arrays. Additionally, fast vectorized string operations > will perform best on arrays of type 'string_' and 'unicode_', though > 'object_' will work, it requires casting all objects to strings along > the way, and could fail thousands of items in to an operation. It's a > "best tool for the job" judgment call, not a "one tool fits all". > Perhaps the above should read: > > "If one needs arrays of strings, use arrays of dtype > string_ or > unicode_. If one needs arrays of variable-length strings, use arrays of > dtype object_." > > Mike > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pgmdevlist at gmail.com Mon Oct 12 13:26:35 2009 From: pgmdevlist at gmail.com (Pierre GM) Date: Mon, 12 Oct 2009 13:26:35 -0400 Subject: [SciPy-dev] Trac + Netbeans/cubeon integration ? Message-ID: <55690D38-CAAE-4B7E-A226-88D9C3D5E6BE@gmail.com> All (but more specifically, the team in charge of the Trac system), Over the last few weeks, I've been ditching Eclipse as my choice IDE for Netbeans 6.7 [http://www.netbeans.org/], which tends to work a bit better on my machine . A plug-in (Cube?n) extends Netbeans to let it communicate with a Trac system [http://code.google.com/p/cubeon/]. Of course I wanted to try it to have access to the numpy/scipy/sciktis tickets directly in my IDE (far easier than having to keep a browser window opened all the time). Unfortunately, I can't get it to work. When telling Cube?n the path of the Trac, I'm getting a 403 (I don't have permission to access http://projects.scipy.org/numpy/login/xmlrpc). Would any of the Enthought team interested in helping me investigating what's going on ? In addition, it looks like the trac system should be patched for Cube?n to work (according to http://code.google.com/p/cubeon/wiki/GSTracRepository ). Is it easy to do ? Thanks a lot in advance P. From perry at stsci.edu Mon Oct 12 14:05:08 2009 From: perry at stsci.edu (Perry Greenfield) Date: Mon, 12 Oct 2009 14:05:08 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: References: <4AD34DFF.6000809@stsci.edu> Message-ID: <939FF245-308B-4235-845C-BDF6F1FC12A4@stsci.edu> On Oct 12, 2009, at 1:09 PM, Ralf Gommers wrote: > > > It would still be nice to have a slightly longer (tutorial-style) > discussion somewhere in the docs. Or otherwise maybe a link to FITS > docs that are illustrative? I still have a hard time understanding > why operating on astronomical images with Python string methods is a > good idea. The short answer is that the FITS package deals with reading table data as well (which is used for all sorts of things like sky target lists or other catalog-like information). Such tables often have fields with string content (e.g., target name). It is common to select rows based on the content of these character fields. These tables can have tens of thousands of rows so efficiency is an issue. Perry From ralf.gommers at googlemail.com Mon Oct 12 15:15:19 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Mon, 12 Oct 2009 21:15:19 +0200 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <939FF245-308B-4235-845C-BDF6F1FC12A4@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> <939FF245-308B-4235-845C-BDF6F1FC12A4@stsci.edu> Message-ID: On Mon, Oct 12, 2009 at 8:05 PM, Perry Greenfield wrote: > > On Oct 12, 2009, at 1:09 PM, Ralf Gommers wrote: > > > > > > It would still be nice to have a slightly longer (tutorial-style) > > discussion somewhere in the docs. Or otherwise maybe a link to FITS > > docs that are illustrative? I still have a hard time understanding > > why operating on astronomical images with Python string methods is a > > good idea. > > The short answer is that the FITS package deals with reading table > data as well (which is used for all sorts of things like sky target > lists or other catalog-like information). Such tables often have > fields with string content (e.g., target name). It is common to select > rows based on the content of these character fields. These tables can > have tens of thousands of rows so efficiency is an issue. > > Thanks, that makes sense. Ralf > Perry > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Mon Oct 12 15:51:54 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Mon, 12 Oct 2009 21:51:54 +0200 Subject: [SciPy-dev] ndimage - docfiller and output_type Message-ID: Hi all, While working on the ndimage docs, I saw that the docfiller decorator in ndimage.filters does not work well with the doc wiki. Basically, in the doc wiki you see the docstring with the decorator applied. Then if you make an edit, all the %(param) markup has been replaced. Compare the `convolve` and `convolve1d` docstrings in trunk to see what happens. Some options to fix this: 1. remove the filters docstrings from the doc wiki 2. make the doc wiki understand the decorator 3. remove the decorator, and do some copy-paste'ing of parameter descriptions 4. remove the decorator, and refer to a single description of those parameters outside the docstring. I would have a preference for 3 or 4, mostly because it's easier and more robust. Would you like a patch for either of those options? The second thing I came across is the output_type keyword in ndimage.interpolation. Specifying anything other than None for that keyword results in a RuntimeError. The code responsible for that is in ndimage._ni_support.py : def _get_output(output, input, output_type = None, shape = None): if output_type is not None: msg = "'output_type' argument is deprecated." msg += " Assign type to 'output' instead." raise RuntimeError, msg <6 more lines that never execute> Is there any reason to keep that keyword around? I can write a patch for that as well. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Mon Oct 12 20:14:35 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Mon, 12 Oct 2009 20:14:35 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910061539n6d6d2f7ar9b3b2701f71460ad@mail.gmail.com> <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: 2009/10/9 Charles R Harris : > > It's not inheritence, it's a class templaate.? Because classes are objects > in python you can return them from functions.with a defined enviroment. > Another way to look at it is completions. A basis would just be another > passed parameter. and the class code would still only heed to be written > once for all the different types of polynnomial basis.. I think it is a > cleaner way to go. Upon reflection, I think you were right about wanting the polynomial code in the polynomial classes themselves. I've implemented this in the git branch "twoclass"; there, when you implement a new representation of polynomials, you write a new subclass of Polynomial and a new subclass of Basis, and polynomials are created with "basis.polynomial(coefficients)". Anne From charlesr.harris at gmail.com Mon Oct 12 22:33:08 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 12 Oct 2009 20:33:08 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: On Mon, Oct 12, 2009 at 6:14 PM, Anne Archibald wrote: > 2009/10/9 Charles R Harris : > > > > It's not inheritence, it's a class templaate. Because classes are > objects > > in python you can return them from functions.with a defined enviroment. > > Another way to look at it is completions. A basis would just be another > > passed parameter. and the class code would still only heed to be written > > once for all the different types of polynnomial basis.. I think it is a > > cleaner way to go. > > Upon reflection, I think you were right about wanting the polynomial > code in the polynomial classes themselves. I've implemented this in > the git branch "twoclass"; there, when you implement a new > representation of polynomials, you write a new subclass of Polynomial > and a new subclass of Basis, and polynomials are created with > "basis.polynomial(coefficients)". > > I wasn't thinking of using subclasses at all, what I wanted was an standard interface on top of the different types of polynomials using a few polynomial specific functions like add, sub, mul, and divide. One way to do that would be to simply copy the interface code into each different polynomial module, but that would make maintenance more difficult. Using a generic_interface function to return a class object (not instance) from a function that accepts the add, sub, mul, and divide functions as arguments has the same effect, but the class code only needs to be written in one place. And since I've already written code for the Chebyshev series I thought I would reuse it, just replacing Chebyshev by some generic name like Poly, which can be done without danger because the class name is local to the factory function. Anyway, I was thinking about polynomials of the Chebyshev or power series type, I don't know that the same interface would be appropriate for the different Lagrange basis. All I wanted for the different polynomial types were the coefficients and domain, whereas for the Lagrange type basis it looks like you will need a list of (x_i, y_i) pairs, or at least the x_i. You could still use a domain for axis scaling/translation, but that doesn't seem the most natural way. What you might could do is pass a list of the x_i to a factory function which would then return a class that could be instantiated using the y_i values. That way the instances with different sample points would not compare as instances of the same class. I've just gotten back and have been playing with the cat/dog/pet thing just to test things out and it looks like it is going to work, I've attached copies of the silly things so you can see an example of what I'm thinking of. Now I'll go take a closer look at what you've been doing, which I probably should have done before running on like this ;) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pet.py Type: text/x-python Size: 291 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cat.py Type: text/x-python Size: 79 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dog.py Type: text/x-python Size: 79 bytes Desc: not available URL: From pav+sp at iki.fi Tue Oct 13 05:12:36 2009 From: pav+sp at iki.fi (Pauli Virtanen) Date: Tue, 13 Oct 2009 09:12:36 +0000 (UTC) Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) References: <45d1ab480910062333o3fe350d2vd3ba3e845e82c13f@mail.gmail.com> Message-ID: Mon, 12 Oct 2009 20:33:08 -0600, Charles R Harris wrote: [clip] > I've just gotten back and have been playing with the cat/dog/pet thing > just to test things out and it looks like it is going to work, I've > attached copies of the silly things so you can see an example of what > I'm thinking of. Now I'll go take a closer look at what you've been > doing, which I probably should have done before running on like this ;) That's sort of magicky: the user ends with instances of class named Polynomial, which are actually not necessarily of the same class. How about using class mixins / multiple inheritance instead? ------------ class CatMixin: def sound(self): return "meow" class DogMixin: def sound(self): return "woof" class Pet: def speak(self): return self.sound() def breed(self, other): if isinstance(other, Pet): return self.__class__() else: raise ValueError("Oh no!") class Cat(Pet, CatMixin): pass class Dog(Pet, DogMixin): pass dog = Dog() cat = Cat() print "Cat:", cat.speak() print "Dog:", dog.speak() mongrel = dog.breed(cat) print "Mongrel:", mongrel.speak() ------------ Now, I'm not 100% opposed to magically generating the Mixins themselves with class factories, as they are a step removed from what's visible to the user. -- Pauli Virtanen From sebastian.walter at gmail.com Tue Oct 13 06:08:32 2009 From: sebastian.walter at gmail.com (Sebastian Walter) Date: Tue, 13 Oct 2009 12:08:32 +0200 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: IMHO one should adhere to the KISS principle and separate the algorithms from the syntactic sugar. It also seems to me that hiding the base of the polynomial from the user is actually a bad thing. As far as I understood, Robert Kern suggested the following a few weeks ago: 1) define the algorithms on the coefficients as class methods to avoid name cluttering 2) have a convenience implementation that uses operator overloading import numpy as np class Polynomial(object): def __init__(self, coeffs): self.coeffs = np.asarray(coeffs) def __mul__(self,rhs): return self.__class__( self.__class__.mul(self.coeffs, rhs.coeffs) ) def __rmul__(self,lhs): return self*lhs def __str__(self): return str(self.coeffs) class PowerPolynomial(Polynomial): @classmethod def mul(cls, lhs_coeffs, rhs_coeffs): return np.convolve(lhs_coeffs, rhs_coeffs, mode='full') if __name__ == '__main__': x = PowerPolynomial([1.,2.]) y = PowerPolynomial([3.,4.]) z = x * y print z print PowerPolynomial.mul([1.,2.],[3.,4.]) --------- output ----------- python poly.py [ 3. 10. 8.] [ 3. 10. 8.] On Tue, Oct 13, 2009 at 11:12 AM, Pauli Virtanen wrote: > Mon, 12 Oct 2009 20:33:08 -0600, Charles R Harris wrote: > [clip] >> I've just gotten back and have been playing with the cat/dog/pet thing >> just to test things out and it looks like it is going to work, I've >> attached copies of the silly things so you can see an example of what >> I'm thinking of. Now I'll go take a closer look at what you've been >> doing, which I probably should have done before running on like this ;) > > That's sort of magicky: the user ends with instances of class named > Polynomial, which are actually not necessarily of the same class. > > How about using class mixins / multiple inheritance instead? > > ------------ > class CatMixin: > def sound(self): > return "meow" > > class DogMixin: > def sound(self): > return "woof" > > class Pet: > def speak(self): > return self.sound() > > def breed(self, other): > if isinstance(other, Pet): > return self.__class__() > else: > raise ValueError("Oh no!") > > class Cat(Pet, CatMixin): > pass > > class Dog(Pet, DogMixin): > pass > > dog = Dog() > cat = Cat() > > print "Cat:", cat.speak() > print "Dog:", dog.speak() > mongrel = dog.breed(cat) > print "Mongrel:", mongrel.speak() > ------------ > > Now, I'm not 100% opposed to magically generating the Mixins themselves > with class factories, as they are a step removed from what's visible to > the user. > > -- > Pauli Virtanen > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From charlesr.harris at gmail.com Tue Oct 13 09:36:26 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 07:36:26 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 4:08 AM, Sebastian Walter < sebastian.walter at gmail.com> wrote: > IMHO one should adhere to the KISS principle and separate the > algorithms from the syntactic sugar. > It also seems to me that hiding the base of the polynomial from the > user is actually a bad thing. > > It is kiss. You have the separate functions (which is what Robert wanted), and an interface that is reused. What it buys is complete code reuse in the interface as well as the ability to limit the operators to classes of the same sort. Inheritance doesn't really work for this because we don't want to operate on a common base class (is a relationship). But I think we do want every class to operate with itself in a similar manner. As far as I understood, Robert Kern suggested the following a few weeks ago: > 1) define the algorithms on the coefficients as class methods to avoid > name cluttering > That was my initial thought: static methods. But I don't think that is what Robert had in mind, I think he just wanted to be sure the basic functions were available as standalone functions and not just methods of some class. > 2) have a convenience implementation that uses operator overloading > > The classes do that. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Oct 13 10:00:30 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 08:00:30 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 3:12 AM, Pauli Virtanen > wrote: > Mon, 12 Oct 2009 20:33:08 -0600, Charles R Harris wrote: > [clip] > > I've just gotten back and have been playing with the cat/dog/pet thing > > just to test things out and it looks like it is going to work, I've > > attached copies of the silly things so you can see an example of what > > I'm thinking of. Now I'll go take a closer look at what you've been > > doing, which I probably should have done before running on like this ;) > > That's sort of magicky: the user ends with instances of class named > Polynomial, which are actually not necessarily of the same class. > > How about using class mixins / multiple inheritance instead? > > I just don't think inheritance is appropriate here. I'll admit the proposed implementation has a magicky flavor, but in my case that arises because the programming idiom using closures is unfamiliar. In C++ what I would have is a templated class, but python doesn't have such, while it does have closures. I suspect folks who do more functional programming might find the approach more familiar. In Python it's really no different than copying the interface code into the various modules and changing the underlying function names with an editor. In one case the result is compiled in the module environment, in the latter in the function environment. The thing is, I can change the function environment by what gets passed in the arguments. > ------------ > class CatMixin: > def sound(self): > return "meow" > > class DogMixin: > def sound(self): > return "woof" > > class Pet: > def speak(self): > return self.sound() > > def breed(self, other): > if isinstance(other, Pet): > return self.__class__() > else: > raise ValueError("Oh no!") > > class Cat(Pet, CatMixin): > pass > > class Dog(Pet, DogMixin): > pass > > dog = Dog() > cat = Cat() > > print "Cat:", cat.speak() > print "Dog:", dog.speak() > mongrel = dog.breed(cat) > print "Mongrel:", mongrel.speak() > The dog/cat cross didn't raise an error as it should. Dog and Cat really need to be different classes. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.walter at gmail.com Tue Oct 13 10:06:59 2009 From: sebastian.walter at gmail.com (Sebastian Walter) Date: Tue, 13 Oct 2009 16:06:59 +0200 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 3:36 PM, Charles R Harris wrote: > > > On Tue, Oct 13, 2009 at 4:08 AM, Sebastian Walter > wrote: >> >> IMHO one should adhere to the KISS principle and separate the >> algorithms from the syntactic sugar. >> It also seems to me that hiding the base of the polynomial from the >> user is actually a bad thing. >> > > It is kiss. You have the separate functions (which is what Robert wanted), > and an interface that is reused. What it buys is complete code reuse in the > interface as well as the ability to limit the operators to classes of the > same sort. Inheritance doesn't really work for this because we don't want to > operate on a common base class (is a relationship). But I think we do want > every class to operate with itself in a similar manner. > >> As far as I understood, Robert Kern suggested the following a few weeks >> ago: >> 1) define the algorithms on the coefficients as class methods to avoid >> name cluttering > > That was my initial thought: static methods. But I don't think that is what > Robert had in mind, I think he just wanted to be sure the basic functions > were available as standalone functions and not just methods of some class. > >> >> 2) have a convenience implementation that uses operator overloading >> > > The classes do that. > Hmm, ok, I think you are right. I changed my code it now uses a class factory. import numpy as np class PowerBaseOps(object): @classmethod def mul(cls,lhs_coeffs, rhs_coeffs): return np.convolve(lhs_coeffs, rhs_coeffs, mode='full') def poly_factory(BaseOps, dtype): """ produces polynomials with dtype coefficients""" class PolyClass(object): def __init__(self, coeffs): self.coeffs = np.asarray(coeffs, dtype=dtype) def __mul__(self,rhs): return PolyClass(BaseOps.mul(self.coeffs, rhs.coeffs)) def __str__(self): return str(self.coeffs) return PolyClass if __name__ == '__main__': PowerPolynomial = poly_factory(PowerBaseOps, float) IntPowerPolynomial = poly_factory(PowerBaseOps, int) x = PowerPolynomial([1.,2.]) y = PowerPolynomial([3.,4.]) z = x * y print z print PowerBaseOps.mul([1.,2.],[3.,4.]) i = IntPowerPolynomial([1,2]) j = IntPowerPolynomial([3,4]) k = i * j print k --------- output ------------ python poly2.py [ 3. 10. 8.] [ 3. 10. 8.] [ 3 10 8] > > Chuck > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From pav+sp at iki.fi Tue Oct 13 10:14:13 2009 From: pav+sp at iki.fi (Pauli Virtanen) Date: Tue, 13 Oct 2009 14:14:13 +0000 (UTC) Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) References: Message-ID: Tue, 13 Oct 2009 08:00:30 -0600, Charles R Harris wrote: [clip] > I just don't think inheritance is appropriate here. I'll admit the > proposed implementation has a magicky flavor, but in my case that > arises because the programming idiom using closures is unfamiliar. In > C++ what I would have is a templated class, but python doesn't have > such, while it does have closures. [clip] If you also change the class names dynamically in the factory, there'll be less user confusion. Anyway, you write the code, you call the shots :) [clip] > The dog/cat cross didn't raise an error as it should. Dog and Cat really > need to be different classes. You're right, maybe breeding dogs with cats is not such a bright idea. -- Pauli Virtanen From charlesr.harris at gmail.com Tue Oct 13 10:49:31 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 08:49:31 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 8:14 AM, Pauli Virtanen > wrote: > Tue, 13 Oct 2009 08:00:30 -0600, Charles R Harris wrote: > [clip] > > I just don't think inheritance is appropriate here. I'll admit the > > proposed implementation has a magicky flavor, but in my case that > > arises because the programming idiom using closures is unfamiliar. In > > C++ what I would have is a templated class, but python doesn't have > > such, while it does have closures. > [clip] > > If you also change the class names dynamically in the factory, there'll > be less user confusion. > > Well, the class identity is set by where it is instantiated. The isinstance function will show that Dog and Cat are different classes, i.e., an instance of Dog will fail in isinstance (mydog, Cat). > Anyway, you write the code, you call the shots :) > > Heh. I am looking for feedback before taking the plunge. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdroe at stsci.edu Tue Oct 13 11:23:37 2009 From: mdroe at stsci.edu (Michael Droettboom) Date: Tue, 13 Oct 2009 11:23:37 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: References: <4AD34DFF.6000809@stsci.edu> Message-ID: <4AD49B79.9080409@stsci.edu> Ralf Gommers wrote: > > > On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom > wrote: > > I was able to make my big chararray commit today. If I understand > correctly, I need to wait 24 hours for the doc editor to sync with > SVN, > and then I should mark all the chararray-related docstrings as "needs > review". > Ok -- I'm a bit lost in the documentation editor. I go to the defchararray module, and I don't see any of the new functions or old methods: http://docs.scipy.org/numpy/docs/numpy.core.defchararray The "Review status" is stuck on "Unimportant". Not sure what that means. I go to the chararray class and I can't see any methods: http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ Pardon my ignorance -- just trying to mark the new docstrings as "needs review". Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA From josef.pktd at gmail.com Tue Oct 13 11:45:07 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Tue, 13 Oct 2009 11:45:07 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <4AD49B79.9080409@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> Message-ID: <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> On Tue, Oct 13, 2009 at 11:23 AM, Michael Droettboom wrote: > Ralf Gommers wrote: >> >> >> On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom > > wrote: >> >> ? ? I was able to make my big chararray commit today. ?If I understand >> ? ? correctly, I need to wait 24 hours for the doc editor to sync with >> ? ? SVN, >> ? ? and then I should mark all the chararray-related docstrings as "needs >> ? ? review". >> > Ok -- I'm a bit lost in the documentation editor. > > I go to the defchararray module, and I don't see any of the new > functions or old methods: > > http://docs.scipy.org/numpy/docs/numpy.core.defchararray > > The "Review status" is stuck on "Unimportant". ?Not sure what that means. > > I go to the chararray class and I can't see any methods: > > http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ I see many methods at the bottom of this page, I don't know the names of the new methods to see whether they are there. links to the methods are e.g. http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray.upper/ Josef > > Pardon my ignorance -- just trying to mark the new docstrings as "needs > review". > > Mike > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From mdroe at stsci.edu Tue Oct 13 11:47:45 2009 From: mdroe at stsci.edu (Michael Droettboom) Date: Tue, 13 Oct 2009 11:47:45 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> Message-ID: <4AD4A121.5060009@stsci.edu> josef.pktd at gmail.com wrote: > On Tue, Oct 13, 2009 at 11:23 AM, Michael Droettboom wrote: > >> Ralf Gommers wrote: >> >>> On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom >> > wrote: >>> >>> I was able to make my big chararray commit today. If I understand >>> correctly, I need to wait 24 hours for the doc editor to sync with >>> SVN, >>> and then I should mark all the chararray-related docstrings as "needs >>> review". >>> >>> >> Ok -- I'm a bit lost in the documentation editor. >> >> I go to the defchararray module, and I don't see any of the new >> functions or old methods: >> >> http://docs.scipy.org/numpy/docs/numpy.core.defchararray >> >> The "Review status" is stuck on "Unimportant". Not sure what that means. >> >> I go to the chararray class and I can't see any methods: >> >> > > >> http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ >> > > I see many methods at the bottom of this page, I don't know the names > of the new methods to see whether they are there. > Odd. They weren't there a moment ago. Sorry for the noise. Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA From josef.pktd at gmail.com Tue Oct 13 11:59:26 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Tue, 13 Oct 2009 11:59:26 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <4AD4A121.5060009@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> Message-ID: <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> On Tue, Oct 13, 2009 at 11:47 AM, Michael Droettboom wrote: > josef.pktd at gmail.com wrote: >> On Tue, Oct 13, 2009 at 11:23 AM, Michael Droettboom wrote: >> >>> Ralf Gommers wrote: >>> >>>> On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom >>> > wrote: >>>> >>>> ? ? I was able to make my big chararray commit today. ?If I understand >>>> ? ? correctly, I need to wait 24 hours for the doc editor to sync with >>>> ? ? SVN, >>>> ? ? and then I should mark all the chararray-related docstrings as "needs >>>> ? ? review". >>>> >>>> >>> Ok -- I'm a bit lost in the documentation editor. >>> >>> I go to the defchararray module, and I don't see any of the new >>> functions or old methods: >>> >>> http://docs.scipy.org/numpy/docs/numpy.core.defchararray >>> >>> The "Review status" is stuck on "Unimportant". ?Not sure what that means. >>> >>> I go to the chararray class and I can't see any methods: >>> >>> >> >> >>> http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ >>> >> >> I see many ?methods at the bottom of this page, I don't know the names >> of the new methods to see whether they are there. >> > Odd. ?They weren't there a moment ago. ?Sorry for the noise. I also found the new functions at http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.char.rst/#id4 but the doc editor is not picking up the docstrings I think they are only picked up if they are in an __all__ currently: defchararray.__all__ = ['chararray'] Josef > > Mike > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From mdroe at stsci.edu Tue Oct 13 12:06:34 2009 From: mdroe at stsci.edu (Michael Droettboom) Date: Tue, 13 Oct 2009 12:06:34 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> Message-ID: <4AD4A58A.5070308@stsci.edu> Ok -- I can add them to __all__ in defchararray.py. Is there a good way for me to check that my changes will work in the documentation editor? I rebuilt the docs in the source tree (by going in to doc/ and running "make html") and these methods *were* picked up. How do other people work and test their changes? I also noticed that the cross-referencing from the chararray methods to the free functions in defchararray is broken in the online documentation editor. Again, mostly just my ignorance of how all the pieces go together. Can someone explain what it should be, and then I'll make all the necessary corrections? E.g. what should the "See also" section of defchararray.chararray.isalnum look like to reference defchararray.isalnum? Also -- How can I change a review status from "Unimportant"? Mike josef.pktd at gmail.com wrote: > On Tue, Oct 13, 2009 at 11:47 AM, Michael Droettboom wrote: > >> josef.pktd at gmail.com wrote: >> >>> On Tue, Oct 13, 2009 at 11:23 AM, Michael Droettboom wrote: >>> >>> >>>> Ralf Gommers wrote: >>>> >>>> >>>>> On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom >>>> > wrote: >>>>> >>>>> I was able to make my big chararray commit today. If I understand >>>>> correctly, I need to wait 24 hours for the doc editor to sync with >>>>> SVN, >>>>> and then I should mark all the chararray-related docstrings as "needs >>>>> review". >>>>> >>>>> >>>>> >>>> Ok -- I'm a bit lost in the documentation editor. >>>> >>>> I go to the defchararray module, and I don't see any of the new >>>> functions or old methods: >>>> >>>> http://docs.scipy.org/numpy/docs/numpy.core.defchararray >>>> >>>> The "Review status" is stuck on "Unimportant". Not sure what that means. >>>> >>>> I go to the chararray class and I can't see any methods: >>>> >>>> >>>> >>> >>>> http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ >>>> >>>> >>> I see many methods at the bottom of this page, I don't know the names >>> of the new methods to see whether they are there. >>> >>> >> Odd. They weren't there a moment ago. Sorry for the noise. >> > > I also found the new functions at > > http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.char.rst/#id4 > > but the doc editor is not picking up the docstrings > > I think they are only picked up if they are in an __all__ > > currently: defchararray.__all__ = ['chararray'] > > Josef > > >> Mike >> >> -- >> Michael Droettboom >> Science Software Branch >> Operations and Engineering Division >> Space Telescope Science Institute >> Operated by AURA for NASA >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA From peridot.faceted at gmail.com Tue Oct 13 12:12:14 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 13 Oct 2009 12:12:14 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/13 Charles R Harris : > > On Tue, Oct 13, 2009 at 8:14 AM, Pauli Virtanen wrote: [snip] >> Anyway, you write the code, you call the shots :) >> > > Heh. I am looking for feedback before taking the plunge. Wow, lots of ideas! But I think before we can evaluate them we need to figure out what our requirements are. Here's a suggested list: (1) Simple. Simple to understand, simple to add new representations, even for users outside scipy. (2) Possible for users to subclass polynomials to provide polynomial objects with more features. (3) Polynomial objects should be able to detect and prevent operations between polynomials in different bases when appropriate (for example, you can substitute any kind of polynomial into a power-basis polynomial, or a Lagrange basis polynomial into anything). (4) Polynomial operations available as functions that operate on coefficient arrays rather than polynomial objects. (5) Possibly substantial data and methods attached to bases - orthogonal polynomial roots and weights, lists of points for Lagrange polynomials. Any others? Are all these really requirements? Here are some key questions: * Do we want the information about what representation a polynomial is in stored in the polynomial's class, in an explicit basis object the polynomial contains, or both? * How should a user obtain a polynomial object? My current implementation fails only (4), and could be modified by moving the code back into basis objects to satisfy (4). Then polynomials of all current representations could be of class Polynomial, with the basis information stored as an explicit basis object. Users obtain polynomials as "mybasis.polynomial(coefficients)"; the Polynomial constructor is also possible but raises an exception if misused. I'm not sure how best to discourage its use. Anne From ralf.gommers at googlemail.com Tue Oct 13 11:45:25 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Tue, 13 Oct 2009 17:45:25 +0200 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <4AD49B79.9080409@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> Message-ID: On Tue, Oct 13, 2009 at 5:23 PM, Michael Droettboom wrote: > Ralf Gommers wrote: > > > > > > On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom > > wrote: > > > > I was able to make my big chararray commit today. If I understand > > correctly, I need to wait 24 hours for the doc editor to sync with > > SVN, > > and then I should mark all the chararray-related docstrings as "needs > > review". > > > Ok -- I'm a bit lost in the documentation editor. > > I go to the defchararray module, and I don't see any of the new > functions or old methods: > > http://docs.scipy.org/numpy/docs/numpy.core.defchararray > The methods are all there, you see them when you click on the chararray class in the link above. see http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ > > The "Review status" is stuck on "Unimportant". Not sure what that means. > Each item has a review status. You were looking at the module docstring. You can see that the class docstring is at "Needs Review" and most methods in the writing/needs review stage as well. I don't see the new functions you added yet, not sure why. All docs in the wiki are visible on this page: http://docs.scipy.org/numpy/docs/ I usually search there for what I need. > > I go to the chararray class and I can't see any methods: > > http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ > I see them now. Maybe something odd was going on with merging in your changes, I had to click "accept merge" on the Merge page. Cheers, Ralf > > Pardon my ignorance -- just trying to mark the new docstrings as "needs > review". > > Mike > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Tue Oct 13 12:22:15 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Tue, 13 Oct 2009 12:22:15 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> Message-ID: <1cd32cbb0910130922g20c7045eo546c542e697829b3@mail.gmail.com> On Tue, Oct 13, 2009 at 11:45 AM, Ralf Gommers wrote: > > > On Tue, Oct 13, 2009 at 5:23 PM, Michael Droettboom wrote: >> >> Ralf Gommers wrote: >> > >> > >> > On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom > > > wrote: >> > >> > ? ? I was able to make my big chararray commit today. ?If I understand >> > ? ? correctly, I need to wait 24 hours for the doc editor to sync with >> > ? ? SVN, >> > ? ? and then I should mark all the chararray-related docstrings as >> > "needs >> > ? ? review". >> > >> Ok -- I'm a bit lost in the documentation editor. >> >> I go to the defchararray module, and I don't see any of the new >> functions or old methods: >> >> http://docs.scipy.org/numpy/docs/numpy.core.defchararray > > The methods are all there, you see them when you click on the chararray > class in the link above. see > http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ >> >> The "Review status" is stuck on "Unimportant". ?Not sure what that means. > > Each item has a review status. You were looking at the module docstring. You > can see that the class docstring is at "Needs Review"? and most methods in > the writing/needs review stage as well. > > I don't see the new functions you added yet, not sure why. > > All docs in the wiki are visible on this page: > http://docs.scipy.org/numpy/docs/ I usually search there for what I need. >> >> I go to the chararray class and I can't see any methods: >> >> http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ > > I see them now. Maybe something odd was going on with merging in your > changes, I had to click "accept merge" on the Merge page. > > Cheers, > Ralf > >> >> Pardon my ignorance -- just trying to mark the new docstrings as "needs >> review". >> >> Mike >> >> -- >> Michael Droettboom >> Science Software Branch >> Operations and Engineering Division >> Space Telescope Science Institute >> Operated by AURA for NASA >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > How are the new string functions imported from python? How are they exposed? I think the doc editor only picks up functions that are exposed in an __all__ and maybe also from a module that doesn't have an __all__ cross linking should work once the doc editor is able to pick up the functions in http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.char.rst/ That's my impression of the doc editor so far. Josef From charlesr.harris at gmail.com Tue Oct 13 12:42:20 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 10:42:20 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 10:12 AM, Anne Archibald wrote: > 2009/10/13 Charles R Harris : > > > > On Tue, Oct 13, 2009 at 8:14 AM, Pauli Virtanen > > wrote: > [snip] > >> Anyway, you write the code, you call the shots :) > >> > > > > Heh. I am looking for feedback before taking the plunge. > > Wow, lots of ideas! But I think before we can evaluate them we need to > figure out what our requirements are. Here's a suggested list: > > (1) Simple. Simple to understand, simple to add new representations, > even for users outside scipy. > (2) Possible for users to subclass polynomials to provide polynomial > objects with more features. > (3) Polynomial objects should be able to detect and prevent operations > between polynomials in different bases when appropriate (for example, > you can substitute any kind of polynomial into a power-basis > polynomial, or a Lagrange basis polynomial into anything). > (4) Polynomial operations available as functions that operate on > coefficient arrays rather than polynomial objects. > (5) Possibly substantial data and methods attached to bases - > orthogonal polynomial roots and weights, lists of points for Lagrange > polynomials. > > Any others? Are all these really requirements? > > I would add "massive code reuse". > Here are some key questions: > > * Do we want the information about what representation a polynomial is > in stored in the polynomial's class, in an explicit basis object the > polynomial contains, or both? > I would say different representations, different classes, because different representations don't mix; you can't even add the coefficients together. This could be carried to an extreme, i.e., different scalings -> different classes, but I think that degree of specificity is too much. Also no class should need to know about any other class in order to work properly, it should stand independently. > * How should a user obtain a polynomial object? > > I was thinking if you do 4), which is the fundamental requirement, then you should be able get an appropriate class on top of that without any extra work. > > My current implementation fails only (4), and could be modified by > moving the code back into basis objects to satisfy (4). Then > polynomials of all current representations could be of class > Polynomial, with the basis information stored as an explicit basis > object. Users obtain polynomials as > I really don't see any natural is-a relationship except for extending a class, say by adding weights. Which is to say I don't see any natural base class for all the different representations. I don't think inheritance is the right concept here. > "mybasis.polynomial(coefficients)"; the Polynomial constructor is also > possible but raises an exception if misused. I'm not sure how best to > discourage its use. > > > Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Tue Oct 13 12:53:26 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Tue, 13 Oct 2009 18:53:26 +0200 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <4AD4A58A.5070308@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> Message-ID: On Tue, Oct 13, 2009 at 6:06 PM, Michael Droettboom wrote: > Ok -- I can add them to __all__ in defchararray.py. > > Is there a good way for me to check that my changes will work in the > documentation editor? I rebuilt the docs in the source tree (by going > in to doc/ and running "make html") and these methods *were* picked up. > How do other people work and test their changes? > If the object is already in the wiki, you can just edit its page, copy in your changes and click preview to check. For new objects, I guess you would have to run the whole wiki on your own computer to try - a bit too much trouble maybe. > > I also noticed that the cross-referencing from the chararray methods to > the free functions in defchararray is broken in the online documentation > editor. Again, mostly just my ignorance of how all the pieces go > together. Can someone explain what it should be, and then I'll make all > the necessary corrections? E.g. what should the "See also" section of > defchararray.chararray.isalnum look like to reference defchararray.isalnum? > >From the changes I saw, it should indeed work once the functions show up. The way it works is that first the current module is searched for an object with the name of the cross-ref you gave, if it is not found then it goes up the chain of modules. If the object you reference from for example numpy.mais in `ma` or in `numpy` it will be found, but if it is in `numpy.fft` you reference it as `fft.yourref`. > Also -- How can I change a review status from "Unimportant"? > > One of the admins has to give you reviewer permissions. Please leave private objects at Unimportant though - besides indicating what needs review the status is also used to guide contributors to the most important docstrings to be done. Stuff only relevant to devs can sometimes be marked Unimportant; that does not mean those docs can't also be improved. Cheers, Ralf > Mike > > josef.pktd at gmail.com wrote: > > On Tue, Oct 13, 2009 at 11:47 AM, Michael Droettboom > wrote: > > > >> josef.pktd at gmail.com wrote: > >> > >>> On Tue, Oct 13, 2009 at 11:23 AM, Michael Droettboom > wrote: > >>> > >>> > >>>> Ralf Gommers wrote: > >>>> > >>>> > >>>>> On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom >>>>> > wrote: > >>>>> > >>>>> I was able to make my big chararray commit today. If I > understand > >>>>> correctly, I need to wait 24 hours for the doc editor to sync > with > >>>>> SVN, > >>>>> and then I should mark all the chararray-related docstrings as > "needs > >>>>> review". > >>>>> > >>>>> > >>>>> > >>>> Ok -- I'm a bit lost in the documentation editor. > >>>> > >>>> I go to the defchararray module, and I don't see any of the new > >>>> functions or old methods: > >>>> > >>>> http://docs.scipy.org/numpy/docs/numpy.core.defchararray > >>>> > >>>> The "Review status" is stuck on "Unimportant". Not sure what that > means. > >>>> > >>>> I go to the chararray class and I can't see any methods: > >>>> > >>>> > >>>> > >>> > >>>> http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ > >>>> > >>>> > >>> I see many methods at the bottom of this page, I don't know the names > >>> of the new methods to see whether they are there. > >>> > >>> > >> Odd. They weren't there a moment ago. Sorry for the noise. > >> > > > > I also found the new functions at > > > > > http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.char.rst/#id4 > > > > but the doc editor is not picking up the docstrings > > > > I think they are only picked up if they are in an __all__ > > > > currently: defchararray.__all__ = ['chararray'] > > > > Josef > > > > > >> Mike > >> > >> -- > >> Michael Droettboom > >> Science Software Branch > >> Operations and Engineering Division > >> Space Telescope Science Institute > >> Operated by AURA for NASA > >> > >> _______________________________________________ > >> Scipy-dev mailing list > >> Scipy-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/scipy-dev > >> > >> > > _______________________________________________ > > Scipy-dev mailing list > > Scipy-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gael.varoquaux at normalesup.org Tue Oct 13 12:55:59 2009 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Tue, 13 Oct 2009 18:55:59 +0200 Subject: [SciPy-dev] chararray docstrings In-Reply-To: References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> Message-ID: <20091013165559.GA20164@phare.normalesup.org> On Tue, Oct 13, 2009 at 06:53:26PM +0200, Ralf Gommers wrote: > One of the admins has to give you reviewer permissions. Done. Ga?l From peridot.faceted at gmail.com Tue Oct 13 14:31:21 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 13 Oct 2009 14:31:21 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/13 Charles R Harris : > > > On Tue, Oct 13, 2009 at 10:12 AM, Anne Archibald > wrote: >> >> 2009/10/13 Charles R Harris : >> > >> > On Tue, Oct 13, 2009 at 8:14 AM, Pauli Virtanen wrote: >> [snip] >> >> Anyway, you write the code, you call the shots :) >> >> >> > >> > Heh. I am looking for feedback before taking the plunge. >> >> Wow, lots of ideas! But I think before we can evaluate them we need to >> figure out what our requirements are. Here's a suggested list: >> >> (1) Simple. Simple to understand, simple to add new representations, >> even for users outside scipy. >> (2) Possible for users to subclass polynomials to provide polynomial >> objects with more features. >> (3) Polynomial objects should be able to detect and prevent operations >> between polynomials in different bases when appropriate (for example, >> you can substitute any kind of polynomial into a power-basis >> polynomial, or a Lagrange basis polynomial into anything). >> (4) Polynomial operations available as functions that operate on >> coefficient arrays rather than polynomial objects. >> (5) Possibly substantial data and methods attached to bases - >> orthogonal polynomial roots and weights, lists of points for Lagrange >> polynomials. >> >> Any others? Are all these really requirements? >> > > I would add "massive code reuse". I agree, which is why I used inheritance to connect bases and polynomial types. >> Here are some key questions: >> >> * Do we want the information about what representation a polynomial is >> in stored in the polynomial's class, in an explicit basis object the >> polynomial contains, or both? > > I would say different representations, different classes, because different > representations don't mix; you can't even add the coefficients together. > This could be carried to an extreme, i.e., different scalings -> different > classes, but I think that degree of specificity is too much. Also no class > should need to know about any other class in order to work properly, it > should stand independently. The idea that they shouldn't mix is certainly correct, but having a polynomial refer to its basis explicitly is another way to achieve this; given the way python works, having separate classes doesn't even allow you to avoid manual type checking. class Foo: def __mul__(self, other): if not isinstance(other, Foo): raise ValueError versus class Bar: def __mul__(self, other): if not (isinstance(other, Bar) and other.basis==self.basis): raise ValueError or even, as with my current implementation: class Bar: def __mul__(self, other): if not self.iscompatible(other): raise ValueError I agree classes should not need to know about each other's internal workings; this is part of requirement (2): we must be able to work fine with polynomial representations added by users outside scipy. >> * How should a user obtain a polynomial object? >> > > I was thinking if you do 4), which is the fundamental requirement, then you > should be able get an appropriate class on top of that without any extra > work. Well, my first attempt was a simple Polynomial object which was used for polynomials in any representation; it delegated all the heavy lifting to the basis object's (4)-style interface. Polynomial objects were constructed in the usual way. Unfortunately, that means there's no way for different bases to provide polynomial objects with more methods. So I switched to using mybasis.polynomial(coefficients), which lets the basis create a subclass if that's appropriate. >> My current implementation fails only (4), and could be modified by >> moving the code back into basis objects to satisfy (4). Then >> polynomials of all current representations could be of class >> Polynomial, with the basis information stored as an explicit basis >> object. Users obtain polynomials as > > I really don't see any natural is-a relationship except for extending a > class, say by adding weights. Which is to say I don't see any natural base > class for all the different representations. I don't think inheritance is > the right concept here. Well, a GradedBasis is-a Basis, and a PowerBasis is-a GradedBasis, and so on, but I think you were talking more about the polynomial objects themselves. If polynomial objects delegate all their calculation to the (4)-style interface, then is there any need for them to be different types at all? After all, the delegation, compatibility, and type-checking code is all common to all polynomial types. In the Basis objects, there is room for massive code reuse - I have generic companion matrix and root-finding code that works for any polynomial representation that admits division, for example. Extending coefficient arrays by padding with zeros works for any GradedBasis (and in fact it's possible to write a division algorithm that works for any GradedBasis). This takes natural advantage of a class hierarchy among the bases. As for polynomial objects, code reuse is most easily accomplished by having them be exactly the same type; all the wrapper gubbins is the same in all bases, apart from raising a few NotImplementedErrors. Anne From charlesr.harris at gmail.com Tue Oct 13 16:09:31 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 14:09:31 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 12:31 PM, Anne Archibald wrote: > 2009/10/13 Charles R Harris : > > > > > > On Tue, Oct 13, 2009 at 10:12 AM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/13 Charles R Harris : > >> > > >> > On Tue, Oct 13, 2009 at 8:14 AM, Pauli Virtanen > > wrote: > >> [snip] > >> >> Anyway, you write the code, you call the shots :) > >> >> > >> > > >> > Heh. I am looking for feedback before taking the plunge. > >> > >> Wow, lots of ideas! But I think before we can evaluate them we need to > >> figure out what our requirements are. Here's a suggested list: > >> > >> (1) Simple. Simple to understand, simple to add new representations, > >> even for users outside scipy. > >> (2) Possible for users to subclass polynomials to provide polynomial > >> objects with more features. > >> (3) Polynomial objects should be able to detect and prevent operations > >> between polynomials in different bases when appropriate (for example, > >> you can substitute any kind of polynomial into a power-basis > >> polynomial, or a Lagrange basis polynomial into anything). > >> (4) Polynomial operations available as functions that operate on > >> coefficient arrays rather than polynomial objects. > >> (5) Possibly substantial data and methods attached to bases - > >> orthogonal polynomial roots and weights, lists of points for Lagrange > >> polynomials. > >> > >> Any others? Are all these really requirements? > >> > > > > I would add "massive code reuse". > > I agree, which is why I used inheritance to connect bases and polynomial > types. > > >> Here are some key questions: > >> > >> * Do we want the information about what representation a polynomial is > >> in stored in the polynomial's class, in an explicit basis object the > >> polynomial contains, or both? > > > > I would say different representations, different classes, because > different > > representations don't mix; you can't even add the coefficients together. > > This could be carried to an extreme, i.e., different scalings -> > different > > classes, but I think that degree of specificity is too much. Also no > class > > should need to know about any other class in order to work properly, it > > should stand independently. > > The idea that they shouldn't mix is certainly correct, but having a > polynomial refer to its basis explicitly is another way to achieve > this; given the way python works, having separate classes doesn't even > allow you to avoid manual type checking. > > class Foo: > def __mul__(self, other): > if not isinstance(other, Foo): > raise ValueError > > versus > > class Bar: > def __mul__(self, other): > if not (isinstance(other, Bar) and other.basis==self.basis): > raise ValueError > > or even, as with my current implementation: > > class Bar: > def __mul__(self, other): > if not self.iscompatible(other): > raise ValueError > > I agree classes should not need to know about each other's internal > workings; this is part of requirement (2): we must be able to work > fine with polynomial representations added by users outside scipy. > > >> * How should a user obtain a polynomial object? > >> > > > > I was thinking if you do 4), which is the fundamental requirement, then > you > > should be able get an appropriate class on top of that without any extra > > work. > > Well, my first attempt was a simple Polynomial object which was used > for polynomials in any representation; it delegated all the heavy > lifting to the basis object's (4)-style interface. Polynomial objects > were constructed in the usual way. Unfortunately, that means there's > no way for different bases to provide polynomial objects with more > methods. So I switched to using mybasis.polynomial(coefficients), > which lets the basis create a subclass if that's appropriate. > > >> My current implementation fails only (4), and could be modified by > >> moving the code back into basis objects to satisfy (4). Then > >> polynomials of all current representations could be of class > >> Polynomial, with the basis information stored as an explicit basis > >> object. Users obtain polynomials as > > > > I really don't see any natural is-a relationship except for extending a > > class, say by adding weights. Which is to say I don't see any natural > base > > class for all the different representations. I don't think inheritance is > > the right concept here. > > Well, a GradedBasis is-a Basis, and a PowerBasis is-a GradedBasis, and > so on, but I think you were talking more about the polynomial objects > themselves. If polynomial objects delegate all their calculation to > the (4)-style interface, then is there any need for them to be > different types at all? After all, the delegation, compatibility, and > type-checking code is all common to all polynomial types > In the Basis objects, there is room for massive code reuse - I have > generic companion matrix and root-finding code that works for any > polynomial representation that admits division, for example. Extending > coefficient arrays by padding with zeros works for any GradedBasis > (and in fact it's possible to write a division algorithm that works > for any GradedBasis). This takes natural advantage of a class > hierarchy among the bases. > > But those basis objects return a ton of NotImplementedErrors, which means that somewhere down the line the methods have to be implemented. That was one of the problems that I wanted to get around, because it leaves too much code to be written. And makes having different implementations for the methods, i.e., cython or Fortran, less transparent than just writing a set of base functions (add, sub, etc.) in those languages. But I think we are starting from somewhat different places. To me, the base functions were primary and the classes conveniences built on top of them. That is more flexible, IMHO, and doesn't tie one into a big system of class inheritance, etc., which is one of the drawbacks I see in a lot in C++ projects. The GNU scientific library would be an example of that where the pieces get tangled up in the whole. I think your method of getting the companion matrix is clever, and having definitions for zero, one, and x is a useful touch that should probably be part of any interface. I'm less convinced by the use of an empty list for zero. > As for polynomial objects, code reuse is most easily accomplished by > having them be exactly the same type; all the wrapper gubbins is the > same in all bases, apart from raising a few NotImplementedErrors. > > > Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Tue Oct 13 17:48:39 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Tue, 13 Oct 2009 14:48:39 -0700 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <20091013165559.GA20164@phare.normalesup.org> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> <20091013165559.GA20164@phare.normalesup.org> Message-ID: <45d1ab480910131448u6547db11xe2706f05eb14a09b@mail.gmail.com> OK, I'm coming into this thread a bit late, but I see that many of the defchararray.chararray docstrings are still at "Being Written" (which is better than "Needs Editing," where a couple still are) but they should all be at "Needs Review," yes? I'm happy to do this, Mike, but if you'd like the "practice" using the doc wiki, what I'd recommend is: 0) login to the wiki and, as suggested by Ralf, go to the list of all docstrings, http://docs.scipy.org/numpy/docs/ (accessible via the Docstrings link atop every wiki page); 1) scroll (way) down to the light gray background "Being written" docstrings and look for the defchararray.chararray docstring links (I count 23 of them; 'twas 24, but I promoted one while writing this out to verify these instructions); :-) 2) click on 'em one at a time (you'll probably want to open them in a new tab or window, so you don't have to keep going back to the docstring page); if you get "bored," let me know and I'll finish 'em off; 3) look for the gray box on the right which says "Review status:" and then in a drop-down has "Being written" in it, and a "Change" button next to that (if you don't see the Change button, you're not logged-in); 4) click on the drop-down triangle to expose the other status options, one of which should be "Needs review" (which should be in black font, assuming you have successfully been granted the privilege to execute this promotion; if not, it will be in gray font) and click on that; 5) then click the Change button and the Review status box should change to pink; 6) you're done! (with one docstring; if you wish to verify that the change "took," reload the "all docstrings" page and visually confirm that the docstring you just promoted is no longer in the "Being written" section and is now in fact in the "Needs review" section). HTH; again, I'm happy to do this once you get bored, just let me know via email (off-list is fine). DG On Tue, Oct 13, 2009 at 9:55 AM, Gael Varoquaux < gael.varoquaux at normalesup.org> wrote: > On Tue, Oct 13, 2009 at 06:53:26PM +0200, Ralf Gommers wrote: > > One of the admins has to give you reviewer permissions. > > Done. > > Ga?l > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Tue Oct 13 18:01:37 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 13 Oct 2009 18:01:37 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/13 Charles R Harris : > > > On Tue, Oct 13, 2009 at 12:31 PM, Anne Archibald > wrote: >> In the Basis objects, there is room for massive code reuse - I have >> generic companion matrix and root-finding code that works for any >> polynomial representation that admits division, for example. Extending >> coefficient arrays by padding with zeros works for any GradedBasis >> (and in fact it's possible to write a division algorithm that works >> for any GradedBasis). This takes natural advantage of a class >> hierarchy among the bases. >> > > But those basis objects return a ton of NotImplementedErrors, which means > that somewhere down the line the methods have to be implemented. That was > one of the problems that I wanted to get around, because it leaves too much > code to be written. And makes having different implementations for the > methods, i.e., cython or Fortran, less transparent than just writing a set > of base functions (add, sub, etc.) in those languages. But I think we are > starting from somewhat different places. To me, the base functions were > primary and the classes conveniences built on top of them. That is more > flexible, IMHO, and doesn't tie one into a big system of class inheritance, > etc., which is one of the drawbacks I see in a lot in C++ projects. The GNU > scientific library would be an example of that where the pieces get tangled > up in the whole. I'm not sure I see why returning all those NotImplementedErrors is a problem - yes, the code needs to be written to override the methods that return NotImplementedError, but that code needs to be written no matter how we set up an interface to it. And having the base class allows us to take any code that is the same between bases - like the companion matrix code - and put it in the base class. So the only "wasted" code here is the one-line wrappers that return NotImplementedError, and these serve as documentation for users - any such method is something they're going to have to implement for their own polynomial representations. If you look at the three different representations I have currently implemented, there isn't really much repeated code. This approach - of having an abstract base class whose operations are defined but not implemented - is a common design, albeit more common in languages that don't use duck-typing. There's no reason I couldn't just omit those methods from the base class, but I think they serve a useful documentary function. > I think your method of getting the companion matrix is clever, and having > definitions for zero, one, and x is a useful touch that should probably be > part of any interface. I'm less convinced by the use of an empty list for > zero. This choice is a natural one in some ways, because it means you never normally have a leading coefficient of zero for polynomials of any degree. It also means degree = len(coefficients)-1, apart from polynomials that are represented in a degree-inflated manner. And the places you need to special-case it are almost all places you would need to special-case zero polynomials anyway. I realize it may not be so convenient for a vector-of-coefficients interface, but I think even there it's the natural approach. Anne From charlesr.harris at gmail.com Tue Oct 13 19:04:30 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 17:04:30 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 4:01 PM, Anne Archibald wrote: > 2009/10/13 Charles R Harris : > > > > > > On Tue, Oct 13, 2009 at 12:31 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> In the Basis objects, there is room for massive code reuse - I have > >> generic companion matrix and root-finding code that works for any > >> polynomial representation that admits division, for example. Extending > >> coefficient arrays by padding with zeros works for any GradedBasis > >> (and in fact it's possible to write a division algorithm that works > >> for any GradedBasis). This takes natural advantage of a class > >> hierarchy among the bases. > >> > > > > But those basis objects return a ton of NotImplementedErrors, which means > > that somewhere down the line the methods have to be implemented. That was > > one of the problems that I wanted to get around, because it leaves too > much > > code to be written. And makes having different implementations for the > > methods, i.e., cython or Fortran, less transparent than just writing a > set > > of base functions (add, sub, etc.) in those languages. But I think we are > > starting from somewhat different places. To me, the base functions were > > primary and the classes conveniences built on top of them. That is more > > flexible, IMHO, and doesn't tie one into a big system of class > inheritance, > > etc., which is one of the drawbacks I see in a lot in C++ projects. The > GNU > > scientific library would be an example of that where the pieces get > tangled > > up in the whole. > > I'm not sure I see why returning all those NotImplementedErrors is a > problem - yes, the code needs to be written to override the methods > that return NotImplementedError, but that code needs to be written no > matter how we set up an interface to it. It's not trivial to do a complete workup. My template class is 698 lines long, but it exists once in one place, and that does the trick for *all* the classes. > And having the base class > allows us to take any code that is the same between bases - like the > companion matrix code - and put it in the base class. So the only > "wasted" code here is the one-line wrappers that return > NotImplementedError, and these serve as documentation for users - any > such method is something they're going to have to implement for their > own polynomial representations. If you look at the three different > representations I have currently implemented, there isn't really much > repeated code. > > But it is also incomplete. And you have to have the separate functions anyway, so it is easier if you just need to write them and then you are done. If you want to change or fix the class interface, you don't have to touch the implementations of the functions. If you want to change the implementations, then you don't need to touch the interface. The two parts become mostly independent, which is generally a good thing. This approach - of having an abstract base class whose operations are > defined but not implemented - is a common design, albeit more common > in languages that don't use duck-typing. There's no reason I couldn't > just omit those methods from the base class, but I think they serve a > useful documentary function. > But it wasn't used much for the C++ STL, which is mostly template based, and there was a reason that approach was chosen over inheritance. And even there if you look at some of the implementations there is some tricky downcasting of types to get generality. I think the functional programming approach solves those particular problems more transparently. > > > I think your method of getting the companion matrix is clever, and having > > definitions for zero, one, and x is a useful touch that should probably > be > > part of any interface. I'm less convinced by the use of an empty list for > > zero. > > This choice is a natural one in some ways, because it means you never > normally have a leading coefficient of zero for polynomials of any > degree. It also means degree = len(coefficients)-1, apart from > polynomials that are represented in a degree-inflated manner. And the > places you need to special-case it are almost all places you would > need to special-case zero polynomials anyway. I realize it may not be > so convenient for a vector-of-coefficients interface, but I think even > there it's the natural approach. > > But does it avoid special casing? Can you add it transparently to another array? In [1]: x = arange(3) In [2]: x + [] --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /home/charris/ in () ValueError: shape mismatch: objects cannot be broadcast to a single shape In [4]: x + [0] Out[4]: array([0, 1, 2]) I think it is a complication. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Tue Oct 13 19:55:18 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 13 Oct 2009 19:55:18 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/13 Charles R Harris : > >> I'm not sure I see why returning all those NotImplementedErrors is a >> problem - yes, the code needs to be written to override the methods >> that return NotImplementedError, but that code needs to be written no >> matter how we set up an interface to it. > > It's not trivial to do a complete workup. My template class is 698 lines > long, but it exists once in one place, and that does the trick for *all* the > classes. Aha. I think I see where the confusion is. In my proposed interface, the coefficient-oriented functions would *be* the ones in the Basis object. So, for example, chebadd(c1,c2,interval) -> ChebyshevBasis(interval).add(c1, c2) or, more likely, you'd have somewhere earlier cheb = ChebyshevBasis(interval) and then it's chebadd(c1,c2,interval) -> cheb.add(c1,c2) So the coefficient-oriented interface that we need to have is what lives in the basis objects, and it is what the Polynomial object uses to implement (e.g.) p1*p2. There would be no separate chebmul, polymul, lagrangemul, .... >> And having the base class >> allows us to take any code that is the same between bases - like the >> companion matrix code - and put it in the base class. So the only >> "wasted" code here is the one-line wrappers that return >> NotImplementedError, and these serve as documentation for users - any >> such method is something they're going to have to implement for their >> own polynomial representations. If you look at the three different >> representations I have currently implemented, there isn't really much >> repeated code. > > But it is also incomplete. And you have to have the separate functions > anyway, so it is easier if you just need to write them and then you are > done. If you want to change or fix the class interface, you don't have to > touch the implementations of the functions. If you want to change the > implementations, then you don't need to touch the interface. The two parts > become mostly independent, which is generally a good thing. The Basis object is effectively just a namespace holding the collection of functions, and possibly any data they need (interval, for example). So the separation you're describing is still present in my proposed interface; it's the separation between Basis objects (which contain the functions but no particular plumbing) and the Polynomial objects (which are all plumbing). >> > I think your method of getting the companion matrix is clever, and >> > having >> > definitions for zero, one, and x is a useful touch that should probably >> > be >> > part of any interface. I'm less convinced by the use of an empty list >> > for >> > zero. >> >> This choice is a natural one in some ways, because it means you never >> normally have a leading coefficient of zero for polynomials of any >> degree. It also means degree = len(coefficients)-1, apart from >> polynomials that are represented in a degree-inflated manner. And the >> places you need to special-case it are almost all places you would >> need to special-case zero polynomials anyway. I realize it may not be >> so convenient for a vector-of-coefficients interface, but I think even >> there it's the natural approach. >> > > But does it avoid special casing? Can you add it transparently to another > array? > > In [1]: x = arange(3) > > In [2]: x + [] > --------------------------------------------------------------------------- > ValueError??????????????????????????????? Traceback (most recent call last) > > /home/charris/ in () > > ValueError: shape mismatch: objects cannot be broadcast to a single shape > > In [4]: x + [0] > Out[4]: array([0, 1, 2]) This is a special case anyway - it's one of the very few instances in which you can add coefficients the way you're describing. Normally you have to extend one or the other so that they're the same length (and in some bases this is a non-trivial operations, rather than simply padding with zeros). If what you mean here is to add a scalar, there's no reason to wrap it in a list; In [3]: np.arange(3)+0 Out[3]: array([0, 1, 2]) In [4]: np.arange(3)+2 Out[4]: array([2, 3, 4]) If what you're adding is actually a vector of coefficients, you should be surprised if it works: In [5]: np.arange(3)+np.arange(2) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /home/peridot/ in () ValueError: shape mismatch: objects cannot be broadcast to a single shape In [6]: np.arange(3)+np.arange(1) Out[6]: array([0, 1, 2]) > I think it is a complication. It's a question of trading one set of special cases for another. If the zero polynomial is the only polynomial with zero leading coefficient, that's going to force a fair number of special cases as well. (Polynomials with zero leading coefficient that are generated as a result of calculations are trouble of a different sort, since it's entirely likely that the leading coefficient will be corrupted by roundoff error and come out to something near zero, where the notion of "near" is unclear.) In any case, it's only really important for people who want to use the coefficient-array interface; the polynomial class can hide this detail under the rug. I'm not terribly attached to this representation. Anne From charlesr.harris at gmail.com Tue Oct 13 20:15:37 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 18:15:37 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 5:55 PM, Anne Archibald wrote: > 2009/10/13 Charles R Harris : > > > >> I'm not sure I see why returning all those NotImplementedErrors is a > >> problem - yes, the code needs to be written to override the methods > >> that return NotImplementedError, but that code needs to be written no > >> matter how we set up an interface to it. > > > > It's not trivial to do a complete workup. My template class is 698 lines > > long, but it exists once in one place, and that does the trick for *all* > the > > classes. > > Aha. I think I see where the confusion is. In my proposed interface, > the coefficient-oriented functions would *be* the ones in the Basis > object. So, for example, > > chebadd(c1,c2,interval) -> ChebyshevBasis(interval).add(c1, c2) > > or, more likely, you'd have somewhere earlier > > cheb = ChebyshevBasis(interval) > > and then it's > > chebadd(c1,c2,interval) -> cheb.add(c1,c2) > > So the coefficient-oriented interface that we need to have is what > lives in the basis objects, and it is what the Polynomial object uses > to implement (e.g.) p1*p2. There would be no separate chebmul, > polymul, lagrangemul, .... > > But the separate functions are what we want. They provide the most flexible low level implementation and make the fewest assumptions as to what the programmer might want to use them for. You don't really have a base class, you essentially have classes derived from different base classes, which isn't so different from what I am proposing. > >> And having the base class > >> allows us to take any code that is the same between bases - like the > >> companion matrix code - and put it in the base class. So the only > >> "wasted" code here is the one-line wrappers that return > >> NotImplementedError, and these serve as documentation for users - any > >> such method is something they're going to have to implement for their > >> own polynomial representations. If you look at the three different > >> representations I have currently implemented, there isn't really much > >> repeated code. > > > > But it is also incomplete. And you have to have the separate functions > > anyway, so it is easier if you just need to write them and then you are > > done. If you want to change or fix the class interface, you don't have to > > touch the implementations of the functions. If you want to change the > > implementations, then you don't need to touch the interface. The two > parts > > become mostly independent, which is generally a good thing. > > The Basis object is effectively just a namespace holding the > collection of functions, and possibly any data they need (interval, > for example). So the separation you're describing is still present in > my proposed interface; it's the separation between Basis objects > (which contain the functions but no particular plumbing) and the > Polynomial objects (which are all plumbing). > Why not let the module hold the functions? It is the more natural namespace for python. > > >> > I think your method of getting the companion matrix is clever, and > >> > having > >> > definitions for zero, one, and x is a useful touch that should > probably > >> > be > >> > part of any interface. I'm less convinced by the use of an empty list > >> > for > >> > zero. > >> > >> This choice is a natural one in some ways, because it means you never > >> normally have a leading coefficient of zero for polynomials of any > >> degree. It also means degree = len(coefficients)-1, apart from > >> polynomials that are represented in a degree-inflated manner. And the > >> places you need to special-case it are almost all places you would > >> need to special-case zero polynomials anyway. I realize it may not be > >> so convenient for a vector-of-coefficients interface, but I think even > >> there it's the natural approach. > >> > > > > But does it avoid special casing? Can you add it transparently to another > > array? > > > > In [1]: x = arange(3) > > > > In [2]: x + [] > > > --------------------------------------------------------------------------- > > ValueError Traceback (most recent call > last) > > > > /home/charris/ in () > > > > ValueError: shape mismatch: objects cannot be broadcast to a single shape > > > > In [4]: x + [0] > > Out[4]: array([0, 1, 2]) > > This is a special case anyway - it's one of the very few instances in > which you can add coefficients the way you're describing. Normally you > have to extend one or the other so that they're the same length (and > in some bases this is a non-trivial operations, rather than simply > padding with zeros). If what you mean here is to add a scalar, there's > no reason to wrap it in a list; > Consistency. Usually these things are thought of as algebras with a unit over the scalars. > > In [3]: np.arange(3)+0 > Out[3]: array([0, 1, 2]) > > In [4]: np.arange(3)+2 > Out[4]: array([2, 3, 4]) > > If what you're adding is actually a vector of coefficients, you should > be surprised if it works: > > In [5]: np.arange(3)+np.arange(2) > --------------------------------------------------------------------------- > ValueError Traceback (most recent call last) > > /home/peridot/ in () > > ValueError: shape mismatch: objects cannot be broadcast to a single shape > > In [6]: np.arange(3)+np.arange(1) > Out[6]: array([0, 1, 2]) > > > I think it is a complication. > > It's a question of trading one set of special cases for another. If > the zero polynomial is the only polynomial with zero leading > coefficient, that's going to force a fair number of special cases as > well. (Polynomials with zero leading coefficient that are generated as > a result of calculations are trouble of a different sort, since it's > entirely likely that the leading coefficient will be corrupted by > roundoff error and come out to something near zero, where the notion > of "near" is unclear.) > Yes, that is always going to be a problem. > > In any case, it's only really important for people who want to use the > coefficient-array interface; the polynomial class can hide this detail > under the rug. I'm not terribly attached to this representation. > > Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Oct 13 20:59:09 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 18:59:09 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: I've broken up the Chebyshev file into a class part and a function part and attached to show how this works. The file names are provisional. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: polyclass.py Type: text/x-python Size: 19313 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: polycheb.py Type: text/x-python Size: 23657 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test_polycheb.py Type: text/x-python Size: 7021 bytes Desc: not available URL: From peridot.faceted at gmail.com Tue Oct 13 21:24:09 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 13 Oct 2009 21:24:09 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/13 Charles R Harris : > > > On Tue, Oct 13, 2009 at 5:55 PM, Anne Archibald > wrote: >> >> 2009/10/13 Charles R Harris : >> > >> >> I'm not sure I see why returning all those NotImplementedErrors is a >> >> problem - yes, the code needs to be written to override the methods >> >> that return NotImplementedError, but that code needs to be written no >> >> matter how we set up an interface to it. >> > >> > It's not trivial to do a complete workup. My template class is 698 lines >> > long, but it exists once in one place, and that does the trick for *all* >> > the >> > classes. >> >> Aha. I think I see where the confusion is. In my proposed interface, >> the coefficient-oriented functions would *be* the ones in the Basis >> object. So, for example, >> >> chebadd(c1,c2,interval) -> ChebyshevBasis(interval).add(c1, c2) >> >> or, more likely, you'd have somewhere earlier >> >> cheb = ChebyshevBasis(interval) >> >> and then it's >> >> chebadd(c1,c2,interval) -> cheb.add(c1,c2) >> >> So the coefficient-oriented interface that we need to have is what >> lives in the basis objects, and it is what the Polynomial object uses >> to implement (e.g.) p1*p2. There would be no separate chebmul, >> polymul, lagrangemul, .... >> > > But the separate functions are what we want. They provide the most flexible > low level implementation and make the fewest assumptions as to what the > programmer might want to use them for. You don't really have a base class, > you essentially have classes derived from different base classes, which > isn't so different from what I am proposing. Do we really want the separate functions, rather than methods on a basis object? Why? I don't really see how they're more flexible or make fewer assumptions, and they become very cumbersome (and prevent some important optimizations) when working with polynomials in the Lagrange basis. It really can be the difference between chebadd(...) and cheb.add(...), in the (common?) case where people aren't specifying the interval. And if people do want to specify the interval, using a basis object makes it much easier to avoid forgetting it and getting nonsensical results (or worse, sensible results until you try a different interval). In this respect, the current interface in chebyshev.py looks error-prone to me: you can't suppy an interval to chebadd and chebmul, but if you forget it for chebval you get bogus answers. (And in fact chebint and chebder are wrong, or at least misleading, if you use a different interval.) I'm not sure what you mean when you say I don't really have a base class: Basis is literally the base class for all Basis objects, and it provides nontrivial functionality (e.g. the power implementation) that would otherwise need to be repeated for each representation. Polynomial is the base class for all polynomials. >> The Basis object is effectively just a namespace holding the >> collection of functions, and possibly any data they need (interval, >> for example). So the separation you're describing is still present in >> my proposed interface; it's the separation between Basis objects >> (which contain the functions but no particular plumbing) and the >> Polynomial objects (which are all plumbing). > > Why not let the module hold the functions? It is the more natural namespace > for python. Because there is important context - intervals, lists of specified points - that must be provided to all the coefficient-array functions. And collections of methods on objects are very natural in python. >> > In [4]: x + [0] >> > Out[4]: array([0, 1, 2]) >> >> This is a special case anyway - it's one of the very few instances in >> which you can add coefficients the way you're describing. Normally you >> have to extend one or the other so that they're the same length (and >> in some bases this is a non-trivial operations, rather than simply >> padding with zeros). If what you mean here is to add a scalar, there's >> no reason to wrap it in a list; > > Consistency. Usually these things are thought of as algebras with a unit > over the scalars. I'm not sure I understand this - consistency with what? It is only scalars you can reasonably add to coefficient arrays by wrapping in a list; other coefficient arrays must be the same length as the other (array) summand or they will fail. And scalars work just as well without wrapping in a list, so why do it? Anne P.S. the current version on github now has the code in the Basis objects, so that their methods can be used to operate on coefficient arrays, and added a file demonstrating this use. -A From peridot.faceted at gmail.com Tue Oct 13 21:39:52 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 13 Oct 2009 21:39:52 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/13 Charles R Harris : > I've broken up the Chebyshev file into a class part and a function part and > attached to show how this works. The file names are provisional. I see what you were getting at now, about using a function to manufacture classes, but I'm not convinced it's a good idea. For example, how would a scipy user produce a polynomial object with some extra operations (a fast convex hull, perhaps, for polynomials in the Bernstein basis)? Wouldn't they have to reimplement polyclass, cutting and pasting most of the code? What about polynomials for which "interval" is not the right extra data to specify? This is the case for both Lagrange polynomials (either an interval, a list of points, or both) and power basis polynomials (where a center point is the relevant information). There also doesn't seem to be any place for extra information about the polynomial's basis (orthogonal roots and weights, Lagrange interpolation weights and derivative matrices) to be stored. Anne From charlesr.harris at gmail.com Tue Oct 13 22:38:05 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 13 Oct 2009 20:38:05 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 7:24 PM, Anne Archibald wrote: > 2009/10/13 Charles R Harris : > > > > > > On Tue, Oct 13, 2009 at 5:55 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/13 Charles R Harris : > >> > > >> >> I'm not sure I see why returning all those NotImplementedErrors is a > >> >> problem - yes, the code needs to be written to override the methods > >> >> that return NotImplementedError, but that code needs to be written no > >> >> matter how we set up an interface to it. > >> > > >> > It's not trivial to do a complete workup. My template class is 698 > lines > >> > long, but it exists once in one place, and that does the trick for > *all* > >> > the > >> > classes. > >> > >> Aha. I think I see where the confusion is. In my proposed interface, > >> the coefficient-oriented functions would *be* the ones in the Basis > >> object. So, for example, > >> > >> chebadd(c1,c2,interval) -> ChebyshevBasis(interval).add(c1, c2) > >> > >> or, more likely, you'd have somewhere earlier > >> > >> cheb = ChebyshevBasis(interval) > >> > >> and then it's > >> > >> chebadd(c1,c2,interval) -> cheb.add(c1,c2) > >> > >> So the coefficient-oriented interface that we need to have is what > >> lives in the basis objects, and it is what the Polynomial object uses > >> to implement (e.g.) p1*p2. There would be no separate chebmul, > >> polymul, lagrangemul, .... > >> > > > > But the separate functions are what we want. They provide the most > flexible > > low level implementation and make the fewest assumptions as to what the > > programmer might want to use them for. You don't really have a base > class, > > you essentially have classes derived from different base classes, which > > isn't so different from what I am proposing. > > Do we really want the separate functions, rather than methods on a > basis object? Why? I don't really see how they're more flexible or > make fewer assumptions, and they become very cumbersome (and prevent > some important optimizations) when working with polynomials in the > Lagrange basis. It really can be the difference between chebadd(...) > and cheb.add(...), in the (common?) case where people aren't > But why do that if you already have the name space when you import the module? It's redundant. > specifying the interval. And if people do want to specify the > interval, using a basis object makes it much easier to avoid > forgetting it and getting nonsensical results (or worse, sensible > results until you try a different interval). > > That is because they are low level building blocks, they should have the absolute minimum of crud stuck on, and that means the interval is [-1, 1]. If you need more, that is what a class is for. That how low level things should work, small functions doing the minimum amount provide the most flexibility. The programmer should be handed enough rope to hang themselves at that level if they so choose. That's why folks use C and not Pascal. > In this respect, the current interface in chebyshev.py looks > error-prone to me: you can't suppy an interval to chebadd and chebmul, > but if you forget it for chebval you get bogus answers. (And in fact > chebint and chebder are wrong, or at least misleading, if you use a > different interval.) > > Tracking things like intervals is best delegated to higher level functions where you can deal with it without worrying about the low level implementation at the same time. Of course there is some loop-de-loop involved to get to the ideal dividing line, but it is important to draw that line. > I'm not sure what you mean when you say I don't really have a base > class: Basis is literally the base class for all Basis objects, and it > provides nontrivial functionality (e.g. the power implementation) that > would otherwise need to be repeated for each representation. > Polynomial is the base class for all polynomials. > > >> The Basis object is effectively just a namespace holding the > >> collection of functions, and possibly any data they need (interval, > >> for example). So the separation you're describing is still present in > >> my proposed interface; it's the separation between Basis objects > >> (which contain the functions but no particular plumbing) and the > >> Polynomial objects (which are all plumbing). > > > > Why not let the module hold the functions? It is the more natural > namespace > > for python. > > Because there is important context - intervals, lists of specified > points - that must be provided to all the coefficient-array functions. > And collections of methods on objects are very natural in python. > > That is higher level stuff and doesn't belong at the bottom, it should be separate IMHO.The functions are the ingredients, not the cake. I've spent too much time pulling apart classes to get to the reusable bits not to want them separated out up front. Now you could start with the separate functions and then construct a basis to pass into the initializer. At that point you and I are operating along the same lines except I don't pass them to the contructor, I put them in the local environment where the class methods will find them, that's what the code completion does and Python handles that efficiently. They are consequently more like static (class) methods than instance methods. That way I end up with a whole new class that uses those functions, inherits from nothing but object, and has a minimal constructor. I'll admit that my class implementation may not be suitable for some of the other types of polynomials you are looking at, I hadn't been thinking about them yet. On the other hand it is a pretty complete implementation that requires nothing more than the functions, raises no NotImplementedErrors and should work with most of the orthogonal polynomial bases. And it can be reworked without reference to the low level functions. > >> > In [4]: x + [0] > >> > Out[4]: array([0, 1, 2]) > >> > >> This is a special case anyway - it's one of the very few instances in > >> which you can add coefficients the way you're describing. Normally you > >> have to extend one or the other so that they're the same length (and > >> in some bases this is a non-trivial operations, rather than simply > >> padding with zeros). If what you mean here is to add a scalar, there's > >> no reason to wrap it in a list; > > > > Consistency. Usually these things are thought of as algebras with a unit > > over the scalars. > > I'm not sure I understand this - consistency with what? It is only > scalars you can reasonably add to coefficient arrays by wrapping in a > list; other coefficient arrays must be the same length as the other > (array) summand or they will fail. And scalars work just as well > without wrapping in a list, so why do it? > Scalars should work with and without wrapping, no? But I think using an empty list for what is actually a polynomial all of whose coefficients are zero is mixing apples and oranges. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Wed Oct 14 00:52:36 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 14 Oct 2009 00:52:36 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> On Tue, Oct 13, 2009 at 10:38 PM, Charles R Harris wrote: > > > On Tue, Oct 13, 2009 at 7:24 PM, Anne Archibald > wrote: >> >> 2009/10/13 Charles R Harris : >> > >> > >> > On Tue, Oct 13, 2009 at 5:55 PM, Anne Archibald >> > >> > wrote: >> >> >> >> 2009/10/13 Charles R Harris : >> >> > >> >> >> I'm not sure I see why returning all those NotImplementedErrors is a >> >> >> problem - yes, the code needs to be written to override the methods >> >> >> that return NotImplementedError, but that code needs to be written >> >> >> no >> >> >> matter how we set up an interface to it. >> >> > >> >> > It's not trivial to do a complete workup. My template class is 698 >> >> > lines >> >> > long, but it exists once in one place, and that does the trick for >> >> > *all* >> >> > the >> >> > classes. >> >> >> >> Aha. I think I see where the confusion is. In my proposed interface, >> >> the coefficient-oriented functions would *be* the ones in the Basis >> >> object. So, for example, >> >> >> >> chebadd(c1,c2,interval) -> ChebyshevBasis(interval).add(c1, c2) >> >> >> >> or, more likely, you'd have somewhere earlier >> >> >> >> cheb = ChebyshevBasis(interval) >> >> >> >> and then it's >> >> >> >> chebadd(c1,c2,interval) -> cheb.add(c1,c2) >> >> >> >> So the coefficient-oriented interface that we need to have is what >> >> lives in the basis objects, and it is what the Polynomial object uses >> >> to implement (e.g.) p1*p2. There would be no separate chebmul, >> >> polymul, lagrangemul, .... >> >> >> > >> > But the separate functions are what we want. They provide the most >> > flexible >> > low level implementation and make the fewest assumptions as to what the >> > programmer might want to use them for. You don't really have a base >> > class, >> > you essentially have classes derived from different base classes, which >> > isn't so different from what I am proposing. >> >> Do we really want the separate functions, rather than methods on a >> basis object? Why? I don't really see how they're more flexible or >> make fewer assumptions, and they become very cumbersome (and prevent >> some important optimizations) when working with polynomials in the >> Lagrange basis. It really can be the difference between chebadd(...) >> and cheb.add(...), in the (common?) case where people aren't > > But why do that if you already have the name space when you import the > module? It's redundant. > >> >> specifying the interval. And if people do want to specify the >> interval, using a basis object makes it much easier to avoid >> forgetting it and getting nonsensical results (or worse, sensible >> results until you try a different interval). >> > > That is because they are low level building blocks, they should have the > absolute minimum of crud stuck on, and that means the interval is [-1, 1]. > If you need more, that is what a class is for. That how low level things > should work, small functions doing the minimum amount provide the most > flexibility. The programmer should be handed enough rope to hang themselves > at that level if they so choose. That's why folks use C and not Pascal. > >> >> In this respect, the current interface in chebyshev.py looks >> error-prone to me: you can't suppy an interval to chebadd and chebmul, >> but if you forget it for chebval you get bogus answers. (And in fact >> chebint and chebder are wrong, or at least misleading, if you use a >> different interval.) >> > > Tracking things like intervals is best delegated to higher level functions > where you can deal with it without worrying about the low level > implementation at the same time. Of course there is some loop-de-loop > involved to get to the ideal dividing line, but it is important to draw that > line. > >> >> I'm not sure what you mean when you say I don't really have a base >> class: Basis is literally the base class for all Basis objects, and it >> provides nontrivial functionality (e.g. the power implementation) that >> would otherwise need to be repeated for each representation. >> Polynomial is the base class for all polynomials. >> >> >> The Basis object is effectively just a namespace holding the >> >> collection of functions, and possibly any data they need (interval, >> >> for example). So the separation you're describing is still present in >> >> my proposed interface; it's the separation between Basis objects >> >> (which contain the functions but no particular plumbing) and the >> >> Polynomial objects (which are all plumbing). >> > >> > Why not let the module hold the functions? It is the more natural >> > namespace >> > for python. >> >> Because there is important context - intervals, lists of specified >> points - that must be provided to all the coefficient-array functions. >> And collections of methods on objects are very natural in python. >> > > That is higher level stuff and doesn't belong at the bottom, it should be > separate IMHO.The functions are the ingredients, not the cake. I've spent > too much time pulling apart classes to get to the reusable bits not to want > them separated out up front. Now you could start with the separate functions > and then construct a basis to pass into the initializer. At that point you > and I are operating along the same lines except I don't pass them to the > contructor, I put them in the local environment where the class methods will > find them, that's what the code completion does and Python handles that > efficiently. They are consequently more like static (class) methods than > instance methods. That way I end up with a whole new class that uses those > functions, inherits from nothing but object, and has a minimal constructor. > > I'll admit that my class implementation may not be suitable for some of the > other types of polynomials you are looking at, I hadn't been thinking about > them yet. On the other hand it is a pretty complete implementation that > requires nothing more than the functions, raises no NotImplementedErrors and > should work with most of the orthogonal polynomial bases. And it can be > reworked without reference to the low level functions. How expensive can _cseries_to_zseries, as_series and similar be? Since you currently have all main calculations in self-contained functions, you need to convert and check the inputs each time. If these were expensive intermediate results, then methods could save time in this. For short series, this doesn't seem to be a problem with polycheb, but inside a loop I'm not so sure. (as an aside: we were also thinking about moving some methods in statsmodels to functions, but in many cases we would need a larger number of function arguments of intermediate results or duplicate calculations, which makes it less useful.) I was playing a bit with the cats, and inheritance seems to work without problems. Following roughly the discussion, I would find it useful if the functions specify the used or assumed domain in the doc strings. Since I'm not very familiar with Chebyshev polynomials, I wouldn't know which functions would require rescaling of the domain. Josef >> >> >> > In [4]: x + [0] >> >> > Out[4]: array([0, 1, 2]) >> >> >> >> This is a special case anyway - it's one of the very few instances in >> >> which you can add coefficients the way you're describing. Normally you >> >> have to extend one or the other so that they're the same length (and >> >> in some bases this is a non-trivial operations, rather than simply >> >> padding with zeros). If what you mean here is to add a scalar, there's >> >> no reason to wrap it in a list; >> > >> > Consistency. Usually these things are thought of as algebras with a unit >> > over the scalars. >> >> I'm not sure I understand this - consistency with what? It is only >> scalars you can reasonably add to coefficient arrays by wrapping in a >> list; other coefficient arrays must be the same length as the other >> (array) summand or they will fail. And scalars work just as well >> without wrapping in a list, so why do it? > > Scalars should work with and without wrapping, no? But I think using an > empty list for what is actually a polynomial all of whose coefficients are > zero is mixing apples and oranges. > > Chuck > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From peridot.faceted at gmail.com Wed Oct 14 01:03:35 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Wed, 14 Oct 2009 01:03:35 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/13 Charles R Harris : > > On Tue, Oct 13, 2009 at 7:24 PM, Anne Archibald > wrote: >> specifying the interval. And if people do want to specify the >> interval, using a basis object makes it much easier to avoid >> forgetting it and getting nonsensical results (or worse, sensible >> results until you try a different interval). >> > > That is because they are low level building blocks, they should have the > absolute minimum of crud stuck on, and that means the interval is [-1, 1]. > If you need more, that is what a class is for. That how low level things > should work, small functions doing the minimum amount provide the most > flexibility. The programmer should be handed enough rope to hang themselves > at that level if they so choose. That's why folks use C and not Pascal. Well, the glib answer would be that we're using python, so we should choose a pythonic way of doing things. But clearly having the low-level functions is important to some potential users - you and Robert Kern, plus surely more other people - so where possible the code should accommodate the as-low-level-as-possible approach. How's this for a compromise? The high-level class interface has a Polynomial object and various Basis subclasses. For representations where having a featureful Basis object is important, the implementation should just write one, inheriting as appropriate. For representations that don't need this extra stuff, people can just write a collection of functions acting on coefficient arrays (as you did for the Chebyshev polynomials) and create a basis object using ChebyshevBasis = generate_basis("ChebyshevBasis", mul=chebmul, ...) I have a demonstration implementation in git now. >> I'm not sure I understand this - consistency with what? It is only >> scalars you can reasonably add to coefficient arrays by wrapping in a >> list; other coefficient arrays must be the same length as the other >> (array) summand or they will fail. And scalars work just as well >> without wrapping in a list, so why do it? > > Scalars should work with and without wrapping, no? But I think using an > empty list for what is actually a polynomial all of whose coefficients are > zero is mixing apples and oranges. No, I don't think wrapping is a sensible thing to do to scalars. I think it's a bizarre quirk (in this instance) that numpy broadcasting allows you to add a one-element array to a three-element array - and in fact it's wrong! In the power basis, [1,2] is the polynomial 1+2*X. [1,2]+[1] = [2,3] is the polynomial 2+3*X - not 1+2*X + 1. So you just plain can't (correctly) add scalars to coefficient arrays, with the sole exception of zero. As for the conceptual representation, every polynomial is implicitly extended by zeros, and every nonzero polynomial normally stops after the last nonzero entry. Using the array of shape (0,) follows these rules exactly, while using the array of shape (1,) containing zero breaks the second, albeit in a not-very-important way (since trailing zeros can occur in non-normalized polynomials). Anne From d.l.goldsmith at gmail.com Wed Oct 14 04:58:51 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 14 Oct 2009 01:58:51 -0700 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <4AD4A58A.5070308@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> Message-ID: <45d1ab480910140158q43d892b9i8a3dea41e69d1339@mail.gmail.com> Hi, Michael. Say, when you get a chance, can you take a look at http://docs.scipy.org/numpy/docs/numpy.core.defchararray? I feel like, at the very least, the Note should be either modified or deleted, but are there any other changes to be made in light of the work you've done? DG On Tue, Oct 13, 2009 at 9:06 AM, Michael Droettboom wrote: > Ok -- I can add them to __all__ in defchararray.py. > > Is there a good way for me to check that my changes will work in the > documentation editor? I rebuilt the docs in the source tree (by going > in to doc/ and running "make html") and these methods *were* picked up. > How do other people work and test their changes? > > I also noticed that the cross-referencing from the chararray methods to > the free functions in defchararray is broken in the online documentation > editor. Again, mostly just my ignorance of how all the pieces go > together. Can someone explain what it should be, and then I'll make all > the necessary corrections? E.g. what should the "See also" section of > defchararray.chararray.isalnum look like to reference defchararray.isalnum? > > Also -- How can I change a review status from "Unimportant"? > > Mike > > josef.pktd at gmail.com wrote: > > On Tue, Oct 13, 2009 at 11:47 AM, Michael Droettboom > wrote: > > > >> josef.pktd at gmail.com wrote: > >> > >>> On Tue, Oct 13, 2009 at 11:23 AM, Michael Droettboom > wrote: > >>> > >>> > >>>> Ralf Gommers wrote: > >>>> > >>>> > >>>>> On Mon, Oct 12, 2009 at 5:40 PM, Michael Droettboom >>>>> > wrote: > >>>>> > >>>>> I was able to make my big chararray commit today. If I > understand > >>>>> correctly, I need to wait 24 hours for the doc editor to sync > with > >>>>> SVN, > >>>>> and then I should mark all the chararray-related docstrings as > "needs > >>>>> review". > >>>>> > >>>>> > >>>>> > >>>> Ok -- I'm a bit lost in the documentation editor. > >>>> > >>>> I go to the defchararray module, and I don't see any of the new > >>>> functions or old methods: > >>>> > >>>> http://docs.scipy.org/numpy/docs/numpy.core.defchararray > >>>> > >>>> The "Review status" is stuck on "Unimportant". Not sure what that > means. > >>>> > >>>> I go to the chararray class and I can't see any methods: > >>>> > >>>> > >>>> > >>> > >>>> http://docs.scipy.org/numpy/docs/numpy.core.defchararray.chararray/ > >>>> > >>>> > >>> I see many methods at the bottom of this page, I don't know the names > >>> of the new methods to see whether they are there. > >>> > >>> > >> Odd. They weren't there a moment ago. Sorry for the noise. > >> > > > > I also found the new functions at > > > > > http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.char.rst/#id4 > > > > but the doc editor is not picking up the docstrings > > > > I think they are only picked up if they are in an __all__ > > > > currently: defchararray.__all__ = ['chararray'] > > > > Josef > > > > > >> Mike > >> > >> -- > >> Michael Droettboom > >> Science Software Branch > >> Operations and Engineering Division > >> Space Telescope Science Institute > >> Operated by AURA for NASA > >> > >> _______________________________________________ > >> Scipy-dev mailing list > >> Scipy-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/scipy-dev > >> > >> > > _______________________________________________ > > Scipy-dev mailing list > > Scipy-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdroe at stsci.edu Wed Oct 14 09:48:22 2009 From: mdroe at stsci.edu (Michael Droettboom) Date: Wed, 14 Oct 2009 09:48:22 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <45d1ab480910131448u6547db11xe2706f05eb14a09b@mail.gmail.com> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> <20091013165559.GA20164@phare.normalesup.org> <45d1ab480910131448u6547db11xe2706f05eb14a09b@mail.gmail.com> Message-ID: <4AD5D6A6.2080209@stsci.edu> Yes -- they should all eventually be at needs review, but I held off yesterday to wait for all of the free functions to show up in the doc editor, and a way to test the cross-referencing from the methods to the functions was correct, since those will probably require editing en masse once I know the solution. I didn't want to have to run through all of them twice. I should be able to find some time to do this today, though perhaps not enough to complete them all. Mike David Goldsmith wrote: > OK, I'm coming into this thread a bit late, but I see that many of the > defchararray.chararray docstrings are still at "Being Written" (which > is better than "Needs Editing," where a couple still are) but they > should all be at "Needs Review," yes? I'm happy to do this, Mike, but > if you'd like the "practice" using the doc wiki, what I'd recommend is: > > 0) login to the wiki and, as suggested by Ralf, go to the list of all > docstrings, http://docs.scipy.org/numpy/docs/ (accessible via the > Docstrings link atop every wiki page); > > 1) scroll (way) down to the light gray background "Being written" > docstrings and look for the defchararray.chararray docstring links (I > count 23 of them; 'twas 24, but I promoted one while writing this out > to verify these instructions); :-) > > 2) click on 'em one at a time (you'll probably want to open them in a > new tab or window, so you don't have to keep going back to the > docstring page); if you get "bored," let me know and I'll finish 'em off; > > 3) look for the gray box on the right which says "Review status:" and > then in a drop-down has "Being written" in it, and a "Change" button > next to that (if you don't see the Change button, you're not logged-in); > > 4) click on the drop-down triangle to expose the other status options, > one of which should be "Needs review" (which should be in black font, > assuming you have successfully been granted the privilege to execute > this promotion; if not, it will be in gray font) and click on that; > > 5) then click the Change button and the Review status box should > change to pink; > > 6) you're done! (with one docstring; if you wish to verify that the > change "took," reload the "all docstrings" page and visually confirm > that the docstring you just promoted is no longer in the "Being > written" section and is now in fact in the "Needs review" section). > > HTH; again, I'm happy to do this once you get bored, just let me know > via email (off-list is fine). > > DG > > On Tue, Oct 13, 2009 at 9:55 AM, Gael Varoquaux > > > wrote: > > On Tue, Oct 13, 2009 at 06:53:26PM +0200, Ralf Gommers wrote: > > One of the admins has to give you reviewer permissions. > > Done. > > Ga?l > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > ------------------------------------------------------------------------ > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA From charlesr.harris at gmail.com Wed Oct 14 11:08:10 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 14 Oct 2009 09:08:10 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: On Tue, Oct 13, 2009 at 10:52 PM, wrote: > On Tue, Oct 13, 2009 at 10:38 PM, Charles R Harris > wrote: > > > > > > On Tue, Oct 13, 2009 at 7:24 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/13 Charles R Harris : > >> > > >> > > >> > On Tue, Oct 13, 2009 at 5:55 PM, Anne Archibald > >> > > >> > wrote: > >> >> > >> >> 2009/10/13 Charles R Harris : > >> >> > > >> >> >> I'm not sure I see why returning all those NotImplementedErrors is > a > >> >> >> problem - yes, the code needs to be written to override the > methods > >> >> >> that return NotImplementedError, but that code needs to be written > >> >> >> no > >> >> >> matter how we set up an interface to it. > >> >> > > >> >> > It's not trivial to do a complete workup. My template class is 698 > >> >> > lines > >> >> > long, but it exists once in one place, and that does the trick for > >> >> > *all* > >> >> > the > >> >> > classes. > >> >> > >> >> Aha. I think I see where the confusion is. In my proposed interface, > >> >> the coefficient-oriented functions would *be* the ones in the Basis > >> >> object. So, for example, > >> >> > >> >> chebadd(c1,c2,interval) -> ChebyshevBasis(interval).add(c1, c2) > >> >> > >> >> or, more likely, you'd have somewhere earlier > >> >> > >> >> cheb = ChebyshevBasis(interval) > >> >> > >> >> and then it's > >> >> > >> >> chebadd(c1,c2,interval) -> cheb.add(c1,c2) > >> >> > >> >> So the coefficient-oriented interface that we need to have is what > >> >> lives in the basis objects, and it is what the Polynomial object uses > >> >> to implement (e.g.) p1*p2. There would be no separate chebmul, > >> >> polymul, lagrangemul, .... > >> >> > >> > > >> > But the separate functions are what we want. They provide the most > >> > flexible > >> > low level implementation and make the fewest assumptions as to what > the > >> > programmer might want to use them for. You don't really have a base > >> > class, > >> > you essentially have classes derived from different base classes, > which > >> > isn't so different from what I am proposing. > >> > >> Do we really want the separate functions, rather than methods on a > >> basis object? Why? I don't really see how they're more flexible or > >> make fewer assumptions, and they become very cumbersome (and prevent > >> some important optimizations) when working with polynomials in the > >> Lagrange basis. It really can be the difference between chebadd(...) > >> and cheb.add(...), in the (common?) case where people aren't > > > > But why do that if you already have the name space when you import the > > module? It's redundant. > > > >> > >> specifying the interval. And if people do want to specify the > >> interval, using a basis object makes it much easier to avoid > >> forgetting it and getting nonsensical results (or worse, sensible > >> results until you try a different interval). > >> > > > > That is because they are low level building blocks, they should have the > > absolute minimum of crud stuck on, and that means the interval is [-1, > 1]. > > If you need more, that is what a class is for. That how low level things > > should work, small functions doing the minimum amount provide the most > > flexibility. The programmer should be handed enough rope to hang > themselves > > at that level if they so choose. That's why folks use C and not Pascal. > > > >> > >> In this respect, the current interface in chebyshev.py looks > >> error-prone to me: you can't suppy an interval to chebadd and chebmul, > >> but if you forget it for chebval you get bogus answers. (And in fact > >> chebint and chebder are wrong, or at least misleading, if you use a > >> different interval.) > >> > > > > Tracking things like intervals is best delegated to higher level > functions > > where you can deal with it without worrying about the low level > > implementation at the same time. Of course there is some loop-de-loop > > involved to get to the ideal dividing line, but it is important to draw > that > > line. > > > >> > >> I'm not sure what you mean when you say I don't really have a base > >> class: Basis is literally the base class for all Basis objects, and it > >> provides nontrivial functionality (e.g. the power implementation) that > >> would otherwise need to be repeated for each representation. > >> Polynomial is the base class for all polynomials. > >> > >> >> The Basis object is effectively just a namespace holding the > >> >> collection of functions, and possibly any data they need (interval, > >> >> for example). So the separation you're describing is still present in > >> >> my proposed interface; it's the separation between Basis objects > >> >> (which contain the functions but no particular plumbing) and the > >> >> Polynomial objects (which are all plumbing). > >> > > >> > Why not let the module hold the functions? It is the more natural > >> > namespace > >> > for python. > >> > >> Because there is important context - intervals, lists of specified > >> points - that must be provided to all the coefficient-array functions. > >> And collections of methods on objects are very natural in python. > >> > > > > That is higher level stuff and doesn't belong at the bottom, it should be > > separate IMHO.The functions are the ingredients, not the cake. I've spent > > too much time pulling apart classes to get to the reusable bits not to > want > > them separated out up front. Now you could start with the separate > functions > > and then construct a basis to pass into the initializer. At that point > you > > and I are operating along the same lines except I don't pass them to the > > contructor, I put them in the local environment where the class methods > will > > find them, that's what the code completion does and Python handles that > > efficiently. They are consequently more like static (class) methods than > > instance methods. That way I end up with a whole new class that uses > those > > functions, inherits from nothing but object, and has a minimal > constructor. > > > > I'll admit that my class implementation may not be suitable for some of > the > > other types of polynomials you are looking at, I hadn't been thinking > about > > them yet. On the other hand it is a pretty complete implementation that > > requires nothing more than the functions, raises no NotImplementedErrors > and > > should work with most of the orthogonal polynomial bases. And it can be > > reworked without reference to the low level functions. > > How expensive can _cseries_to_zseries, as_series and similar be? > Since you currently have all main calculations in self-contained > functions, you need to convert and check the inputs each time. If > these were expensive intermediate results, then methods could save > time in this. For short series, this doesn't seem to be a problem with > polycheb, but inside a loop I'm not so sure. > (as an aside: we were also thinking about moving some methods in > statsmodels to functions, but in many cases we would need a larger > number of function arguments of intermediate results or duplicate > calculations, which makes it less useful.) > > One reason for the low level functions is that they are short and can be easily converted to cython functions or be library functions written in fortran or whatever. The z series could be done that way, but at the cython level they could probably be dispensed with and the underlying Cheybyshev series used instead, that's one reason they are private functions. What they brought to the python case was some vectorization of the algorithms. Another way to look at them is as symbolic Fourier transforms. Finding the proper collection of small routines can be hard work, which seems like it might be the case in the statsmodels. Sometimes profiling will reveal small bits that are bottlenecks and they are potential candidates. Another view that helps is to separate the data shuffling and verification to the python level and think of that as an interface to some underlying routines in a faster language. I don't know enough about statsmodels to make any more useful suggestions ;) > I was playing a bit with the cats, and inheritance seems to work > without problems. > > Things can be done with inheritance, but in this case the need was for a common mass produced interface and the desire to avoid work. Inheritance isn't the only way to build interfaces, unless it happens to be the only tool at hand. Templated classes are another way that is closer to the duck typing model and they seem to work better in some circumstances. For instance, in the C++ STL templated classes are provided so that you can have, say, lists of different (uniform) types, but the lists all have the same methods. That works better than trying to use inheritance. I think that in general the templated class method is preferable because the design is flatter and more flexible, and it doesn't use hokey virtual base classes and such. Inheritance works best when you need to call a bunch of different objects through a common base class, i.e., the objects are all "is a" base class, and I don't think that requirement is there for the polynomial classes. The temptation of inheritance is to use it for implementation, and more often than not that leads to trouble down the line. The upshot is that I was looking for the python equivalent of templated classes and completions looked to be the best bet, so I gave it a shot to see how it looked. I think it looks OK, but the functional approach is a bit strange to folks like myself who have spent most of our lives working with imperative languages. Following roughly the discussion, I would find it useful if the > functions specify the used or assumed domain in the doc strings. Since > I'm not very familiar with Chebyshev polynomials, I wouldn't know > which functions would require rescaling of the domain. > > Yes, that should be documented. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Wed Oct 14 12:21:33 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 14 Oct 2009 10:21:33 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Tue, Oct 13, 2009 at 11:03 PM, Anne Archibald wrote: > 2009/10/13 Charles R Harris : > > > > On Tue, Oct 13, 2009 at 7:24 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> specifying the interval. And if people do want to specify the > >> interval, using a basis object makes it much easier to avoid > >> forgetting it and getting nonsensical results (or worse, sensible > >> results until you try a different interval). > >> > > > > That is because they are low level building blocks, they should have the > > absolute minimum of crud stuck on, and that means the interval is [-1, > 1]. > > If you need more, that is what a class is for. That how low level things > > should work, small functions doing the minimum amount provide the most > > flexibility. The programmer should be handed enough rope to hang > themselves > > at that level if they so choose. That's why folks use C and not Pascal. > > Well, the glib answer would be that we're using python, so we should > choose a pythonic way of doing things. But clearly having the > low-level functions is important to some potential users - you and > Robert Kern, plus surely more other people - so where possible the > code should accommodate the as-low-level-as-possible approach. > > How's this for a compromise? The high-level class interface has a > Polynomial object and various Basis subclasses. For representations > where having a featureful Basis object is important, the > implementation should just write one, inheriting as appropriate. For > representations that don't need this extra stuff, people can just > write a collection of functions acting on coefficient arrays (as you > did for the Chebyshev polynomials) and create a basis object using > > I'm trying to get you to consider not using inheritance at all and see how things look. We might not want to go that way in the end, but inheritance tends to be overused. Probably because it is the first tool that comes to mind and because it looks like a cheap way to reuse code. But it is often not the best tool so folks need to step back and see if there aren't better ways to get things done. The fact that you are putting some functionality in the base class indicates that you are thinking along the lines of implementation. Passing a Basis object into a constructor also doesn't smell right. The question to ask is what role you want the Basis object to play and if it doesn't more naturally belong somewhere else if you weren't constrained by inheritance. In a sense it looks like you would like to derive the interface class from the basis object, but somehow that doesn't seem to work well with inheritance because then it is hard for the derived objects to share code. ChebyshevBasis = generate_basis("ChebyshevBasis", mul=chebmul, ...) > > Heh! Well it's does look like a compromise, or rather, design by committee ;) But it is something to think about. > I have a demonstration implementation in git now. > > OK. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Wed Oct 14 14:28:20 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 14 Oct 2009 14:28:20 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: <1cd32cbb0910141128j29e758bdy2971015cb369e88d@mail.gmail.com> On Wed, Oct 14, 2009 at 11:08 AM, Charles R Harris wrote: > > > On Tue, Oct 13, 2009 at 10:52 PM, wrote: >> >> On Tue, Oct 13, 2009 at 10:38 PM, Charles R Harris >> wrote: >> > >> > >> > On Tue, Oct 13, 2009 at 7:24 PM, Anne Archibald >> > >> > wrote: >> >> >> >> 2009/10/13 Charles R Harris : >> >> > >> >> > >> >> > On Tue, Oct 13, 2009 at 5:55 PM, Anne Archibald >> >> > >> >> > wrote: >> >> >> >> >> >> 2009/10/13 Charles R Harris : >> >> >> > >> >> >> >> I'm not sure I see why returning all those NotImplementedErrors >> >> >> >> is a >> >> >> >> problem - yes, the code needs to be written to override the >> >> >> >> methods >> >> >> >> that return NotImplementedError, but that code needs to be >> >> >> >> written >> >> >> >> no >> >> >> >> matter how we set up an interface to it. >> >> >> > >> >> >> > It's not trivial to do a complete workup. My template class is 698 >> >> >> > lines >> >> >> > long, but it exists once in one place, and that does the trick for >> >> >> > *all* >> >> >> > the >> >> >> > classes. >> >> >> >> >> >> Aha. I think I see where the confusion is. In my proposed interface, >> >> >> the coefficient-oriented functions would *be* the ones in the Basis >> >> >> object. So, for example, >> >> >> >> >> >> chebadd(c1,c2,interval) -> ChebyshevBasis(interval).add(c1, c2) >> >> >> >> >> >> or, more likely, you'd have somewhere earlier >> >> >> >> >> >> cheb = ChebyshevBasis(interval) >> >> >> >> >> >> and then it's >> >> >> >> >> >> chebadd(c1,c2,interval) -> cheb.add(c1,c2) >> >> >> >> >> >> So the coefficient-oriented interface that we need to have is what >> >> >> lives in the basis objects, and it is what the Polynomial object >> >> >> uses >> >> >> to implement (e.g.) p1*p2. There would be no separate chebmul, >> >> >> polymul, lagrangemul, .... >> >> >> >> >> > >> >> > But the separate functions are what we want. They provide the most >> >> > flexible >> >> > low level implementation and make the fewest assumptions as to what >> >> > the >> >> > programmer might want to use them for. You don't really have a base >> >> > class, >> >> > you essentially have classes derived from different base classes, >> >> > which >> >> > isn't so different from what I am proposing. >> >> >> >> Do we really want the separate functions, rather than methods on a >> >> basis object? Why? I don't really see how they're more flexible or >> >> make fewer assumptions, and they become very cumbersome (and prevent >> >> some important optimizations) when working with polynomials in the >> >> Lagrange basis. It really can be the difference between chebadd(...) >> >> and cheb.add(...), in the (common?) case where people aren't >> > >> > But why do that if you already have the name space when you import the >> > module? It's redundant. >> > >> >> >> >> specifying the interval. And if people do want to specify the >> >> interval, using a basis object makes it much easier to avoid >> >> forgetting it and getting nonsensical results (or worse, sensible >> >> results until you try a different interval). >> >> >> > >> > That is because they are low level building blocks, they should have the >> > absolute minimum of crud stuck on, and that means the interval is [-1, >> > 1]. >> > If you need more, that is what a class is for. That how low level things >> > should work, small functions doing the minimum amount provide the most >> > flexibility. The programmer should be handed enough rope to hang >> > themselves >> > at that level if they so choose. That's why folks use C and not Pascal. >> > >> >> >> >> In this respect, the current interface in chebyshev.py looks >> >> error-prone to me: you can't suppy an interval to chebadd and chebmul, >> >> but if you forget it for chebval you get bogus answers. (And in fact >> >> chebint and chebder are wrong, or at least misleading, if you use a >> >> different interval.) >> >> >> > >> > Tracking things like intervals is best delegated to higher level >> > functions >> > where you can deal with it without worrying about the low level >> > implementation at the same time. Of course there is some loop-de-loop >> > involved to get to the ideal dividing line, but it is important to draw >> > that >> > line. >> > >> >> >> >> I'm not sure what you mean when you say I don't really have a base >> >> class: Basis is literally the base class for all Basis objects, and it >> >> provides nontrivial functionality (e.g. the power implementation) that >> >> would otherwise need to be repeated for each representation. >> >> Polynomial is the base class for all polynomials. >> >> >> >> >> The Basis object is effectively just a namespace holding the >> >> >> collection of functions, and possibly any data they need (interval, >> >> >> for example). So the separation you're describing is still present >> >> >> in >> >> >> my proposed interface; it's the separation between Basis objects >> >> >> (which contain the functions but no particular plumbing) and the >> >> >> Polynomial objects (which are all plumbing). >> >> > >> >> > Why not let the module hold the functions? It is the more natural >> >> > namespace >> >> > for python. >> >> >> >> Because there is important context - intervals, lists of specified >> >> points - that must be provided to all the coefficient-array functions. >> >> And collections of methods on objects are very natural in python. >> >> >> > >> > That is higher level stuff and doesn't belong at the bottom, it should >> > be >> > separate IMHO.The functions are the ingredients, not the cake. I've >> > spent >> > too much time pulling apart classes to get to the reusable bits not to >> > want >> > them separated out up front. Now you could start with the separate >> > functions >> > and then construct a basis to pass into the initializer. At that point >> > you >> > and I are operating along the same lines except I don't pass them to the >> > contructor, I put them in the local environment where the class methods >> > will >> > find them, that's what the code completion does and Python handles that >> > efficiently. They are consequently more like static (class) methods than >> > instance methods. That way I end up with a whole new class that uses >> > those >> > functions, inherits from nothing but object, and has a minimal >> > constructor. >> > >> > I'll admit that my class implementation may not be suitable for some of >> > the >> > other types of polynomials you are looking at, I hadn't been thinking >> > about >> > them yet. On the other hand it is a pretty complete implementation that >> > requires nothing more than the functions, raises no NotImplementedErrors >> > and >> > should work with most of the orthogonal polynomial bases. And it can be >> > reworked without reference to the low level functions. >> >> How expensive can _cseries_to_zseries, as_series and similar be? >> Since you currently have all main calculations in self-contained >> functions, you need to convert and check the inputs each time. ?If >> these were expensive intermediate results, then methods could save >> time in this. For short series, this doesn't seem to be a problem with >> polycheb, but inside a loop I'm not so sure. >> (as an aside: we were also thinking about moving some methods in >> statsmodels to functions, but in many cases we would need a larger >> number of function arguments of intermediate results or duplicate >> calculations, which makes it less useful.) >> > > One reason for the low level functions is that they are short and can be > easily converted to cython functions or be library functions written in > fortran or whatever. The z series could be done that way, but at the cython > level they could probably be dispensed with and the underlying Cheybyshev > series used instead, that's one reason they are private functions. What they > brought to the python case was some vectorization of the algorithms. Another > way to look at them is as symbolic Fourier transforms. > > Finding the proper collection of small routines can be hard work, which > seems like it might be the case in the statsmodels. Sometimes profiling will > reveal small bits that are bottlenecks and they are potential candidates. > Another view that helps is to separate the data shuffling and verification > to the python level and think of that as an interface to some underlying > routines in a faster language. I don't know enough about statsmodels to make > any more useful suggestions ;) > >> >> I was playing a bit with the cats, and inheritance seems to work >> without problems. >> > > Things can be done with inheritance, but in this case the need was for a > common mass produced interface and the desire to avoid work. Inheritance > isn't the only way to build interfaces, unless it happens to be the only > tool at hand. Templated classes are another way that is closer to the duck > typing model and they seem to work better in some circumstances. For > instance, in the C++ STL templated classes are provided so that you can > have, say, lists of different (uniform) types, but the lists all have the > same methods. That works better than trying to use inheritance. I think that > in general the templated class method is preferable because the design is > flatter and more flexible, and it doesn't use hokey virtual base classes and > such. Inheritance works best when you need to call a bunch of different > objects through a common base class, i.e., the objects are all "is a" base > class, and I don't think that requirement is there for the polynomial > classes. The temptation of inheritance is to use it for implementation, and > more often than not that leads to trouble down the line. The upshot is that > I was looking for the python equivalent of templated classes and completions > looked to be the best bet, so I gave it a shot to see how it looked. I think > it looks OK, but the functional approach is a bit strange to folks like > myself who have spent most of our lives working with imperative languages. I didn't know polynomials are going into mass production. I like subclasses, I think I would have a much harder time keeping track of 90 stats.distribution classes if they were generated with a factory function than as subclasses. The current structure keeps all the information of each distribution together and I don't have to chase down many individual functions. It is also relatively easy to tell whether a problem is in the generic superclass (interface) code or in the individual distributions. I tried to convert the cats example to using metaclasses and it looks ok. Your class factory approach or the metaclasses look useful for generating non-linear transformations of distributions on the fly, where the transformation functions are not known in advance. Josef > >> Following roughly the discussion, I would find it useful if the >> functions specify the used or assumed domain in the doc strings. Since >> I'm not very familiar with Chebyshev polynomials, I wouldn't know >> which functions would require rescaling of the domain. >> > > Yes, that should be documented. > > Chuck > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From peridot.faceted at gmail.com Wed Oct 14 14:43:47 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Wed, 14 Oct 2009 14:43:47 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/14 Charles R Harris : > > > On Tue, Oct 13, 2009 at 11:03 PM, Anne Archibald > wrote: >> >> 2009/10/13 Charles R Harris : >> > >> > On Tue, Oct 13, 2009 at 7:24 PM, Anne Archibald >> > >> > wrote: >> >> specifying the interval. And if people do want to specify the >> >> interval, using a basis object makes it much easier to avoid >> >> forgetting it and getting nonsensical results (or worse, sensible >> >> results until you try a different interval). >> >> >> > >> > That is because they are low level building blocks, they should have the >> > absolute minimum of crud stuck on, and that means the interval is [-1, >> > 1]. >> > If you need more, that is what a class is for. That how low level things >> > should work, small functions doing the minimum amount provide the most >> > flexibility. The programmer should be handed enough rope to hang >> > themselves >> > at that level if they so choose. That's why folks use C and not Pascal. >> >> Well, the glib answer would be that we're using python, so we should >> choose a pythonic way of doing things. But clearly having the >> low-level functions is important to some potential users - you and >> Robert Kern, plus surely more other people - so where possible the >> code should accommodate the as-low-level-as-possible approach. >> >> How's this for a compromise? The high-level class interface has a >> Polynomial object and various Basis subclasses. For representations >> where having a featureful Basis object is important, the >> implementation should just write one, inheriting as appropriate. For >> representations that don't need this extra stuff, people can just >> write a collection of functions acting on coefficient arrays (as you >> did for the Chebyshev polynomials) and create a basis object using >> > > I'm trying to get you to consider not using inheritance at all and see how > things look. We might not want to go that way in the end, but inheritance > tends to be overused. Probably because it is the first tool that comes to > mind and because it looks like a cheap way to reuse code. But it is often > not the best tool so folks need to step back and see if there aren't better > ways to get things done. The fact that you are putting some functionality in > the base class indicates that you are thinking along the lines of > implementation. Passing a Basis object into a constructor also doesn't smell > right. The question to ask is what role you want the Basis object to play > and if it doesn't more naturally belong somewhere else if you weren't > constrained by inheritance. In a sense it looks like you would like to > derive the interface class from the basis object, but somehow that doesn't > seem to work well with inheritance because then it is hard for the derived > objects to share code. Well, conceptually there are two different objects for a given representation: a basis, and a polynomial represented as a linear combination of basis elements. The polynomial object should obviously have methods for things like multiplication and addition. I agree with your idea that it should be immutable. But it must have some way of knowing what basis its coefficient refers to. The basis object is a natural place to put information like roots of basis functions, weights for barycentric interpolation, and derivative matrices. Having basis objects is also a natural way to answer the question "are these polynomials represented in the same basis?", necessary for type-checking; it also lets user code create new objects in the same basis as a polynomial it is handed. Being able to define a custom __eq__ allows users to create bases in different places that are nonetheless compatible. (Incidentally, my intent is that users should not ever call the Polynomial constructor; the preferred way to make a polynomial object is to hand a basis object a coefficient array.) Given that we need these two kinds of object, how should the code be arranged? We have, so far, three different representations of polynomials to work with, so we can look at how much code can reasonably be shared between them. Addition can be identical for all three, provided that we have an operation to "extend" a coefficient array to a basis with more elements. This extension operation is identical - simply padding with zeros - for Chebyshev and power bases, but different for the Lagrange basis. Evaluation is different for all three bases (but will be similar for the various orthogonal-polynomial bases if we implement them). Multiplication and division are different for all three bases, though I know how to write a generic division that should both work and be numerically stable (albeit not as efficient as one might wish) for all representations. Power can be implemented once for all types, but has a more efficient specialized implementation for Lagrange polynomials. Companion matrices and roots can be done generically for any representation that allows division, but will have special implementations in most bases. All the various adapter plumbing to make operators work on polynomials is identical for all representations. The convert() operation to change the representation of a polynomial, can probably work fine given the destination basis and a polynomial object, but will need to be different for each basis. So to summarize: we have some code that should be shared by all representations, some that can be written generically but some representations will provide specialized implementations, some that is shared between only some representations, and some that is unique to each representation. User representations, if they occur, may share some code with existing representations, but it's not clear which. So where do we put the code? Whether any given bit of code should be in the polynomial objects or the basis objects is not clear; some bits need to be in the basis objects (conversion code, zero(), one(), X()) and some bits need to be in the polynomial objects (type checking, operator implementation). But the actual meat of a new representation could in principle go in either place or in some third place (module namespace?). What decides this, I think, should be how we handle the code sharing. My answer is to handle the code sharing with inheritance - it is a natural, well-supported, flexible mechanism built into python for doing exactly this sort of code sharing. All basis objects share some features (zero(), __neq__()), and some share more (any graded basis can sensibly act like a sequence, returning the ith basis function). So it seems to me that inheritance is a sensible way to connect basis objects and allow them to share code. I've tried two different places to put the "meat" functions, directly in polynomial classes, or in basis classes. The latter allows all current polynomial representations to share a common type, which makes implementing new representations a little tidier. Then the basis class hierarchy allows code sharing between the "meat" functions too (e.g. power, roots). Could the basis objects share code in some other way? I don't really see another good approach - for example I don't see how your loose-functions-plus-class-maker would handle the issue of power(), which is generic for some implementations but specialized in others. I expect roots() to be the same. >> ChebyshevBasis = generate_basis("ChebyshevBasis", mul=chebmul, ...) >> > > Heh! Well it's does look like a compromise, or rather, design by committee > ;) But it is something to think about. You should have seen my first attempt - I tried to write a function declare_function(globals(),"lag",LagrangeBasis) that would create functional interfaces and add them to the module namespace, but I got lost in the bowels of python's introspection system. Joking aside, this really is applying your technique to my preferred structure of polynomial object plus basis object. I think there's a real need for a basis object, and if you want to be able to create one simply by specifying a handful of low-level functions, this will do it for you. I should also say that, having implemented KroghInterpolator and BarycentricInterpolator - polynomial objects in fact, albeit with a different goal and very limited features - it's really not difficult to write fast compiled code in cython that works in an object-oriented way. Anne From d.l.goldsmith at gmail.com Wed Oct 14 18:38:42 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 14 Oct 2009 15:38:42 -0700 Subject: [SciPy-dev] Proposed generic docstring for attributes of class generic Message-ID: <45d1ab480910141538p10d1c812jcb76fed378035e46@mail.gmail.com> Short summary: Not implemented (virtual attribute) Extended summary: Class generic exists solely to derive numpy scalars from, and possesses, albeit unimplemented, all the attributes of the ndarray class so as to provide a uniform API. See Also: the corresponding attribute of the derived class of interest. Comments? DG -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Wed Oct 14 19:38:09 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 14 Oct 2009 17:38:09 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Wed, Oct 14, 2009 at 12:43 PM, Anne Archibald wrote: > 2009/10/14 Charles R Harris : > > > > > > On Tue, Oct 13, 2009 at 11:03 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> 2009/10/13 Charles R Harris : > >> > > >> > On Tue, Oct 13, 2009 at 7:24 PM, Anne Archibald > >> > > >> > wrote: > >> >> specifying the interval. And if people do want to specify the > >> >> interval, using a basis object makes it much easier to avoid > >> >> forgetting it and getting nonsensical results (or worse, sensible > >> >> results until you try a different interval). > >> >> > >> > > >> > That is because they are low level building blocks, they should have > the > >> > absolute minimum of crud stuck on, and that means the interval is [-1, > >> > 1]. > >> > If you need more, that is what a class is for. That how low level > things > >> > should work, small functions doing the minimum amount provide the most > >> > flexibility. The programmer should be handed enough rope to hang > >> > themselves > >> > at that level if they so choose. That's why folks use C and not > Pascal. > >> > >> Well, the glib answer would be that we're using python, so we should > >> choose a pythonic way of doing things. But clearly having the > >> low-level functions is important to some potential users - you and > >> Robert Kern, plus surely more other people - so where possible the > >> code should accommodate the as-low-level-as-possible approach. > >> > >> How's this for a compromise? The high-level class interface has a > >> Polynomial object and various Basis subclasses. For representations > >> where having a featureful Basis object is important, the > >> implementation should just write one, inheriting as appropriate. For > >> representations that don't need this extra stuff, people can just > >> write a collection of functions acting on coefficient arrays (as you > >> did for the Chebyshev polynomials) and create a basis object using > >> > > > > I'm trying to get you to consider not using inheritance at all and see > how > > things look. We might not want to go that way in the end, but inheritance > > tends to be overused. Probably because it is the first tool that comes to > > mind and because it looks like a cheap way to reuse code. But it is often > > not the best tool so folks need to step back and see if there aren't > better > > ways to get things done. The fact that you are putting some functionality > in > > the base class indicates that you are thinking along the lines of > > implementation. Passing a Basis object into a constructor also doesn't > smell > > right. The question to ask is what role you want the Basis object to > play > > and if it doesn't more naturally belong somewhere else if you weren't > > constrained by inheritance. In a sense it looks like you would like to > > derive the interface class from the basis object, but somehow that > doesn't > > seem to work well with inheritance because then it is hard for the > derived > > objects to share code. > > Well, conceptually there are two different objects for a given > representation: a basis, and a polynomial represented as a linear > combination of basis elements. > > The polynomial object should obviously have methods for things like > multiplication and addition. I agree with your idea that it should be > immutable. But it must have some way of knowing what basis its > coefficient refers to. > > The basis object is a natural place to put information like roots of > basis functions, weights for barycentric interpolation, and derivative > matrices. Having basis objects is also a natural way to answer the > question "are these polynomials represented in the same basis?", > necessary for type-checking; it also lets user code create new objects > in the same basis as a polynomial it is handed. Being able to define a > custom __eq__ allows users to create bases in different places that > are nonetheless compatible. (Incidentally, my intent is that users > should not ever call the Polynomial constructor; the preferred way to > make a polynomial object is to hand a basis object a coefficient > array.) > > Given that we need these two kinds of object, how should the code be > arranged? We have, so far, three different representations of > polynomials to work with, so we can look at how much code can > reasonably be shared between them. > > Addition can be identical for all three, provided that we have an > operation to "extend" a coefficient array to a basis with more > elements. This extension operation is identical - simply padding with > zeros - for Chebyshev and power bases, but different for the Lagrange > basis. > > Evaluation is different for all three bases (but will be similar for > the various orthogonal-polynomial bases if we implement them). > > Multiplication and division are different for all three bases, though > I know how to write a generic division that should both work and be > numerically stable (albeit not as efficient as one might wish) for all > representations. > > Power can be implemented once for all types, but has a more efficient > specialized implementation for Lagrange polynomials. > > Companion matrices and roots can be done generically for any > representation that allows division, but will have special > implementations in most bases. > > All the various adapter plumbing to make operators work on polynomials > is identical for all representations. > > The convert() operation to change the representation of a polynomial, > can probably work fine given the destination basis and a polynomial > object, but will need to be different for each basis. > > So to summarize: we have some code that should be shared by all > representations, some that can be written generically but some > representations will provide specialized implementations, some that is > shared between only some representations, and some that is unique to > each representation. User representations, if they occur, may share > some code with existing representations, but it's not clear which. > > So where do we put the code? Whether any given bit of code should be > in the polynomial objects or the basis objects is not clear; some bits > need to be in the basis objects (conversion code, zero(), one(), X()) > and some bits need to be in the polynomial objects (type checking, > operator implementation). But the actual meat of a new representation > could in principle go in either place or in some third place (module > namespace?). > > What decides this, I think, should be how we handle the code sharing. > My answer is to handle the code sharing with inheritance - it is a > natural, well-supported, flexible mechanism built into python for > doing exactly this sort of code sharing. All basis objects share some > features (zero(), __neq__()), and some share more (any graded basis > can sensibly act like a sequence, returning the ith basis function). > So it seems to me that inheritance is a sensible way to connect basis > objects and allow them to share code. > > I've tried two different places to put the "meat" functions, directly > in polynomial classes, or in basis classes. The latter allows all > current polynomial representations to share a common type, which makes > implementing new representations a little tidier. Then the basis class > hierarchy allows code sharing between the "meat" functions too (e.g. > power, roots). > > Could the basis objects share code in some other way? I don't really > see another good approach - for example I don't see how your > loose-functions-plus-class-maker would handle the issue of power(), > which is generic for some implementations but specialized in others. I > expect roots() to be the same. > > I think we should look at Josef's suggestions of metaclasses also. > >> ChebyshevBasis = generate_basis("ChebyshevBasis", mul=chebmul, ...) > >> > > > > Heh! Well it's does look like a compromise, or rather, design by > committee > > ;) But it is something to think about. > > You should have seen my first attempt - I tried to write a function > declare_function(globals(),"lag",LagrangeBasis) that would create > functional interfaces and add them to the module namespace, but I got > lost in the bowels of python's introspection system. > > Yes, I think the thing to keep in mind is what we want to do and then find the best tool for the job. > Joking aside, this really is applying your technique to my preferred > structure of polynomial object plus basis object. I think there's a > real need for a basis object, and if you want to be able to create one > simply by specifying a handful of low-level functions, this will do it > for you. > > I don't like the idea of passing the basis into the instance constructor, I think it would be preferable to pass it into the constructor of a class creator. That is why Josef's suggestion of a metaclass rouses my curiosity. Now if I can only figure out what metaclasses are... > I should also say that, having implemented KroghInterpolator and > BarycentricInterpolator - polynomial objects in fact, albeit with a > different goal and very limited features - it's really not difficult > to write fast compiled code in cython that works in an object-oriented > way. > > Yes. Although I expect the bottlenecks are few and those few can be pure C (cython) code while the rest is just plain old python. Which cython will compile also, of course ;) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Wed Oct 14 21:09:08 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Wed, 14 Oct 2009 21:09:08 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/14 Charles R Harris : > > > On Wed, Oct 14, 2009 at 12:43 PM, Anne Archibald > wrote: >> >> Could the basis objects share code in some other way? I don't really >> see another good approach - for example I don't see how your >> loose-functions-plus-class-maker would handle the issue of power(), >> which is generic for some implementations but specialized in others. I >> expect roots() to be the same. >> > > I think we should look at Josef's suggestions of metaclasses also. >> Joking aside, this really is applying your technique to my preferred >> structure of polynomial object plus basis object. I think there's a >> real need for a basis object, and if you want to be able to create one >> simply by specifying a handful of low-level functions, this will do it >> for you. >> > > I don't like the idea of passing the basis into the instance constructor, I > think it would be preferable to pass it into the constructor of a class > creator. That is why Josef's suggestion of a metaclass rouses my curiosity. > Now if I can only figure out what metaclasses are... I think passing it to the constructor is kind of a side issue (as I said, users are never supposed to call the constructor); if I understand correctly, what you're objecting to is the fact that every polynomial has a .basis attribute that points to the basis its coefficients represent. Certainly every polynomial needs to know what basis it is in, and many users will also need to know. But it would be possible to create a class for every different basis and store the basis object in the class, so that a polynomial contains only its coefficients and a pointer to its class. Then you could extract a polynomial's basis by doing, say, mypoly.__class__.basis. Apart from the ugliness of doing this, I find it rather unpleasant to have potentially thousands of class objects floating around - remember PowerBasis(0) is different from PowerBasis(0.32). Also, you have to do some careful work to check polynomial compatibility: people can, and often will, create different basis objects that represent the same object. Either you need to implement a flyweight pattern for your basis objects (or classes - ugh), or you need to have a way of detecting when two polynomials with different classes are actually compatible (because the Basis objects used to manufacture the classes are equal but not the same object). It just seems simpler to me to have the basis as a full-fledged attribute of the polynomial class. Given that polynomial classes are supposed to be immutable, that means it has to be passed in the constructor. As for metaclasses, well, in older versions of python if you took an object's __class__ you got an object representing the object. It mostly bundled together a bunch of methods, most notably the constructor for instances. But the type of this object was always the same. Metaclasses let you use a different type for this object representing the type of an instance. For example, if you wanted every class someone defined to be registered in some central registry, you could make a metaclass that upon instantiation registered itself. Then every time someone created a new class whose metaclass was this, the new class would be registered. It allows you to effectively change the behaviour of the "class" statement. For a sillier example, if you wanted, for some reason, to have classes print "spam" when asked for the value of an undefined class method instead of raising an exception, you'd just make sure the classes used a metaclass that fabricated class objects that did that. For a more concrete example, metaclasses can be used to prevent you from instantiating an abstract base class without providing implementations for all the abstract methods. It does this by changing the way class objects are made. I have to say, though, that an implementation that is hard to understand won't be very friendly to maintainers or users of the package. And metaclasses and class-producing functions are straying dangerously into that territory. Limited though inheritance is, it is at least a very standard well-understood way of sharing code in python. >> I should also say that, having implemented KroghInterpolator and >> BarycentricInterpolator - polynomial objects in fact, albeit with a >> different goal and very limited features - it's really not difficult >> to write fast compiled code in cython that works in an object-oriented >> way. > > Yes. Although I expect the bottlenecks are few and those few can be pure C > (cython) code while the rest is just plain old python. Which cython will > compile also, of course ;) And the important speedups will be algorithmic - caching the weights in a barycentric interpolator reduces evaluation from O(degree**2) to O(degree), for example. I haven't done this yet because I want to hash out the interface before I start having too much code to shovel around. Anne From charlesr.harris at gmail.com Wed Oct 14 23:02:39 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 14 Oct 2009 21:02:39 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Wed, Oct 14, 2009 at 7:09 PM, Anne Archibald wrote: > 2009/10/14 Charles R Harris : > > > > > > On Wed, Oct 14, 2009 at 12:43 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> Could the basis objects share code in some other way? I don't really > >> see another good approach - for example I don't see how your > >> loose-functions-plus-class-maker would handle the issue of power(), > >> which is generic for some implementations but specialized in others. I > >> expect roots() to be the same. > >> > > > > I think we should look at Josef's suggestions of metaclasses also. > > > > >> Joking aside, this really is applying your technique to my preferred > >> structure of polynomial object plus basis object. I think there's a > >> real need for a basis object, and if you want to be able to create one > >> simply by specifying a handful of low-level functions, this will do it > >> for you. > >> > > > > I don't like the idea of passing the basis into the instance constructor, > I > > think it would be preferable to pass it into the constructor of a class > > creator. That is why Josef's suggestion of a metaclass rouses my > curiosity. > > Now if I can only figure out what metaclasses are... > > I think passing it to the constructor is kind of a side issue (as I > said, users are never supposed to call the constructor); if I > understand correctly, what you're objecting to is the fact that every > polynomial has a .basis attribute that points to the basis its > coefficients represent. > > Wait a minute, let's get this straight. Users aren't supposed to use the class object, but only instances generated from some primeval instance created out of nothingness in the module? That sounds like a factory object, but not of the traditional sort. Mathematicians may theoretically start with the empty set and a couple of axioms, but most don't work that way ;) How does repr work, since it is supposed to generate a string that can be run to recreate the object? Doesn't it need to call the constructor? Same with unpickling. > Certainly every polynomial needs to know what basis it is in, and many > users will also need to know. But it would be possible to create a > class for every different basis and store the basis object in the > class, so that a polynomial contains only its coefficients and a > pointer to its class. Then you could extract a polynomial's basis by > doing, say, mypoly.__class__.basis. Apart from the ugliness of doing > this, I find it rather unpleasant to have potentially thousands of > class objects floating around Well, identifying different classes by attaching a name attribute, in this case a basis, is how some folks identified classes back in the paleolithic period of C++, Borland comes to mind, IIRC, they used to attach an identifier to plain old structures too. It works, sorta, but it is just a lightweight version of having lots of different classes. But it does seem to me that your contruction is oriented towards Lagrange bases and barycentric interpolation and the desire to have it also incorporate graded polynomial basis and such has complicated it. It would be simpler to just build a different classes for the different sorts of polynomials. > - remember PowerBasis(0) is different > from PowerBasis(0.32). Also, you have to do some careful work to check > polynomial compatibility: people can, and often will, create different > basis objects that represent the same object. Either you need to > implement a flyweight pattern for your basis objects (or classes - > ugh), or you need to have a way of detecting when two polynomials with > different classes are actually compatible (because the Basis objects > used to manufacture the classes are equal but not the same object). > > How do different classes end up with the same basis. And where do all the different basis come from if the user doesn't generate them and call the constructor? > It just seems simpler to me to have the basis as a full-fledged > attribute of the polynomial class. Given that polynomial classes are > supposed to be immutable, that means it has to be passed in the > constructor. > > I think is would be simpler to seek less generality. You have an abstract Basis class, then a derived GradedBasis class, then a factory function to generate a GeneratedBasis class derived from GradedBasis, and that class has to be instantiated somewhere. Then there is a Polynomial class that accepts a basis (or basis class?). How is that simpler then a factory function that just builds specific classes for specific instances of graded polynomials out of a few defined functions and inherits only from object? The setup you have looks a bit over designed. > As for metaclasses, well, in older versions of python if you took an > object's __class__ you got an object representing the object. It > mostly bundled together a bunch of methods, most notably the > constructor for instances. But the type of this object was always the > same. Metaclasses let you use a different type for this object > representing the type of an instance. For example, if you wanted every > class someone defined to be registered in some central registry, you > could make a metaclass that upon instantiation registered itself. That's one example, but there are others. In any case, after doing some reading, metaclasses don't seem to me to be the way to go. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Thu Oct 15 00:53:55 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Thu, 15 Oct 2009 00:53:55 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/14 Charles R Harris : > > On Wed, Oct 14, 2009 at 7:09 PM, Anne Archibald > wrote: >> >> I think passing it to the constructor is kind of a side issue (as I >> said, users are never supposed to call the constructor); if I >> understand correctly, what you're objecting to is the fact that every >> polynomial has a .basis attribute that points to the basis its >> coefficients represent. >> > > Wait a minute, let's get this straight. Users aren't supposed to use the > class object, but only instances generated from some primeval instance > created out of nothingness in the module? That sounds like a factory object, > but not of the traditional sort. Mathematicians may theoretically start with > the empty set and a couple of axioms, but most don't work that way ;) How > does repr work, since it is supposed to generate a string that can be run to > recreate the object? Doesn't it need to call the constructor? Same with > unpickling. Here's a typical use: b = ChebyshevBasis((0,1)) X = b.polynomial([0,5,0.5]) f = polyfit(basis=b, degree=1, x=[0,0.5,1], y=[1,2,1]) Y = X**2 + f print Y([0,0.5,1]) The constructor does indeed get called, but by the basis object (inside its polynomial method). It's still not possible to construct a polynomial without specifying its basis, of course; after all a coefficient array is meaningless without a basis in which to interpret it. So this may not resolve your objection to passing a basis to the constructor. >> Certainly every polynomial needs to know what basis it is in, and many >> users will also need to know. But it would be possible to create a >> class for every different basis and store the basis object in the >> class, so that a polynomial contains only its coefficients and a >> pointer to its class. Then you could extract a polynomial's basis by >> doing, say, mypoly.__class__.basis. Apart from the ugliness of doing >> this, I find it rather unpleasant to have potentially thousands of >> class objects floating around > > Well, identifying different classes by attaching a name attribute, in this > case a basis, is how some folks identified classes back in the paleolithic > period of C++, Borland comes to mind, IIRC, they used to attach an > identifier to plain old structures too. It works, sorta, but it is just a > lightweight version of having lots of different classes.? But it does seem > to me that your contruction is oriented towards Lagrange bases and > barycentric interpolation and the desire to have it also incorporate graded > polynomial basis and such has complicated it. It would be simpler to just > build a different classes for the different sorts of polynomials. Certainly my design is intended to accommodate the Lagrange representation. But it is also shaped by a desire to be able to deal sensibly with families of orthogonal polynomials, providing auxiliary information about them. There is also a certain amount of code that can be shared between the Lagrange basis polynomials and other polynomial implementations - all the type checking plumbing, for example. I'd be reluctant to pull them apart and duplicate that code. (I also rather like being able to use isinstance(x,Polynomial) but I realize that's not as pythonic as one might wish.) >> - remember PowerBasis(0) is different >> from PowerBasis(0.32). Also, you have to do some careful work to check >> polynomial compatibility: people can, and often will, create different >> basis objects that represent the same object. Either you need to >> implement a flyweight pattern for your basis objects (or classes - >> ugh), or you need to have a way of detecting when two polynomials with >> different classes are actually compatible (because the Basis objects >> used to manufacture the classes are equal but not the same object). >> > How do different classes end up with the same basis. And where do all the > different > basis come from if the user doesn't generate them and call the constructor? Well, in an ideal world, users would simply create one basis and use it for all the polynomials they produce in that basis. But it's already clear from writing the test code that we're going to have things like (in my syntax): p1 = PowerBasis(center=3).polynomial([1,1]) ... many lines of code ... p2 = PowerBasis(center=3).polynomial([3,5,6]) In this situation p1 and p2 have received different (i.e. id(A)!=id(B)) basis objects that are actually equal. As for different classes with the same basis, I think I misunderstood your proposal. Since your code requires bases to be specified completely by an implementation and an interval, and you store the interval in the polynomial object rather than the class, you never do end up with many classes. If I've understood you correctly, you do require the user to specify an interval any time they create a polynomial (inlcuding zero(), one(), and X()), and this is then checked for compatibility. I'm not sure I see how your code can be adapted to sensibly handle the power basis (specifying a center and maybe a scale) or some of the orthogonal polynomial bases (some of which need an endpoint and scale, or possibly also one or two parameters specifying the family of polynomials). >> It just seems simpler to me to have the basis as a full-fledged >> attribute of the polynomial class. Given that polynomial classes are >> supposed to be immutable, that means it has to be passed in the >> constructor. >> > > I think is would be simpler to seek less generality. You have an abstract > Basis class, then a derived GradedBasis class, then a factory function to > generate a GeneratedBasis class derived from GradedBasis,? and that class > has to be instantiated somewhere. Then there is a Polynomial class that > accepts a basis (or basis class?). How is that simpler then a factory > function that just builds specific classes for specific instances of graded > polynomials out of a few defined functions and inherits only from object? > The setup you have looks a bit over designed. One reason it looks over-designed is that it only has three polynomial types; it should at least have one more for families of orthogonal polynomials, and perhaps another for Bernstein polynomials. (The orthogonal polynomials will inherit from GradedBasis but probably can't be produced using GeneratedBasis; the Bernstein basis is not graded.) The GeneratedBasis I would be inclined to do without, except that if people want to easily make polynomials given collections of coefficient-operating functions, it will let them do that. I should point out that, for example, there's no need to implement chebadd and chebsub, since those implementations are automatically provided by GradedBasis. Leaving aside the GeneratedBasis, I think it's fairly simple. >From a user's point of view, they pick a polynomial representation they want to work with by creating an appropriate Basis object, then use that to create the polynomial, perhaps directly from coefficients (with basis.polynomial()) or roots (basis.from_roots()), perhaps using existing ones (basis.X(), basis.convert(other_polynomial)), or by using some polynomial-returning function (e.g. poly.polyfit(basis, ...)). They can then use that polynomial as an immutable type. >From an implementer's point of view, adding a new polynomial type is simple: they define a new basis class inheriting from the most similar existing basis class, and implement a few operations on coefficient arrays. Addition and power they can leave out; if they don't care about division they can leave it out; if they implement division or companion matrices they get root-finding for free. They'll probably want to write a routine to convert arbitrary polynomials to their representation; the other direction is already implemented. If they've got low-level functions like chebmul to work with, they can simply write MyBasis = GeneratedBasis(...functions...) and lo and behold they've got a reasonably functional polynomial class. Or if we ditch that as too messy, they write a class with some one- and two-line wrappers. If you want to compare our two designs, mine uses a single class for all polynomials and uses the different classes of Basis to provide different implementations, while yours puts the code directly in the polynomial classes. Mine uses inheritance to share code between representations, while yours shares code by writing it into the class generator function. I think that to the extent mine is more complicated, it's because it wants to allow more different implementations of polynomial classes. Yours is very specialized to orthogonal polynomials defined on a finite interval. You'll need to generalize it somewhat to handle power-basis polynomials at least. I find the class-generating code fairly opaque, and I'm not sure how well it will interact with debugging and introspection. If these aren't a problem, and we really don't need more polynomial types, then maybe it does make sense to go with yours as the simpler. I'd argue, though, that a general, efficient, numerically-stable polynomial class could replace KroghInterpolator and BarycentricInterpolator and serve to make PiecewisePolynomial more powerful, for example. The orthogonal polynomial code in scipy.special could return polynomial objects, ideally in their own basis, that allowed fast evaluation, integration, differentiation, and whose basis objects carried information about the roots and weights needed for Gaussian quadrature. (In fact, those basis objects themselves could live in scipy.special and serve to produce the orthogonal polynomials as needed.) Anne From ralf.gommers at googlemail.com Thu Oct 15 08:57:36 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Thu, 15 Oct 2009 14:57:36 +0200 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Thu, Oct 15, 2009 at 3:09 AM, Anne Archibald wrote: > > I have to say, though, that an implementation that is hard to > understand won't be very friendly to maintainers or users of the > package. And metaclasses and class-producing functions are straying > dangerously into that territory. Limited though inheritance is, it is > at least a very standard well-understood way of sharing code in > python. > > This whole discussion is very interesting, and I have looked at both your code to learn something. Here I would like to give my 2c: Charles solution is not that difficult to understand. To me it was clear very quickly where to look in the code, and also clear (in principle) how to implement a new basis. You also brought up concerns like being able to use isinstance, and introspection. As far as I can tell it works as expected. Try running the following: ############################################################ from polycheb import * MyCheb = Chebyshev([1, 2, 3]) # check some simple attributes print 'Roots : ', MyCheb.roots() print 'Coefs : ', MyCheb.coef # we can check if our instance is really a Chebyshev class print 'Is MyCheb instance of Chebyshev? : ', isinstance(MyCheb, Chebyshev) # in what module is our class defined? print 'Class defined in module : ', Chebyshev.__module__ # what does it inherit from? print 'Inherits from : ', Chebyshev.__bases__ # The docstring is a bit uninformative, should link to chebroots links_to_func = (MyCheb.roots.__doc__.find('chebroots') != -1) print 'Does docstring link to actual implementation? : ', links_to_func ############################################################ Printed results: Roots : Cheb(2) Coefs : [ 1. 2. 3.] Is MyCheb instance of Chebyshev? : True Class defined in module : polyclass Inherits from : (,) Does docstring link to actual implementation? : False Except for the docstring which is easy to fix, it looks nice. There's of course nothing wrong with inheritance either, but when looking at your ChebyshevBasis I could already use an inheritance diagram. I do see you have more bases in there so maybe it's not a fair comparison. Just my 2c, please keep the insights coming! Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Oct 15 11:55:38 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 15 Oct 2009 09:55:38 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Wed, Oct 14, 2009 at 10:53 PM, Anne Archibald wrote: > 2009/10/14 Charles R Harris : > > > > On Wed, Oct 14, 2009 at 7:09 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> I think passing it to the constructor is kind of a side issue (as I > >> said, users are never supposed to call the constructor); if I > >> understand correctly, what you're objecting to is the fact that every > >> polynomial has a .basis attribute that points to the basis its > >> coefficients represent. > >> > > > > Wait a minute, let's get this straight. Users aren't supposed to use the > > class object, but only instances generated from some primeval instance > > created out of nothingness in the module? That sounds like a factory > object, > > but not of the traditional sort. Mathematicians may theoretically start > with > > the empty set and a couple of axioms, but most don't work that way ;) How > > does repr work, since it is supposed to generate a string that can be run > to > > recreate the object? Doesn't it need to call the constructor? Same with > > unpickling. > > Here's a typical use: > > b = ChebyshevBasis((0,1)) > > X = b.polynomial([0,5,0.5]) > > f = polyfit(basis=b, degree=1, x=[0,0.5,1], y=[1,2,1]) > > Y = X**2 + f > > print Y([0,0.5,1]) > > The constructor does indeed get called, but by the basis object > (inside its polynomial method). It's still not possible to construct a > polynomial without specifying its basis, of course; after all a > coefficient array is meaningless without a basis in which to interpret > it. So this may not resolve your objection to passing a basis to the > constructor. > > >> Certainly every polynomial needs to know what basis it is in, and many > >> users will also need to know. But it would be possible to create a > >> class for every different basis and store the basis object in the > >> class, so that a polynomial contains only its coefficients and a > >> pointer to its class. Then you could extract a polynomial's basis by > >> doing, say, mypoly.__class__.basis. Apart from the ugliness of doing > >> this, I find it rather unpleasant to have potentially thousands of > >> class objects floating around > > > > Well, identifying different classes by attaching a name attribute, in > this > > case a basis, is how some folks identified classes back in the > paleolithic > > period of C++, Borland comes to mind, IIRC, they used to attach an > > identifier to plain old structures too. It works, sorta, but it is just a > > lightweight version of having lots of different classes. But it does > seem > > to me that your contruction is oriented towards Lagrange bases and > > barycentric interpolation and the desire to have it also incorporate > graded > > polynomial basis and such has complicated it. It would be simpler to just > > build a different classes for the different sorts of polynomials. > > Certainly my design is intended to accommodate the Lagrange > representation. But it is also shaped by a desire to be able to deal > sensibly with families of orthogonal polynomials, providing auxiliary > information about them. There is also a certain amount of code that > can be shared between the Lagrange basis polynomials and other > polynomial implementations - all the type checking plumbing, for > example. I'd be reluctant to pull them apart and duplicate that code. > (I also rather like being able to use isinstance(x,Polynomial) but I > realize that's not as pythonic as one might wish.) > > >> - remember PowerBasis(0) is different > >> from PowerBasis(0.32). Also, you have to do some careful work to check > >> polynomial compatibility: people can, and often will, create different > >> basis objects that represent the same object. Either you need to > >> implement a flyweight pattern for your basis objects (or classes - > >> ugh), or you need to have a way of detecting when two polynomials with > >> different classes are actually compatible (because the Basis objects > >> used to manufacture the classes are equal but not the same object). > >> > > How do different classes end up with the same basis. And where do all the > > different > > basis come from if the user doesn't generate them and call the > constructor? > > Well, in an ideal world, users would simply create one basis and use > it for all the polynomials they produce in that basis. But it's > already clear from writing the test code that we're going to have > things like (in my syntax): > > p1 = PowerBasis(center=3).polynomial([1,1]) > ... many lines of code ... > p2 = PowerBasis(center=3).polynomial([3,5,6]) > > In this situation p1 and p2 have received different (i.e. > id(A)!=id(B)) basis objects that are actually equal. > > As for different classes with the same basis, I think I misunderstood > your proposal. Since your code requires bases to be specified > completely by an implementation and an interval, and you store the > interval in the polynomial object rather than the class, you never do > end up with many classes. If I've understood you correctly, you do > require the user to specify an interval any time they create a > polynomial (inlcuding zero(), one(), and X()), and this is then > checked for compatibility. > > I'm not sure I see how your code can be adapted to sensibly handle the > power basis (specifying a center and maybe a scale) or some of the > orthogonal polynomial bases (some of which need an endpoint and scale, > or possibly also one or two parameters specifying the family of > polynomials). > > >> It just seems simpler to me to have the basis as a full-fledged > >> attribute of the polynomial class. Given that polynomial classes are > >> supposed to be immutable, that means it has to be passed in the > >> constructor. > >> > > > > I think is would be simpler to seek less generality. You have an abstract > > Basis class, then a derived GradedBasis class, then a factory function to > > generate a GeneratedBasis class derived from GradedBasis, and that class > > has to be instantiated somewhere. Then there is a Polynomial class that > > accepts a basis (or basis class?). How is that simpler then a factory > > function that just builds specific classes for specific instances of > graded > > polynomials out of a few defined functions and inherits only from object? > > The setup you have looks a bit over designed. > > One reason it looks over-designed is that it only has three polynomial > types; it should at least have one more for families of orthogonal > polynomials, and perhaps another for Bernstein polynomials. (The > orthogonal polynomials will inherit from GradedBasis but probably > can't be produced using GeneratedBasis; the Bernstein basis is not > graded.) The GeneratedBasis I would be inclined to do without, except > that if people want to easily make polynomials given collections of > coefficient-operating functions, it will let them do that. I should > point out that, for example, there's no need to implement chebadd and > chebsub, since those implementations are automatically provided by > GradedBasis. > > Leaving aside the GeneratedBasis, I think it's fairly simple. > > From a user's point of view, they pick a polynomial representation > they want to work with by creating an appropriate Basis object, then > use that to create the polynomial, perhaps directly from coefficients > (with basis.polynomial()) or roots (basis.from_roots()), perhaps using > existing ones (basis.X(), basis.convert(other_polynomial)), or by > using some polynomial-returning function (e.g. poly.polyfit(basis, > ...)). They can then use that polynomial as an immutable type. > > From an implementer's point of view, adding a new polynomial type is > simple: they define a new basis class inheriting from the most similar > existing basis class, and implement a few operations on coefficient > arrays. Addition and power they can leave out; if they don't care > about division they can leave it out; if they implement division or > companion matrices they get root-finding for free. They'll probably > want to write a routine to convert arbitrary polynomials to their > representation; the other direction is already implemented. > > If they've got low-level functions like chebmul to work with, they can > simply write MyBasis = GeneratedBasis(...functions...) and lo and > behold they've got a reasonably functional polynomial class. Or if we > ditch that as too messy, they write a class with some one- and > two-line wrappers. > > If you want to compare our two designs, mine uses a single class for > all polynomials and uses the different classes of Basis to provide > different implementations, while yours puts the code directly in the > polynomial classes. Mine uses inheritance to share code between > representations, while yours shares code by writing it into the class > generator function. > > I think that to the extent mine is more complicated, it's because it > wants to allow more different implementations of polynomial classes. > Yours is very specialized to orthogonal polynomials defined on a > finite interval. You'll need to generalize it somewhat to handle > power-basis polynomials at least. I find the class-generating code > fairly opaque, and I'm not sure how well it will interact with > debugging and introspection. If these aren't a problem, and we really > don't need more polynomial types, then maybe it does make sense to go > with yours as the simpler. > > I'd argue, though, that a general, efficient, numerically-stable > polynomial class could replace KroghInterpolator and > BarycentricInterpolator and serve to make PiecewisePolynomial more > powerful, for example. The orthogonal polynomial code in scipy.special > could return polynomial objects, ideally in their own basis, that > allowed fast evaluation, integration, differentiation, and whose basis > objects carried information about the roots and weights needed for > Gaussian quadrature. (In fact, those basis objects themselves could > live in scipy.special and serve to produce the orthogonal polynomials > as needed.) > > > Anne > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Oct 15 12:16:59 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 15 Oct 2009 10:16:59 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Wed, Oct 14, 2009 at 10:53 PM, Anne Archibald wrote: > 2009/10/14 Charles R Harris : > > > > On Wed, Oct 14, 2009 at 7:09 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> I think passing it to the constructor is kind of a side issue (as I > >> said, users are never supposed to call the constructor); if I > >> understand correctly, what you're objecting to is the fact that every > >> polynomial has a .basis attribute that points to the basis its > >> coefficients represent. > >> > > > > Wait a minute, let's get this straight. Users aren't supposed to use the > > class object, but only instances generated from some primeval instance > > created out of nothingness in the module? That sounds like a factory > object, > > but not of the traditional sort. Mathematicians may theoretically start > with > > the empty set and a couple of axioms, but most don't work that way ;) How > > does repr work, since it is supposed to generate a string that can be run > to > > recreate the object? Doesn't it need to call the constructor? Same with > > unpickling. > > Here's a typical use: > > b = ChebyshevBasis((0,1)) > > X = b.polynomial([0,5,0.5]) > > f = polyfit(basis=b, degree=1, x=[0,0.5,1], y=[1,2,1]) > > Y = X**2 + f > > print Y([0,0.5,1]) > > The constructor does indeed get called, but by the basis object > (inside its polynomial method). It's still not possible to construct a > polynomial without specifying its basis, of course; after all a > coefficient array is meaningless without a basis in which to interpret > it. So this may not resolve your objection to passing a basis to the > constructor. > > >> Certainly every polynomial needs to know what basis it is in, and many > >> users will also need to know. But it would be possible to create a > >> class for every different basis and store the basis object in the > >> class, so that a polynomial contains only its coefficients and a > >> pointer to its class. Then you could extract a polynomial's basis by > >> doing, say, mypoly.__class__.basis. Apart from the ugliness of doing > >> this, I find it rather unpleasant to have potentially thousands of > >> class objects floating around > > > > Well, identifying different classes by attaching a name attribute, in > this > > case a basis, is how some folks identified classes back in the > paleolithic > > period of C++, Borland comes to mind, IIRC, they used to attach an > > identifier to plain old structures too. It works, sorta, but it is just a > > lightweight version of having lots of different classes. But it does > seem > > to me that your contruction is oriented towards Lagrange bases and > > barycentric interpolation and the desire to have it also incorporate > graded > > polynomial basis and such has complicated it. It would be simpler to just > > build a different classes for the different sorts of polynomials. > > Certainly my design is intended to accommodate the Lagrange > representation. But it is also shaped by a desire to be able to deal > sensibly with families of orthogonal polynomials, providing auxiliary > information about them. There is also a certain amount of code that > can be shared between the Lagrange basis polynomials and other > polynomial implementations - all the type checking plumbing, for > example. I'd be reluctant to pull them apart and duplicate that code. > (I also rather like being able to use isinstance(x,Polynomial) but I > realize that's not as pythonic as one might wish.) > > >> - remember PowerBasis(0) is different > >> from PowerBasis(0.32). Also, you have to do some careful work to check > >> polynomial compatibility: people can, and often will, create different > >> basis objects that represent the same object. Either you need to > >> implement a flyweight pattern for your basis objects (or classes - > >> ugh), or you need to have a way of detecting when two polynomials with > >> different classes are actually compatible (because the Basis objects > >> used to manufacture the classes are equal but not the same object). > >> > > How do different classes end up with the same basis. And where do all the > > different > > basis come from if the user doesn't generate them and call the > constructor? > > Well, in an ideal world, users would simply create one basis and use > it for all the polynomials they produce in that basis. But it's > already clear from writing the test code that we're going to have > things like (in my syntax): > > p1 = PowerBasis(center=3).polynomial([1,1]) > ... many lines of code ... > p2 = PowerBasis(center=3).polynomial([3,5,6]) > > In this situation p1 and p2 have received different (i.e. > id(A)!=id(B)) basis objects that are actually equal. > > As for different classes with the same basis, I think I misunderstood > your proposal. Since your code requires bases to be specified > completely by an implementation and an interval, and you store the > interval in the polynomial object rather than the class, you never do > end up with many classes. If I've understood you correctly, you do > require the user to specify an interval any time they create a > polynomial (inlcuding zero(), one(), and X()), and this is then > checked for compatibility. > > I'm not sure I see how your code can be adapted to sensibly handle the > power basis (specifying a center and maybe a scale) or some of the > orthogonal polynomial bases (some of which need an endpoint and scale, > or possibly also one or two parameters specifying the family of > polynomials). > > The interval defines an affine map from the specified interval to [-1,1], and hence gives scale and center, it was just a guestion of choosing a representation of that map. Since I almost always end up having to calculate center and scale from two points, the interval approach seemed the most natural. Thus plain old power series would be defined in the interval [-1, 1] by default, just like Chebyshev series. This breaks down a bit for the Bernstein polynomials, which could just as easily be defined on [-1, 1] by using the form (x + 1)^i * (x + 1)^(n - i), but that wouldn't be what folks were expecting. This could be fixed by making the default interval a class attribute. > >> It just seems simpler to me to have the basis as a full-fledged > >> attribute of the polynomial class. Given that polynomial classes are > >> supposed to be immutable, that means it has to be passed in the > >> constructor. > >> > > > > I think is would be simpler to seek less generality. You have an abstract > > Basis class, then a derived GradedBasis class, then a factory function to > > generate a GeneratedBasis class derived from GradedBasis, and that class > > has to be instantiated somewhere. Then there is a Polynomial class that > > accepts a basis (or basis class?). How is that simpler then a factory > > function that just builds specific classes for specific instances of > graded > > polynomials out of a few defined functions and inherits only from object? > > The setup you have looks a bit over designed. > > One reason it looks over-designed is that it only has three polynomial > types; it should at least have one more for families of orthogonal > polynomials, and perhaps another for Bernstein polynomials. (The > orthogonal polynomials will inherit from GradedBasis but probably > can't be produced using GeneratedBasis; the Bernstein basis is not > graded.) The GeneratedBasis I would be inclined to do without, except > that if people want to easily make polynomials given collections of > coefficient-operating functions, it will let them do that. I should > point out that, for example, there's no need to implement chebadd and > chebsub, since those implementations are automatically provided by > GradedBasis. > > Leaving aside the GeneratedBasis, I think it's fairly simple. > > I think it would be interesting to see an implementation restricted to Lagrange basis only and as simple as you could make it. > From a user's point of view, they pick a polynomial representation > they want to work with by creating an appropriate Basis object, then > use that to create the polynomial, perhaps directly from coefficients > (with basis.polynomial()) or roots (basis.from_roots()), perhaps using > existing ones (basis.X(), basis.convert(other_polynomial)), or by > using some polynomial-returning function (e.g. poly.polyfit(basis, > ...)). They can then use that polynomial as an immutable type. > > From an implementer's point of view, adding a new polynomial type is > simple: they define a new basis class inheriting from the most similar > existing basis class, and implement a few operations on coefficient > arrays. Addition and power they can leave out; if they don't care > about division they can leave it out; if they implement division or > companion matrices they get root-finding for free. They'll probably > want to write a routine to convert arbitrary polynomials to their > representation; the other direction is already implemented. > > If they've got low-level functions like chebmul to work with, they can > simply write MyBasis = GeneratedBasis(...functions...) and lo and > behold they've got a reasonably functional polynomial class. Or if we > ditch that as too messy, they write a class with some one- and > two-line wrappers. > > If you want to compare our two designs, mine uses a single class for > all polynomials and uses the different classes of Basis to provide > different implementations, while yours puts the code directly in the > polynomial classes. Mine uses inheritance to share code between > representations, while yours shares code by writing it into the class > generator function. > > I think that to the extent mine is more complicated, it's because it > wants to allow more different implementations of polynomial classes. > Yours is very specialized to orthogonal polynomials defined on a > finite interval. You'll need to generalize it somewhat to handle > power-basis polynomials at least. I find the class-generating code > fairly opaque, and I'm not sure how well it will interact with > debugging and introspection. If these aren't a problem, and we really > don't need more polynomial types, then maybe it does make sense to go > with yours as the simpler. > > I'd argue, though, that a general, efficient, numerically-stable > polynomial class could replace KroghInterpolator and > BarycentricInterpolator and serve to make PiecewisePolynomial more > powerful, for example. I agree completely on this point. That is one reason I think a "simplest" class for Lagrange bases would be interesting to look at. > The orthogonal polynomial code in scipy.special > could return polynomial objects, ideally in their own basis, that > allowed fast evaluation, integration, differentiation, and whose basis > objects carried information about the roots and weights needed for > Gaussian quadrature. (In fact, those basis objects themselves could > live in scipy.special and serve to produce the orthogonal polynomials > as needed.) > > All you need to do for the Gaussian quadrature is to carry along a weight function. But the quadrature module already has that bit of infrastructure, it uses the standard companion matrix defined by the recursion along with the weight function. My feeling was that having that as part of the basic polynomial class was starting to overburden the class with accessories. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Oct 15 13:45:29 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 15 Oct 2009 11:45:29 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: Just to throw another stick into the fire, I note that it is possible to reuse the same toplevel code while creating classes that derive from different bases. Using the type metaclass it goes something like this: class_code_text = """blah blah blah""" class_dict = {} exec class_code_text in globals(), class_dict MyVerySpecialClass = type("MyVerySpecialClass", (MyBasis,), class_dict) I think the first three lines could be in their own module. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Thu Oct 15 16:54:13 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Thu, 15 Oct 2009 16:54:13 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: 2009/10/15 Charles R Harris : > I think it would be interesting to see an implementation restricted to > Lagrange basis only and as simple as you could make it. See the branch lagrangeonly in git. The simplification is not that great, since Lagrange polynomials really need a Basis object to keep the list of points they're defined on around. I'd also hope that there was some way to identify all polynomial objects, since (for example) a Lagrange polynomial can be multiplied with a polynomial in any representation to give another Lagrange polynomial... >> I'd argue, though, that a general, efficient, numerically-stable >> polynomial class could replace KroghInterpolator and >> BarycentricInterpolator and serve to make PiecewisePolynomial more >> powerful, for example. > > I agree completely on this point. That is one reason I think a "simplest" > class for Lagrange bases would be interesting to look at. I'm not sure whether Lagrange polynomials alone would qualify - their derivative and integral methods are kind of expensive, and they are not necessarily the most stable available, even in barycentric form. It's also not easy to impose derivative conditions on them, which is needed for PiecewisePolynomial in many applications. This is why I was thinking of multiple representations... Anne From charlesr.harris at gmail.com Thu Oct 15 18:28:42 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 15 Oct 2009 16:28:42 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: Message-ID: On Thu, Oct 15, 2009 at 2:54 PM, Anne Archibald wrote: > 2009/10/15 Charles R Harris : > > > I think it would be interesting to see an implementation restricted to > > Lagrange basis only and as simple as you could make it. > > See the branch lagrangeonly in git. The simplification is not that > great, since Lagrange polynomials really need a Basis object to keep > the list of points they're defined on around. I'd also hope that there > was some way to identify all polynomial objects, since (for example) a > Lagrange polynomial can be multiplied with a polynomial in any > representation to give another Lagrange polynomial... > > OK. I've also attached a "both a dessert topping and a floor wax" version of things that I tried out to see what it looked like. Like your earlier proposal, the standalone functions are @staticmethods of a base class (roughly equivalent to your Bases), and a new class that inherits from the base class is then generated using a common template for the derived class. The resulting class contains both the standalone functions, which can be called from the class object itself, and the syntactic sugar for the class instances, i.e., In [3]: from polycheb import * In [4]: a = Chebyshev([1,2,3]) In [5]: a + a Out[5]: Chebyshev(array([ 2., 4., 6.]), [-1, 1]) In [6]: Chebyshev.add([1,2,3],[1,2,3]) Out[6]: array([ 2., 4., 6.]) In [7]: type(a) Out[7]: It's something of a curiosity and I don't claim it is bug free, it is the result of a bit of quick and dirty editing. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: polytemplate.py Type: application/x-python Size: 11041 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: polycheb.py Type: application/x-python Size: 29018 bytes Desc: not available URL: From fperez.net at gmail.com Thu Oct 15 18:35:23 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 15 Oct 2009 15:35:23 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: Hi all, On Wed, Oct 14, 2009 at 8:08 AM, Charles R Harris wrote: > One reason for the low level functions is that they are short and can be > easily converted to cython functions or be library functions written in > fortran or whatever. The z series could be done that way, but at the cython > level they could probably be dispensed with and the underlying Cheybyshev > series used instead, that's one reason they are private functions. What they > brought to the python case was some vectorization of the algorithms. Another > way to look at them is as symbolic Fourier transforms. I don't know if this will genuinely add anything to this (very interesting discussion), but Ariel, Mike and I thought a lot about fairly similar problems in the nitime package, and went for an approach with: - a purely functional library (even if the functions end up having long signatures and don't cache things they could), - very simple data cointainers whose design evolves very conservatively - analysis objects that expose a nice, high-level interface and manage the containers and library functions. These are what users should use 90% of the time, but if we get their design wrong, little is lost because most of the complexities are in the functional library. This is all summarized in our preprint for the scipy proceedings (currently under review): http://fperez.org/papers/nitime09_ro-tr-pe.pdf and the nitime docs can be found here: http://nipy.sourceforge.net/nitime/ while the master branch lives at github: http://github.com/fperez/nitime So far we're quite happy with this approach, and I hope this example may be of some use to you guys as you sort out the poly design. BTW, I'm very excited to see this coming along, many thanks to Anne and Chuck for so much hard work! Cheers, f From charlesr.harris at gmail.com Thu Oct 15 21:36:32 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 15 Oct 2009 19:36:32 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: On Thu, Oct 15, 2009 at 4:35 PM, Fernando Perez wrote: > Hi all, > > On Wed, Oct 14, 2009 at 8:08 AM, Charles R Harris > wrote: > > One reason for the low level functions is that they are short and can be > > easily converted to cython functions or be library functions written in > > fortran or whatever. The z series could be done that way, but at the > cython > > level they could probably be dispensed with and the underlying Cheybyshev > > series used instead, that's one reason they are private functions. What > they > > brought to the python case was some vectorization of the algorithms. > Another > > way to look at them is as symbolic Fourier transforms. > > I don't know if this will genuinely add anything to this (very > interesting discussion), but Ariel, Mike and I thought a lot about > fairly similar problems in the nitime package, and went for an > approach with: > > - a purely functional library (even if the functions end up having > long signatures and don't cache things they could), > > - very simple data cointainers whose design evolves very conservatively > > - analysis objects that expose a nice, high-level interface and manage > the containers and library functions. These are what users should use > 90% of the time, but if we get their design wrong, little is lost > because most of the complexities are in the functional library. > > How did you build the analysis objects? Part of what we have been discussing it the easiest way to expose a consistent interface while reusing the maximum amount of code. And thanks for the links. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Oct 15 21:57:34 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 15 Oct 2009 18:57:34 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: On Thu, Oct 15, 2009 at 6:36 PM, Charles R Harris wrote: > > How did you build the analysis objects? Part of what we have been discussing > it the easiest way to expose a consistent interface while reusing the > maximum amount of code. See http://github.com/fperez/nitime/blob/master/nitime/timeseries.py starting around line 504. The analyzers store a reference to a time series object and then expose properties (that are replaced in-place by a static attribute once computed, as explained in the paper) that make the library calls. All the actual numerics are in http://github.com/fperez/nitime/blob/master/nitime/algorithms.py I'm not saying this will exactly solve your current design issues, but it may provide some useful ideas to think about. Cheers, f From peridot.faceted at gmail.com Fri Oct 16 15:58:07 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Fri, 16 Oct 2009 15:58:07 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: 2009/10/15 Fernando Perez : > On Thu, Oct 15, 2009 at 6:36 PM, Charles R Harris > wrote: >> >> How did you build the analysis objects? Part of what we have been discussing >> it the easiest way to expose a consistent interface while reusing the >> maximum amount of code. > > See > > http://github.com/fperez/nitime/blob/master/nitime/timeseries.py > > starting around line 504. ?The analyzers store a reference to a time > series object and then expose properties (that are replaced in-place > by a static attribute once computed, as explained in the paper) that > make the library calls. ?All the actual numerics are in > > http://github.com/fperez/nitime/blob/master/nitime/algorithms.py > > I'm not saying this will exactly solve your current design issues, but > it may provide some useful ideas to think about. Well, I think I'm convinced of the necessity for a low-level procedural interface, so I've set up a draft one in git. There's some question of just how bare-bones it should really be - for example should it include an add function, since that's just using the coefficient-extension function then adding? What about multiplying, which is the same thing, though this is not obvious? As it stands I've only included implementations of functions with nontrivial algorithms. This has the advantage that, realistically, these functions will not be shared between different polynomial representations. If I start including more straight-forward functions, I will need to figure out how to share the implementation of "add" between different representations if I want to avoid it being boilerplate code for all representations. If we're implementing the heavy lifting in a procedural interface, then the object implementations will just be plumbing (as I see your analyzer objects are). So on the one hand there's not much code to be shared, but on the other it's all boilerplate that would benefit from code sharing. I think, though, that the next important decision has nothing direct to do with code sharing; we need to settle on what polynomial objects will look like to users. Some things are clear: * Polynomial objects will be immutable. * Polynomial objects will support + - * / % ** divmod and __call__ * Polynomial objects will be able to report their roots * Polynomial objects will be able to return polynomials representing their derivatives and antiderivatives * Some polynomial objects will have extra features, like the ability to trim small coefficients from Chebyshev polynomials * We need some method of converting from one representation to another * We should have convenience objects zero, one, and X, to build polynomials. * There should be a least-squares polynomial fitting function. * Some polynomials will have functions that generate them (e.g. Chebyshev series for a function) * We need at least power basis, Chebyshev, and Lagrange polynomials. * Ultimately the polynomial class should be able to replace KroghInterpolator and BarycentricInterpolator and be stably returned from the orthogonal polynomial routines Less clear: * Should polynomials store a degree attribute or should it be inferred from the size of their coefficient array? (Sometimes it's handy to work with degree-inflated polynomials, but deflating their degree is a nontrivial operation.) * Should we try to accommodate complex variables? Complex coefficients? * Should we try to accommodate vector values? * Should polynomials have a method to evaluate derivatives and antiderivatives directly without constructing the derivative/antiderivative object first? * Do we need Lagrange-Hermite (some derivatives specified as well as values) polynomials? (KroghInterpolator implements these) * How should code that works for any polynomial type (e.g. multiplication by a Lagrange polynomial) identify that an object it has been given is some sort of polynomial? Callable and presence of a degree attribute? Subclass of Polynomial? * How should users implement their own polynomial types so they interoperate with scipy's built-in ones? Still debated: * I think we need a first-class object to represent the basis used to express a polynomial. Charles Harris disagrees; he wants to encode that information in the class of the polynomial plus an interval stored in the polynomial. Some different scheme would necessarily be used for Lagrange polynomials. * How should the zero polynomial be represented? I think its coefficient array should be np.zeros(0); Charles Harris prefers np.zeros(1). Anne From josh.k.lawrence at gmail.com Fri Oct 16 17:11:43 2009 From: josh.k.lawrence at gmail.com (Josh Lawrence) Date: Fri, 16 Oct 2009 17:11:43 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: <8C1B28D8-CC03-43DD-9358-EB3A54F03837@gmail.com> Hey all, I'm chiming in here after following this post. I am highly interested and excited about the resulting tools. Keep up the good work! On Oct 16, 2009, at 3:58 PM, Anne Archibald wrote: > * Should we try to accommodate complex variables? Complex > coefficients? FWIW, scipy's polynomial stuff would be must less valuable (not quite worthless, but close) to me if in general they did not support complex domains and ranges. Contour integration stuffs are just too valuable for a lot of my research. So a huge +1 to complex domains and ranges for these polynomial objects. -Josh From peridot.faceted at gmail.com Fri Oct 16 19:23:41 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Fri, 16 Oct 2009 19:23:41 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <8C1B28D8-CC03-43DD-9358-EB3A54F03837@gmail.com> References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> <8C1B28D8-CC03-43DD-9358-EB3A54F03837@gmail.com> Message-ID: 2009/10/16 Josh Lawrence : > Hey all, > > I'm chiming in here after following this post. I am highly interested > and excited about the resulting tools. Keep up the good work! > > On Oct 16, 2009, at 3:58 PM, Anne Archibald wrote: > >> * Should we try to accommodate complex variables? Complex >> coefficients? > > > FWIW, scipy's polynomial stuff would be must less valuable (not quite > worthless, but close) to me if in general they did not support complex > domains and ranges. Contour integration stuffs are just too valuable > for a lot of my research. So a huge +1 to complex domains and ranges > for these polynomial objects. Okay, we can definitely allow these, with just a little care about dtypes. But all the clever stability and approximation work I've seen has been for real polynomials - for example, the Chebyshev polynomials are in some sense optimal for approximating real functions, and I choose points for the Lagrange polynomials so as to get good stability on a real interval. For really good complex polynomial support we're going to have to do some reading in the literature - do you have any suggestions? For that matter, would you like to describe in a bit more detail how you'd use a polynomial class and what you want from it? There's definitely a danger of us producing a polynomial class that does just the few things we need it to... Anne From charlesr.harris at gmail.com Fri Oct 16 22:24:21 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 16 Oct 2009 20:24:21 -0600 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: On Fri, Oct 16, 2009 at 1:58 PM, Anne Archibald wrote: > 2009/10/15 Fernando Perez : > > On Thu, Oct 15, 2009 at 6:36 PM, Charles R Harris > > wrote: > >> > >> How did you build the analysis objects? Part of what we have been > discussing > >> it the easiest way to expose a consistent interface while reusing the > >> maximum amount of code. > > > > See > > > > http://github.com/fperez/nitime/blob/master/nitime/timeseries.py > > > > starting around line 504. The analyzers store a reference to a time > > series object and then expose properties (that are replaced in-place > > by a static attribute once computed, as explained in the paper) that > > make the library calls. All the actual numerics are in > > > > http://github.com/fperez/nitime/blob/master/nitime/algorithms.py > > > > I'm not saying this will exactly solve your current design issues, but > > it may provide some useful ideas to think about. > > Well, I think I'm convinced of the necessity for a low-level > procedural interface, so I've set up a draft one in git. There's some > question of just how bare-bones it should really be - for example > should it include an add function, since that's just using the > coefficient-extension function then adding? What about multiplying, > which is the same thing, though this is not obvious? > > It should definitely include add. For my personal use the following tend to be most important: Stuff I use: - evaluate - differentiate - integrate - leastsquares - zeros Stuff I don't use much but should be there: - add - subtract - multiply - divide Stuff for completeness: - powers Conveniences? - weight function - one - zero - X > As it stands I've only included implementations of functions with > nontrivial algorithms. Include them all, even the trivial ones. It may look silly, but more often than not it will save hassles down the line. Remember, you aren't doing this for you and *your* classes, you are doing it for someone else and *their* classes. Why should they have to worry about how you do addition? If nothing else the meaning of the inputs will be documented. > This has the advantage that, realistically, > these functions will not be shared between different polynomial > representations. If I start including more straight-forward functions, > I will need to figure out how to share the implementation of "add" > between different representations if I want to avoid it being > boilerplate code for all representations. > > Write a polyutils module and define: my_utterly_trivial_addition_function_that_bores_me = common_utterly_trivial_addition_function If we're implementing the heavy lifting in a procedural interface, > then the object implementations will just be plumbing (as I see your > analyzer objects are). So on the one hand there's not much code to be > shared, but on the other it's all boilerplate that would benefit from > code sharing. > > This is where the polynomial code differs from the nipy code: they have analyser objects with different interfaces that use the same basic functions on different datasets. We want to have a common "analyser" interface that uses different basic function. > I think, though, that the next important decision has nothing direct > to do with code sharing; we need to settle on what polynomial objects > will look like to users. > > Some things are clear: > > * Polynomial objects will be immutable. > * Polynomial objects will support + - * / % ** divmod and __call__ > * Polynomial objects will be able to report their roots > * Polynomial objects will be able to return polynomials representing > their derivatives and antiderivatives > * Some polynomial objects will have extra features, like the ability > to trim small coefficients from Chebyshev polynomials > * We need some method of converting from one representation to another > Since the Chebyshev class can operate with objects, evaluating the Chebyshev series for the "X" polynomial object does the conversion. This works for all the graded polynomial classes and might could be made to work with Lagrange basis. Hopefully, that is something that isn't used a whole lot anyway. Efficient evaluation for numeric arguments need to be reasonably efficient, evaluation for objects less so. > * We should have convenience objects zero, one, and X, to build > polynomials. > * There should be a least-squares polynomial fitting function. > +1 Maybe a standard interval also? > * Some polynomials will have functions that generate them (e.g. > Chebyshev series for a function) > Isn't this a special case of fitting? > * We need at least power basis, Chebyshev, and Lagrange polynomials. > * Ultimately the polynomial class should be able to replace > KroghInterpolator and BarycentricInterpolator and be stably returned > from the orthogonal polynomial routines > > We probably want trigonometric polynomial class(es) also. I use barycentric forms of these for the complex remez algorithm. They are just streamlined versions of a lagrange basis with the sample points on the unit circle in the complex plane or the two-sheeted Riemann surface equivalent. > Less clear: > > * Should polynomials store a degree attribute or should it be inferred > from the size of their coefficient array? (Sometimes it's handy to > work with degree-inflated polynomials, but deflating their degree is a > nontrivial operation.) > I choose the latter because it is more dynamic. The coefficients are public attributes and if some fool wants to assign a different array to them I figure thing should still work. Computing it is unlikely to be a performance bottleneck in any case. * Should we try to accommodate complex variables? Complex coefficients? > Complex coefficients, definitely. I might not use them, but no doubt someone will want them. If there are stability problems, that is part of the game and maybe the user can offer improvements. Caveat emptor. > * Should we try to accommodate vector values? > If you mean evaluate at a vector of points, then yes, it is the only way to overcome function calling overhead if you need a ton of values. > * Should polynomials have a method to evaluate derivatives and > antiderivatives directly without constructing the > derivative/antiderivative object first? > I think that is unnecessary. > * Do we need Lagrange-Hermite (some derivatives specified as well as > values) polynomials? (KroghInterpolator implements these) > Don't know > * How should code that works for any polynomial type (e.g. > multiplication by a Lagrange polynomial) identify that an object it > has been given is some sort of polynomial? Callable and presence of a > degree attribute? Subclass of Polynomial? > I don't think the classes need to interoperate, in fact I would discourage it because it complicates everything for a need I don't see. But if there are such functions, then deriving everything from a dummy Polynomial class would be the best approach IMHO. We could do that anyway to leave the door open to future changes. class Polynomial: pass > * How should users implement their own polynomial types so they > interoperate with scipy's built-in ones? > > What do you mean by interoperate? > Still debated: > > * I think we need a first-class object to represent the basis used to > express a polynomial. Charles Harris disagrees; he wants to encode > that information in the class of the polynomial plus an interval > stored in the polynomial. Some different scheme would necessarily be > used for Lagrange polynomials. > Things are encoded in 1) an array of coefficients 2) an interval, and 3) the base functions and 4) the class name. In the Nipy model, 1) and 2) would be the data containers. > * How should the zero polynomial be represented? I think its > coefficient array should be np.zeros(0); Charles Harris prefers > np.zeros(1). > > You know where I stand ;) For a Lagrange basis of indeterminate degree I think is should be an array of zeros of the appropriate dimension. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Oct 17 03:26:45 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 17 Oct 2009 00:26:45 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: On Fri, Oct 16, 2009 at 12:58 PM, Anne Archibald wrote: > > If we're implementing the heavy lifting in a procedural interface, > then the object implementations will just be plumbing (as I see your > analyzer objects are). So on the one hand there's not much code to be > shared, but on the other it's all boilerplate that would benefit from > code sharing. Yes, in our case it was a deliberate design decision: we were in a sense hedging our bets. Since it's so easy to go down the wrong design path with complex objects, we basically punted and made the user-visible objects trivially simple from an algorithmic/computational perspective. This puts all the 'smarts' in more cumbersome procedural interfaces, but it also means that the same procedural foundation can support more than one OO design. So if either we get our design horribly wrong, or it's simply not a good fit for someone, there's no major loss, as they can just build their own interface on top of the same machinery, and very little code is lost. I'm not sure this idea works really well in the long run, but we got there after being bitten (many times) by complex OO codes that end up 'trapping' a lot of procedural smarts within, and thus condemning them to be lost wherever the OO design doesn't work well. We thus tried to separate the two concerns a little bit. More experience will tell us whether the approach does work well, and what its limitations in practice turn out to be. Cheers, f From josef.pktd at gmail.com Sat Oct 17 10:01:34 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Sat, 17 Oct 2009 10:01:34 -0400 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> Message-ID: <1cd32cbb0910170701q2546f903u39b7eb77e4fbc973@mail.gmail.com> On Sat, Oct 17, 2009 at 3:26 AM, Fernando Perez wrote: > On Fri, Oct 16, 2009 at 12:58 PM, Anne Archibald > wrote: >> >> If we're implementing the heavy lifting in a procedural interface, >> then the object implementations will just be plumbing (as I see your >> analyzer objects are). So on the one hand there's not much code to be >> shared, but on the other it's all boilerplate that would benefit from >> code sharing. > > Yes, in our case it was a deliberate design decision: we were in a > sense hedging our bets. ?Since it's so easy to go down the wrong > design path with complex objects, we basically punted and made the > user-visible objects trivially simple from an > algorithmic/computational perspective. ?This puts all the 'smarts' in > more cumbersome procedural interfaces, but it also means that the same > procedural foundation can support more than one OO design. ?So if > either we get our design horribly wrong, or it's simply not a good fit > for someone, there's no major loss, as they can just build their own > interface on top of the same machinery, and very little code is lost. > > I'm not sure this idea works really well in the long run, but we got > there after being bitten (many times) by complex OO codes that end up > 'trapping' a lot of procedural smarts within, and thus condemning them > to be lost wherever the OO design doesn't work well. ?We thus tried to > ?separate the two concerns a little bit. Just to point out that OO design and "framework independence" are not necessarily exclusive: In statsmodels, we still use classes and subclasses to benefit from inheritance and OO, but the interface is in plain numpy arrays. This way, any data framework or data structure (nitime time series, scikits.timeseries, pandas, tabular, ...) or formula framework can use the classes, but has to write the conversion code from a data frame object to the ndarray design matrix and back again themselves. However, we are still not sure what or where the boundaries should be, and we still have methods like statistical tests, that would be useful as standalone functions so they can be reused without requiring that the estimation is done by the statsmodels classes. In the nitime case, whether ``algorithms`` uses classes wouldn't really matter for the easy usage from outside of neuroimaging as long as it doesn't force the user of the algorithms to use the nitime timeseries class as the Analyzers in timeseries do. Josef > > More experience will tell us whether the approach does work well, and > what its limitations in practice turn out to be. > > Cheers, > > f > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From fperez.net at gmail.com Sun Oct 18 01:18:22 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 17 Oct 2009 22:18:22 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: <1cd32cbb0910170701q2546f903u39b7eb77e4fbc973@mail.gmail.com> References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> <1cd32cbb0910170701q2546f903u39b7eb77e4fbc973@mail.gmail.com> Message-ID: 2009/10/17 josef.pktd : > Just to point out that OO design and "framework independence" are > not necessarily exclusive: > > In statsmodels, we still use classes and subclasses to benefit from > inheritance and OO, but the interface is in plain numpy arrays. > This way, any data framework or data structure (nitime time series, > scikits.timeseries, pandas, tabular, ...) or formula framework > can use the classes, but has to write the conversion code from > a data frame object to the ndarray design matrix and back again > themselves. [...] I agree with you, and in fact in nitime we obviously do use 'objects' in the sense that the procedural interface operates on python base objects like lists, numbers and strings, and numpy arrays. What I meant to say, and perhaps wasn't precise enough, was that we wanted this base layer to not depend on new, custom objects introduced by us. In this manner, this base functional layer could be used to build new class hierarchies independent from ours while reusing the base algorithms. In terms of dependencies, we wanted our chain to be (I suck at ascii art): 0. python objects, nd arrays 1.a algorithms 1.b data containers 2. Analyzers that need 1a, 1b. > In the nitime case, whether ``algorithms`` uses classes wouldn't > really matter for the easy usage from outside of neuroimaging as > long as it doesn't force the user of the algorithms to use the > nitime timeseries class as the Analyzers in timeseries do. The point was to have algorithms only rely on objects that would be familiar and available to anyone using already numpy. Part of it is a code dependency issue, part of it is cognitive load: if you need to learn a lot of new object APIs before you can start using a library, the cost for you to start using it goes up. We've found that so far, we can keep the algorithmic library depending only on numpy/scipy numerical tools. We haven't set in stone avoiding more objects at that layer at all costs, but we do want to keep it simple to use and understand. We are gradually making the other objects richer, but that is driven by careful discussion of real use cases: this week we started working with a Berkeley group that has a lot of code for the analysis of single-cell recording data, and we're going to grow the object design to accommodate such uses. We're very happy about this, because it means our objects will grow in functionality but driven by usage-proven needs. Design isn't a black-and-white world of rights and wrongs, but rather a collection of compromises that, if successful, when put together provide something useful. We've taken one approach to attempt that, based on prior experience (especially prior mistakes we want to avoid), but I'm not trying to say it's the *only* way to do this. Just providing feedback that may be useful to others facing similar design questions. Cheers, f From fperez.net at gmail.com Sun Oct 18 01:22:09 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 17 Oct 2009 22:22:09 -0700 Subject: [SciPy-dev] Generic polynomials class (was Re: Volunteer for Scipy Project) In-Reply-To: References: <1cd32cbb0910132152j41a8f7aan3ef6ae5c87f18da2@mail.gmail.com> <1cd32cbb0910170701q2546f903u39b7eb77e4fbc973@mail.gmail.com> Message-ID: 2009/10/17 Fernando Perez : [ unreadable run-in text ] I have no idea why gmail mangled that reply as it did, destroying all paragraph structure. Sorry for that mess, I'd never seen a gmail reply like that. Cheers, f From arkapravobhaumik at gmail.com Sun Oct 18 20:34:28 2009 From: arkapravobhaumik at gmail.com (Arkapravo Bhaumik) Date: Mon, 19 Oct 2009 01:34:28 +0100 Subject: [SciPy-dev] Issues with square roots of complex numbers Message-ID: Dear Friends I faced some issues in square root of iota, and evaluated it using cmath, scipy and Matlab. The details are included on the following link from my blogspot,http://programming-unlimited.blogspot.com/2009/10/square-root-of-iota.html . Please feel free to comment or criticise. Regards Arkapravo -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Sun Oct 18 21:05:51 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Sun, 18 Oct 2009 21:05:51 -0400 Subject: [SciPy-dev] Issues with square roots of complex numbers In-Reply-To: References: Message-ID: <1cd32cbb0910181805o11e9138fy4db70aef45d02e4b@mail.gmail.com> On Sun, Oct 18, 2009 at 8:34 PM, Arkapravo Bhaumik wrote: > Dear Friends > > I faced some issues in square root of iota, and evaluated it using cmath, > scipy and Matlab. > > The details are included on the following link from my blogspot, > http://programming-unlimited.blogspot.com/2009/10/square-root-of-iota.html . > Please feel free to comment or criticise. > > Regards > > Arkapravo > Looks to me like standard floating point issues, matlab is not more precise because it displays only 4 digits >>> np.sqrt([1j]) array([ 0.7071+0.7071j]) >>> np.array([1j])**(0.5) array([ 0.7071+0.7071j]) However, it seems like printoptions doesn't work for scalars (?) >>> np.set_printoptions(precision=4) >>> np.sqrt(1j) (0.70710678118654746+0.70710678118654757j) >>> print np.sqrt(2) 1.41421356237 Josef From josef.pktd at gmail.com Mon Oct 19 11:20:34 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Mon, 19 Oct 2009 11:20:34 -0400 Subject: [SciPy-dev] test failures in trunk in special Message-ID: <1cd32cbb0910190820q36a68b80ia9f82f4ef50ec8bc@mail.gmail.com> Are these known failures? (WindowsXP, MingW) Josef >>> scipy.test() Running unit tests for scipy NumPy version 1.4.0.dev7539 NumPy is installed in C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\ site-packages\numpy SciPy version 0.8.0.dev5970 SciPy is installed in C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\ site-packages\scipy Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Int el)] nose version 0.11.1 ====================================================================== FAIL: test_boost_data.test_all(,) ---------------------------------------------------------------------- Traceback (most recent call last): File "c:\josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ nose-0.11.1-py2.5.egg\nose\case.py", line 183, in runTest self.test(*self.arg) File "C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ scipy\special\tests\test_boost_data.py", line 196, in _test_factory test.check(dtype=dtype) File "C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ scipy\special\tests\test_boost_data.py", line 332, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 8.53484e-016 Max |rdiff|: 2.61501e-012 Bad results for the following points (in output 0): 1.4615000486373901 => -0.00012783367977313529 != -0. 000127833679772801 (rdiff 2.6150122207678078e-012) ====================================================================== FAIL: test_boost_data.test_all(,) ---------------------------------------------------------------------- Traceback (most recent call last): File "c:\josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ nose-0.11.1-py2.5.egg\nose\case.py", line 183, in runTest self.test(*self.arg) File "C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ scipy\special\tests\test_boost_data.py", line 196, in _test_factory test.check(dtype=dtype) File "C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ scipy\special\tests\test_boost_data.py", line 332, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 8.53484e-016 Max |rdiff|: 2.61501e-012 Bad results for the following points (in output 0): (1.4615000486373901+0j) => (-0.00012783367977313529+0j) != -0. 000127833679772801 (rdiff 2.6150122207678078e-012) ---------------------------------------------------------------------- Ran 3880 tests in 77.688s FAILED (KNOWNFAIL=6, SKIP=29, failures=2) From nwagner at iam.uni-stuttgart.de Mon Oct 19 17:13:43 2009 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Mon, 19 Oct 2009 23:13:43 +0200 Subject: [SciPy-dev] test failures in trunk in special In-Reply-To: <1cd32cbb0910190820q36a68b80ia9f82f4ef50ec8bc@mail.gmail.com> References: <1cd32cbb0910190820q36a68b80ia9f82f4ef50ec8bc@mail.gmail.com> Message-ID: I can reproduce some failures on linux (opensuse 11.1) ====================================================================== FAIL: test_boost_data.test_all(,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.11.2.dev-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 196, in _test_factory test.check(dtype=dtype) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 332, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 9.29812e-16 Max |rdiff|: 3.15103e-12 Bad results for the following points (in output 0): 1.4609999656677246 => -0.00061191937930571427 != -0.00061191937930632739 (rdiff 1.0019560570799086e-12) 1.4615000486373901 => -0.00012783367977320381 != -0.000127833679772801 (rdiff 3.151033537081845e-12) ====================================================================== FAIL: test_boost_data.test_all(,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.11.2.dev-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 196, in _test_factory test.check(dtype=dtype) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 332, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 9.29812e-16 Max |rdiff|: 3.15103e-12 Bad results for the following points (in output 0): (1.4609999656677246+0j) => (-0.00061191937930571427+0j) != -0.00061191937930632739 (rdiff 1.0019560570799086e-12) (1.4615000486373901+0j) => (-0.00012783367977320381+0j) != -0.000127833679772801 (rdiff 3.151033537081845e-12) ====================================================================== FAIL: test_boost_data.test_all(,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.11.2.dev-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 196, in _test_factory test.check(dtype=dtype) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 332, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 2.27374e-13 Max |rdiff|: 1.97027e-15 Bad results for the following points (in output 0): 0.0017721911426633596 -0.38366591930389404 => 0.001772191006114956 != 0.0017721910061149527 (rdiff 1.8353588897773323e-15) 0.0022177286446094513 -0.38366591930389404 => 0.0022177283770131993 != 0.0022177283770131949 (rdiff 1.9555184191595052e-15) 0.0074444999918341637 -0.38366591930389404 => 0.0074444898700397055 != 0.0074444898700396925 (rdiff 1.7476585094415186e-15) 0.014336004853248596 -0.38366591930389404 => 0.014335932572318438 != 0.014335932572318417 (rdiff 1.4520633106155296e-15) 0.017609169706702232 -0.38366591930389404 => 0.017609035755112572 != 0.017609035755112537 (rdiff 1.9702651526199038e-15) ====================================================================== FAIL: test_boost_data.test_all(,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.11.2.dev-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 196, in _test_factory test.check(dtype=dtype) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 332, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 1.76256e-14 Max |rdiff|: 2.24484e-09 Bad results for the following points (in output 0): (9.4190988540649414+0j) => (7.8515817802937704e-06+0j) != 7.8515817979193266e-06 (rdiff 2.2448414456281229e-09) ====================================================================== FAIL: test_boost_data.test_all(,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.11.2.dev-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 196, in _test_factory test.check(dtype=dtype) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 332, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 8.67362e-19 Max |rdiff|: 1.11795e-15 Bad results for the following points (in output 0): 18.0 9.4190988540649414 => 3.0306769539617701e-06 != 3.0306769539617735e-06 (rdiff 1.117945541700891e-15) ====================================================================== FAIL: test_boost_data.test_all(,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.11.2.dev-py2.6.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 196, in _test_factory test.check(dtype=dtype) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_boost_data.py", line 332, in check assert False, "\n".join(msg) AssertionError: Max |adiff|: 1.91079e-16 Max |rdiff|: 1.1286e-10 Bad results for the following points (in output 0): 1.9999923706054688 => -3.225569724207844e-06 != -3.2255697240167648e-06 (rdiff 5.9238898645735525e-11) 1.9999961853027344 => -1.6127895543505133e-06 != -1.6127895545325333e-06 (rdiff 1.1286033463889784e-10) ---------------------------------------------------------------------- Ran 3868 tests in 95.566s FAILED (KNOWNFAIL=6, SKIP=17, failures=6) Cheers, Nils On Mon, 19 Oct 2009 11:20:34 -0400 josef.pktd at gmail.com wrote: > Are these known failures? (WindowsXP, MingW) > > Josef > >>>> scipy.test() > Running unit tests for scipy > NumPy version 1.4.0.dev7539 > NumPy is installed in >C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\ > site-packages\numpy > SciPy version 0.8.0.dev5970 > SciPy is installed in >C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\ > site-packages\scipy > Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) >[MSC v.1310 32 bit (Int > el)] > nose version 0.11.1 > > > ====================================================================== >FAIL: test_boost_data.test_all(digamma_root_data_ipp-digamm > a_root_data>,) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File >"c:\josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ > nose-0.11.1-py2.5.egg\nose\case.py", line 183, in >runTest > self.test(*self.arg) > File >"C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ > scipy\special\tests\test_boost_data.py", line 196, in >_test_factory > test.check(dtype=dtype) > File >"C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ > scipy\special\tests\test_boost_data.py", line 332, in >check > assert False, "\n".join(msg) > AssertionError: > Max |adiff|: 8.53484e-016 > Max |rdiff|: 2.61501e-012 > Bad results for the following points (in output 0): > 1.4615000486373901 => > -0.00012783367977313529 != -0. > 000127833679772801 (rdiff > 2.6150122207678078e-012) > > ====================================================================== >FAIL: test_boost_data.test_all((complex): digamma_root_data_ > ipp-digamma_root_data>,) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File >"c:\josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ > nose-0.11.1-py2.5.egg\nose\case.py", line 183, in >runTest > self.test(*self.arg) > File >"C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ > scipy\special\tests\test_boost_data.py", line 196, in >_test_factory > test.check(dtype=dtype) > File >"C:\Josef\_progs\virtualpy25\testscipy\numpyscipyoct09\lib\site-packages\ > scipy\special\tests\test_boost_data.py", line 332, in >check > assert False, "\n".join(msg) > AssertionError: > Max |adiff|: 8.53484e-016 > Max |rdiff|: 2.61501e-012 > Bad results for the following points (in output 0): > (1.4615000486373901+0j) => > (-0.00012783367977313529+0j) != -0. > 000127833679772801 (rdiff > 2.6150122207678078e-012) > > ---------------------------------------------------------------------- > Ran 3880 tests in 77.688s > >FAILED (KNOWNFAIL=6, SKIP=29, failures=2) > failures=2> > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev From drizzd at aon.at Mon Oct 19 18:20:24 2009 From: drizzd at aon.at (Clemens Buchacher) Date: Tue, 20 Oct 2009 00:20:24 +0200 Subject: [SciPy-dev] [mlabwrap] linker path Message-ID: <20091019222024.GA26523@localhost> Hi, If I understand correctly, mlabwrap is now maintained as part of the scikits project. Since the latest release 1.1-pre from the old project site does not even compile out of the box, I'm assuming the latest version from the SVN repo is the most usable one. However, contrary to 1.1-pre (after fixing it up with the patch below), it requires matlab to be added to the linker path. I really don't want that, because matlab contains old versions of standard libraries such as zlib and libstdc++. Since we only need libeng.so, libmat.so, libmx.so and libut.so, wouldn't it be much nicer to create a directory with symlinks to those instead? Cheers, Clemens -->o-- Subject: [PATCH] non-const required by Py_DECREF --- mlabraw.cpp | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/mlabraw.cpp b/mlabraw.cpp index 18a972d..429102d 100644 --- a/mlabraw.cpp +++ b/mlabraw.cpp @@ -310,7 +310,7 @@ static inline void copyCplxNumeric2Mx(T *p,int size,double *pRData,double *pIDat } } -static mxArray *makeMxFromNumeric(const PyArrayObject *pSrc) +static mxArray *makeMxFromNumeric(PyArrayObject *pSrc) { npy_intp lRows=0, lCols=0; bool lIsComplex; @@ -320,7 +320,7 @@ static mxArray *makeMxFromNumeric(const PyArrayObject *pSrc) mxArray *lRetval = NULL; mwSize dims[NPY_MAXDIMS]; mwSize nDims = pSrc->nd; - const PyArrayObject *ap=NULL; + PyArrayObject *ap=NULL; switch (pSrc->nd) { case 0: // XXX the evil 0D @@ -353,7 +353,7 @@ static mxArray *makeMxFromNumeric(const PyArrayObject *pSrc) // converts to fortran order if not already if(!PyArray_ISFORTRAN(pSrc)){ - ap = (const PyArrayObject *)PyArray_FromArray((PyArrayObject*)pSrc,NULL,NPY_ALIGNED|NPY_F_CONTIGUOUS); + ap = (PyArrayObject *)PyArray_FromArray((PyArrayObject*)pSrc,NULL,NPY_ALIGNED|NPY_F_CONTIGUOUS); } else{ ap = pSrc; @@ -501,13 +501,13 @@ static mxArray *numeric2mx(PyObject *pSrc) pyassert(PyArray_API, "Unable to perform this function without NumPy installed"); if (PyArray_Check(pSrc)) { - lDst = makeMxFromNumeric((const PyArrayObject *)pSrc); + lDst = makeMxFromNumeric((PyArrayObject *)pSrc); } else if (PySequence_Check(pSrc)) { lDst = makeMxFromSeq(pSrc); } else if (PyObject_HasAttrString(pSrc, "__array__")) { PyObject *arp; arp = PyObject_CallMethod(pSrc, "__array__", NULL); - lDst = makeMxFromNumeric((const PyArrayObject *)arp); + lDst = makeMxFromNumeric((PyArrayObject *)arp); Py_DECREF(arp); // FIXME check this is correct; } else if (PyInt_Check(pSrc) || PyLong_Check(pSrc) || -- 1.6.5.1 From dwf at cs.toronto.edu Mon Oct 19 18:08:27 2009 From: dwf at cs.toronto.edu (David Warde-Farley) Date: Mon, 19 Oct 2009 18:08:27 -0400 Subject: [SciPy-dev] [mlabwrap] linker path In-Reply-To: <20091019222024.GA26523@localhost> References: <20091019222024.GA26523@localhost> Message-ID: <20091019220827.GA18383@rodimus> On Tue, Oct 20, 2009 at 12:20:24AM +0200, Clemens Buchacher wrote: > However, contrary to 1.1-pre (after fixing it up with the patch below), it > requires matlab to be added to the linker path. I really don't want that, > because matlab contains old versions of standard libraries such as zlib and > libstdc++. Since we only need libeng.so, libmat.so, libmx.so and libut.so, > wouldn't it be much nicer to create a directory with symlinks to those > instead? This may not work, since the libraries you specify may depend on specific versions of zlib, libstdc++, etc. The version installed in your system's link path may be ABI-incompatible with the one that the Matlab guys built against, hence the reason for including them in the first place. David From david at ar.media.kyoto-u.ac.jp Mon Oct 19 23:25:51 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 20 Oct 2009 12:25:51 +0900 Subject: [SciPy-dev] [mlabwrap] linker path In-Reply-To: <20091019220827.GA18383@rodimus> References: <20091019222024.GA26523@localhost> <20091019220827.GA18383@rodimus> Message-ID: <4ADD2DBF.1020607@ar.media.kyoto-u.ac.jp> David Warde-Farley wrote: > On Tue, Oct 20, 2009 at 12:20:24AM +0200, Clemens Buchacher wrote: > > >> However, contrary to 1.1-pre (after fixing it up with the patch below), it >> requires matlab to be added to the linker path. I really don't want that, >> because matlab contains old versions of standard libraries such as zlib and >> libstdc++. Since we only need libeng.so, libmat.so, libmx.so and libut.so, >> wouldn't it be much nicer to create a directory with symlinks to those >> instead? >> > > This may not work, since the libraries you specify may depend on specific versions of zlib, libstdc++, etc. The > version installed in your system's link path may be ABI-incompatible with the one that the Matlab guys built against, > hence the reason for including them in the first place. > That's a major problem with the matlab C API BTW - since matlab uses quite a few well known libraries internally (atlas, fftw), if your own extension uses say fftw as well, it is very unlikely to work. That's a consequence of the dreadful flat namespace in the conventional unix model. On the "bright" side, I doubt that the new mlabwrap changes anything: when mlabwrap used ctypes and loaded the matlab engine through ctypes, the old zlib was likely loaded as well through rpath links. I think that at least in theory, this could be solved with symbol versioning, but this is likely to be quite hairy, if not impossible since you don't have matlab sources. I doubt it worths it for most cases. David From angel.yanguas at gmail.com Tue Oct 20 00:08:48 2009 From: angel.yanguas at gmail.com (Angel Yanguas-Gil) Date: Mon, 19 Oct 2009 23:08:48 -0500 Subject: [SciPy-dev] contributing to numpy/scipy documentation Message-ID: <11f4deca0910192108m6d345ceaob0303d9653026447@mail.gmail.com> Hi! I'd like to contribute to the numpy/scipy documentation. Could I get edit rights? My user name is: anglyan Thanks! Angel From gael.varoquaux at normalesup.org Tue Oct 20 00:39:17 2009 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Tue, 20 Oct 2009 06:39:17 +0200 Subject: [SciPy-dev] contributing to numpy/scipy documentation In-Reply-To: <11f4deca0910192108m6d345ceaob0303d9653026447@mail.gmail.com> References: <11f4deca0910192108m6d345ceaob0303d9653026447@mail.gmail.com> Message-ID: <20091020043917.GA25011@phare.normalesup.org> On Mon, Oct 19, 2009 at 11:08:48PM -0500, Angel Yanguas-Gil wrote: > Hi! > I'd like to contribute to the numpy/scipy documentation. Could I get > edit rights? My user name is: anglyan Done! Thank you for your interest. Ga?l From peridot.faceted at gmail.com Tue Oct 20 01:11:17 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Tue, 20 Oct 2009 01:11:17 -0400 Subject: [SciPy-dev] contributing to numpy/scipy documentation In-Reply-To: <11f4deca0910192108m6d345ceaob0303d9653026447@mail.gmail.com> References: <11f4deca0910192108m6d345ceaob0303d9653026447@mail.gmail.com> Message-ID: I don't think I have write permission on the Wiki either. Could I please be added? My user name is aarchiba. Thanks! Anne 2009/10/20 Angel Yanguas-Gil : > Hi! > > I'd like to contribute to the numpy/scipy documentation. Could I get > edit rights? My user name is: anglyan > > Thanks! > > Angel > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From gael.varoquaux at normalesup.org Tue Oct 20 01:14:58 2009 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Tue, 20 Oct 2009 07:14:58 +0200 Subject: [SciPy-dev] contributing to numpy/scipy documentation In-Reply-To: References: <11f4deca0910192108m6d345ceaob0303d9653026447@mail.gmail.com> Message-ID: <20091020051458.GC25011@phare.normalesup.org> On Tue, Oct 20, 2009 at 01:11:17AM -0400, Anne Archibald wrote: > I don't think I have write permission on the Wiki either. Could I > please be added? My user name is aarchiba. Done. Ga?l From scott.sinclair.za at gmail.com Tue Oct 20 01:36:44 2009 From: scott.sinclair.za at gmail.com (Scott Sinclair) Date: Tue, 20 Oct 2009 07:36:44 +0200 Subject: [SciPy-dev] contributing to numpy/scipy documentation In-Reply-To: <20091020043917.GA25011@phare.normalesup.org> References: <11f4deca0910192108m6d345ceaob0303d9653026447@mail.gmail.com> <20091020043917.GA25011@phare.normalesup.org> Message-ID: <6a17e9ee0910192236n5bd7a3fcjf983f6ae408279a7@mail.gmail.com> >2009/10/20 Gael Varoquaux : > On Mon, Oct 19, 2009 at 11:08:48PM -0500, Angel Yanguas-Gil wrote: >> Hi! > >> I'd like to contribute to the numpy/scipy documentation. Could I get >> edit rights? My user name is: anglyan > > Done! Thank you for your interest. There is a sprint on this weekend: http://projects.scipy.org/scipy/wiki/SciPySprint200910 There will be people working on the docs and grateful for some help... Cheers, Scott From pav+sp at iki.fi Tue Oct 20 04:17:39 2009 From: pav+sp at iki.fi (Pauli Virtanen) Date: Tue, 20 Oct 2009 08:17:39 +0000 (UTC) Subject: [SciPy-dev] test failures in trunk in special References: <1cd32cbb0910190820q36a68b80ia9f82f4ef50ec8bc@mail.gmail.com> Message-ID: Mon, 19 Oct 2009 23:13:43 +0200, Nils Wagner wrote: > > I can reproduce some failures on linux (opensuse 11.1) The test tolerances were a bit too tight. I relaxed them slightly. Please report again if you see more of these, thanks! -- Pauli Virtanen From ralf.gommers at googlemail.com Tue Oct 20 05:56:46 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Tue, 20 Oct 2009 11:56:46 +0200 Subject: [SciPy-dev] Proposed generic docstring for attributes of class generic In-Reply-To: <45d1ab480910141538p10d1c812jcb76fed378035e46@mail.gmail.com> References: <45d1ab480910141538p10d1c812jcb76fed378035e46@mail.gmail.com> Message-ID: On Thu, Oct 15, 2009 at 12:38 AM, David Goldsmith wrote: > Short summary: Not implemented (virtual attribute) > > Extended summary: Class generic exists solely to derive numpy scalars from, > and possesses, albeit unimplemented, all the attributes of the ndarray class > so as to provide a uniform API. > > See Also: the corresponding attribute of the derived class of interest. > > Comments? > I saw you put this in the wiki already. One thing that is wrong is the See Also on each page, instead of the default text it needs the right link, i.e. for numpy.generic.sum refer to ndarray.sum. Other than that I have no idea. Please ask again during the sprint if you don't get a better answer. Cheers, Ralf > DG > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nwagner at iam.uni-stuttgart.de Tue Oct 20 12:16:55 2009 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Tue, 20 Oct 2009 18:16:55 +0200 Subject: [SciPy-dev] test failures in trunk in special In-Reply-To: References: <1cd32cbb0910190820q36a68b80ia9f82f4ef50ec8bc@mail.gmail.com> Message-ID: On Tue, 20 Oct 2009 08:17:39 +0000 (UTC) Pauli Virtanen wrote: > Mon, 19 Oct 2009 23:13:43 +0200, Nils Wagner wrote: >> >> I can reproduce some failures on linux (opensuse 11.1) > > The test tolerances were a bit too tight. I relaxed them >slightly. Please > report again if you see more of these, thanks! > > -- > Pauli Virtanen > Great ! Ran 3868 tests in 93.999s OK (KNOWNFAIL=6, SKIP=17) Nils > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org From drizzd at aon.at Tue Oct 20 18:01:29 2009 From: drizzd at aon.at (Clemens Buchacher) Date: Wed, 21 Oct 2009 00:01:29 +0200 Subject: [SciPy-dev] [mlabwrap] linker path In-Reply-To: <4ADD2DBF.1020607@ar.media.kyoto-u.ac.jp> References: <20091019222024.GA26523@localhost> <20091019220827.GA18383@rodimus> <4ADD2DBF.1020607@ar.media.kyoto-u.ac.jp> Message-ID: <20091020220129.GA10895@localhost> Is it common practice not to Cc the OP on this list? Luckily, I just registered yesterday. On Tue, Oct 20, 2009 at 12:25:51PM +0900, David Cournapeau wrote: > On the "bright" side, I doubt that the new mlabwrap changes anything: > when mlabwrap used ctypes and loaded the matlab engine through ctypes, > the old zlib was likely loaded as well through rpath links. Yeah, pre-1.1 has the matlab path hardcoded. I'm not sure if I like that or not. > I think that at least in theory, this could be solved with symbol > versioning, but this is likely to be quite hairy, if not impossible > since you don't have matlab sources. I doubt it worths it for most cases. Well, I'm helping myself by setting LD_LIBRARY_PATH=/lib:/usr/lib:/path/to/matlab/bin/glnxa64 for now. Seems to work so far. Clemens From robert.kern at gmail.com Tue Oct 20 18:11:32 2009 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 20 Oct 2009 17:11:32 -0500 Subject: [SciPy-dev] [mlabwrap] linker path In-Reply-To: <20091020220129.GA10895@localhost> References: <20091019222024.GA26523@localhost> <20091019220827.GA18383@rodimus> <4ADD2DBF.1020607@ar.media.kyoto-u.ac.jp> <20091020220129.GA10895@localhost> Message-ID: <3d375d730910201511k41a76bcx98b5e9b82b254c33@mail.gmail.com> On Tue, Oct 20, 2009 at 17:01, Clemens Buchacher wrote: > Is it common practice not to Cc the OP on this list? Yes. Please do not Cc people when you respond to the list. If you expect a response, we expect you to read the list. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From david at ar.media.kyoto-u.ac.jp Tue Oct 20 20:45:41 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 21 Oct 2009 09:45:41 +0900 Subject: [SciPy-dev] [mlabwrap] linker path In-Reply-To: <20091020220129.GA10895@localhost> References: <20091019222024.GA26523@localhost> <20091019220827.GA18383@rodimus> <4ADD2DBF.1020607@ar.media.kyoto-u.ac.jp> <20091020220129.GA10895@localhost> Message-ID: <4ADE59B5.8050809@ar.media.kyoto-u.ac.jp> Clemens Buchacher wrote: > Is it common practice not to Cc the OP on this list? Luckily, I just > registered yesterday. > > On Tue, Oct 20, 2009 at 12:25:51PM +0900, David Cournapeau wrote: > > >> On the "bright" side, I doubt that the new mlabwrap changes anything: >> when mlabwrap used ctypes and loaded the matlab engine through ctypes, >> the old zlib was likely loaded as well through rpath links. >> > > Yeah, pre-1.1 has the matlab path hardcoded. I'm not sure if I like that or > not. > > That's not what I mean: when you load libeng.so, the loader automatically loads the dependend libraries, including libz, according to the rpath. On my machine, ldd libeng.so returns: linux-gate.so.1 => (0xb7ef0000) libut.so => /home/speech/matlab/bin/glnx86/./libut.so (0xb7e4b000) libmx.so => /home/speech/matlab/bin/glnx86/./libmx.so (0xb7df7000) libmat.so => /home/speech/matlab/bin/glnx86/./libmat.so (0xb7dee000) libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7db4000) libstdc++.so.5 => /home/speech/matlab/bin/glnx86/./../../sys/os/glnx86/libstdc++.so.5 (0xb7d0f000) libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7ce9000) libgcc_s.so.1 => /home/speech/matlab/bin/glnx86/./../../sys/os/glnx86/libgcc_s.so.1 (0xb7ce1000) libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7b7d000) librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0xb7b74000) libicudata.so.24 => /home/speech/matlab/bin/glnx86/./libicudata.so.24 (0xb7b72000) libicui18n.so.24 => /home/speech/matlab/bin/glnx86/./libicui18n.so.24 (0xb7ab6000) libicuuc.so.24 => /home/speech/matlab/bin/glnx86/./libicuuc.so.24 (0xb7a1f000) libustdio.so.24 => /home/speech/matlab/bin/glnx86/./libustdio.so.24 (0xb7a0f000) libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7a0b000) libz.so => /home/speech/matlab/bin/glnx86/./libz.so (0xb79fc000) /lib/ld-linux.so.2 (0xb7ef1000) Note the full paths - this is most likely a consequence of using rpath when linking libeng.so. Those paths have precedence over LD_LIBRARY_PATH. I am not sure what happens if you load libeng.so dynamically through dlopen and co (as does ctype), and that prior to calling, dlopen, you already have libz from /usr/lib. The ELF model is such as any library is shared through the whole binary, and I don't know how rpath interacts with this. David From d.l.goldsmith at gmail.com Wed Oct 21 12:52:14 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 21 Oct 2009 09:52:14 -0700 Subject: [SciPy-dev] Proposed generic docstring for attributes of class generic In-Reply-To: References: <45d1ab480910141538p10d1c812jcb76fed378035e46@mail.gmail.com> Message-ID: <45d1ab480910210952t42dbdb01r5a73497c39b97fc5@mail.gmail.com> On Tue, Oct 20, 2009 at 2:56 AM, Ralf Gommers wrote: > > On Thu, Oct 15, 2009 at 12:38 AM, David Goldsmith > wrote: > >> Short summary: Not implemented (virtual attribute) >> >> Extended summary: Class generic exists solely to derive numpy scalars >> from, and possesses, albeit unimplemented, all the attributes of the ndarray >> class so as to provide a uniform API. > > >> See Also: the corresponding attribute of the derived class of interest. >> >> Comments? >> > > I saw you put this in the wiki already. One thing that is wrong is the See > Also on each page, instead of the default text it needs the right link, i.e. > for numpy.generic.sum refer to ndarray.sum. > Umm, why, exactly? Generic is the base class for scalars and doesn't really have anything to do w/ ndarray. Other than that I have no idea. Please ask again during the sprint if you > don't get a better answer. > As you yourself have observed, I can ask 'til I'm blue in the face, but I don't think I'm going to get anyone who knows better than the people who've already read the docstring on scipy-dev. DG > > Cheers, > Ralf > > >> DG >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Wed Oct 21 13:05:29 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 21 Oct 2009 10:05:29 -0700 Subject: [SciPy-dev] Any chance I can get a head count... Message-ID: <45d1ab480910211005u12d838f1rf15d0243afc8b835@mail.gmail.com> ...of who all plans on participating in the documentation portion of this weekend's sprint? Also, as near as I can determine, the sprint never got announced on scipy-user nor on numpy-discussion - is there a reason, or just an oversight? DG -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Wed Oct 21 13:14:47 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Wed, 21 Oct 2009 19:14:47 +0200 Subject: [SciPy-dev] Proposed generic docstring for attributes of class generic In-Reply-To: <45d1ab480910210952t42dbdb01r5a73497c39b97fc5@mail.gmail.com> References: <45d1ab480910141538p10d1c812jcb76fed378035e46@mail.gmail.com> <45d1ab480910210952t42dbdb01r5a73497c39b97fc5@mail.gmail.com> Message-ID: On Wed, Oct 21, 2009 at 6:52 PM, David Goldsmith wrote: > On Tue, Oct 20, 2009 at 2:56 AM, Ralf Gommers > wrote: > >> >> On Thu, Oct 15, 2009 at 12:38 AM, David Goldsmith < >> d.l.goldsmith at gmail.com> wrote: >> >>> Short summary: Not implemented (virtual attribute) >>> >>> Extended summary: Class generic exists solely to derive numpy scalars >>> from, and possesses, albeit unimplemented, all the attributes of the ndarray >>> class so as to provide a uniform API. >> >> >>> See Also: the corresponding attribute of the derived class of interest. >>> >>> Comments? >>> >> >> I saw you put this in the wiki already. One thing that is wrong is the See >> Also on each page, instead of the default text it needs the right link, i.e. >> for numpy.generic.sum refer to ndarray.sum. >> > > Umm, why, exactly? Generic is the base class for scalars and doesn't > really have anything to do w/ ndarray. > Just saying that it makes no sense to have broken links in html docs. Look at it here: http://docs.scipy.org/numpy/docs/numpy.generic.sum/ See Also can be deleted or fixed, either one is fine with me. Cheers, Ralf > > Other than that I have no idea. Please ask again during the sprint if you >> don't get a better answer. >> > > As you yourself have observed, I can ask 'til I'm blue in the face, but I > don't think I'm going to get anyone who knows better than the people who've > already read the docstring on scipy-dev. > > DG > >> >> Cheers, >> Ralf >> >> >>> DG >>> >>> _______________________________________________ >>> Scipy-dev mailing list >>> Scipy-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>> >>> >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Wed Oct 21 13:29:21 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 21 Oct 2009 10:29:21 -0700 Subject: [SciPy-dev] Proposed generic docstring for attributes of class generic In-Reply-To: References: <45d1ab480910141538p10d1c812jcb76fed378035e46@mail.gmail.com> <45d1ab480910210952t42dbdb01r5a73497c39b97fc5@mail.gmail.com> Message-ID: <45d1ab480910211029ked58e48mc3975d9ccf7b81fd@mail.gmail.com> On Wed, Oct 21, 2009 at 10:14 AM, Ralf Gommers wrote: > > > On Wed, Oct 21, 2009 at 6:52 PM, David Goldsmith wrote: > >> On Tue, Oct 20, 2009 at 2:56 AM, Ralf Gommers < >> ralf.gommers at googlemail.com> wrote: >> >>> >>> On Thu, Oct 15, 2009 at 12:38 AM, David Goldsmith < >>> d.l.goldsmith at gmail.com> wrote: >>> >>>> Short summary: Not implemented (virtual attribute) >>>> >>>> Extended summary: Class generic exists solely to derive numpy scalars >>>> from, and possesses, albeit unimplemented, all the attributes of the ndarray >>>> class so as to provide a uniform API. >>> >>> >>>> See Also: the corresponding attribute of the derived class of interest. >>>> >>>> Comments? >>>> >>> >>> I saw you put this in the wiki already. One thing that is wrong is the >>> See Also on each page, instead of the default text it needs the right link, >>> i.e. for numpy.generic.sum refer to ndarray.sum. >>> >> >> Umm, why, exactly? Generic is the base class for scalars and doesn't >> really have anything to do w/ ndarray. >> > > Just saying that it makes no sense to have broken links in html docs. Look > at it here: > http://docs.scipy.org/numpy/docs/numpy.generic.sum/ > > See Also can be deleted or fixed, either one is fine with me. > Ah, gotchya. OK, not that it appears to be attracting much attention, but I'll make this a question - what to do about these particular broken links - on the Q & A page and see if anyone notices... DG > > Cheers, > Ralf > > >> >> Other than that I have no idea. Please ask again during the sprint if you >>> don't get a better answer. >>> >> >> As you yourself have observed, I can ask 'til I'm blue in the face, but I >> don't think I'm going to get anyone who knows better than the people who've >> already read the docstring on scipy-dev. >> >> DG >> >>> >>> Cheers, >>> Ralf >>> >>> >>>> DG >>>> >>>> _______________________________________________ >>>> Scipy-dev mailing list >>>> Scipy-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>>> >>>> >>> >>> _______________________________________________ >>> Scipy-dev mailing list >>> Scipy-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>> >>> >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Wed Oct 21 22:42:50 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 21 Oct 2009 19:42:50 -0700 Subject: [SciPy-dev] Fwd: scipy.test() failures In-Reply-To: <41412F55-9289-44CE-8005-6F0EC91907F1@unimelb.edu.au> References: <8C5415C2-66B9-40C8-AFBD-47C93BD6EB91@unimelb.edu.au> <41412F55-9289-44CE-8005-6F0EC91907F1@unimelb.edu.au> Message-ID: <45d1ab480910211942q29c4e498n9a1e4d72a80d596d@mail.gmail.com> Hi! I had to admit to this guy that I don't know specifically who our list admin is; can someone please help him (I already instructed him to login and make sure none of his settings might be blocking delivery). Thanks! DG ---------- Forwarded message ---------- From: Neil Killeen Date: Wed, Oct 21, 2009 at 4:53 PM Subject: Fwd: scipy.test() failures To: d.l.goldsmith at gmail.com Cc: Neil Killeen Dear David sorry to write to you out of the list, but perhaps you can help ! I signed up to the dev list, but I can't post; my messages do not turn up and I don't get a bounce or a copy. I have written twice to the administrator (below) but no answer. Do you happen to know who the admin is (they could be away of course) or suggest a course of action ? You might even know the solution to my problem (below) thanks Neil Killeen p.s. I only picked you as you are the last poster ! Begin forwarded message: *From: *Neil Killeen *Date: *21 October 2009 10:12:14 AM *To: *Neil Killeen *Cc: *scipy-dev-owner at scipy.org *Subject: **Re: scipy.test() failures* Hi I tried again without attachments but still no success. Can you please resolve why I can't post to the list ? thanks Neil On 19/10/2009, at 9:59 , Neil Killeen wrote: Hi I am a new member of the list. I posted this last week but it has not turned up on the list It must have been rejected for some reason. can you have a look ? thanks Neil Begin forwarded message: *From: *Neil Killeen *Date: *15 October 2009 5:49:53 PM *To: *scipy-dev at scipy.org *Cc: *Neil Killeen *Subject: **scipy.test() failures* Hi - I am operating on an SGI Altix 3700 BX2 Itaniam IA64 system running SUSE/Novelle Linux SLES10 - I am trying to install scipy V 0.7.1 - I am using python 2.4 - blas and lapack are installed via system rpms in /usr/lib soma:/usr/lib # ls -al *blas* *lapack* -rw-r--r-- 1 root root 929506 2007-04-14 02:16 libblas.a lrwxrwxrwx 1 root root 12 2008-03-12 12:20 libblas.so -> libblas.so.3 lrwxrwxrwx 1 root root 16 2008-03-12 12:20 libblas.so.3 -> libblas.so.3.0.0 -rwxr-xr-x 1 root root 678024 2007-04-14 02:16 libblas.so.3.0.0 lrwxrwxrwx 1 root root 20 2008-12-02 22:52 libgslcblas.so.0 -> libgslcblas.so.0.0.0 -rwxr-xr-x 1 root root 908816 2007-04-14 02:30 libgslcblas.so.0.0.0 -rw-r--r-- 1 root root 11208466 2007-04-14 02:16 liblapack.a lrwxrwxrwx 1 root root 14 2008-03-12 12:20 liblapack.so -> liblapack.so.3 lrwxrwxrwx 1 root root 18 2008-03-12 12:20 liblapack.so.3 -> liblapack.so.3.0.0 -rwxr-xr-x 1 root root 7503568 2007-04-14 02:16 liblapack.so.3.0.0 1. I did a vanilla build (python setup.py build/install) which appeared to work ok. 2. When I tried to run scipy.test() I saw errors (see attachment 1). They are in two categories: ERROR: Failure: ImportError (/usr/local/lib/python2.4/site-packages/scipy/linalg/fblas.so: undefined symbol: srotmg_) and ImportError: cannot import name {linalg, calc_lwork} Perhaps the latter are due to the former ? 3. I read in the FAQ that the solution to the srotmg error is to install blas myself either from netlib or via ATLAS 4. I decided to do the easy thing and installed blas from netlib in /usr/local/lib/libblas.a 5. I then deleted the scipy installation (/usr/local/lib/python2.4/site-packges/scipy) and rebuilt and installed afresh. YOu can see in attachment 2 (the build log) that it appears to have found the /usr/local/lib/libblas.a library E.g. blas_info: FOUND: libraries = ['blas'] library_dirs = ['/usr/local/lib'] language = f77 FOUND: libraries = ['blas'] library_dirs = ['/usr/local/lib'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77 6. Then I - set env variable BLAS=/usr/local/lib/libblas.a (not sure useful since static library so should be no run-time dependency) - ran the test again with the same result as the initial. Can someone advise what more I must do to eradicate these run-time errors ? thanks Neil -------------- next part -------------- An HTML attachment was scrubbed... URL: From angel.yanguas at gmail.com Wed Oct 21 23:05:13 2009 From: angel.yanguas at gmail.com (Angel Yanguas-Gil) Date: Wed, 21 Oct 2009 22:05:13 -0500 Subject: [SciPy-dev] consistency for documenting mathematical functions in numpy.core.umath Message-ID: <11f4deca0910212005r7b053e70i66b4e7d40ade5ee@mail.gmail.com> Hi! I have noticed that the structure of the docstrings (sections, etc.) of the mathematical functions defined in numpy.core.umath change from function to function. Maybe it is an old topic but, would it be worth to unify them - i.e., using the same reference sources, same section structure, consistent cross-reference between methods etc. ? Angel From charlesr.harris at gmail.com Wed Oct 21 23:24:36 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 21 Oct 2009 21:24:36 -0600 Subject: [SciPy-dev] Fwd: scipy.test() failures In-Reply-To: <45d1ab480910211942q29c4e498n9a1e4d72a80d596d@mail.gmail.com> References: <8C5415C2-66B9-40C8-AFBD-47C93BD6EB91@unimelb.edu.au> <41412F55-9289-44CE-8005-6F0EC91907F1@unimelb.edu.au> <45d1ab480910211942q29c4e498n9a1e4d72a80d596d@mail.gmail.com> Message-ID: On Wed, Oct 21, 2009 at 8:42 PM, David Goldsmith wrote: > Hi! I had to admit to this guy that I don't know specifically who our list > admin is; can someone please help him (I already instructed him to login and > make sure none of his settings might be blocking delivery). Thanks! > > It might be an attachment problem if the attachments excede 40K. In that case a note will be sent off list to the sender and might get missed or filtered out as spam. A simple "testing" post test if that is the case. As to the list admin... Hey, I don't get mail on svn commits and the only ticket notifications I see are for those I post to. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Wed Oct 21 23:33:18 2009 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 21 Oct 2009 22:33:18 -0500 Subject: [SciPy-dev] consistency for documenting mathematical functions in numpy.core.umath In-Reply-To: <11f4deca0910212005r7b053e70i66b4e7d40ade5ee@mail.gmail.com> References: <11f4deca0910212005r7b053e70i66b4e7d40ade5ee@mail.gmail.com> Message-ID: <3d375d730910212033k46703c90p7e23c6391811dbdc@mail.gmail.com> On Wed, Oct 21, 2009 at 22:05, Angel Yanguas-Gil wrote: > Hi! > > I have noticed that the structure of the docstrings (sections, etc.) > of the mathematical functions defined in numpy.core.umath change from > function to function. Maybe it is an old topic but, would it be worth > to unify them - i.e., using the same reference sources, same section > structure, consistent cross-reference between methods etc. ? The standard is here: http://projects.scipy.org/numpy/browser/trunk/doc/HOWTO_DOCUMENT.txt You can help fix nonconforming docstrings here: http://docs.scipy.org/numpy/Front%20Page/ You may be looking at older versions. There has been substantial effort put into improving this aspect of the docstrings recently. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From josef.pktd at gmail.com Wed Oct 21 23:55:21 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 21 Oct 2009 23:55:21 -0400 Subject: [SciPy-dev] scipy doc milestones duplicates and triplicates Message-ID: <1cd32cbb0910212055y52d99d2ev8f0a9c3e570b5400@mail.gmail.com> I looked at http://docs.scipy.org/scipy/Milestones/Milestones_11/ and the list is huge, however, many functions are in there either as duplicates (those imported into scipy.stats.*) or as triplicates (distributions). Are functions supposed to be listed in the module, e.g. scipy.stats.stats, or the package name, scipy.stats? Individual distributions are listed as class, and twice as instances # scipy.stats.distributions.nbinom # scipy.stats.distributions.nbinom_gen # scipy.stats.nbinom None of the docstrings of the individual distributions can actually be edited since they are autogenerated (except for the extradoc). Any idea what the milestones should look like in this case, or is it too early to ask? Josef From d.l.goldsmith at gmail.com Thu Oct 22 01:03:28 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 21 Oct 2009 22:03:28 -0700 Subject: [SciPy-dev] consistency for documenting mathematical functions in numpy.core.umath In-Reply-To: <3d375d730910212033k46703c90p7e23c6391811dbdc@mail.gmail.com> References: <11f4deca0910212005r7b053e70i66b4e7d40ade5ee@mail.gmail.com> <3d375d730910212033k46703c90p7e23c6391811dbdc@mail.gmail.com> Message-ID: <45d1ab480910212203y4a51ead9ib2b81583852637c0@mail.gmail.com> On Wed, Oct 21, 2009 at 8:33 PM, Robert Kern wrote: > On Wed, Oct 21, 2009 at 22:05, Angel Yanguas-Gil > wrote: > > Hi! > > > > I have noticed that the structure of the docstrings (sections, etc.) > > of the mathematical functions defined in numpy.core.umath change from > > function to function. Maybe it is an old topic but, would it be worth > > to unify them - i.e., using the same reference sources, same section > > structure, consistent cross-reference between methods etc. ? > > The standard is here: > > http://projects.scipy.org/numpy/browser/trunk/doc/HOWTO_DOCUMENT.txt > > You can help fix nonconforming docstrings here: > > http://docs.scipy.org/numpy/Front%20Page/ > > You may be looking at older versions. There has been substantial > effort put into improving this aspect of the docstrings recently. > > -- > Robert Kern > In particular, the docstrings you generally refer to *should* all have been modified to conform to the standard Robert refers to above (note, however, that many of the sections are optional: if a section is irrelevant for a particular object, e.g., Returns often is, or is empty, e.g., References often is, then we just omit it altogether). However, generally, we make such mods using our special wiki app (Robert's second link above), not in the code itself; changes made in the wiki are not *automatically* merged into the code; and we have a large backlog of docstring changes (as well as code changes) for which merge is pending. If you can be more specific as to which docstrings do not conform to the above standard, we (or you) can check to see if they are at least compliant in the wiki, in which case they will be merged, hopefully this weekend, and if they aren't, well then, we have some substantive work to do. DG -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Thu Oct 22 01:10:07 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Wed, 21 Oct 2009 22:10:07 -0700 Subject: [SciPy-dev] scipy doc milestones duplicates and triplicates In-Reply-To: <1cd32cbb0910212055y52d99d2ev8f0a9c3e570b5400@mail.gmail.com> References: <1cd32cbb0910212055y52d99d2ev8f0a9c3e570b5400@mail.gmail.com> Message-ID: <45d1ab480910212210jdb8c8co4cfdfca266ab1156@mail.gmail.com> On Wed, Oct 21, 2009 at 8:55 PM, wrote: > I looked at http://docs.scipy.org/scipy/Milestones/Milestones_11/ > > and the list is huge, however, many functions are in there either as > duplicates (those imported into scipy.stats.*) or as triplicates > (distributions). > > Are functions supposed to be listed in the module, e.g. > scipy.stats.stats, or the package name, scipy.stats? > > Individual distributions are listed as class, and twice as instances > > # scipy.stats.distributions.nbinom > # scipy.stats.distributions.nbinom_gen > # scipy.stats.nbinom > > None of the docstrings of the individual distributions can actually be > edited since they are autogenerated (except for the extradoc). > > Any idea what the milestones should look like in this case, or is it > too early to ask? > It's definitely not too early to ask, and as a "matter of policy," we don't want the same things needing to be documented in more than one place due to the risk of divergence down the line, so yours is a good, ripe question, requiring a solution. The numpy side of the wiki has a Q&A page that was set up to facilitate collaborative answering of these kinds of questions, however, I don't think it's well-trafficked, and this is important enough that I think we should just go ahead and discuss it here. So, what do people think about this? DG > > Josef > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Thu Oct 22 03:31:56 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Thu, 22 Oct 2009 09:31:56 +0200 Subject: [SciPy-dev] consistency for documenting mathematical functions in numpy.core.umath In-Reply-To: <45d1ab480910212203y4a51ead9ib2b81583852637c0@mail.gmail.com> References: <11f4deca0910212005r7b053e70i66b4e7d40ade5ee@mail.gmail.com> <3d375d730910212033k46703c90p7e23c6391811dbdc@mail.gmail.com> <45d1ab480910212203y4a51ead9ib2b81583852637c0@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 7:03 AM, David Goldsmith wrote: > changes made in the wiki are not *automatically* merged into the code; and > we have a large backlog of docstring changes (as well as code changes) for > which merge is pending. > Not any more. Thanks to Pauli and Jarrod pretty much all of it is merged now. Cheers, Ralf > > DG > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Thu Oct 22 04:13:45 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Thu, 22 Oct 2009 01:13:45 -0700 Subject: [SciPy-dev] consistency for documenting mathematical functions in numpy.core.umath In-Reply-To: References: <11f4deca0910212005r7b053e70i66b4e7d40ade5ee@mail.gmail.com> <3d375d730910212033k46703c90p7e23c6391811dbdc@mail.gmail.com> <45d1ab480910212203y4a51ead9ib2b81583852637c0@mail.gmail.com> Message-ID: <45d1ab480910220113w2c2e5e2cyfe7abfa184e408be@mail.gmail.com> Thanks Pauli and Jarrod! Of course that means one would have to check-out the bleeding-edge svn source to see those merges in the code, correct? DG On Thu, Oct 22, 2009 at 12:31 AM, Ralf Gommers wrote: > > > On Thu, Oct 22, 2009 at 7:03 AM, David Goldsmith wrote: > >> changes made in the wiki are not *automatically* merged into the code; and >> we have a large backlog of docstring changes (as well as code changes) for >> which merge is pending. >> > > Not any more. Thanks to Pauli and Jarrod pretty much all of it is merged > now. > > Cheers, > Ralf > > >> >> DG >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Thu Oct 22 04:16:28 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Thu, 22 Oct 2009 10:16:28 +0200 Subject: [SciPy-dev] scipy doc milestones duplicates and triplicates In-Reply-To: <1cd32cbb0910212055y52d99d2ev8f0a9c3e570b5400@mail.gmail.com> References: <1cd32cbb0910212055y52d99d2ev8f0a9c3e570b5400@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 5:55 AM, wrote: > I looked at http://docs.scipy.org/scipy/Milestones/Milestones_11/ > > and the list is huge, however, many functions are in there either as > duplicates (those imported into scipy.stats.*) or as triplicates > (distributions). > > Are functions supposed to be listed in the module, e.g. > scipy.stats.stats, or the package name, scipy.stats? > Can you give an example of duplicates here? I see functions like cov and itemfreq only listed once. The pages are under scipy.stats.stats but the ones that are imported to scipy.stats should be listed only there. That's how the NumPy Milestones work anyway. > > Individual distributions are listed as class, and twice as instances > > # scipy.stats.distributions.nbinom > # scipy.stats.distributions.nbinom_gen > # scipy.stats.nbinom > > This looks like a bug in pydocweb to me, since >>> sp.stats.nbinom is sp.stats.distributions.nbinom True but there are actually separate pages: http://docs.scipy.org/scipy/docs/scipy.stats.distributions.nbinom/ http://docs.scipy.org/scipy/docs/scipy.stats.nbinom/ None of the docstrings of the individual distributions can actually be > edited since they are autogenerated (except for the extradoc). > They can be edited in the wiki, but you're right that there will be a problem when trying to merge wiki edits back into trunk. There's a similar problem in ndimage, where the wiki collides with docstring manipulation in the code. No solution for that yet, but it should be fixed somehow. Cheers, Ralf > Any idea what the milestones should look like in this case, or is it > too early to ask? > > Josef > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Thu Oct 22 04:17:54 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Thu, 22 Oct 2009 10:17:54 +0200 Subject: [SciPy-dev] consistency for documenting mathematical functions in numpy.core.umath In-Reply-To: <45d1ab480910220113w2c2e5e2cyfe7abfa184e408be@mail.gmail.com> References: <11f4deca0910212005r7b053e70i66b4e7d40ade5ee@mail.gmail.com> <3d375d730910212033k46703c90p7e23c6391811dbdc@mail.gmail.com> <45d1ab480910212203y4a51ead9ib2b81583852637c0@mail.gmail.com> <45d1ab480910220113w2c2e5e2cyfe7abfa184e408be@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 10:13 AM, David Goldsmith wrote: > Thanks Pauli and Jarrod! Of course that means one would have to check-out > the bleeding-edge svn source to see those merges in the code, correct? > > Correct. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan at sun.ac.za Thu Oct 22 05:28:49 2009 From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=) Date: Thu, 22 Oct 2009 11:28:49 +0200 Subject: [SciPy-dev] ANN: SciPy October Sprint In-Reply-To: <9457e7c80910070446k2c1ae895u4011d242abc224b@mail.gmail.com> References: <9457e7c80910070446k2c1ae895u4011d242abc224b@mail.gmail.com> Message-ID: <9457e7c80910220228v15638e9awc4b8096b8d7e960e@mail.gmail.com> Hi all, The weekend is just around the corner, and we're looking forward to the sprint! Here is the detail again: """ Our patch queue keeps getting longer and longer, so here is an opportunity to do some spring cleaning (it's spring in South Africa, at least)! Please join us for an October SciPy sprint: ? ?* Date: 24/25 October 2009 (Sat/Sun) ? ?* More information: http://projects.scipy.org/scipy/wiki/SciPySprint200910 We are looking for volunteers to write documentation, review code, fix bugs or design marketing material. New contributors are most welcome, and mentoring will be available. """ See you there, Regards St?fan From nkilleen at unimelb.edu.au Thu Oct 22 05:58:14 2009 From: nkilleen at unimelb.edu.au (Neil Killeen) Date: Thu, 22 Oct 2009 20:58:14 +1100 Subject: [SciPy-dev] Test post In-Reply-To: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> Message-ID: <12A95E78-35B2-41DC-A597-201A96B1C625@unimelb.edu.au> Test post From nkilleen at unimelb.edu.au Thu Oct 22 18:23:57 2009 From: nkilleen at unimelb.edu.au (Neil Killeen) Date: Fri, 23 Oct 2009 09:23:57 +1100 Subject: [SciPy-dev] scipy.test() failures References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> Message-ID: Hi all Hi - I am operating on an SGI Altix 3700 BX2 Itanium IA64 system running SUSE/Novelle Linux SLES10 - I am trying to install scipy V 0.7.1 - I am using python 2.4 - blas and lapack are installed via system rpms in /usr/lib soma:/usr/lib # ls -al *blas* *lapack* -rw-r--r-- 1 root root 929506 2007-04-14 02:16 libblas.a lrwxrwxrwx 1 root root 12 2008-03-12 12:20 libblas.so -> libblas.so.3 lrwxrwxrwx 1 root root 16 2008-03-12 12:20 libblas.so.3 -> libblas.so.3.0.0 -rwxr-xr-x 1 root root 678024 2007-04-14 02:16 libblas.so.3.0.0 lrwxrwxrwx 1 root root 20 2008-12-02 22:52 libgslcblas.so.0 -> libgslcblas.so.0.0.0 -rwxr-xr-x 1 root root 908816 2007-04-14 02:30 libgslcblas.so.0.0.0 -rw-r--r-- 1 root root 11208466 2007-04-14 02:16 liblapack.a lrwxrwxrwx 1 root root 14 2008-03-12 12:20 liblapack.so -> liblapack.so.3 lrwxrwxrwx 1 root root 18 2008-03-12 12:20 liblapack.so.3 -> liblapack.so.3.0.0 -rwxr-xr-x 1 root root 7503568 2007-04-14 02:16 liblapack.so.3.0.0 1. I did a vanilla build (python setup.py build/install) which appeared to work ok. 2. When I tried to run scipy.test() I saw errors. They are in two categories: ERROR: Failure: ImportError (/usr/local/lib/python2.4/site-packages/ scipy/linalg/fblas.so: undefined symbol: srotmg_) and ImportError: cannot import name {linalg, calc_lwork} Perhaps the latter are due to the former ? 3. I read in the FAQ that the solution to the srotmg error is to install blas myself either from netlib or via ATLAS 4. I decided to do the easy thing and installed blas from netlib in / usr/local/lib/libblas.a 5. I then deleted the scipy installation (/usr/local/lib/python2.4/ site-packges/scipy) and rebuilt and installed afresh. In the build log one can see that it appears to have found the /usr/ local/lib/libblas.a library E.g. blas_info: FOUND: libraries = ['blas'] library_dirs = ['/usr/local/lib'] language = f77 FOUND: libraries = ['blas'] library_dirs = ['/usr/local/lib'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77 6. Then I - set env variable BLAS=/usr/local/lib/libblas.a (not sure useful since static library so should be no run-time dependency) - ran the test again with the same errors resulting as the initial. Can someone advise what more I must do to eradicate these run-time errors ? thanks Neil p.s. tried to post this last week but message did not arrive (perhaps as I included attachments which I have left off this time) From matthew.brett at gmail.com Fri Oct 23 18:16:10 2009 From: matthew.brett at gmail.com (Matthew Brett) Date: Fri, 23 Oct 2009 15:16:10 -0700 Subject: [SciPy-dev] git cloning? Message-ID: <1e2af89e0910231516s1f6f0f6cm4f344345e0d44d3e@mail.gmail.com> Hi, I was just trying the new-fangled git mirror, and: git clone --origin svn http://projects.scipy.org/git/scipy.git scipy -> Initialized empty Git repository in /home/mb312/dev_trees/scipy/.git/ /usr/bin/git-clone: 374: curl: not found Sure enough: http://projects.scipy.org/git/scipy.git/ in my browser, gives a 403: Forbidden - You don't have permission to access /git/scipy.git/ on this server. Have I missed a step somewhere? Best, Matthew From arkapravobhaumik at gmail.com Fri Oct 23 20:10:15 2009 From: arkapravobhaumik at gmail.com (Arkapravo Bhaumik) Date: Sat, 24 Oct 2009 01:10:15 +0100 Subject: [SciPy-dev] Scipy-dev Digest, Vol 72, Issue 51 In-Reply-To: References: Message-ID: Dear Friends Is there any facility or module in scipy / numpy for (1) conformal mapping (2) chaotic systems. Regards Arkapravo 2009/10/23 > Send Scipy-dev mailing list submissions to > scipy-dev at scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request at scipy.org > > You can reach the person managing the list at > scipy-dev-owner at scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Scipy-dev digest..." > > > Today's Topics: > > 1. scipy.test() failures (Neil Killeen) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 23 Oct 2009 09:23:57 +1100 > From: Neil Killeen > Subject: [SciPy-dev] scipy.test() failures > To: scipy-dev at scipy.org > Message-ID: > Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes > > Hi all > > Hi > > - I am operating on an SGI Altix 3700 BX2 Itanium IA64 system running > SUSE/Novelle Linux SLES10 > - I am trying to install scipy V 0.7.1 > - I am using python 2.4 > - blas and lapack are installed via system rpms in /usr/lib > > > soma:/usr/lib # ls -al *blas* *lapack* > -rw-r--r-- 1 root root 929506 2007-04-14 02:16 libblas.a > lrwxrwxrwx 1 root root 12 2008-03-12 12:20 libblas.so -> > libblas.so.3 > lrwxrwxrwx 1 root root 16 2008-03-12 12:20 libblas.so.3 -> > libblas.so.3.0.0 > -rwxr-xr-x 1 root root 678024 2007-04-14 02:16 libblas.so.3.0.0 > lrwxrwxrwx 1 root root 20 2008-12-02 22:52 libgslcblas.so.0 -> > libgslcblas.so.0.0.0 > -rwxr-xr-x 1 root root 908816 2007-04-14 02:30 libgslcblas.so.0.0.0 > -rw-r--r-- 1 root root 11208466 2007-04-14 02:16 liblapack.a > lrwxrwxrwx 1 root root 14 2008-03-12 12:20 liblapack.so -> > liblapack.so.3 > lrwxrwxrwx 1 root root 18 2008-03-12 12:20 liblapack.so.3 -> > liblapack.so.3.0.0 > -rwxr-xr-x 1 root root 7503568 2007-04-14 02:16 liblapack.so.3.0.0 > > > > > 1. I did a vanilla build (python setup.py build/install) which > appeared to work ok. > > 2. When I tried to run scipy.test() I saw errors. They are in two > categories: > > ERROR: Failure: ImportError (/usr/local/lib/python2.4/site-packages/ > scipy/linalg/fblas.so: undefined symbol: srotmg_) > > and > > ImportError: cannot import name {linalg, calc_lwork} > > Perhaps the latter are due to the former ? > > > 3. I read in the FAQ that the solution to the srotmg error is to > install blas myself either from netlib or via ATLAS > > 4. I decided to do the easy thing and installed blas from netlib in / > usr/local/lib/libblas.a > > 5. I then deleted the scipy installation (/usr/local/lib/python2.4/ > site-packges/scipy) and rebuilt and installed afresh. > > In the build log one can see that it appears to have found the /usr/ > local/lib/libblas.a library > > E.g. > > blas_info: > FOUND: > libraries = ['blas'] > library_dirs = ['/usr/local/lib'] > language = f77 > > FOUND: > libraries = ['blas'] > library_dirs = ['/usr/local/lib'] > define_macros = [('NO_ATLAS_INFO', 1)] > language = f77 > > > > > 6. Then I > - set env variable BLAS=/usr/local/lib/libblas.a (not sure useful > since static library so should be no run-time dependency) > - ran the test again with the same errors resulting as the initial. > > > Can someone advise what more I must do to eradicate these run-time > errors ? > > > thanks > Neil > > p.s. tried to post this last week but message did not arrive (perhaps > as I included attachments > which I have left off this time) > > > ------------------------------ > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > End of Scipy-dev Digest, Vol 72, Issue 51 > ***************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Fri Oct 23 20:18:30 2009 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 23 Oct 2009 19:18:30 -0500 Subject: [SciPy-dev] Scipy-dev Digest, Vol 72, Issue 51 In-Reply-To: References: Message-ID: <3d375d730910231718v4ff2554bxd967bd88a1b7efc3@mail.gmail.com> Please do not reply to Digest messages. If you wish to start a new thread, please just send mail to scipy-dev at scipy.org with an appropriate Subject line. Please subscribe normally to the list if you wish to have discussions and reply to messages. Only use the Digest if you just want to read the list and not participate in the conversation. Thank you. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From cournape at gmail.com Fri Oct 23 23:26:32 2009 From: cournape at gmail.com (David Cournapeau) Date: Sat, 24 Oct 2009 12:26:32 +0900 Subject: [SciPy-dev] git cloning? In-Reply-To: <1e2af89e0910231516s1f6f0f6cm4f344345e0d44d3e@mail.gmail.com> References: <1e2af89e0910231516s1f6f0f6cm4f344345e0d44d3e@mail.gmail.com> Message-ID: <5b8d13220910232026u2a00fd9ai794fcccde89007cd@mail.gmail.com> Hi Matthew, On Sat, Oct 24, 2009 at 7:16 AM, Matthew Brett wrote: > Hi, > > I was just trying the new-fangled git mirror, and: > > git clone --origin svn http://projects.scipy.org/git/scipy.git scipy > > -> > > Initialized empty Git repository in /home/mb312/dev_trees/scipy/.git/ > /usr/bin/git-clone: 374: curl: not found It looks like you have an old git version. Curl is a command line to fetch file from http protocols (and other). I strongly advise you to use git 1.6.0 or later. On Linux, installing git from sources is not too difficult (IIRC, you need to install libcurl and openssl dev packages to have a full git). Building the doc is a PITA, but you can find manpages and html doc on the git webpage (manpages are necessary for git help subcommand to work). David From stefan at sun.ac.za Sat Oct 24 12:33:51 2009 From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=) Date: Sat, 24 Oct 2009 18:33:51 +0200 Subject: [SciPy-dev] CeCILL license Message-ID: <9457e7c80910240933h7c2c2f6frcef06c469f5b8f9f@mail.gmail.com> Hi all, As far as I know, only the CeCILL-B license is similar to our modified BSD. Does that mean that we can't use code from Scientific Python, for example? I ask in relation to http://projects.scipy.org/scipy/ticket/354#comment:7 Regards St?fan From gael.varoquaux at normalesup.org Sat Oct 24 12:55:29 2009 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sat, 24 Oct 2009 18:55:29 +0200 Subject: [SciPy-dev] CeCILL license In-Reply-To: <9457e7c80910240933h7c2c2f6frcef06c469f5b8f9f@mail.gmail.com> References: <9457e7c80910240933h7c2c2f6frcef06c469f5b8f9f@mail.gmail.com> Message-ID: <20091024165529.GA19508@phare.normalesup.org> On Sat, Oct 24, 2009 at 06:33:51PM +0200, St?fan van der Walt wrote: > Hi all, > As far as I know, only the CeCILL-B license is similar to our modified > BSD. Does that mean that we can't use code from Scientific Python, > for example? I am not sure. The CeCILL licenses are GPL and BSD equivalents adapted to French law (but I am not sure which one is which). Why don't you simply ask Konrad, he is a very nice man. Ga?l From bpederse at gmail.com Sat Oct 24 16:14:10 2009 From: bpederse at gmail.com (Brent Pedersen) Date: Sat, 24 Oct 2009 13:14:10 -0700 Subject: [SciPy-dev] edit rights for docs Message-ID: hi, i'm with the berkeley group doing the doc sprint following the instructions for editing docs here: http://docs.scipy.org/numpy/Front%20Page/ could i get permissions to edit? username: brentp thanks. From robert.kern at gmail.com Sat Oct 24 17:30:03 2009 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 24 Oct 2009 16:30:03 -0500 Subject: [SciPy-dev] CeCILL license In-Reply-To: <9457e7c80910240933h7c2c2f6frcef06c469f5b8f9f@mail.gmail.com> References: <9457e7c80910240933h7c2c2f6frcef06c469f5b8f9f@mail.gmail.com> Message-ID: <3d375d730910241430gc3452e5y256aeab452702f27@mail.gmail.com> 2009/10/24 St?fan van der Walt : > Hi all, > > As far as I know, only the CeCILL-B license is similar to our modified > BSD. ?Does that mean that we can't use code from Scientific Python, > for example? None of the CeCILL licenses are suitable for code in scipy, in my opinion. The CeCILL-B is closest, but it has attribution requirements that are stronger in some ways than the GPL-incompatible advertising clause in the original BSD license. I'm pretty sure it is not GPL-compatible, and I doubt it will be considered DFSG-free or OSI approved. The CeCILL license (~GPL) and the CeCILL-C license (~LGPL) are not suitable for obvious reasons. > I ask in relation to > > http://projects.scipy.org/scipy/ticket/354#comment:7 While it is worthwhile for the author of the patch to ask Konrad, I'm pretty sure that code existed in ScientificPython before Konrad switched from the BSD license to the CeCILL license and could be forked from a previous version of ScientificPython. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From chlewis at berkeley.edu Sat Oct 24 17:51:24 2009 From: chlewis at berkeley.edu (Chloe Lewis) Date: Sat, 24 Oct 2009 14:51:24 -0700 Subject: [SciPy-dev] edit rights on the docs wiki, please? Message-ID: username chlewis; will join the docs sprint thanks Chloe Lewis Graduate student, Amundson Lab Ecosystem Sciences 137 Mulford Hall Berkeley, CA 94720-3114 http://nature.berkeley.edu/~chlewis From gael.varoquaux at normalesup.org Sat Oct 24 18:30:08 2009 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 25 Oct 2009 00:30:08 +0200 Subject: [SciPy-dev] edit rights for docs In-Reply-To: References: Message-ID: <20091024223008.GA22427@phare.normalesup.org> On Sat, Oct 24, 2009 at 01:14:10PM -0700, Brent Pedersen wrote: > hi, i'm with the berkeley group doing the doc sprint following the > instructions for editing docs here: > http://docs.scipy.org/numpy/Front%20Page/ > could i get permissions to edit? > username: brentp I added you. Cheers, Ga?l From gael.varoquaux at normalesup.org Sat Oct 24 18:39:54 2009 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 25 Oct 2009 00:39:54 +0200 Subject: [SciPy-dev] CeCILL license In-Reply-To: <3d375d730910241430gc3452e5y256aeab452702f27@mail.gmail.com> References: <9457e7c80910240933h7c2c2f6frcef06c469f5b8f9f@mail.gmail.com> <3d375d730910241430gc3452e5y256aeab452702f27@mail.gmail.com> Message-ID: <20091024223954.GB22427@phare.normalesup.org> On Sat, Oct 24, 2009 at 04:30:03PM -0500, Robert Kern wrote: > None of the CeCILL licenses are suitable for code in scipy, in my > opinion. The CeCILL-B is closest, but it has attribution requirements > that are stronger in some ways than the GPL-incompatible advertising > clause in the original BSD license. I'm pretty sure it is not > GPL-compatible, and I doubt it will be considered DFSG-free or OSI > approved. I do believe it is DFSG-free. If not, I would love to know about an example. According to the wikipedia page, CeCILL is GPL compatible, and CeCILL-B is compatible with BSD, BIT and X11. However, the CeCILL webpage does look suspicious. If you have any had facts, or previous legal or informal cases involving the interpretation of the attribution closes, I am interested, as I can try loby the legal office of my two employers, which are 2 of the 3 CeCILL founder to make changes for the next version. > While it is worthwhile for the author of the patch to ask Konrad, I'm > pretty sure that code existed in ScientificPython before Konrad > switched from the BSD license to the CeCILL license and could be > forked from a previous version of ScientificPython. +1 on that. I doubt that Konrad would mind relicensing a bit of code. Cheers, Ga?l From gael.varoquaux at normalesup.org Sat Oct 24 18:40:39 2009 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 25 Oct 2009 00:40:39 +0200 Subject: [SciPy-dev] edit rights on the docs wiki, please? In-Reply-To: References: Message-ID: <20091024224039.GC22427@phare.normalesup.org> On Sat, Oct 24, 2009 at 02:51:24PM -0700, Chloe Lewis wrote: > username chlewis; will join the docs sprint I have made you editors. Ga?l From robert.kern at gmail.com Sat Oct 24 19:06:18 2009 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 24 Oct 2009 18:06:18 -0500 Subject: [SciPy-dev] CeCILL license In-Reply-To: <20091024223954.GB22427@phare.normalesup.org> References: <9457e7c80910240933h7c2c2f6frcef06c469f5b8f9f@mail.gmail.com> <3d375d730910241430gc3452e5y256aeab452702f27@mail.gmail.com> <20091024223954.GB22427@phare.normalesup.org> Message-ID: <3d375d730910241606w4a25caacib7322845a03ddc18@mail.gmail.com> On Sat, Oct 24, 2009 at 17:39, Gael Varoquaux wrote: > On Sat, Oct 24, 2009 at 04:30:03PM -0500, Robert Kern wrote: >> None of the CeCILL licenses are suitable for code in scipy, in my >> opinion. The CeCILL-B is closest, but it has attribution requirements >> that are stronger in some ways than the GPL-incompatible advertising >> clause in the original BSD license. I'm pretty sure it is not >> GPL-compatible, and I doubt it will be considered DFSG-free or OSI >> approved. > > I do believe it is DFSG-free. If not, I would love to know about an > example. According to the wikipedia page, CeCILL is GPL compatible, and > CeCILL-B is compatible with BSD, BIT and X11. However, the CeCILL webpage > does look suspicious. If you have any had facts, or previous legal or > informal cases involving the interpretation of the attribution closes, I > am interested, as I can try loby the legal office of my two employers, > which are 2 of the 3 CeCILL founder to make changes for the next version. Section "5.3.4 CREDITS" contains this clause: "3. mention, on a freely accessible website describing the Modified Software, at least throughout the distribution term thereof, that it is based on the Software licensed hereunder, and reproduce the Software intellectual property notice," I believe this fails the Dissident Test. http://wiki.debian.org/DissidentTest Also, this section's interactions with "External Modules", which are defined as "any or all Modules, not derived from the Software, so that this Module and the Software run in separate address spaces, with one calling the other when they are run", probably runs afoul of the "License Must Not Contaminate Other Software" provision: "If the Software, whether or not modified, is distributed with an External Module designed for use in connection with the Software, the Licensee shall submit said External Module to the foregoing obligations." While the CeCILL-B license claims to be *similar* to the BSD license with respect to its obligations about redistributing source code, it can hardly be said to be "compatible" with it in the way that the CeCILL license is compatible with the GPL. It has far too many terms above and beyond the BSD license. I would encourage you to encourage the CeCILL powers-that-be to submit their licenses for OSI approval and through debian-legal to get their licenses declared DFSG-free. Those processes will shake out a whole ton(ne) of issues. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From gael.varoquaux at normalesup.org Sun Oct 25 03:59:26 2009 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 25 Oct 2009 08:59:26 +0100 Subject: [SciPy-dev] CeCILL license In-Reply-To: <3d375d730910241606w4a25caacib7322845a03ddc18@mail.gmail.com> References: <9457e7c80910240933h7c2c2f6frcef06c469f5b8f9f@mail.gmail.com> <3d375d730910241430gc3452e5y256aeab452702f27@mail.gmail.com> <20091024223954.GB22427@phare.normalesup.org> <3d375d730910241606w4a25caacib7322845a03ddc18@mail.gmail.com> Message-ID: <20091025075926.GA29867@phare.normalesup.org> On Sat, Oct 24, 2009 at 06:06:18PM -0500, Robert Kern wrote: > I would encourage you to encourage the CeCILL powers-that-be to submit > their licenses for OSI approval and through debian-legal to get their > licenses declared DFSG-free. Those processes will shake out a whole > ton(ne) of issues. Yes, it does seem way more dodgy than it claims. I'll try and talk to people. Ga?l From gnurser at googlemail.com Sun Oct 25 04:53:39 2009 From: gnurser at googlemail.com (George Nurser) Date: Sun, 25 Oct 2009 08:53:39 +0000 Subject: [SciPy-dev] test failures in trunk in special In-Reply-To: References: <1cd32cbb0910190820q36a68b80ia9f82f4ef50ec8bc@mail.gmail.com> Message-ID: <1d1e6ea70910250153n77287410u6f608b2e664be30a@mail.gmail.com> I still have a couple of problems here. mac OS X 10.5.8, python 2.5.2 from python.org compiled with gcc4.0.1 numpy SVN v 7539 scipy SVN v 5997 (both compiled with Apple gcc 4.2.1) ====================================================================== FAIL: test_random_real (test_basic.TestSingleIFFT) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/fftpack/tests/test_basic.py", line 205, in test_random_real assert_array_almost_equal (y1, x) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/testing/utils.py", line 763, in assert_array_almost_equal header='Arrays are not almost equal') File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/testing/utils.py", line 607, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 0.900900900901%) x: array([ 0.42236230 +1.61093638e-09j, 0.29044548 -8.72590533e-09j, 0.14066364 -4.29583018e-08j, 0.79427350 +8.41708481e-09j, 0.23108852 -3.32202674e-08j, 0.85690206 +5.95867569e-08j,... y: array([ 0.42236236, 0.29044554, 0.14066353, 0.79427326, 0.23108847, 0.85690218, 0.90094191, 0.51615924, 0.31402779, 0.94050783, 0.43960547, 0.24419551, 0.80371237, 0.52694863, 0.48202211,... ====================================================================== FAIL: test_iv_cephes_vs_amos_mass_test (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/special/tests/test_basic.py", line 1714, in test_iv_cephes_vs_amos_mass_test assert dc[k] < 1e-9, (iv(v[k], x[k]), iv(v[k], x[k]+0j)) AssertionError: (1.8320048963545875e+306, (inf+0j)) --George 2009/10/20 Nils Wagner : > On Tue, 20 Oct 2009 08:17:39 +0000 (UTC) > ?Pauli Virtanen wrote: >> Mon, 19 Oct 2009 23:13:43 +0200, Nils Wagner wrote: >>> >>> I can reproduce some failures on linux (opensuse 11.1) >> >> The test tolerances were a bit too tight. I relaxed them >>slightly. Please >> report again if you see more of these, thanks! >> >> -- >> Pauli Virtanen >> > > Great ! > > Ran 3868 tests in 93.999s > > OK (KNOWNFAIL=6, SKIP=17) > > > Nils > >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From ralf.gommers at googlemail.com Sun Oct 25 06:44:20 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Sun, 25 Oct 2009 11:44:20 +0100 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <4AD5D6A6.2080209@stsci.edu> References: <4AD34DFF.6000809@stsci.edu> <4AD49B79.9080409@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> <20091013165559.GA20164@phare.normalesup.org> <45d1ab480910131448u6547db11xe2706f05eb14a09b@mail.gmail.com> <4AD5D6A6.2080209@stsci.edu> Message-ID: On Wed, Oct 14, 2009 at 2:48 PM, Michael Droettboom wrote: > Yes -- they should all eventually be at needs review, but I held off > yesterday to wait for all of the free functions to show up in the doc > editor, and a way to test the cross-referencing from the methods to the > functions was correct, since those will probably require editing en > masse once I know the solution. I didn't want to have to run through > all of them twice. > > > Hi Mike, I had a look at all the new chararray docstrings and it all looks pretty good, thanks! There is one minor thing that requires some mass-editing: the type description "array_like of string_ or unicode_" needs to be "array_like of str or unicode". Also similar usage of string_ and unicode_ elsewhere. The trailing backslash is Sphinx markup for a link, so they all turn into broken links. The other problem I noticed is the intersphinx links to the Python docs (str.methodname) do not work. The intersphinx extension seems to be enabled, not sure why it does not work. Maybe Pauli can shed some light on that. Cheers, Ralf Mike > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Sun Oct 25 07:18:41 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Sun, 25 Oct 2009 12:18:41 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: Message-ID: On Mon, Oct 12, 2009 at 8:51 PM, Ralf Gommers wrote: > Hi all, > > While working on the ndimage docs, I saw that the docfiller decorator in > ndimage.filters does not work well with the doc wiki. Basically, in the doc > wiki you see the docstring with the decorator applied. Then if you make an > edit, all the %(param) markup has been replaced. Compare the `convolve` and > `convolve1d` docstrings in trunk to see what happens. > > Some options to fix this: > 1. remove the filters docstrings from the doc wiki > 2. make the doc wiki understand the decorator > 3. remove the decorator, and do some copy-paste'ing of parameter > descriptions > 4. remove the decorator, and refer to a single description of those > parameters outside the docstring. > > I would have a preference for 3 or 4, mostly because it's easier and more > robust. Would you like a patch for either of those options? > > Update: someone decided to work on the filters docs last night during the sprint, so the @docfiller markup will be wiped out by the next doc merge unless we decide it is important and fix it. Also see this related ticket: http://code.google.com/p/pydocweb/issues/detail?id=33 Cheers, Ralf > > > Cheers, > Ralf > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Sun Oct 25 07:58:48 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Sun, 25 Oct 2009 12:58:48 +0100 Subject: [SciPy-dev] module docstrings Message-ID: Hi all, The doc standard does not say anything about module docstrings, and before starting to clean those up it would be nice to agree on a format. Right now there are modules that: - do not have any docstring (matrixlib, fromnumeric, ..) - have license/author info in the docstring (scipy.interpolate.fitpack, ..) - list their routines (linalg, core, lib, ..) * some of these have a routines rst doc: routines.linalg.rst exists, routines.lib.rst does not. - have a single summary line (distutils, ..) - have some basic explanation (ma, scimath, ..) What do you all think about the following? : 1. every module needs a docstring with at least a summary line. 2. follow the doc standard where relevant, so typically: - summary line - extended summary paragraph(s) - see also, if needed - notes, if needed - references, if needed 3. no routine listing in the docstring, each module gets a corresponding routines.module.rst file 4. license and author info can be in the source file, but not in the docstring. (same principle for author info in reST docs by the way). Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Sun Oct 25 12:27:53 2009 From: matthew.brett at gmail.com (Matthew Brett) Date: Sun, 25 Oct 2009 09:27:53 -0700 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: Message-ID: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> Hi, >> Some options to fix this: >> 1. remove the filters docstrings from the doc wiki >> 2. make the doc wiki understand the decorator >> 3. remove the decorator, and do some copy-paste'ing of parameter >> descriptions >> 4. remove the decorator, and refer to a single description of those >> parameters outside the docstring. >> >> I would have a preference for 3 or 4, mostly because it's easier and more >> robust. Would you like a patch for either of those options? >> > Update: someone decided to work on the filters docs last night during the > sprint, so the @docfiller markup will be wiped out by the next doc merge > unless we decide it is important and fix it. I wrote the original docfiller, so I should probably say that: For this situation, where there are many functions with the same set of parameters, and the parameters require significant explanation, it would be good to have some mechanism to keep track of the overlap. If we separate the docstrings, it's difficult to imagine that they will stay in sync, if people are editing them on the doc wiki. The docfiller pattern is one I'd already used on the matlab IO code, for the same reason. I think it's relatively easy to keep track of what's going on using SVN, so it would be a shame if we lost the ability to sync docstring components by using the wiki... Best, Matthew From d.l.goldsmith at gmail.com Sun Oct 25 13:14:06 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 10:14:06 -0700 Subject: [SciPy-dev] module docstrings In-Reply-To: References: Message-ID: <45d1ab480910251014g4f4056b3q73fd0d932fae28d3@mail.gmail.com> "Works for me." :-) Are there *any* that presently conform to that standard? (Or do they consequently all need editing?) DG On Sun, Oct 25, 2009 at 4:58 AM, Ralf Gommers wrote: > Hi all, > > The doc standard does not say anything about module docstrings, and before > starting to clean those up it would be nice to agree on a format. > > Right now there are modules that: > - do not have any docstring (matrixlib, fromnumeric, ..) > - have license/author info in the docstring (scipy.interpolate.fitpack, ..) > - list their routines (linalg, core, lib, ..) > * some of these have a routines rst doc: routines.linalg.rst exists, > routines.lib.rst does not. > - have a single summary line (distutils, ..) > - have some basic explanation (ma, scimath, ..) > > > What do you all think about the following? : > > 1. every module needs a docstring with at least a summary line. > > 2. follow the doc standard where relevant, so typically: > - summary line > - extended summary paragraph(s) > - see also, if needed > - notes, if needed > - references, if needed > > 3. no routine listing in the docstring, each module gets a corresponding > routines.module.rst file > > 4. license and author info can be in the source file, but not in the > docstring. (same principle for author info in reST docs by the way). > > > Cheers, > Ralf > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Sun Oct 25 13:25:56 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Sun, 25 Oct 2009 18:25:56 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 5:27 PM, Matthew Brett wrote: > > Update: someone decided to work on the filters docs last night during the > > sprint, so the @docfiller markup will be wiped out by the next doc merge > > unless we decide it is important and fix it. > > > I wrote the original docfiller, so I should probably say that: > > For this situation, where there are many functions with the same set > of parameters, and the parameters require significant explanation, it > would be good to have some mechanism to keep track of the overlap. Agreed. If we separate the docstrings, it's difficult to imagine that they will > stay in sync, if people are editing them on the doc wiki. Yes, they won't stay in sync. > The docfiller pattern is one I'd already used on the matlab IO code, for > the same reason. As noted in pydocweb issue 33 http://code.google.com/p/pydocweb/issues/detail?id=33 , it is possible to support this. But then this needs to be a standard, and the only such way to manipulate docstrings. I think in scipy.stats there's a different mechanism already. Could they be unified? > I think it's relatively easy to keep track of > what's going on using SVN, so it would be a shame if we lost the > ability to sync docstring components by using the wiki... > Keeping track is easy, but fixing it by hand each time after wiki merges will not work. docfiller markup has already been lost. Could you comment on the flexibility of docfiller? Would it work for scipy.stats? Would it need a NEP to document its behavior and specify that this is *the* way to modify docstrings? Thanks, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Sun Oct 25 13:32:16 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Sun, 25 Oct 2009 18:32:16 +0100 Subject: [SciPy-dev] module docstrings In-Reply-To: <45d1ab480910251014g4f4056b3q73fd0d932fae28d3@mail.gmail.com> References: <45d1ab480910251014g4f4056b3q73fd0d932fae28d3@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 6:14 PM, David Goldsmith wrote: > "Works for me." :-) Are there *any* that presently conform to that > standard? (Or do they consequently all need editing?) > Pretty much all of them still need editing. The only one I've worked on is numpy.lib.format, which is maybe too long for a docstring. numpy.ma and numpy.scimath are pretty good already. Ralf > > DG > > On Sun, Oct 25, 2009 at 4:58 AM, Ralf Gommers > wrote: > >> Hi all, >> >> The doc standard does not say anything about module docstrings, and before >> starting to clean those up it would be nice to agree on a format. >> >> Right now there are modules that: >> - do not have any docstring (matrixlib, fromnumeric, ..) >> - have license/author info in the docstring (scipy.interpolate.fitpack, >> ..) >> - list their routines (linalg, core, lib, ..) >> * some of these have a routines rst doc: routines.linalg.rst exists, >> routines.lib.rst does not. >> - have a single summary line (distutils, ..) >> - have some basic explanation (ma, scimath, ..) >> >> >> What do you all think about the following? : >> >> 1. every module needs a docstring with at least a summary line. >> >> 2. follow the doc standard where relevant, so typically: >> - summary line >> - extended summary paragraph(s) >> - see also, if needed >> - notes, if needed >> - references, if needed >> >> 3. no routine listing in the docstring, each module gets a corresponding >> routines.module.rst file >> >> 4. license and author info can be in the source file, but not in the >> docstring. (same principle for author info in reST docs by the way). >> >> >> Cheers, >> Ralf >> >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Sun Oct 25 15:20:43 2009 From: matthew.brett at gmail.com (Matthew Brett) Date: Sun, 25 Oct 2009 12:20:43 -0700 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> Message-ID: <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> Hi, > As noted in pydocweb issue 33 > http://code.google.com/p/pydocweb/issues/detail?id=33 , it is possible to > support this. But then this needs to be a standard, and the only such way to > manipulate docstrings. I think in scipy.stats there's a different mechanism > already. Could they be unified? [see below] >> ? I think it's relatively easy to keep track of >> what's going on using SVN, so it would be a shame if we lost the >> ability to sync docstring components by using the wiki... > > Keeping track is easy, but fixing it by hand each time after wiki merges > will not work. docfiller markup has already been lost. Yes, right... > Could you comment on the flexibility of docfiller? Would it work for > scipy.stats? Would it need a NEP to document its behavior and specify that > this is *the* way to modify docstrings? Ah - now you're asking me to think, on a Sunday morning. Just a review for those not reading the code for pleasure: docfiller does this: (scipy.ndimage.doccer, scipy.ndimage.filters): # give docstrings for particular parameters as dictionary values. # usually they will be multi-line of course docdict = {'param1': 'param1 : int, optional', 'param2': 'param2 : bool, optional'} # make a decorator to fill in those strings when they appear in the docstring docfiller = doccer.filldoc(docdict) # use decorator... @docfiller def func(param1=0, param2=False): """ A function to do things Parameters ---------------- %(param1)s %(param2)s '''' pass scipy.stats (sorry I read this fast) does a similar thing by defining an object to contain a particular type of distribution function (e.g rv_continuous), and you can pass things like the function short and long names, allowable input shape and any extra text you want to add to the docstring, and the object __init__ fills the docstring for you. Docfiller allows you to enter any repeated piece of text, requiring you only to define the %(text_here)s placeholders. scipy.stats in docfiller flavor would be: docdict = {'name': 'normal', 'longname': 'the Gaussian (normal) distribution'} # etc scipy.stats does some processing of the input parameters like shape, to determine the docstring, but that could easily be put in your decorator. So, I'm not sure docfiller is the only or best solution, but it looks like something like it recurs in the code, and I think it could work for scipy.stats... See you, Matthew From d.l.goldsmith at gmail.com Sun Oct 25 16:10:29 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 13:10:29 -0700 Subject: [SciPy-dev] module docstrings In-Reply-To: References: <45d1ab480910251014g4f4056b3q73fd0d932fae28d3@mail.gmail.com> Message-ID: <45d1ab480910251310l1289fb61mf68718d0fed74b4e@mail.gmail.com> OK, leave some for me. ;-) DG On Sun, Oct 25, 2009 at 10:32 AM, Ralf Gommers wrote: > > On Sun, Oct 25, 2009 at 6:14 PM, David Goldsmith wrote: > >> "Works for me." :-) Are there *any* that presently conform to that >> standard? (Or do they consequently all need editing?) >> > > Pretty much all of them still need editing. The only one I've worked on is > numpy.lib.format, which is maybe too long for a docstring. > > numpy.ma and numpy.scimath are pretty good already. > > Ralf > > >> >> DG >> >> On Sun, Oct 25, 2009 at 4:58 AM, Ralf Gommers < >> ralf.gommers at googlemail.com> wrote: >> >>> Hi all, >>> >>> The doc standard does not say anything about module docstrings, and >>> before starting to clean those up it would be nice to agree on a format. >>> >>> Right now there are modules that: >>> - do not have any docstring (matrixlib, fromnumeric, ..) >>> - have license/author info in the docstring (scipy.interpolate.fitpack, >>> ..) >>> - list their routines (linalg, core, lib, ..) >>> * some of these have a routines rst doc: routines.linalg.rst exists, >>> routines.lib.rst does not. >>> - have a single summary line (distutils, ..) >>> - have some basic explanation (ma, scimath, ..) >>> >>> >>> What do you all think about the following? : >>> >>> 1. every module needs a docstring with at least a summary line. >>> >>> 2. follow the doc standard where relevant, so typically: >>> - summary line >>> - extended summary paragraph(s) >>> - see also, if needed >>> - notes, if needed >>> - references, if needed >>> >>> 3. no routine listing in the docstring, each module gets a corresponding >>> routines.module.rst file >>> >>> 4. license and author info can be in the source file, but not in the >>> docstring. (same principle for author info in reST docs by the way). >>> >>> >>> Cheers, >>> Ralf >>> >>> >>> _______________________________________________ >>> Scipy-dev mailing list >>> Scipy-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>> >>> >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Sun Oct 25 16:14:13 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 13:14:13 -0700 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> Message-ID: <45d1ab480910251314k4ed5c51ch6fcc519c2a80b3a4@mail.gmail.com> NEP = ? DG On Sun, Oct 25, 2009 at 10:25 AM, Ralf Gommers wrote: > > > On Sun, Oct 25, 2009 at 5:27 PM, Matthew Brett wrote: > >> > Update: someone decided to work on the filters docs last night during >> the >> > sprint, so the @docfiller markup will be wiped out by the next doc merge >> > unless we decide it is important and fix it. >> >> >> I wrote the original docfiller, so I should probably say that: >> >> For this situation, where there are many functions with the same set >> of parameters, and the parameters require significant explanation, it >> would be good to have some mechanism to keep track of the overlap. > > > Agreed. > > If we separate the docstrings, it's difficult to imagine that they will >> stay in sync, if people are editing them on the doc wiki. > > > Yes, they won't stay in sync. > > >> The docfiller pattern is one I'd already used on the matlab IO code, for >> the same reason. > > > As noted in pydocweb issue 33 > http://code.google.com/p/pydocweb/issues/detail?id=33 , it is possible to > support this. But then this needs to be a standard, and the only such way to > manipulate docstrings. I think in scipy.stats there's a different mechanism > already. Could they be unified? > > >> I think it's relatively easy to keep track of >> what's going on using SVN, so it would be a shame if we lost the >> ability to sync docstring components by using the wiki... >> > > Keeping track is easy, but fixing it by hand each time after wiki merges > will not work. docfiller markup has already been lost. > > Could you comment on the flexibility of docfiller? Would it work for > scipy.stats? Would it need a NEP to document its behavior and specify that > this is *the* way to modify docstrings? > > Thanks, > Ralf > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Sun Oct 25 16:15:51 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 13:15:51 -0700 Subject: [SciPy-dev] chararray docstrings In-Reply-To: References: <4AD34DFF.6000809@stsci.edu> <1cd32cbb0910130845l7635c393x3f1e35aeda2a6a90@mail.gmail.com> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> <20091013165559.GA20164@phare.normalesup.org> <45d1ab480910131448u6547db11xe2706f05eb14a09b@mail.gmail.com> <4AD5D6A6.2080209@stsci.edu> Message-ID: <45d1ab480910251315k63ae297fo58468cde4a0e67e3@mail.gmail.com> On Sun, Oct 25, 2009 at 3:44 AM, Ralf Gommers wrote: > > > On Wed, Oct 14, 2009 at 2:48 PM, Michael Droettboom wrote: > >> Yes -- they should all eventually be at needs review, but I held off >> yesterday to wait for all of the free functions to show up in the doc >> editor, and a way to test the cross-referencing from the methods to the >> functions was correct, since those will probably require editing en >> masse once I know the solution. I didn't want to have to run through >> all of them twice. >> >> >> Hi Mike, > > I had a look at all the new chararray docstrings and it all looks pretty > good, thanks! There is one minor thing that requires some mass-editing: the > type description "array_like of string_ or unicode_" needs to be "array_like > of str or unicode". Also similar usage of string_ and unicode_ elsewhere. > The trailing backslash is Sphinx markup for a link, so they all turn into > broken links. > I imagine this problem is even more widespread - I can take care of it (or what remains of it). DG The other problem I noticed is the intersphinx links to the Python docs > (str.methodname) do not work. The intersphinx extension seems to be enabled, > not sure why it does not work. Maybe Pauli can shed some light on that. > > Cheers, > Ralf > > > Mike >> >> > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From millman at berkeley.edu Sun Oct 25 16:25:22 2009 From: millman at berkeley.edu (Jarrod Millman) Date: Sun, 25 Oct 2009 13:25:22 -0700 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <45d1ab480910251314k4ed5c51ch6fcc519c2a80b3a4@mail.gmail.com> References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <45d1ab480910251314k4ed5c51ch6fcc519c2a80b3a4@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 1:14 PM, David Goldsmith wrote: > NEP = ? NumPy Enhancement Proposal (like a PEP): http://projects.scipy.org/numpy/browser/trunk/doc/neps -- Jarrod Millman Helen Wills Neuroscience Institute 10 Giannini Hall, UC Berkeley http://cirl.berkeley.edu/ From ralf.gommers at googlemail.com Sun Oct 25 16:35:08 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Sun, 25 Oct 2009 21:35:08 +0100 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <45d1ab480910251315k63ae297fo58468cde4a0e67e3@mail.gmail.com> References: <4AD34DFF.6000809@stsci.edu> <4AD4A121.5060009@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> <20091013165559.GA20164@phare.normalesup.org> <45d1ab480910131448u6547db11xe2706f05eb14a09b@mail.gmail.com> <4AD5D6A6.2080209@stsci.edu> <45d1ab480910251315k63ae297fo58468cde4a0e67e3@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 9:15 PM, David Goldsmith wrote: > On Sun, Oct 25, 2009 at 3:44 AM, Ralf Gommers > wrote: > >> I had a look at all the new chararray docstrings and it all looks pretty >> good, thanks! There is one minor thing that requires some mass-editing: the >> type description "array_like of string_ or unicode_" needs to be "array_like >> of str or unicode". Also similar usage of string_ and unicode_ elsewhere. >> The trailing backslash is Sphinx markup for a link, so they all turn into >> broken links. >> >> > I imagine this problem is even more widespread - I can take care of it (or > what remains of it). > > Well, you have to visit ~50 wiki pages. For Mike it's probably some global replaces, and an SVN commit..... Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Sun Oct 25 17:05:56 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 14:05:56 -0700 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <45d1ab480910251314k4ed5c51ch6fcc519c2a80b3a4@mail.gmail.com> Message-ID: <45d1ab480910251405hec069cx88f59715faa223b0@mail.gmail.com> Thanks! DG On Sun, Oct 25, 2009 at 1:25 PM, Jarrod Millman wrote: > On Sun, Oct 25, 2009 at 1:14 PM, David Goldsmith > wrote: > > NEP = ? > > NumPy Enhancement Proposal (like a PEP): > http://projects.scipy.org/numpy/browser/trunk/doc/neps > > -- > Jarrod Millman > Helen Wills Neuroscience Institute > 10 Giannini Hall, UC Berkeley > http://cirl.berkeley.edu/ > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Sun Oct 25 17:11:30 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 14:11:30 -0700 Subject: [SciPy-dev] chararray docstrings In-Reply-To: References: <4AD34DFF.6000809@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> <20091013165559.GA20164@phare.normalesup.org> <45d1ab480910131448u6547db11xe2706f05eb14a09b@mail.gmail.com> <4AD5D6A6.2080209@stsci.edu> <45d1ab480910251315k63ae297fo58468cde4a0e67e3@mail.gmail.com> Message-ID: <45d1ab480910251411s118703een44bfb1de299787b2@mail.gmail.com> On Sun, Oct 25, 2009 at 1:35 PM, Ralf Gommers wrote: > > > On Sun, Oct 25, 2009 at 9:15 PM, David Goldsmith wrote: > >> On Sun, Oct 25, 2009 at 3:44 AM, Ralf Gommers < >> ralf.gommers at googlemail.com> wrote: >> >>> I had a look at all the new chararray docstrings and it all looks pretty >>> good, thanks! There is one minor thing that requires some mass-editing: the >>> type description "array_like of string_ or unicode_" needs to be "array_like >>> of str or unicode". Also similar usage of string_ and unicode_ elsewhere. >>> The trailing backslash is Sphinx markup for a link, so they all turn into >>> broken links. >>> >>> >> I imagine this problem is even more widespread - I can take care of it (or >> what remains of it). >> >> Well, you have to visit ~50 wiki pages. For Mike it's probably some global > replaces, and an SVN commit..... > My thinking is that global replaces will be "dangerous" in that they'll replace the word string where that *is* what's wanted (e.g., in a narrative). That said, I'll take care of non-chararray occurrences first and if Mike hasn't gotten to the chararray/"new chararray" occurrences by the time I'm done, I'll start in on those. DG > > Ralf > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Sun Oct 25 17:33:49 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Sun, 25 Oct 2009 22:33:49 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 8:20 PM, Matthew Brett wrote: > > Ah - now you're asking me to think, on a Sunday morning. > > Sorry. Hope it wasn't too painful :) > > scipy.stats (sorry I read this fast) does a similar thing by defining > an object to contain a particular type of distribution function (e.g > rv_continuous), and you can pass things like the function short and > long names, allowable input shape and any extra text you want to add > to the docstring, and the object __init__ fills the docstring for you. > > Okay, now it makes sense to me. The scipy.stats docs are pretty much completely auto-generated, even the examples. Then the "extradoc" keyword contains extra info that is just tacked on at the end (below the examples). There seems to be room for improvement there. Docfiller allows you to enter any repeated piece of text, requiring > you only to define the %(text_here)s placeholders. scipy.stats in > docfiller flavor would be: > > docdict = {'name': 'normal', 'longname': 'the Gaussian (normal) > distribution'} # etc > > scipy.stats does some processing of the input parameters like shape, > to determine the docstring, but that could easily be put in your > decorator. > > So, I'm not sure docfiller is the only or best solution, but it looks > like something like it recurs in the code, and I think it could work > for scipy.stats... > Thanks Matthew. Searching for how other people/projects solve this same problem I found nothing much fancier than docfiller, so I think I will try to get an idea first of how much work it will be to change the scipy.stats docs to use docfiller throughout. Then of course it would be good to hear from Pauli roughly how much work it would be to support this in pydocweb. The ticket says "rewrite of pydoc-tool.py" so it may be non-trivial.... Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Sun Oct 25 17:37:59 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 14:37:59 -0700 Subject: [SciPy-dev] Documenting function-like classes Message-ID: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> Quoting http://docs.scipy.org/numpy/Questions+Answers/#how-to-document-the-class-numpy-ufuncs : " How to document (the class) numpy.ufuncs Though a class, numpy.ufuncs is presently documented, albeit not "to spec," as if it were a function; how *should* we document it? *David Goldsmith, 2009-07-21* - I vote for as a class. As a consequence, I think, much of what's in there now can/should be disposed of: it more properly belongs in the docstrings of specific ufunc instances anyway, IMO. *David Goldsmith, 2009-07-21* - As a function. This goes for all classes that pretend to be functions IMHO. If a class has a __call__ method for example, it needs a Returns section. Pydocweb will complain right now, but this can be fixed. - Ralf Gommers, 2009-10-25* " What *is* the design motivation behind having a function-like class anyway? DG -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Sun Oct 25 18:11:13 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Sun, 25 Oct 2009 18:11:13 -0400 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> Message-ID: <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> On Sun, Oct 25, 2009 at 5:33 PM, Ralf Gommers wrote: > > > On Sun, Oct 25, 2009 at 8:20 PM, Matthew Brett > wrote: >> >> Ah - now you're asking me to think, on a Sunday morning. >> > Sorry. Hope it wasn't too painful :) >> >> scipy.stats (sorry I read this fast) does a similar thing by defining >> an object to contain a particular type of distribution function (e.g >> rv_continuous), and you can pass things like the function short and >> long names, allowable input shape and any extra text you want to add >> to the docstring, and the object __init__ fills the docstring for you. >> > Okay, now it makes sense to me. The scipy.stats docs are pretty much > completely auto-generated, even the examples. Then the "extradoc" keyword > contains extra info that is just tacked on at the end (below the examples). > There seems to be room for improvement there. > >> Docfiller allows you to enter any repeated piece of text, requiring >> you only to define the %(text_here)s placeholders. ?scipy.stats in >> docfiller flavor would be: >> >> docdict = {'name': 'normal', 'longname': 'the Gaussian (normal) >> distribution'} # etc >> >> scipy.stats does some processing of the input parameters like shape, >> to determine the docstring, but that could easily be put in your >> decorator. >> >> So, I'm not sure docfiller is the only or best solution, but it looks >> like something like it recurs in the code, and I think it could work >> for scipy.stats... > > Thanks Matthew. Searching for how other people/projects solve this same > problem I found nothing much fancier than docfiller, so I think I will try > to get an idea first of how much work it will be to change the scipy.stats > docs to use docfiller throughout. > > Then of course it would be good to hear from Pauli roughly how much work it > would be to support this in pydocweb. The ticket says "rewrite of > pydoc-tool.py" so it may be non-trivial.... > > Cheers, > Ralf One of the differences between ndimage and stats.distributions is that in ndimage functions are decorated, while the individual distributions (kind of) inherit the adjusted docstring from the basic distribution classes. The documentation can be improved (maybe with better templating as in the ndimage decorator), and I don't know how pydocweb can handle it, but I don't think we need to start to decorate individual distributions when we can use the class hierarchy for generating the docs. One useful enhancement to the distribution docs would be to get autogenerated links to the description of the individual distributions that has been added to the scipy.stats.tutorial. Josef > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From ralf.gommers at googlemail.com Sun Oct 25 19:00:36 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Mon, 26 Oct 2009 00:00:36 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 11:11 PM, wrote: > On Sun, Oct 25, 2009 at 5:33 PM, Ralf Gommers > wrote: > > > > > > On Sun, Oct 25, 2009 at 8:20 PM, Matthew Brett > >> > >> scipy.stats does some processing of the input parameters like shape, > >> to determine the docstring, but that could easily be put in your > >> decorator. > >> > >> So, I'm not sure docfiller is the only or best solution, but it looks > >> like something like it recurs in the code, and I think it could work > >> for scipy.stats... > > > > Thanks Matthew. Searching for how other people/projects solve this same > > problem I found nothing much fancier than docfiller, so I think I will > try > > to get an idea first of how much work it will be to change the > scipy.stats > > docs to use docfiller throughout. > > > > Then of course it would be good to hear from Pauli roughly how much work > it > > would be to support this in pydocweb. The ticket says "rewrite of > > pydoc-tool.py" so it may be non-trivial.... > > > > Cheers, > > Ralf > > One of the differences between ndimage and stats.distributions is that in > ndimage functions are decorated, while the individual distributions (kind > of) > inherit the adjusted docstring from the basic distribution classes. > > The documentation can be improved (maybe with better templating as in the > ndimage decorator), and I don't know how pydocweb can > handle it, but I don't think we need to start to decorate individual > distributions > when we can use the class hierarchy for generating the docs. > For scipy.stats it might not be any better than the current method, but it also would not be worse. The point is that it will be a lot of work to make the wiki play nice with even *one* templating system. Since a decorator is more widely applicable than the scipy.stats inheritance, it could be a good candidate to be applied everywhere in NumPy/SciPy and be recognized by pydocweb. > One useful enhancement to the distribution docs would be to get > autogenerated > links to the description of the individual distributions that has been > added to the > scipy.stats.tutorial. > Yeah, that would help. Linking to .rst files in general could be improved still. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From pav at iki.fi Sun Oct 25 20:56:44 2009 From: pav at iki.fi (Pauli Virtanen) Date: Mon, 26 Oct 2009 02:56:44 +0200 Subject: [SciPy-dev] RFR: fixing scipy.special.orthogonal Message-ID: <1256518603.6665.16.camel@idol> Hi, http://codereview.appspot.com/140053 The orthogonal polynomial routines in scipy.special.orthogonal are cumbersome to use if you only want *values* of the polynomials at a few points. This patch adds separate vectorized eval_* routines for them. What is more controversial, it also hooks these routines in orthopoly1d, so that >>> special.chebyt(100)(0.98) 0.37728975480815219 instead of the current >>> special.chebyt(100)(0.98) -5.2908451051968135e+20 Any arithmetic on the returned polynomial objects, however, discards the stabler evaluation routines. (Maybe when the scipy/scikits.polynomial stuff gets ready, we can address also this.) Comments? Good to go in? -- Pauli Virtanen From charlesr.harris at gmail.com Sun Oct 25 21:23:28 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 25 Oct 2009 19:23:28 -0600 Subject: [SciPy-dev] RFR: fixing scipy.special.orthogonal In-Reply-To: <1256518603.6665.16.camel@idol> References: <1256518603.6665.16.camel@idol> Message-ID: On Sun, Oct 25, 2009 at 6:56 PM, Pauli Virtanen wrote: > Hi, > > http://codereview.appspot.com/140053 > > The orthogonal polynomial routines in scipy.special.orthogonal are > cumbersome to use if you only want *values* of the polynomials at a few > points. > > I'm not familiar with these routines. In what way is the evaluation at a few points cumbersome? > This patch adds separate vectorized eval_* routines for them. > > What is more controversial, it also hooks these routines in orthopoly1d, > so that > > >>> special.chebyt(100)(0.98) > 0.37728975480815219 > > instead of the current > > >>> special.chebyt(100)(0.98) > -5.2908451051968135e+20 > > Well, the first certainly looks like an improvement ;) I'm about to commit a chebyshev class in numpy, along with a revised polynomial class. At the moment they need more documentation and testing. Apropos that commit in general, I was going to put them in their own directory, "polynomial", at the same level as fft etc., but not have them automatically loaded when numpy is imported. That is, one would have to explicitly do import numpy.polynomial. The modules also generate the class on the fly using a standard string.Template class. Any arithmetic on the returned polynomial objects, however, discards the > stabler evaluation routines. (Maybe when the scipy/scikits.polynomial > stuff gets ready, we can address also this.) > > Comments? Good to go in? > > I can't think of a good reason not to fix that up if you think it appropriate. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From pav at iki.fi Sun Oct 25 21:37:19 2009 From: pav at iki.fi (Pauli Virtanen) Date: Mon, 26 Oct 2009 03:37:19 +0200 Subject: [SciPy-dev] RFR: fixing scipy.special.orthogonal In-Reply-To: References: <1256518603.6665.16.camel@idol> Message-ID: <1256521039.6665.25.camel@idol> su, 2009-10-25 kello 19:23 -0600, Charles R Harris kirjoitti: > On Sun, Oct 25, 2009 at 6:56 PM, Pauli Virtanen wrote: > http://codereview.appspot.com/140053 > > The orthogonal polynomial routines in scipy.special.orthogonal > are > cumbersome to use if you only want *values* of the polynomials > at a few > points. > > I'm not familiar with these routines. In what way is the evaluation at > a few points cumbersome? They IIRC compute polynomial roots via eigenvalues of the companion matrix, so chebyt(1000)(x) takes ages, and it's not blazingly fast for smaller n either. Pauli From robert.kern at gmail.com Sun Oct 25 21:41:45 2009 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 25 Oct 2009 20:41:45 -0500 Subject: [SciPy-dev] Documenting function-like classes In-Reply-To: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> References: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> Message-ID: <3d375d730910251841h50fa6f73ib4c44a6c65840055@mail.gmail.com> On Sun, Oct 25, 2009 at 16:37, David Goldsmith wrote: > What *is* the design motivation behind having a function-like class anyway? Ufuncs have data and behavior bundled together. That's a classic motivation for using classes. It just so happens that one of those behaviors is very like a function call, so one of the methods is __call__. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From peridot.faceted at gmail.com Sun Oct 25 21:42:31 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Sun, 25 Oct 2009 21:42:31 -0400 Subject: [SciPy-dev] Documenting function-like classes In-Reply-To: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> References: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> Message-ID: 2009/10/25 David Goldsmith : > Quoting > http://docs.scipy.org/numpy/Questions+Answers/#how-to-document-the-class-numpy-ufuncs: > > " How to document (the class) numpy.ufuncs > > Though a class, numpy.ufuncs is presently documented, albeit not "to spec," > as if it were a function; how should we document it? > > David Goldsmith, 2009-07-21 > > I vote for as a class. As a consequence, I think, much of what's in there > now can/should be disposed of: it more properly belongs in the docstrings of > specific ufunc instances anyway, IMO. > > David Goldsmith, 2009-07-21 > > As a function. This goes for all classes that pretend to be functions IMHO. > If a class has a __call__ method for example, it needs a Returns section. > Pydocweb will complain right now, but this can be fixed. > > Ralf Gommers, 2009-10-25* > > " Er, just to be clear: what you are describing is a class whose *instances* behave like functions, right? Because all normal classes behave like functions: MyClass() generates an instance of MyClass. > What *is* the design motivation behind having a function-like class anyway? Generally, I use one when an object I am representing with a class has a natural evaluation map. So, for example, an object representing a polynomial ought to be a class - you can multiply it, you can compute derivatives and antiderivatives, you can look at its coefficients - but you should also be able to use it like a function, to evaluate at a particular place. When documenting such a thing, it seems to me you should treat __call__ like __init__ or any other method whose name is a bit obscure. Which, as I understand it, means that the class docstring should mention the existence of them but the bulk of the documentation should be in the method docstring. Anne From d.l.goldsmith at gmail.com Sun Oct 25 21:46:54 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 18:46:54 -0700 Subject: [SciPy-dev] Documenting function-like classes In-Reply-To: References: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> Message-ID: <45d1ab480910251846y5f3debd2p1df19ae2f783c8a9@mail.gmail.com> On Sun, Oct 25, 2009 at 6:42 PM, Anne Archibald wrote: > 2009/10/25 David Goldsmith : > > Quoting > > > http://docs.scipy.org/numpy/Questions+Answers/#how-to-document-the-class-numpy-ufuncs > : > > > > " How to document (the class) numpy.ufuncs > > > > Though a class, numpy.ufuncs is presently documented, albeit not "to > spec," > > as if it were a function; how should we document it? > > > > David Goldsmith, 2009-07-21 > > > > I vote for as a class. As a consequence, I think, much of what's in there > > now can/should be disposed of: it more properly belongs in the docstrings > of > > specific ufunc instances anyway, IMO. > > > > David Goldsmith, 2009-07-21 > > > > As a function. This goes for all classes that pretend to be functions > IMHO. > > If a class has a __call__ method for example, it needs a Returns section. > > Pydocweb will complain right now, but this can be fixed. > > > > Ralf Gommers, 2009-10-25* > > > > " > > Er, just to be clear: what you are describing is a class whose > *instances* behave like functions, right? Because all normal classes > behave like functions: MyClass() generates an instance of MyClass. > > > What *is* the design motivation behind having a function-like class > anyway? > > Generally, I use one when an object I am representing with a class has > a natural evaluation map. So, for example, an object representing a > polynomial ought to be a class - you can multiply it, you can compute > derivatives and antiderivatives, you can look at its coefficients - > but you should also be able to use it like a function, to evaluate at > a particular place. > > When documenting such a thing, it seems to me you should treat > __call__ like __init__ or any other method whose name is a bit > obscure. Which, as I understand it, means that the class docstring > should mention the existence of them but the bulk of the documentation > should be in the method docstring. > > Anne > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > Thanks, Robert and Anne. Would either of you care to opine: document these as classes or as functions? Thanks again, DG -------------- next part -------------- An HTML attachment was scrubbed... URL: From cournape at gmail.com Sun Oct 25 21:48:40 2009 From: cournape at gmail.com (David Cournapeau) Date: Mon, 26 Oct 2009 10:48:40 +0900 Subject: [SciPy-dev] RFR: fixing scipy.special.orthogonal In-Reply-To: References: <1256518603.6665.16.camel@idol> Message-ID: <5b8d13220910251848x7245308bj228852d44b79495d@mail.gmail.com> On Mon, Oct 26, 2009 at 10:23 AM, Charles R Harris wrote: > > > On Sun, Oct 25, 2009 at 6:56 PM, Pauli Virtanen wrote: >> >> Hi, >> >> http://codereview.appspot.com/140053 >> >> The orthogonal polynomial routines in scipy.special.orthogonal are >> cumbersome to use if you only want *values* of the polynomials at a few >> points. >> > > I'm not familiar with these routines. In what way is the evaluation at a few > points cumbersome? > >> >> This patch adds separate vectorized eval_* routines for them. >> >> What is more controversial, it also hooks these routines in orthopoly1d, >> so that >> >> ? ? ? ?>>> special.chebyt(100)(0.98) >> ? ? ? ?0.37728975480815219 >> >> instead of the current >> >> ? ? ? ?>>> special.chebyt(100)(0.98) >> ? ? ? ?-5.2908451051968135e+20 >> > > Well, the first certainly looks like an improvement ;) I'm about to commit a > chebyshev class in numpy, along with a revised polynomial class. At the > moment they need more documentation and testing. > > Apropos that commit in general, I was going to put them in their own > directory, "polynomial", at the same level as fft etc., but not have them > automatically loaded when numpy is imported. That is, one would have to > explicitly do? import numpy.polynomial. The modules also generate the class > on the fly using a standard string.Template class. I have not followed the polynomial discussion - why was it decided that it would be included in numpy ? I thought that new features for numpy should be avoided ? Otherwise, even if they are included in numpy, they should not be loaded automatically - fft is loaded automatically for compatibility with numarray/numeric IIRC. David From d.l.goldsmith at gmail.com Sun Oct 25 21:57:22 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 18:57:22 -0700 Subject: [SciPy-dev] Documenting function-like classes In-Reply-To: References: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> Message-ID: <45d1ab480910251857m71ef3844uc71881bd6409d1e8@mail.gmail.com> And a follow up... On Sun, Oct 25, 2009 at 6:42 PM, Anne Archibald wrote: > Generally, I use one when an object I am representing with a class has > a natural evaluation map. So, for example, an object representing a > polynomial ought to be a class - you can multiply it, you can compute > derivatives and antiderivatives, you can look at its coefficients - > but you should also be able to use it like a function, to evaluate at > a particular place. > However, since the syntax myClass() is already "spoken for" by the c-tor, for the function-like behavior don't you still have to: A) define a method attribute, and B) call it using '.' resolution syntax? DG When documenting such a thing, it seems to me you should treat > __call__ like __init__ or any other method whose name is a bit > obscure. Which, as I understand it, means that the class docstring > should mention the existence of them but the bulk of the documentation > should be in the method docstring. > > Anne > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Sun Oct 25 21:58:50 2009 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 25 Oct 2009 20:58:50 -0500 Subject: [SciPy-dev] Documenting function-like classes In-Reply-To: <45d1ab480910251857m71ef3844uc71881bd6409d1e8@mail.gmail.com> References: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> <45d1ab480910251857m71ef3844uc71881bd6409d1e8@mail.gmail.com> Message-ID: <3d375d730910251858u7cb08e40teb0047ea56e2b421@mail.gmail.com> On Sun, Oct 25, 2009 at 20:57, David Goldsmith wrote: > And a follow up... > > On Sun, Oct 25, 2009 at 6:42 PM, Anne Archibald > wrote: >> >> Generally, I use one when an object I am representing with a class has >> a natural evaluation map. So, for example, an object representing a >> polynomial ought to be a class - you can multiply it, you can compute >> derivatives and antiderivatives, you can look at its coefficients - >> but you should also be able to use it like a function, to evaluate at >> a particular place. > > However, since the syntax myClass() is already "spoken for" by the c-tor, > for the function-like behavior don't you still have to: A) define a method > attribute, and B) call it using '.' resolution syntax? No. You call the instance that is created by the constructor, not the class. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From d.l.goldsmith at gmail.com Sun Oct 25 22:03:34 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Sun, 25 Oct 2009 19:03:34 -0700 Subject: [SciPy-dev] Documenting function-like classes In-Reply-To: <3d375d730910251858u7cb08e40teb0047ea56e2b421@mail.gmail.com> References: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> <45d1ab480910251857m71ef3844uc71881bd6409d1e8@mail.gmail.com> <3d375d730910251858u7cb08e40teb0047ea56e2b421@mail.gmail.com> Message-ID: <45d1ab480910251903u5971f845r847680ceccb9493c@mail.gmail.com> Duh, that's what the __call__ method supplies, isn't it. Sorry for being dense. :-( DG On Sun, Oct 25, 2009 at 6:58 PM, Robert Kern wrote: > On Sun, Oct 25, 2009 at 20:57, David Goldsmith > wrote: > > And a follow up... > > > > On Sun, Oct 25, 2009 at 6:42 PM, Anne Archibald < > peridot.faceted at gmail.com> > > wrote: > >> > >> Generally, I use one when an object I am representing with a class has > >> a natural evaluation map. So, for example, an object representing a > >> polynomial ought to be a class - you can multiply it, you can compute > >> derivatives and antiderivatives, you can look at its coefficients - > >> but you should also be able to use it like a function, to evaluate at > >> a particular place. > > > > However, since the syntax myClass() is already "spoken for" by the c-tor, > > for the function-like behavior don't you still have to: A) define a > method > > attribute, and B) call it using '.' resolution syntax? > > No. You call the instance that is created by the constructor, not the > class. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma that is made terrible by our own mad attempt to interpret it as > though it had an underlying truth." > -- Umberto Eco > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Sun Oct 25 22:19:11 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 25 Oct 2009 20:19:11 -0600 Subject: [SciPy-dev] RFR: fixing scipy.special.orthogonal In-Reply-To: <5b8d13220910251848x7245308bj228852d44b79495d@mail.gmail.com> References: <1256518603.6665.16.camel@idol> <5b8d13220910251848x7245308bj228852d44b79495d@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 7:48 PM, David Cournapeau wrote: > On Mon, Oct 26, 2009 at 10:23 AM, Charles R Harris > wrote: > > > > > > On Sun, Oct 25, 2009 at 6:56 PM, Pauli Virtanen wrote: > >> > >> Hi, > >> > >> http://codereview.appspot.com/140053 > >> > >> The orthogonal polynomial routines in scipy.special.orthogonal are > >> cumbersome to use if you only want *values* of the polynomials at a few > >> points. > >> > > > > I'm not familiar with these routines. In what way is the evaluation at a > few > > points cumbersome? > > > >> > >> This patch adds separate vectorized eval_* routines for them. > >> > >> What is more controversial, it also hooks these routines in orthopoly1d, > >> so that > >> > >> >>> special.chebyt(100)(0.98) > >> 0.37728975480815219 > >> > >> instead of the current > >> > >> >>> special.chebyt(100)(0.98) > >> -5.2908451051968135e+20 > >> > > > > Well, the first certainly looks like an improvement ;) I'm about to > commit a > > chebyshev class in numpy, along with a revised polynomial class. At the > > moment they need more documentation and testing. > > > > Apropos that commit in general, I was going to put them in their own > > directory, "polynomial", at the same level as fft etc., but not have them > > automatically loaded when numpy is imported. That is, one would have to > > explicitly do import numpy.polynomial. The modules also generate the > class > > on the fly using a standard string.Template class. > > I have not followed the polynomial discussion - why was it decided > that it would be included in numpy ? I thought that new features for > numpy should be avoided ? > > Well, there is the polynomial class already, and really folks should be using Chebyshev polynomials for many of those things, they are just one of the numerical basics. I also wanted them for returning L_inf fits from some scipy routines I am also going to commit "real soon now". I got sidetracked ;) Speaking of which, where do you think the generalized remez algorithm should go? It has more uses than filter design for signal processing and might should go into optimize. Although if there was a "fitting" directory I would put it there. > Otherwise, even if they are included in numpy, they should not be > loaded automatically - fft is loaded automatically for compatibility > with numarray/numeric IIRC. > > Yeah, that's what I figured. I also wanted to avoid conflicts with the current polynomial stuff. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From peridot.faceted at gmail.com Sun Oct 25 22:20:31 2009 From: peridot.faceted at gmail.com (Anne Archibald) Date: Sun, 25 Oct 2009 22:20:31 -0400 Subject: [SciPy-dev] Documenting function-like classes In-Reply-To: <45d1ab480910251846y5f3debd2p1df19ae2f783c8a9@mail.gmail.com> References: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> <45d1ab480910251846y5f3debd2p1df19ae2f783c8a9@mail.gmail.com> Message-ID: 2009/10/25 David Goldsmith : > On Sun, Oct 25, 2009 at 6:42 PM, Anne Archibald > wrote: >> >> 2009/10/25 David Goldsmith : >> > Quoting >> > >> > http://docs.scipy.org/numpy/Questions+Answers/#how-to-document-the-class-numpy-ufuncs: >> > >> > " How to document (the class) numpy.ufuncs >> > >> > Though a class, numpy.ufuncs is presently documented, albeit not "to >> > spec," >> > as if it were a function; how should we document it? >> > >> > David Goldsmith, 2009-07-21 >> > >> > I vote for as a class. As a consequence, I think, much of what's in >> > there >> > now can/should be disposed of: it more properly belongs in the >> > docstrings of >> > specific ufunc instances anyway, IMO. >> > >> > David Goldsmith, 2009-07-21 >> > >> > As a function. This goes for all classes that pretend to be functions >> > IMHO. >> > If a class has a __call__ method for example, it needs a Returns >> > section. >> > Pydocweb will complain right now, but this can be fixed. >> > >> > Ralf Gommers, 2009-10-25* >> > >> > " >> >> Er, just to be clear: what you are describing is a class whose >> *instances* behave like functions, right? Because all normal classes >> behave like functions: MyClass() generates an instance of MyClass. >> >> > What *is* the design motivation behind having a function-like class >> > anyway? >> >> Generally, I use one when an object I am representing with a class has >> a natural evaluation map. So, for example, an object representing a >> polynomial ought to be a class - you can multiply it, you can compute >> derivatives and antiderivatives, you can look at its coefficients - >> but you should also be able to use it like a function, to evaluate at >> a particular place. >> >> When documenting such a thing, it seems to me you should treat >> __call__ like __init__ or any other method whose name is a bit >> obscure. Which, as I understand it, means that the class docstring >> should mention the existence of them but the bulk of the documentation >> should be in the method docstring. >> >> Anne >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev > > Thanks, Robert and Anne.? Would either of you care to opine: document these > as classes or as functions?? Thanks again, Well, IMHO, these objects are classes, and so should be documented as such. The function-like documentation should be the documentation for __call__. That said, I haven't been working on the docstrings, so I don't know how people have been documenting classes; it seems to me there's always some question of what goes in the class docstring and what goes in the method docstrings. In the case of weirdly-named methods like __call__ and __init__, one has to be sure that the user can find them and knows how to call them, but that's a problem when documenting all classes. I would guess that the confusion is arising because you're looking at classes that don't really do much other than act like functions? Maybe work through something like poly1d that has substantial data and other methods first. Anne > DG > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From charlesr.harris at gmail.com Sun Oct 25 22:21:29 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 25 Oct 2009 20:21:29 -0600 Subject: [SciPy-dev] RFR: fixing scipy.special.orthogonal In-Reply-To: <1256521039.6665.25.camel@idol> References: <1256518603.6665.16.camel@idol> <1256521039.6665.25.camel@idol> Message-ID: On Sun, Oct 25, 2009 at 7:37 PM, Pauli Virtanen wrote: > su, 2009-10-25 kello 19:23 -0600, Charles R Harris kirjoitti: > > On Sun, Oct 25, 2009 at 6:56 PM, Pauli Virtanen wrote: > > http://codereview.appspot.com/140053 > > > > The orthogonal polynomial routines in scipy.special.orthogonal > > are > > cumbersome to use if you only want *values* of the polynomials > > at a few > > points. > > > > I'm not familiar with these routines. In what way is the evaluation at > > a few points cumbersome? > > They IIRC compute polynomial roots via eigenvalues of the companion > matrix, so chebyt(1000)(x) takes ages, and it's not blazingly fast for > smaller n either. > > Ah, that's right. Pretty much anything would be an improvement there. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From arokem at berkeley.edu Sun Oct 25 22:50:49 2009 From: arokem at berkeley.edu (Ariel Rokem) Date: Sun, 25 Oct 2009 19:50:49 -0700 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. Message-ID: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> Hi everyone, I have been working on some fixes to the functions in scipy.stats which calculate variance/error and related quantities. In particular, in order to comply with the deprecation warnings that appear in use of scipy.stats.samplevar/scipy.stats.samplestd, I have replaced use of these functions with calls to np.std/np.var. I have also cleaned up the documentation a bit. This can all be found here: http://codereview.appspot.com/141051 Cheers, Ariel -- Ariel Rokem Helen Wills Neuroscience Institute University of California, Berkeley http://argentum.ucbso.berkeley.edu/ariel From josef.pktd at gmail.com Sun Oct 25 23:16:35 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Sun, 25 Oct 2009 23:16:35 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> Message-ID: <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> On Sun, Oct 25, 2009 at 10:50 PM, Ariel Rokem wrote: > Hi everyone, > > I have been working on some fixes to the functions in scipy.stats > which calculate variance/error and related quantities. In particular, > in order to comply with the deprecation warnings that appear in use of > scipy.stats.samplevar/scipy.stats.samplestd, I have replaced use of > these functions with calls to np.std/np.var. I have also cleaned up > the documentation a bit. > > This can all be found here: http://codereview.appspot.com/141051 > > Cheers, > > Ariel I just gave it a quick look, looks good so far in def zs looks like a shape error for axis>0 "return (a-mu)/sigma" def zs changes definition, before it normalized with raveled mean, std not by axis - mu = np.mean(a,None) - sigma = samplestd(a) - return (array(a)-mu)/sigma + a,axis = _chk_asarray(a,axis) + mu = np.mean(a,axis) + sigma = np.std(a,axis) + return (a-mu)/sigma I never looked closely at these, zmap has a description I don't understand. z, zs, zm ??? Which is which? they look a bit inconsistent, population might refer to dof correction in z ? Is there a standard terminology for z scores? I think for axis, I have seen more "int or None" ? Josef > -- > Ariel Rokem > Helen Wills Neuroscience Institute > University of California, Berkeley > http://argentum.ucbso.berkeley.edu/ariel > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From arokem at berkeley.edu Sun Oct 25 23:49:16 2009 From: arokem at berkeley.edu (Ariel Rokem) Date: Sun, 25 Oct 2009 20:49:16 -0700 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> Message-ID: <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> Hi Josef and all, thank for looking. Concerning the z-score functions - I am also confused by those and I would suggest unifying them under one function. In particular, I can't imagine what the function 'z' is for. However, I don't want to just remove these without discussion. What do you think about this? Another, more general thing, concerning the axis - I am wondering: why is the default axis for scipy is 0, while the default for numpy (in np.mean, for example) is None? I think that it would be good to have one convention for both libraries. I think that the more parsimonious one is the one using "None" as the default value. This doesn't favor any of the dimensions of an array over others, by default. I don't know - how wide-spread is this convention within scipy? Cheers, Ariel On Sun, Oct 25, 2009 at 8:16 PM, wrote: > On Sun, Oct 25, 2009 at 10:50 PM, Ariel Rokem wrote: >> Hi everyone, >> >> I have been working on some fixes to the functions in scipy.stats >> which calculate variance/error and related quantities. In particular, >> in order to comply with the deprecation warnings that appear in use of >> scipy.stats.samplevar/scipy.stats.samplestd, I have replaced use of >> these functions with calls to np.std/np.var. I have also cleaned up >> the documentation a bit. >> >> This can all be found here: http://codereview.appspot.com/141051 >> >> Cheers, >> >> Ariel > > I just gave it a quick look, looks good so far > > in ?def zs ?looks like a shape error for axis>0 > "return (a-mu)/sigma" > > > def zs ? changes definition, before it normalized with raveled mean, > std not by axis > > - mu = np.mean(a,None) > - sigma = samplestd(a) > - return (array(a)-mu)/sigma > > + a,axis = _chk_asarray(a,axis) > + mu = np.mean(a,axis) > + sigma = np.std(a,axis) > + return (a-mu)/sigma > > I never looked closely at these, > zmap has a description I don't understand. > > z, zs, zm ???? > > Which is which? they look a bit inconsistent, population might refer > to dof correction in z ? > Is there a standard terminology for z scores? > > I think for axis, I have seen more "int or None" ? > > Josef > > > > >> -- >> Ariel Rokem >> Helen Wills Neuroscience Institute >> University of California, Berkeley >> http://argentum.ucbso.berkeley.edu/ariel >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -- Ariel Rokem Helen Wills Neuroscience Institute University of California, Berkeley http://argentum.ucbso.berkeley.edu/ariel From josef.pktd at gmail.com Mon Oct 26 00:19:54 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Mon, 26 Oct 2009 00:19:54 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> Message-ID: <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> On Sun, Oct 25, 2009 at 11:49 PM, Ariel Rokem wrote: > Hi Josef and all, > > thank for looking. Concerning the z-score functions - I am also > confused by those and I would suggest unifying them under one > function. In particular, I can't imagine what the function 'z' is for. > However, I don't want to just remove these without discussion. What do > you think about this? > > Another, more general thing, concerning the axis - I am wondering: why > is the default axis for scipy is 0, while the default for numpy (in > np.mean, for example) is None? I think that it would be good to have > one convention for both libraries. I think that the more parsimonious > one is the one using "None" as the default value. This doesn't favor > any of the dimensions of an array over others, by default. I don't > know - how wide-spread is this convention within scipy? I had to run after the last message. My impression was that maybe in one of the changes the ddof=1 got lost, i.e. the distinction that was in scipy stats for population versus sample statistics. z and zmap look the same to me from the intended (?) calculation but zmap mixes up the axis arguments. (mean with "axis", std with hardcoded axis=0). Maybe the intention will be clearer when I look at the trac history or the original stats package. >From looking at the three function, I would assume that the combined function would have a signature like def zscore(a, compare=None, axis=0, ddof=0) or two functions, one with compare, one without ? About default axis=0: I think this is scipy.stats specific. We had a brief discussion a year ago, where Jarrod agreed that default for stats should remain axis=0. In statistics, you almost never want to ravel data, not mixing apples and cars, or prices and quantities. So the default should be reducing along an axis, e.g. mean over all observations by variable. axis=0 versus axis=-1, this is traditional in statistics/econometrics. Both from other matrix packages (gauss, matlab) and from the textbook treatment (of books that I know). Switching to -1 for the data would be a big mental break and would require axis translation of the textbook formulas, e.g solve X'X beta = X'Y >From my perspective loosing axis=0 as default is the main disadvantage of removing mean, var, and so on, from scipy.stats. eg. I need to create a lambda function if I want mean(x, axis=0) as a callback function. Cheers, Josef > > Cheers, > > Ariel > > On Sun, Oct 25, 2009 at 8:16 PM, ? wrote: >> On Sun, Oct 25, 2009 at 10:50 PM, Ariel Rokem wrote: >>> Hi everyone, >>> >>> I have been working on some fixes to the functions in scipy.stats >>> which calculate variance/error and related quantities. In particular, >>> in order to comply with the deprecation warnings that appear in use of >>> scipy.stats.samplevar/scipy.stats.samplestd, I have replaced use of >>> these functions with calls to np.std/np.var. I have also cleaned up >>> the documentation a bit. >>> >>> This can all be found here: http://codereview.appspot.com/141051 >>> >>> Cheers, >>> >>> Ariel >> >> I just gave it a quick look, looks good so far >> >> in ?def zs ?looks like a shape error for axis>0 >> "return (a-mu)/sigma" >> >> >> def zs ? changes definition, before it normalized with raveled mean, >> std not by axis >> >> - mu = np.mean(a,None) >> - sigma = samplestd(a) >> - return (array(a)-mu)/sigma >> >> + a,axis = _chk_asarray(a,axis) >> + mu = np.mean(a,axis) >> + sigma = np.std(a,axis) >> + return (a-mu)/sigma >> >> I never looked closely at these, >> zmap has a description I don't understand. >> >> z, zs, zm ???? >> >> Which is which? they look a bit inconsistent, population might refer >> to dof correction in z ? >> Is there a standard terminology for z scores? >> >> I think for axis, I have seen more "int or None" ? >> >> Josef >> >> >> >> >>> -- >>> Ariel Rokem >>> Helen Wills Neuroscience Institute >>> University of California, Berkeley >>> http://argentum.ucbso.berkeley.edu/ariel >>> _______________________________________________ >>> Scipy-dev mailing list >>> Scipy-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > > > > -- > Ariel Rokem > Helen Wills Neuroscience Institute > University of California, Berkeley > http://argentum.ucbso.berkeley.edu/ariel > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From josef.pktd at gmail.com Mon Oct 26 00:59:45 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Mon, 26 Oct 2009 00:59:45 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> Message-ID: <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> On Mon, Oct 26, 2009 at 12:19 AM, wrote: > On Sun, Oct 25, 2009 at 11:49 PM, Ariel Rokem wrote: >> Hi Josef and all, >> >> thank for looking. Concerning the z-score functions - I am also >> confused by those and I would suggest unifying them under one >> function. In particular, I can't imagine what the function 'z' is for. >> However, I don't want to just remove these without discussion. What do >> you think about this? >> >> Another, more general thing, concerning the axis - I am wondering: why >> is the default axis for scipy is 0, while the default for numpy (in >> np.mean, for example) is None? I think that it would be good to have >> one convention for both libraries. I think that the more parsimonious >> one is the one using "None" as the default value. This doesn't favor >> any of the dimensions of an array over others, by default. I don't >> know - how wide-spread is this convention within scipy? > > I had to run after the last message. My impression was that maybe in > one of the changes the ddof=1 got lost, i.e. the distinction that was > in scipy stats for population versus sample statistics. > z and zmap look the same to me from the intended (?) calculation > but zmap mixes up the axis arguments. (mean with "axis", std with > hardcoded axis=0). Maybe the intention will be clearer when I look > at the trac history or the original stats package. > > From looking at the three function, I would assume that the combined > function would have a signature like > > def zscore(a, compare=None, axis=0, ddof=0) > > or two functions, one with compare, one without ? see: http://projects.scipy.org/scipy/browser/trunk/Lib/stats/stats.py?rev=2028#L1174 zs was the list version for the zscore using z to calculate, the translation in the next changeset is correct only for 1d or raveled arrays, but it is missing an axis argument. It looks like z was a helper function for a scalar score. zmap got imported in this form in revision 71. stats.mstats has the same functions, but they look like literal translations since they have the same (ambiguous) treatment of axis if it's not 1d. stats.mstats.z has ddof=1, the others ddof=0 With broadcasting and adjustment of the dimension of min and std, only a single score function seems necessary, the current functions look a bit like historical relics. Josef > > > About default axis=0: > > I think this is scipy.stats specific. We had a brief discussion a year > ago, where Jarrod agreed that default for stats should remain axis=0. > > In statistics, you almost never want to ravel data, not mixing apples > and cars, or prices and quantities. So the default should be reducing > along an axis, e.g. mean over all observations by variable. > > axis=0 versus axis=-1, this is traditional in statistics/econometrics. Both > from other matrix packages (gauss, matlab) and from the textbook > treatment (of books that I know). Switching to -1 for the data would > be a big mental break and would require axis translation of the > textbook formulas, e.g solve X'X beta = X'Y > > From my perspective loosing axis=0 as default is the main disadvantage > of removing mean, var, and so on, from scipy.stats. eg. I need to create > a lambda function if I want mean(x, axis=0) as a callback function. > > Cheers, > > Josef > >> >> Cheers, >> >> Ariel >> >> On Sun, Oct 25, 2009 at 8:16 PM, ? wrote: >>> On Sun, Oct 25, 2009 at 10:50 PM, Ariel Rokem wrote: >>>> Hi everyone, >>>> >>>> I have been working on some fixes to the functions in scipy.stats >>>> which calculate variance/error and related quantities. In particular, >>>> in order to comply with the deprecation warnings that appear in use of >>>> scipy.stats.samplevar/scipy.stats.samplestd, I have replaced use of >>>> these functions with calls to np.std/np.var. I have also cleaned up >>>> the documentation a bit. >>>> >>>> This can all be found here: http://codereview.appspot.com/141051 >>>> >>>> Cheers, >>>> >>>> Ariel >>> >>> I just gave it a quick look, looks good so far >>> >>> in ?def zs ?looks like a shape error for axis>0 >>> "return (a-mu)/sigma" >>> >>> >>> def zs ? changes definition, before it normalized with raveled mean, >>> std not by axis >>> >>> - mu = np.mean(a,None) >>> - sigma = samplestd(a) >>> - return (array(a)-mu)/sigma >>> >>> + a,axis = _chk_asarray(a,axis) >>> + mu = np.mean(a,axis) >>> + sigma = np.std(a,axis) >>> + return (a-mu)/sigma >>> >>> I never looked closely at these, >>> zmap has a description I don't understand. >>> >>> z, zs, zm ???? >>> >>> Which is which? they look a bit inconsistent, population might refer >>> to dof correction in z ? >>> Is there a standard terminology for z scores? >>> >>> I think for axis, I have seen more "int or None" ? >>> >>> Josef >>> >>> >>> >>> >>>> -- >>>> Ariel Rokem >>>> Helen Wills Neuroscience Institute >>>> University of California, Berkeley >>>> http://argentum.ucbso.berkeley.edu/ariel >>>> _______________________________________________ >>>> Scipy-dev mailing list >>>> Scipy-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>>> >>> _______________________________________________ >>> Scipy-dev mailing list >>> Scipy-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>> >> >> >> >> -- >> Ariel Rokem >> Helen Wills Neuroscience Institute >> University of California, Berkeley >> http://argentum.ucbso.berkeley.edu/ariel >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > From josef.pktd at gmail.com Mon Oct 26 01:13:04 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Mon, 26 Oct 2009 01:13:04 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> Message-ID: <1cd32cbb0910252213m720e9f8er6ca2ef655089bde@mail.gmail.com> On Mon, Oct 26, 2009 at 12:59 AM, wrote: > On Mon, Oct 26, 2009 at 12:19 AM, ? wrote: >> On Sun, Oct 25, 2009 at 11:49 PM, Ariel Rokem wrote: >>> Hi Josef and all, >>> >>> thank for looking. Concerning the z-score functions - I am also >>> confused by those and I would suggest unifying them under one >>> function. In particular, I can't imagine what the function 'z' is for. >>> However, I don't want to just remove these without discussion. What do >>> you think about this? >>> >>> Another, more general thing, concerning the axis - I am wondering: why >>> is the default axis for scipy is 0, while the default for numpy (in >>> np.mean, for example) is None? I think that it would be good to have >>> one convention for both libraries. I think that the more parsimonious >>> one is the one using "None" as the default value. This doesn't favor >>> any of the dimensions of an array over others, by default. I don't >>> know - how wide-spread is this convention within scipy? >> >> I had to run after the last message. My impression was that maybe in >> one of the changes the ddof=1 got lost, i.e. the distinction that was >> in scipy stats for population versus sample statistics. >> z and zmap look the same to me from the intended (?) calculation >> but zmap mixes up the axis arguments. (mean with "axis", std with >> hardcoded axis=0). Maybe the intention will be clearer when I look >> at the trac history or the original stats package. >> >> From looking at the three function, I would assume that the combined >> function would have a signature like >> >> def zscore(a, compare=None, axis=0, ddof=0) >> >> or two functions, one with compare, one without ? > > see: > http://projects.scipy.org/scipy/browser/trunk/Lib/stats/stats.py?rev=2028#L1174 > > zs was the list version for the zscore using z to calculate, the translation in > the next changeset is correct only for 1d or raveled arrays, but it is missing > an axis argument. It looks like z was a helper function for a scalar score. > zmap got imported in this form in revision 71. > > stats.mstats has the same functions, but they look like literal translations > since they have the same (ambiguous) treatment of axis if it's not 1d. > stats.mstats.z has ddof=1, the others ddof=0 > > With broadcasting and adjustment of the dimension of min and std, only > a single score function seems necessary, the current functions look a bit > like historical relics. > > Josef > >> >> >> About default axis=0: >> >> I think this is scipy.stats specific. We had a brief discussion a year >> ago, where Jarrod agreed that default for stats should remain axis=0. >> >> In statistics, you almost never want to ravel data, not mixing apples >> and cars, or prices and quantities. So the default should be reducing >> along an axis, e.g. mean over all observations by variable. >> >> axis=0 versus axis=-1, this is traditional in statistics/econometrics. Both >> from other matrix packages (gauss, matlab) and from the textbook >> treatment (of books that I know). Switching to -1 for the data would >> be a big mental break and would require axis translation of the >> textbook formulas, e.g solve X'X beta = X'Y >> >> From my perspective loosing axis=0 as default is the main disadvantage >> of removing mean, var, and so on, from scipy.stats. eg. I need to create >> a lambda function if I want mean(x, axis=0) as a callback function. digging a bit in the history, switch from axis=-1 to axis=0: "Fixed functions in stats.py to have default axis 0" http://projects.scipy.org/scipy/changeset/1465 Josef >> >> Cheers, >> >> Josef >> >>> >>> Cheers, >>> >>> Ariel >>> >>> On Sun, Oct 25, 2009 at 8:16 PM, ? wrote: >>>> On Sun, Oct 25, 2009 at 10:50 PM, Ariel Rokem wrote: >>>>> Hi everyone, >>>>> >>>>> I have been working on some fixes to the functions in scipy.stats >>>>> which calculate variance/error and related quantities. In particular, >>>>> in order to comply with the deprecation warnings that appear in use of >>>>> scipy.stats.samplevar/scipy.stats.samplestd, I have replaced use of >>>>> these functions with calls to np.std/np.var. I have also cleaned up >>>>> the documentation a bit. >>>>> >>>>> This can all be found here: http://codereview.appspot.com/141051 >>>>> >>>>> Cheers, >>>>> >>>>> Ariel >>>> >>>> I just gave it a quick look, looks good so far >>>> >>>> in ?def zs ?looks like a shape error for axis>0 >>>> "return (a-mu)/sigma" >>>> >>>> >>>> def zs ? changes definition, before it normalized with raveled mean, >>>> std not by axis >>>> >>>> - mu = np.mean(a,None) >>>> - sigma = samplestd(a) >>>> - return (array(a)-mu)/sigma >>>> >>>> + a,axis = _chk_asarray(a,axis) >>>> + mu = np.mean(a,axis) >>>> + sigma = np.std(a,axis) >>>> + return (a-mu)/sigma >>>> >>>> I never looked closely at these, >>>> zmap has a description I don't understand. >>>> >>>> z, zs, zm ???? >>>> >>>> Which is which? they look a bit inconsistent, population might refer >>>> to dof correction in z ? >>>> Is there a standard terminology for z scores? >>>> >>>> I think for axis, I have seen more "int or None" ? >>>> >>>> Josef >>>> >>>> >>>> >>>> >>>>> -- >>>>> Ariel Rokem >>>>> Helen Wills Neuroscience Institute >>>>> University of California, Berkeley >>>>> http://argentum.ucbso.berkeley.edu/ariel >>>>> _______________________________________________ >>>>> Scipy-dev mailing list >>>>> Scipy-dev at scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>>>> >>>> _______________________________________________ >>>> Scipy-dev mailing list >>>> Scipy-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>>> >>> >>> >>> >>> -- >>> Ariel Rokem >>> Helen Wills Neuroscience Institute >>> University of California, Berkeley >>> http://argentum.ucbso.berkeley.edu/ariel >>> _______________________________________________ >>> Scipy-dev mailing list >>> Scipy-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>> >> > From pgmdevlist at gmail.com Mon Oct 26 01:18:09 2009 From: pgmdevlist at gmail.com (Pierre GM) Date: Mon, 26 Oct 2009 01:18:09 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> Message-ID: <56E37DA4-4F34-4E08-B064-F5D40F4CBCB4@gmail.com> On Oct 26, 2009, at 12:59 AM, josef.pktd at gmail.com wrote: > > zs was the list version for the zscore using z to calculate, the > translation in > the next changeset is correct only for 1d or raveled arrays, but it > is missing > an axis argument. It looks like z was a helper function for a scalar > score. > zmap got imported in this form in revision 71. > > stats.mstats has the same functions, but they look like literal > translations > since they have the same (ambiguous) treatment of axis if it's not 1d. > stats.mstats.z has ddof=1, the others ddof=0 well, maybe it's time to start cleaning up mstats. For the z functions, that should be straightforward, provided we don't lose the mask with np.asarray (a np.asanyarray would be sufficient). In that case, we could probably drop support for them in mstats. At least, we should make sure that the mstats versions have the same defaults as the stats ones. From arokem at berkeley.edu Mon Oct 26 01:31:30 2009 From: arokem at berkeley.edu (Ariel Rokem) Date: Sun, 25 Oct 2009 22:31:30 -0700 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> Message-ID: <43958ee60910252231u158857e4m88e8113879d8703b@mail.gmail.com> Hi Josef - > > >From looking at the three function, I would assume that the combined > function would have a signature like > > def zscore(a, compare=None, axis=0, ddof=0) > > or two functions, one with compare, one without ? Yes - I think that would be best. After all, someone wrote zmap with some usecase in mind (I assume), so we would still want that functionality to live on explicitly. So, I suggest (see attached diff) to have two functions: one will be zscore and the other would be zscore_compare. In the attached diff, I have decorated all these functions with a deprecation warning and added these two new functions, zscore (with the new, by-axis behavior. This makes more sense to me, somehow) and zscore_compare. > > > About default axis=0: > ... Thanks for the explanation and for digging into the history of this. I still think that in the long run it would be preferable to have these things be internally consistent (that is consistent between numpy and scipy), rather than consistent with other tools. Finally - I have tried to combine sem and stderr into one function, under sem. Notice in particular the correction for ddof. My understanding is that this should produce per default the result std/sqrt(n-1), which is what we usually want for the sem. Is that correct? Cheers, Ariel -------------- next part -------------- A non-text attachment was scrubbed... Name: stats_var_cleanup102509_b.diff Type: application/octet-stream Size: 7597 bytes Desc: not available URL: From josef.pktd at gmail.com Mon Oct 26 01:32:49 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Mon, 26 Oct 2009 01:32:49 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <56E37DA4-4F34-4E08-B064-F5D40F4CBCB4@gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> <56E37DA4-4F34-4E08-B064-F5D40F4CBCB4@gmail.com> Message-ID: <1cd32cbb0910252232w26a5ebdcxd0522fde66074fb1@mail.gmail.com> On Mon, Oct 26, 2009 at 1:18 AM, Pierre GM wrote: > > On Oct 26, 2009, at 12:59 AM, josef.pktd at gmail.com wrote: >> >> zs was the list version for the zscore using z to calculate, the >> translation in >> the next changeset is correct only for 1d or raveled arrays, but it >> is missing >> an axis argument. It looks like z was a helper function for a scalar >> score. >> zmap got imported in this form in revision 71. >> >> stats.mstats has the same functions, but they look like literal >> translations >> since they have the same (ambiguous) treatment of axis if it's not 1d. >> stats.mstats.z has ddof=1, the others ddof=0 > > well, maybe it's time to start cleaning up mstats. For the z > functions, that should be straightforward, provided we don't lose the > mask with np.asarray (a np.asanyarray would be sufficient). In that > case, we could probably drop support for them in mstats. At least, we > should make sure that the mstats versions have the same defaults as > the stats ones. I was thinking about whether we can use a version of _chk_asarray with _chk_asanyarray that preserves the subclass and then use only methods in the function, a.mean, a.std,... for the functions that could handle different subclasses. asanyarray also preserves matrices, so I don't know yet how the dimension handling works with matrices. I never use them. And we would have to watch out with multiplication. BTW: I would prefer "standardize" to zscore as a name. It's more informative, than zscore (and similar for zprob) But, I don't care too much once the docstring tells what the function actually does. Josef > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From arokem at berkeley.edu Mon Oct 26 01:32:25 2009 From: arokem at berkeley.edu (Ariel Rokem) Date: Sun, 25 Oct 2009 22:32:25 -0700 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <56E37DA4-4F34-4E08-B064-F5D40F4CBCB4@gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> <56E37DA4-4F34-4E08-B064-F5D40F4CBCB4@gmail.com> Message-ID: <43958ee60910252232hb0d496ag8880f31c76a7d2bc@mail.gmail.com> Hi Pierre, I agree - let's see how we work things out for stats and then, we can copy over whatever behavior we settle on to mstats as well. Cheers, Ariel On Sun, Oct 25, 2009 at 10:18 PM, Pierre GM wrote: > > On Oct 26, 2009, at 12:59 AM, josef.pktd at gmail.com wrote: >> >> zs was the list version for the zscore using z to calculate, the >> translation in >> the next changeset is correct only for 1d or raveled arrays, but it >> is missing >> an axis argument. It looks like z was a helper function for a scalar >> score. >> zmap got imported in this form in revision 71. >> >> stats.mstats has the same functions, but they look like literal >> translations >> since they have the same (ambiguous) treatment of axis if it's not 1d. >> stats.mstats.z has ddof=1, the others ddof=0 > > well, maybe it's time to start cleaning up mstats. For the z > functions, that should be straightforward, provided we don't lose the > mask with np.asarray (a np.asanyarray would be sufficient). In that > case, we could probably drop support for them in mstats. At least, we > should make sure that the mstats versions have the same defaults as > the stats ones. > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -- Ariel Rokem Helen Wills Neuroscience Institute University of California, Berkeley http://argentum.ucbso.berkeley.edu/ariel From arokem at berkeley.edu Mon Oct 26 01:40:44 2009 From: arokem at berkeley.edu (Ariel Rokem) Date: Sun, 25 Oct 2009 22:40:44 -0700 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <1cd32cbb0910252232w26a5ebdcxd0522fde66074fb1@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> <56E37DA4-4F34-4E08-B064-F5D40F4CBCB4@gmail.com> <1cd32cbb0910252232w26a5ebdcxd0522fde66074fb1@mail.gmail.com> Message-ID: <43958ee60910252240y77fc1c3fs515c409483f6e748@mail.gmail.com> Hi - by the way - I also put the recent diff, which I attached as an additional patch here: http://codereview.appspot.com/141051 Cheers, Ariel On Sun, Oct 25, 2009 at 10:32 PM, wrote: > On Mon, Oct 26, 2009 at 1:18 AM, Pierre GM wrote: >> >> On Oct 26, 2009, at 12:59 AM, josef.pktd at gmail.com wrote: >>> >>> zs was the list version for the zscore using z to calculate, the >>> translation in >>> the next changeset is correct only for 1d or raveled arrays, but it >>> is missing >>> an axis argument. It looks like z was a helper function for a scalar >>> score. >>> zmap got imported in this form in revision 71. >>> >>> stats.mstats has the same functions, but they look like literal >>> translations >>> since they have the same (ambiguous) treatment of axis if it's not 1d. >>> stats.mstats.z has ddof=1, the others ddof=0 >> > >> well, maybe it's time to start cleaning up mstats. For the z >> functions, that should be straightforward, provided we don't lose the >> mask with np.asarray (a np.asanyarray would be sufficient). In that >> case, we could probably drop support for them in mstats. At least, we >> should make sure that the mstats versions have the same defaults as >> the stats ones. > > I was thinking about whether we can use a version of _chk_asarray > with _chk_asanyarray ?that preserves the subclass and then use only > methods in the function, a.mean, a.std,... for the functions that could > handle different subclasses. > > asanyarray also preserves matrices, so I don't know yet how the > dimension handling works with matrices. I never use them. > And we would have to watch out with multiplication. > > > BTW: I would prefer "standardize" to zscore as a name. > It's more informative, than zscore (and similar for zprob) > But, I don't care too much once the docstring tells what the > function actually does. > > > Josef > > > >> >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -- Ariel Rokem Helen Wills Neuroscience Institute University of California, Berkeley http://argentum.ucbso.berkeley.edu/ariel From josef.pktd at gmail.com Mon Oct 26 01:51:44 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Mon, 26 Oct 2009 01:51:44 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <43958ee60910252231u158857e4m88e8113879d8703b@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <43958ee60910252231u158857e4m88e8113879d8703b@mail.gmail.com> Message-ID: <1cd32cbb0910252251i52700df5q99351db99c09f8cd@mail.gmail.com> On Mon, Oct 26, 2009 at 1:31 AM, Ariel Rokem wrote: > Hi Josef - > >> >> >From looking at the three function, I would assume that the combined >> function would have a signature like >> >> def zscore(a, compare=None, axis=0, ddof=0) >> >> or two functions, one with compare, one without ? > > Yes - I think that would be best. After all, someone wrote zmap with > some usecase in mind (I assume), so we would still want that > functionality to live on explicitly. So, I suggest (see attached diff) > to have two functions: one will be zscore and the other would be > zscore_compare. In the attached diff, I have decorated all these > functions with a deprecation warning and added these two new > functions, zscore (with the new, by-axis behavior. This makes more > sense to me, somehow) and zscore_compare. > >> >> >> About default axis=0: >> > ... > > Thanks for the explanation and for digging into the history of this. I > still think that in the long run it would be preferable to have these > things be internally consistent (that is consistent between numpy and > scipy), rather than consistent with other tools. > > Finally - I have tried to combine sem and stderr into one function, > under sem. Notice in particular the correction for ddof. My > understanding is that this should produce per default the result > std/sqrt(n-1), which is what we usually want for the sem. Is that > correct? Yes, I had to check the ttests, that's when I spend more time checking the degrees of freedom. It looks like the denominator needs one "n" and one "n-1" v = np.var(a, axis, ddof=1) t = d / np.sqrt(v/float(n)) sem(a, ddof=1, axis=0) should have ddof as last argument to match np.var. your axis handling is still incorrect in zscore for 2d arrays if axis=1 then we need to add an axis a.mean(1)[:,None] there is a function in numpy to do this, expand_axis (?) that works for general axis. There was also a recent discussion on the numpy list for getting the axis back after a reduce. Josef > > ?Cheers, > > Ariel > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From josef.pktd at gmail.com Mon Oct 26 02:07:09 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Mon, 26 Oct 2009 02:07:09 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <1cd32cbb0910252251i52700df5q99351db99c09f8cd@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <43958ee60910252231u158857e4m88e8113879d8703b@mail.gmail.com> <1cd32cbb0910252251i52700df5q99351db99c09f8cd@mail.gmail.com> Message-ID: <1cd32cbb0910252307j3546e6fav511b2e87cfd25fa@mail.gmail.com> On Mon, Oct 26, 2009 at 1:51 AM, wrote: > On Mon, Oct 26, 2009 at 1:31 AM, Ariel Rokem wrote: >> Hi Josef - >> >>> >>> >From looking at the three function, I would assume that the combined >>> function would have a signature like >>> >>> def zscore(a, compare=None, axis=0, ddof=0) >>> >>> or two functions, one with compare, one without ? >> >> Yes - I think that would be best. After all, someone wrote zmap with >> some usecase in mind (I assume), so we would still want that >> functionality to live on explicitly. So, I suggest (see attached diff) >> to have two functions: one will be zscore and the other would be >> zscore_compare. In the attached diff, I have decorated all these >> functions with a deprecation warning and added these two new >> functions, zscore (with the new, by-axis behavior. This makes more >> sense to me, somehow) and zscore_compare. >> >>> >>> >>> About default axis=0: >>> >> ... >> >> Thanks for the explanation and for digging into the history of this. I >> still think that in the long run it would be preferable to have these >> things be internally consistent (that is consistent between numpy and >> scipy), rather than consistent with other tools. >> >> Finally - I have tried to combine sem and stderr into one function, >> under sem. Notice in particular the correction for ddof. My >> understanding is that this should produce per default the result >> std/sqrt(n-1), which is what we usually want for the sem. Is that >> correct? > > > Yes, I had to check the ttests, that's when I spend more time checking the > degrees of freedom. It looks like the denominator needs one "n" and one > "n-1" > > ?v = np.var(a, axis, ddof=1) > ?t = d / np.sqrt(v/float(n)) > > sem(a, ddof=1, axis=0) should have ddof as last argument to match np.var. > > your axis handling is still incorrect in zscore for 2d arrays > > if axis=1 then we need to add an axis > a.mean(1)[:,None] > > there is a function in numpy to do this, expand_axis (?) that > works for general axis. There was also a recent discussion > on the numpy list for getting the axis back after a reduce. (codereview is much easier to read than a diff wordpad) for 2 arrays as in zscore_compare, you can also use _chk2_asarray zscore_compare should match the axis argument of zscore, I think. Cheers (and I'm off) Josef > > Josef > > > >> >> ?Cheers, >> >> Ariel >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > From ralf.gommers at googlemail.com Mon Oct 26 05:17:25 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Mon, 26 Oct 2009 10:17:25 +0100 Subject: [SciPy-dev] Documenting function-like classes In-Reply-To: References: <45d1ab480910251437s3f4a9f98gee2304718a0695d2@mail.gmail.com> <45d1ab480910251846y5f3debd2p1df19ae2f783c8a9@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 3:20 AM, Anne Archibald wrote: > Well, IMHO, these objects are classes, and so should be documented as > such. The function-like documentation should be the documentation for > __call__. > > That said, I haven't been working on the docstrings, so I don't know > how people have been documenting classes; it seems to me there's > always some question of what goes in the class docstring and what goes > in the method docstrings. In the case of weirdly-named methods like > __call__ and __init__, one has to be sure that the user can find them > and knows how to call them, but that's a problem when documenting all > classes. > We decided a while ago that the parameters for __init__ go in the class docstring, not in __init__. Only public methods get substantial documentation in their own docstrings, and get their own wiki page. __call__ should be the same as __init__ IMO. Same for __getitem__ and similar such things. They can be clarified with examples without mentioning them by name. See for example poly1d (__call__) or mgrid (__getitem__): http://docs.scipy.org/numpy/docs/numpy.lib.polynomial.poly1d/ http://docs.scipy.org/numpy/docs/numpy.lib.index_tricks.mgrid/ We should not expect most users to know, or care much about, the details of how such special (double-underscore) class methods work. Cheers, Ralf > I would guess that the confusion is arising because you're looking at > classes that don't really do much other than act like functions? Maybe > work through something like poly1d that has substantial data and other > methods first. > > Anne > > > DG > > > > > > _______________________________________________ > > Scipy-dev mailing list > > Scipy-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdroe at stsci.edu Mon Oct 26 09:23:30 2009 From: mdroe at stsci.edu (Michael Droettboom) Date: Mon, 26 Oct 2009 09:23:30 -0400 Subject: [SciPy-dev] chararray docstrings In-Reply-To: <45d1ab480910251411s118703een44bfb1de299787b2@mail.gmail.com> References: <4AD34DFF.6000809@stsci.edu> <1cd32cbb0910130859o6f010fe4g49cee0dbf0cfda46@mail.gmail.com> <4AD4A58A.5070308@stsci.edu> <20091013165559.GA20164@phare.normalesup.org> <45d1ab480910131448u6547db11xe2706f05eb14a09b@mail.gmail.com> <4AD5D6A6.2080209@stsci.edu> <45d1ab480910251315k63ae297fo58468cde4a0e67e3@mail.gmail.com> <45d1ab480910251411s118703een44bfb1de299787b2@mail.gmail.com> Message-ID: <4AE5A2D2.4080107@stsci.edu> Just seeing this thread now. I'll make these "array_like of string_ or unicode_" -> "array_like of str or unicode" changes in the source and commit to SVN. Thanks for the heads up. Mike David Goldsmith wrote: > On Sun, Oct 25, 2009 at 1:35 PM, Ralf Gommers > > wrote: > > > > On Sun, Oct 25, 2009 at 9:15 PM, David Goldsmith > > wrote: > > On Sun, Oct 25, 2009 at 3:44 AM, Ralf Gommers > > wrote: > > I had a look at all the new chararray docstrings and it > all looks pretty good, thanks! There is one minor thing > that requires some mass-editing: the type description > "array_like of string_ or unicode_" needs to be > "array_like of str or unicode". Also similar usage of > string_ and unicode_ elsewhere. The trailing backslash is > Sphinx markup for a link, so they all turn into broken links. > > > I imagine this problem is even more widespread - I can take > care of it (or what remains of it). > > Well, you have to visit ~50 wiki pages. For Mike it's probably > some global replaces, and an SVN commit..... > > > My thinking is that global replaces will be "dangerous" in that > they'll replace the word string where that *is* what's wanted (e.g., > in a narrative). That said, I'll take care of non-chararray > occurrences first and if Mike hasn't gotten to the chararray/"new > chararray" occurrences by the time I'm done, I'll start in on those. > > DG > > > > Ralf > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > ------------------------------------------------------------------------ > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA From josef.pktd at gmail.com Mon Oct 26 09:58:53 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Mon, 26 Oct 2009 09:58:53 -0400 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <1cd32cbb0910252307j3546e6fav511b2e87cfd25fa@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <43958ee60910252231u158857e4m88e8113879d8703b@mail.gmail.com> <1cd32cbb0910252251i52700df5q99351db99c09f8cd@mail.gmail.com> <1cd32cbb0910252307j3546e6fav511b2e87cfd25fa@mail.gmail.com> Message-ID: <1cd32cbb0910260658y1ded1101m85a15c093a3e601b@mail.gmail.com> On Mon, Oct 26, 2009 at 2:07 AM, wrote: > On Mon, Oct 26, 2009 at 1:51 AM, ? wrote: >> On Mon, Oct 26, 2009 at 1:31 AM, Ariel Rokem wrote: >>> Hi Josef - >>> >>>> >>>> >From looking at the three function, I would assume that the combined >>>> function would have a signature like >>>> >>>> def zscore(a, compare=None, axis=0, ddof=0) >>>> >>>> or two functions, one with compare, one without ? >>> >>> Yes - I think that would be best. After all, someone wrote zmap with >>> some usecase in mind (I assume), so we would still want that >>> functionality to live on explicitly. So, I suggest (see attached diff) >>> to have two functions: one will be zscore and the other would be >>> zscore_compare. In the attached diff, I have decorated all these >>> functions with a deprecation warning and added these two new >>> functions, zscore (with the new, by-axis behavior. This makes more >>> sense to me, somehow) and zscore_compare. >>> >>>> >>>> >>>> About default axis=0: >>>> >>> ... >>> >>> Thanks for the explanation and for digging into the history of this. I >>> still think that in the long run it would be preferable to have these >>> things be internally consistent (that is consistent between numpy and >>> scipy), rather than consistent with other tools. I hope the "long run" is very long. axis=0 as default is consistent with views to structured arrays or recarrays and to reading data from a csv file in the common orientation (genfromtxt). I think, it's more a convention for data analysis than a questionn whether you work with a package that has c orientation instead of fortran orientation. One exception to this is panel data, especially with a 3d array. axis=-1 looks like a pain because it doesn't stay fixed when I add a third dimension. axis=None is pretty much useless with a dataset of variables with mixed units. Practicality beats Purity (especially if Purity is defined by the software and not the problem) and (almost) all of statsmodels is based on variables are columns. Josef >>> Finally - I have tried to combine sem and stderr into one function, >>> under sem. Notice in particular the correction for ddof. My >>> understanding is that this should produce per default the result >>> std/sqrt(n-1), which is what we usually want for the sem. Is that >>> correct? >> >> >> Yes, I had to check the ttests, that's when I spend more time checking the >> degrees of freedom. It looks like the denominator needs one "n" and one >> "n-1" >> >> ?v = np.var(a, axis, ddof=1) >> ?t = d / np.sqrt(v/float(n)) >> >> sem(a, ddof=1, axis=0) should have ddof as last argument to match np.var. >> >> your axis handling is still incorrect in zscore for 2d arrays >> >> if axis=1 then we need to add an axis >> a.mean(1)[:,None] >> >> there is a function in numpy to do this, expand_axis (?) that >> works for general axis. There was also a recent discussion >> on the numpy list for getting the axis back after a reduce. > > (codereview is much easier to read than a diff wordpad) > > for 2 arrays as in zscore_compare, you can also use _chk2_asarray > zscore_compare should match the axis argument of zscore, I think. > > Cheers (and I'm off) > > Josef > > >> >> Josef >> >> >> >>> >>> ?Cheers, >>> >>> Ariel >>> >>> _______________________________________________ >>> Scipy-dev mailing list >>> Scipy-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/scipy-dev >>> >>> >> > From bsouthey at gmail.com Mon Oct 26 12:39:15 2009 From: bsouthey at gmail.com (Bruce Southey) Date: Mon, 26 Oct 2009 11:39:15 -0500 Subject: [SciPy-dev] RFR: Proposed fixes in scipy.stats functions for calculation of variance/error/etc. In-Reply-To: <43958ee60910252232hb0d496ag8880f31c76a7d2bc@mail.gmail.com> References: <43958ee60910251950y1ca8ed4dnf6e8dbd504d67abb@mail.gmail.com> <1cd32cbb0910252016x1bba48b9mfbd99f3e69e7214e@mail.gmail.com> <43958ee60910252049q305b6ae6xa5550d505fe50a8f@mail.gmail.com> <1cd32cbb0910252119t5abf949aq450728dec9aa160c@mail.gmail.com> <1cd32cbb0910252159l666c8f12n867499bfa558fe60@mail.gmail.com> <56E37DA4-4F34-4E08-B064-F5D40F4CBCB4@gmail.com> <43958ee60910252232hb0d496ag8880f31c76a7d2bc@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 12:32 AM, Ariel Rokem wrote: > Hi Pierre, > > I agree - let's see how we work things out for stats and then, we can > copy over whatever behavior we settle on to mstats as well. > > Cheers, > > Ariel > > On Sun, Oct 25, 2009 at 10:18 PM, Pierre GM wrote: >> >> On Oct 26, 2009, at 12:59 AM, josef.pktd at gmail.com wrote: >>> >>> zs was the list version for the zscore using z to calculate, the >>> translation in >>> the next changeset is correct only for 1d or raveled arrays, but it >>> is missing >>> an axis argument. It looks like z was a helper function for a scalar >>> score. >>> zmap got imported in this form in revision 71. >>> >>> stats.mstats has the same functions, but they look like literal >>> translations >>> since they have the same (ambiguous) treatment of axis if it's not 1d. >>> stats.mstats.z has ddof=1, the others ddof=0 >> >> well, maybe it's time to start cleaning up mstats. For the z >> functions, that should be straightforward, provided we don't lose the >> mask with np.asarray (a np.asanyarray would be sufficient). In that >> case, we could probably drop support for them in mstats. At least, we >> should make sure that the mstats versions have the same defaults as >> the stats ones. >> >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > > > > -- > Ariel Rokem > Helen Wills Neuroscience Institute > University of California, Berkeley > http://argentum.ucbso.berkeley.edu/ariel > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > Hi, There is no difference between stats.z, stats.zs and stats.zmap in whether or not the input array is masked or not. So there is no need to have different versions for every type of ndarray. So please create a single version that works with at least the 'standard' array and masked array. I would agree to have a standardization function such as stats.zs() to return the standardized value or normal deviate or z-score function. But this need to allow standardization along the given axis. But I am not sure the utility of the other two functions as stats.zmap() is an extension of stats.z(). Ignoring axis, all these functions are of the form: (input-constant)/standard deviation Where value is a array-like input that can be converted into an array so can be a scalar or array constant is some constant used to center the input. It can be the mean of the input or some user-supplied value such as a scalar or computed from an array-like input. standard deviation is some constant to 'standardize' the input (not sure of the correct terminology). It can be the standard deviation of the input or some user-supplied value such as a scalar or computed from an array-like input. So current three functions are: stats.zs(a): input='a', constant=a.mean() and standard deviation =a.std() stats.z(a, score): input=score, constant=a.mean() and standard deviation =a.std() stats.zmap(scores, compare): input=score, constant=compare.mean() and standard deviation =compare.std() Assuming the axis works correctly, perhaps you can do something like this: def zscore(a, constant=None, stddev=None, axis=0, ddof=1): a=np.asarray(a) # I prefer this over a=asarray(a) to ensure that a is an ndarray or otherwise convert it to an array if constant==None: constant=a.mean(axis=axis) else: constant=np.mean(constant, axis=axis) if stddev==None: if constant==None: stddev=a.std(axis=axis, ddof=1) else: stddev=np.std(constant, axis=axis, ddof=1) elif len(stddev) > 1: # Inappropriate but requires something in case user wants to use the std of a different array-like input stddev=np.std(stddev, axis=axis, ddof=1) return (a-constant)/stddev Bruce From jrennie at gmail.com Mon Oct 26 13:07:50 2009 From: jrennie at gmail.com (Jason Rennie) Date: Mon, 26 Oct 2009 13:07:50 -0400 Subject: [SciPy-dev] L0 norm Message-ID: <75c31b2a0910261007h633715f7g194c902c0708c38a@mail.gmail.com> SciPy.linalg.basic.norm (version 0.7.0) raises a division-by-zero error when an ord value of zero is used. Yet, the L0 norm is commonly defined as the number of non-zero elements in a vector (see wikipedia link below). Has there been any discussion of adding an if statement to scipy.linalg.basic.norm to handle this case? Or, is this the behavior in a future version of scipy? If this functionality hasn't yet been added, would anyone object to a ticket being created requesting this feature? http://en.wikipedia.org/wiki/L0_norm#Zero_norm Cheers, Jason -- Jason Rennie Research Scientist, ITA Software 617-714-2645 http://www.itasoftware.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.l.goldsmith at gmail.com Mon Oct 26 13:14:02 2009 From: d.l.goldsmith at gmail.com (David Goldsmith) Date: Mon, 26 Oct 2009 10:14:02 -0700 Subject: [SciPy-dev] L0 norm In-Reply-To: <75c31b2a0910261007h633715f7g194c902c0708c38a@mail.gmail.com> References: <75c31b2a0910261007h633715f7g194c902c0708c38a@mail.gmail.com> Message-ID: <45d1ab480910261014t4c7382c6x22df0416d03e5192@mail.gmail.com> Please do file a ticket: http://scipy.org/BugReport. Thanks! DG On Mon, Oct 26, 2009 at 10:07 AM, Jason Rennie wrote: > SciPy.linalg.basic.norm (version 0.7.0) raises a division-by-zero error > when an ord value of zero is used. Yet, the L0 norm is commonly defined as > the number of non-zero elements in a vector (see wikipedia link below). Has > there been any discussion of adding an if statement to > scipy.linalg.basic.norm to handle this case? Or, is this the behavior in a > future version of scipy? If this functionality hasn't yet been added, would > anyone object to a ticket being created requesting this feature? > > http://en.wikipedia.org/wiki/L0_norm#Zero_norm > > Cheers, > > Jason > > -- > Jason Rennie > Research Scientist, ITA Software > 617-714-2645 > http://www.itasoftware.com/ > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jrennie at gmail.com Mon Oct 26 14:31:28 2009 From: jrennie at gmail.com (Jason Rennie) Date: Mon, 26 Oct 2009 14:31:28 -0400 Subject: [SciPy-dev] L0 norm In-Reply-To: <45d1ab480910261014t4c7382c6x22df0416d03e5192@mail.gmail.com> References: <75c31b2a0910261007h633715f7g194c902c0708c38a@mail.gmail.com> <45d1ab480910261014t4c7382c6x22df0416d03e5192@mail.gmail.com> Message-ID: <75c31b2a0910261131k59b22c87n613a603b818c61bd@mail.gmail.com> Done. http://projects.scipy.org/scipy/ticket/1037 Cheers, Jason On Mon, Oct 26, 2009 at 1:14 PM, David Goldsmith wrote: > Please do file a ticket: http://scipy.org/BugReport. Thanks! > > DG > > On Mon, Oct 26, 2009 at 10:07 AM, Jason Rennie wrote: > >> SciPy.linalg.basic.norm (version 0.7.0) raises a division-by-zero error >> when an ord value of zero is used. Yet, the L0 norm is commonly defined as >> the number of non-zero elements in a vector (see wikipedia link below). Has >> there been any discussion of adding an if statement to >> scipy.linalg.basic.norm to handle this case? Or, is this the behavior in a >> future version of scipy? If this functionality hasn't yet been added, would >> anyone object to a ticket being created requesting this feature? >> >> http://en.wikipedia.org/wiki/L0_norm#Zero_norm >> >> Cheers, >> >> Jason >> >> -- >> Jason Rennie >> Research Scientist, ITA Software >> 617-714-2645 >> http://www.itasoftware.com/ >> >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -- Jason Rennie Research Scientist, ITA Software 617-714-2645 http://www.itasoftware.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From nkilleen at unimelb.edu.au Mon Oct 26 18:33:33 2009 From: nkilleen at unimelb.edu.au (Neil Killeen) Date: Tue, 27 Oct 2009 09:33:33 +1100 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> Message-ID: <1BA568F3-F63E-48F0-99C7-E35A32DECCAF@unimelb.edu.au> Hi all can anyone make any suggestions; is this the right list for this question ? thanks Neil On 23/10/2009, at 9:23 AM, Neil Killeen wrote: > Hi all > > > > - I am operating on an SGI Altix 3700 BX2 Itanium IA64 system > running SUSE/Novelle Linux SLES10 > - I am trying to install scipy V 0.7.1 > - I am using python 2.4 > - blas and lapack are installed via system rpms in /usr/lib > > > soma:/usr/lib # ls -al *blas* *lapack* > -rw-r--r-- 1 root root 929506 2007-04-14 02:16 libblas.a > lrwxrwxrwx 1 root root 12 2008-03-12 12:20 libblas.so -> > libblas.so.3 > lrwxrwxrwx 1 root root 16 2008-03-12 12:20 libblas.so.3 -> > libblas.so.3.0.0 > -rwxr-xr-x 1 root root 678024 2007-04-14 02:16 libblas.so.3.0.0 > lrwxrwxrwx 1 root root 20 2008-12-02 22:52 libgslcblas.so.0 -> > libgslcblas.so.0.0.0 > -rwxr-xr-x 1 root root 908816 2007-04-14 02:30 libgslcblas.so.0.0.0 > -rw-r--r-- 1 root root 11208466 2007-04-14 02:16 liblapack.a > lrwxrwxrwx 1 root root 14 2008-03-12 12:20 liblapack.so -> > liblapack.so.3 > lrwxrwxrwx 1 root root 18 2008-03-12 12:20 liblapack.so.3 -> > liblapack.so.3.0.0 > -rwxr-xr-x 1 root root 7503568 2007-04-14 02:16 liblapack.so.3.0.0 > > > > > 1. I did a vanilla build (python setup.py build/install) which > appeared to work ok. > > 2. When I tried to run scipy.test() I saw errors. They are in two > categories: > > ERROR: Failure: ImportError (/usr/local/lib/python2.4/site-packages/ > scipy/linalg/fblas.so: undefined symbol: srotmg_) > > and > > ImportError: cannot import name {linalg, calc_lwork} > > Perhaps the latter are due to the former ? > > > 3. I read in the FAQ that the solution to the srotmg error is to > install blas myself either from netlib or via ATLAS > > 4. I decided to do the easy thing and installed blas from netlib in / > usr/local/lib/libblas.a > > 5. I then deleted the scipy installation (/usr/local/lib/python2.4/ > site-packges/scipy) and rebuilt and installed afresh. > > In the build log one can see that it appears to have found the /usr/ > local/lib/libblas.a library > > E.g. > > blas_info: > FOUND: > libraries = ['blas'] > library_dirs = ['/usr/local/lib'] > language = f77 > > FOUND: > libraries = ['blas'] > library_dirs = ['/usr/local/lib'] > define_macros = [('NO_ATLAS_INFO', 1)] > language = f77 > > > > > 6. Then I > - set env variable BLAS=/usr/local/lib/libblas.a (not sure useful > since static library so should be no run-time dependency) > - ran the test again with the same errors resulting as the initial. > > > Can someone advise what more I must do to eradicate these run-time > errors ? > > > thanks > Neil > > p.s. tried to post this last week but message did not arrive > (perhaps as I included attachments > which I have left off this time) From dwf at cs.toronto.edu Mon Oct 26 19:06:26 2009 From: dwf at cs.toronto.edu (David Warde-Farley) Date: Mon, 26 Oct 2009 19:06:26 -0400 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> Message-ID: <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> On 22-Oct-09, at 6:23 PM, Neil Killeen wrote: > 2. When I tried to run scipy.test() I saw errors. They are in two > categories: > > ERROR: Failure: ImportError (/usr/local/lib/python2.4/site-packages/ > scipy/linalg/fblas.so: undefined symbol: srotmg_) > > and > > ImportError: cannot import name {linalg, calc_lwork} > > Perhaps the latter are due to the former ? > > > 3. I read in the FAQ that the solution to the srotmg error is to > install blas myself either from netlib or via ATLAS > > 4. I decided to do the easy thing and installed blas from netlib in / > usr/local/lib/libblas.a How did you install it? Did you compile from source? What about lapack? Can you paste the output of numpy.show_config() and scipy.show_config()? David From nkilleen at unimelb.edu.au Mon Oct 26 19:18:15 2009 From: nkilleen at unimelb.edu.au (Neil Killeen) Date: Tue, 27 Oct 2009 10:18:15 +1100 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> Message-ID: <8546A13F-5C2A-44A4-8EB4-93161C410FA1@unimelb.edu.au> David On 27/10/2009, at 10:06 AM, David Warde-Farley wrote: > On 22-Oct-09, at 6:23 PM, Neil Killeen wrote: > >> 2. When I tried to run scipy.test() I saw errors. They are in two >> categories: >> >> ERROR: Failure: ImportError (/usr/local/lib/python2.4/site-packages/ >> scipy/linalg/fblas.so: undefined symbol: srotmg_) >> >> and >> >> ImportError: cannot import name {linalg, calc_lwork} >> >> Perhaps the latter are due to the former ? >> >> >> 3. I read in the FAQ that the solution to the srotmg error is to >> install blas myself either from netlib or via ATLAS >> >> 4. I decided to do the easy thing and installed blas from netlib in / >> usr/local/lib/libblas.a > > How did you install it? Did you compile from source? yes from source > > What about lapack? i am using the system version, /usr/lib The FAQ states that the srotmg error is a BLAS error and that I should install from netlib which I did. I didn't see any other LAPACK errors but of course they may be obscured by the blas error. > > Can you paste the output of numpy.show_config() and > scipy.show_config()? Sure: >>> scipy.show_config() blas_info: libraries = ['blas'] library_dirs = ['/usr/local/lib'] language = f77 lapack_info: libraries = ['lapack'] library_dirs = ['/usr/lib'] language = f77 atlas_threads_info: NOT AVAILABLE blas_opt_info: libraries = ['blas'] library_dirs = ['/usr/local/lib'] language = f77 define_macros = [('NO_ATLAS_INFO', 1)] atlas_blas_threads_info: NOT AVAILABLE umfpack_info: NOT AVAILABLE lapack_opt_info: libraries = ['lapack', 'blas'] library_dirs = ['/usr/lib', '/usr/local/lib'] language = f77 define_macros = [('NO_ATLAS_INFO', 1)] atlas_info: NOT AVAILABLE lapack_mkl_info: NOT AVAILABLE blas_mkl_info: NOT AVAILABLE atlas_blas_info: NOT AVAILABLE mkl_info: NOT AVAILABLE so it's picking up the /usr/local/lib version of blas that I installed >>> numpy.show_config() blas_info: libraries = ['blas'] library_dirs = ['/usr/lib'] language = f77 lapack_info: libraries = ['lapack'] library_dirs = ['/usr/lib'] language = f77 atlas_threads_info: NOT AVAILABLE blas_opt_info: libraries = ['blas'] library_dirs = ['/usr/lib'] language = f77 define_macros = [('NO_ATLAS_INFO', 1)] atlas_blas_threads_info: NOT AVAILABLE lapack_opt_info: libraries = ['lapack', 'blas'] library_dirs = ['/usr/lib'] language = f77 define_macros = [('NO_ATLAS_INFO', 1)] atlas_info: NOT AVAILABLE lapack_mkl_info: NOT AVAILABLE blas_mkl_info: NOT AVAILABLE atlas_blas_info: NOT AVAILABLE mkl_info: NOT AVAILABLE This was built prior to scipy and uses the /usr/lib versions. numpy passes all of its internal tests and I assume scipy can utilize whatever functional installation is present ? thanks Neil > > David > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev From dwf at cs.toronto.edu Mon Oct 26 20:45:43 2009 From: dwf at cs.toronto.edu (David Warde-Farley) Date: Mon, 26 Oct 2009 20:45:43 -0400 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: <8546A13F-5C2A-44A4-8EB4-93161C410FA1@unimelb.edu.au> References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> <8546A13F-5C2A-44A4-8EB4-93161C410FA1@unimelb.edu.au> Message-ID: <20091027004543.GA1622@rodimus> On Tue, Oct 27, 2009 at 10:18:15AM +1100, Neil Killeen wrote: > >> > >> ERROR: Failure: ImportError (/usr/local/lib/python2.4/site-packages/ > >> scipy/linalg/fblas.so: undefined symbol: srotmg_) > >> > >> and > >> > >> ImportError: cannot import name {linalg, calc_lwork} > >> > >> Perhaps the latter are due to the former ? > >> > >> > yes from source > > > > > What about lapack? > > i am using the system version, /usr/lib The FAQ states that the > srotmg error is a BLAS > error and that I should install from netlib which I did Disclaimer: I usually build ATLAS and LAPACK from source, I've never really played with the netlib BLAS much. liblapack.a may have certain BLAS functions inside the archive (I think) if you installed ATLAS packages from your distribution. I'd recommend a clean LAPACK as well, and building numpy and scipy against the same set of libraries, just to be safe... From show_config() it looks like everything should be in order, I think. Are you getting the exact same errors after building against your source built BLAS? David From nkilleen at unimelb.edu.au Mon Oct 26 22:04:53 2009 From: nkilleen at unimelb.edu.au (Neil Killeen) Date: Tue, 27 Oct 2009 13:04:53 +1100 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: <20091027004543.GA1622@rodimus> References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> <8546A13F-5C2A-44A4-8EB4-93161C410FA1@unimelb.edu.au> <20091027004543.GA1622@rodimus> Message-ID: <0D82DC45-4F98-4309-B8B2-E27D1CEAFDBA@unimelb.edu.au> On 27/10/2009, at 11:45 , David Warde-Farley wrote: > On Tue, Oct 27, 2009 at 10:18:15AM +1100, Neil Killeen wrote: >>>> >>>> ERROR: Failure: ImportError (/usr/local/lib/python2.4/site- >>>> packages/ >>>> scipy/linalg/fblas.so: undefined symbol: srotmg_) >>>> >>>> and >>>> >>>> ImportError: cannot import name {linalg, calc_lwork} >>>> >>>> Perhaps the latter are due to the former ? >>>> >>>> > >> yes from source >> >>> >>> What about lapack? >> >> i am using the system version, /usr/lib The FAQ states that the >> srotmg error is a BLAS >> error and that I should install from netlib which I did > > Disclaimer: I usually build ATLAS and LAPACK from source, I've > never really played with the netlib BLAS much. I'm a bit confused by all the various packagings (bear in mind I am just installing this for end users, I'm not a consumer myself). By going to netlib, which I thought was the home of BLAS, I *did* install from source. ATLAS seems to be repackaging/reimpl of LAPACK and BLAS ? Installing ATLAS seemed rather a lot of bother so I was trying the easy path, > > liblapack.a may have certain BLAS functions inside the archive (I > think) > if you installed ATLAS packages from your distribution. > I'd recommend a clean LAPACK as well, and building numpy and scipy > against the same set of libraries, just to be safe... From > show_config() > it looks like everything should be in order, I think. ok i will try that. I gather though buiding from the reference implementations may bring poor performance and that ATLAS seeks to redress that ? Do you recommend I bite the bullet and try to make an ATLAS install ? > > Are you getting the exact same errors after building against your > source built BLAS? yes > > David > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev cheers Neil -------------- next part -------------- An HTML attachment was scrubbed... URL: From cournape at gmail.com Mon Oct 26 22:33:06 2009 From: cournape at gmail.com (David Cournapeau) Date: Tue, 27 Oct 2009 11:33:06 +0900 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: <0D82DC45-4F98-4309-B8B2-E27D1CEAFDBA@unimelb.edu.au> References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> <8546A13F-5C2A-44A4-8EB4-93161C410FA1@unimelb.edu.au> <20091027004543.GA1622@rodimus> <0D82DC45-4F98-4309-B8B2-E27D1CEAFDBA@unimelb.edu.au> Message-ID: <5b8d13220910261933x1d6042a0k8adb2f25ef9e803e@mail.gmail.com> On Tue, Oct 27, 2009 at 11:04 AM, Neil Killeen wrote: > > I'm a bit confused by all the various packagings (bear in mind I am > just installing this for end users, I'm not a consumer myself). > By going to netlib, which I thought was the home of BLAS, > I *did* install from source. > ATLAS seems to be repackaging/reimpl of LAPACK and BLAS ? > Installing ATLAS seemed rather a lot of bother so I was trying > the easy path, You should not bother with atlas at first - I am sure building it for Itanium will be a pain. There are several things to check for with your build against the version from NETLIB website: - to check which libraries are actually linked, look at the output of ldd on scipy/lapack/fblas.so - to check for symbols, the output of nm is very useful as well. In particular, depending on the fortran compiler, option and architecture, the actual name of the function may be different, this is important to know as well: is it srotgm, _srotgm_, _srotgm, etc... Comparing the name from your custom built libblas.a against the name in scipy/lapack/fblas.so should hint at the problem as well. David From nkilleen at unimelb.edu.au Mon Oct 26 23:04:03 2009 From: nkilleen at unimelb.edu.au (Neil Killeen) Date: Tue, 27 Oct 2009 14:04:03 +1100 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: <5b8d13220910261933x1d6042a0k8adb2f25ef9e803e@mail.gmail.com> References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> <8546A13F-5C2A-44A4-8EB4-93161C410FA1@unimelb.edu.au> <20091027004543.GA1622@rodimus> <0D82DC45-4F98-4309-B8B2-E27D1CEAFDBA@unimelb.edu.au> <5b8d13220910261933x1d6042a0k8adb2f25ef9e803e@mail.gmail.com> Message-ID: David On 27/10/2009, at 1:33 PM, David Cournapeau wrote: > On Tue, Oct 27, 2009 at 11:04 AM, Neil Killeen > wrote: >> >> I'm a bit confused by all the various packagings (bear in mind I am >> just installing this for end users, I'm not a consumer myself). >> By going to netlib, which I thought was the home of BLAS, >> I *did* install from source. >> ATLAS seems to be repackaging/reimpl of LAPACK and BLAS ? >> Installing ATLAS seemed rather a lot of bother so I was trying >> the easy path, > > You should not bother with atlas at first - I am sure building it for > Itanium will be a pain. > > There are several things to check for with your build against the > version from NETLIB website: > - to check which libraries are actually linked, look at the output of > ldd on scipy/lapack/fblas.so First, there appear to be 2 blas shareble objects in the build. nkilleen@ /usr/local/lib/python2.4/site-packages/scipy> find . -name fblas.so ./lib/blas/fblas.so ./linalg/fblas.so and 1) nkilleen@/usr/local/lib/python2.4/site-packages/scipy> ldd linalg/ fblas.so linux-gate.so.1 => (0xa000000000000000) liblapack.so.3 => /usr/lib/liblapack.so.3 (0x20000008000d4000) libblas.so.3 => /usr/lib/libblas.so.3 (0x2000000800908000) libgfortran.so.1 => /usr/lib/libgfortran.so.1 (0x20000008009bc000) libm.so.6.1 => /lib/libm.so.6.1 (0x2000000800aec000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2000000800bbc000) libunwind.so.7 => /lib/libunwind.so.7 (0x2000000800bdc000) libc.so.6.1 => /lib/libc.so.6.1 (0x2000000800c28000) /lib/ld-linux-ia64.so.2 (0x2000000800000000) 2)nkilleen@/usr/local/lib/python2.4/site-packages/scipy> ldd lib/blas/ fblas.so linux-gate.so.1 => (0xa000000000000000) libgfortran.so.1 => /usr/lib/libgfortran.so.1 (0x20000008000fc000) libm.so.6.1 => /lib/libm.so.6.1 (0x200000080022c000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x20000008002fc000) libunwind.so.7 => /lib/libunwind.so.7 (0x200000080031c000) libc.so.6.1 => /lib/libc.so.6.1 (0x2000000800368000) /lib/ld-linux-ia64.so.2 (0x2000000800000000) So you can see one of them points to the native blas in /usr/lib and one doesn't point anywhere. Now I only built /usr/local/lib/libblas.a which the install script seemed to find (see first post which shows log) > - to check for symbols, the output of nm is very useful as well. In > particular, depending on the fortran compiler, option and > architecture, the actual name of the function may be different, this > is important to know as well: is it srotgm, _srotgm_, _srotgm, etc... The two fblas.so in the scipy install tree show nkilleen@/usr/local/lib/python2.4/site-packages/scipy/lib/blas> nm fblas.so | grep rotm 000000000007f418 d doc_f2py_rout_fblas_drotm 000000000007e898 d doc_f2py_rout_fblas_drotmg 000000000007f1e8 d doc_f2py_rout_fblas_srotm 000000000007e7c8 d doc_f2py_rout_fblas_srotmg 000000000004a300 T drotm_ 00000000000495f0 T drotmg_ 0000000000018c50 t f2py_rout_fblas_drotm 0000000000028460 t f2py_rout_fblas_drotmg 0000000000019870 t f2py_rout_fblas_srotm 000000000002e5e0 t f2py_rout_fblas_srotmg 0000000000053720 T srotm_ 0000000000052b90 T srotmg_ and nkilleen@/usr/local/lib/python2.4/site-packages/scipy/linalg> nm fblas.so | grep rotm 00000000000592f8 d doc_f2py_rout_fblas_drotm 0000000000058778 d doc_f2py_rout_fblas_drotmg 00000000000590c8 d doc_f2py_rout_fblas_srotm 00000000000586a8 d doc_f2py_rout_fblas_srotmg U drotm_ U drotmg_ 0000000000018410 t f2py_rout_fblas_drotm 0000000000027c80 t f2py_rout_fblas_drotmg 0000000000019030 t f2py_rout_fblas_srotm 000000000002de40 t f2py_rout_fblas_srotmg U srotm_ U srotmg_ I assume 'U' means undefined ? The static library nkilleen@/usr/local/lib # nm libblas.a | grep rotm drotmg.o: 0000000000000000 T drotmg_ drotm.o: 0000000000000000 T drotm_ srotmg.o: 0000000000000000 T srotmg_ srotm.o: 0000000000000000 T srotm_ > Comparing the name from your custom built libblas.a against the name > in scipy/lapack/fblas.so should hint at the problem as well. so all this suggests that the build or run-time environment is not correctly picking up the static blas library and is still using the /usr/ lib .so despite what the build log says. > > David Questions: 1) why are there two fblas.so objects ? 2) what is the correct way to get scipy and numpy to pick up the / usr/local/lib/libblas.a in both the build and at run-time. The scipy docs don't comment on this... thanks again Neil > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev From david at ar.media.kyoto-u.ac.jp Mon Oct 26 23:09:58 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 27 Oct 2009 12:09:58 +0900 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> <8546A13F-5C2A-44A4-8EB4-93161C410FA1@unimelb.edu.au> <20091027004543.GA1622@rodimus> <0D82DC45-4F98-4309-B8B2-E27D1CEAFDBA@unimelb.edu.au> <5b8d13220910261933x1d6042a0k8adb2f25ef9e803e@mail.gmail.com> Message-ID: <4AE66486.2060807@ar.media.kyoto-u.ac.jp> Neil Killeen wrote: > > First, there appear to be 2 blas shareble objects in the build. > > nkilleen@ /usr/local/lib/python2.4/site-packages/scipy> find . -name > fblas.so > ./lib/blas/fblas.so > ./linalg/fblas.so > Yes, there are two fblas libraries - the lib one is redundant (it was started as an effort to put blas/lapack into its own package so that linalg would use those, but this was never finished AFAIK). > > nkilleen@/usr/local/lib/python2.4/site-packages/scipy/lib/blas> nm > fblas.so | grep rotm > 000000000007f418 d doc_f2py_rout_fblas_drotm > 000000000007e898 d doc_f2py_rout_fblas_drotmg > 000000000007f1e8 d doc_f2py_rout_fblas_srotm > 000000000007e7c8 d doc_f2py_rout_fblas_srotmg > 000000000004a300 T drotm_ > 00000000000495f0 T drotmg_ > 0000000000018c50 t f2py_rout_fblas_drotm > 0000000000028460 t f2py_rout_fblas_drotmg > 0000000000019870 t f2py_rout_fblas_srotm > 000000000002e5e0 t f2py_rout_fblas_srotmg > 0000000000053720 T srotm_ > 0000000000052b90 T srotmg_ > It looks like one was linked against the static library whereas the other one was linked against the shared version. That's odd. > I assume 'U' means undefined ? > Yes it does - it would be resolved once liblapack.so is loaded. > 1) why are there two fblas.so objects ? > 2) what is the correct way to get scipy and numpy to pick up the / > usr/local/lib/libblas.a > in both the build and at run-time. The scipy docs don't > comment on this... > That's not easy to do. By far, the simplest is to put the static libraries in a directory where there is no shared version, and to edit site.cfg so that those directories are looked for at first. Don't trust too much what the build output says, except for the actual command executed. The scipy build output is messy, and a lot of information are not accurate or just plain wrong. cheers, David From ralf.gommers at googlemail.com Wed Oct 28 08:10:48 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Wed, 28 Oct 2009 13:10:48 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 12:00 AM, Ralf Gommers wrote: > > > On Sun, Oct 25, 2009 at 11:11 PM, wrote: > >> On Sun, Oct 25, 2009 at 5:33 PM, Ralf Gommers >> wrote: >> > >> > >> > On Sun, Oct 25, 2009 at 8:20 PM, Matthew Brett > > >> >> >> >> scipy.stats does some processing of the input parameters like shape, >> >> to determine the docstring, but that could easily be put in your >> >> decorator. >> >> >> >> So, I'm not sure docfiller is the only or best solution, but it looks >> >> like something like it recurs in the code, and I think it could work >> >> for scipy.stats... >> > >> > Thanks Matthew. Searching for how other people/projects solve this same >> > problem I found nothing much fancier than docfiller, so I think I will >> try >> > to get an idea first of how much work it will be to change the >> scipy.stats >> > docs to use docfiller throughout. >> > >> > Then of course it would be good to hear from Pauli roughly how much work >> it >> > would be to support this in pydocweb. The ticket says "rewrite of >> > pydoc-tool.py" so it may be non-trivial.... >> > >> > Cheers, >> > Ralf >> >> One of the differences between ndimage and stats.distributions is that in >> ndimage functions are decorated, while the individual distributions (kind >> of) >> inherit the adjusted docstring from the basic distribution classes. >> >> The documentation can be improved (maybe with better templating as in the >> ndimage decorator), and I don't know how pydocweb can >> handle it, but I don't think we need to start to decorate individual >> distributions >> when we can use the class hierarchy for generating the docs. >> > > For scipy.stats it might not be any better than the current method, but it > also would not be worse. The point is that it will be a lot of work to make > the wiki play nice with even *one* templating system. > > Since a decorator is more widely applicable than the scipy.stats > inheritance, it could be a good candidate to be applied everywhere in > NumPy/SciPy and be recognized by pydocweb. > > Hi Josef and others, I have got a prototype of a more flexible docstring system for scipy.stats.distributions working, based on the same string substitution as used in ndimage and io.matlab. I looked at improving the current template as well, but it is simply not possible to get very good results when you generate lots of docstrings from a single generic string with a few words swapped in for placeholders. The new system is backwards-compatible, meaning the current way of generating docstrings from a generic template continues to work if the `distname_gen` class does not have its own docstring. So no extra work for the people working on scipy.stats to change things around. The way it works is that you can use defaults for most elements of a docstring, but add custom descriptions where necessary, add sections, links, references, etc. An example: """A Maxwell continuous random variable. Methods ------- %(rvs)s pdf(x, loc=0, scale=1) Probability density function. Given by :math:`\sqrt(2/pi)*x^2 * exp(-x**2/2)` for ``x>0``. %(cdf)s %(sf)s %(ppf)s %(isf)s %(stats)s %(entropy)s %(fit)s Notes ----- A special case of a `chi` distribution, with ``df = 3``, ``loc = 0.0``, and given ``scale = 1.0 / sqrt(a)``, where a is the parameter used in the Mathworld description [1]_. References ---------- .. [1] http://mathworld.wolfram.com/MaxwellDistribution.html Examples -------- %(default_example)s """ The code can be found at http://github.com/rgommers/doctemple. The oldstats.py file shows the basics of how docstring generation currently works in scipy.stats.distributions, and newstats.py contains the new functionality. Try this with: In [1]: run newstats.py In [2]: maxwell? Please let me know what you think. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Wed Oct 28 09:52:37 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 28 Oct 2009 09:52:37 -0400 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> Message-ID: <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> On Wed, Oct 28, 2009 at 8:10 AM, Ralf Gommers wrote: > > > On Mon, Oct 26, 2009 at 12:00 AM, Ralf Gommers > wrote: >> >> >> On Sun, Oct 25, 2009 at 11:11 PM, wrote: >>> >>> On Sun, Oct 25, 2009 at 5:33 PM, Ralf Gommers >>> wrote: >>> > >>> > >>> > On Sun, Oct 25, 2009 at 8:20 PM, Matthew Brett >>> > >>> >> >>> >> scipy.stats does some processing of the input parameters like shape, >>> >> to determine the docstring, but that could easily be put in your >>> >> decorator. >>> >> >>> >> So, I'm not sure docfiller is the only or best solution, but it looks >>> >> like something like it recurs in the code, and I think it could work >>> >> for scipy.stats... >>> > >>> > Thanks Matthew. Searching for how other people/projects solve this same >>> > problem I found nothing much fancier than docfiller, so I think I will >>> > try >>> > to get an idea first of how much work it will be to change the >>> > scipy.stats >>> > docs to use docfiller throughout. >>> > >>> > Then of course it would be good to hear from Pauli roughly how much >>> > work it >>> > would be to support this in pydocweb. The ticket says "rewrite of >>> > pydoc-tool.py" so it may be non-trivial.... >>> > >>> > Cheers, >>> > Ralf >>> >>> One of the differences between ndimage and stats.distributions is that in >>> ndimage ?functions are decorated, while the individual distributions >>> (kind of) >>> inherit the adjusted docstring from the basic distribution classes. >>> >>> The documentation can be improved (maybe with better templating as in the >>> ndimage decorator), and I don't know how pydocweb can >>> handle it, but I don't think we need to start to decorate individual >>> distributions >>> when we can use the class hierarchy for generating the docs. >> >> For scipy.stats it might not be any better than the current method, but it >> also would not be worse. The point is that it will be a lot of work to make >> the wiki play nice with even *one* templating system. >> >> Since a decorator is more widely applicable than the scipy.stats >> inheritance, it could be a good candidate to be applied everywhere in >> NumPy/SciPy and be recognized by pydocweb. >> > > Hi Josef and others, > > I have got a prototype of a more flexible docstring system for > scipy.stats.distributions working, based on the same string substitution as > used in ndimage and io.matlab. I looked at improving the current template as > well, but it is simply not possible to get very good results when you > generate lots of docstrings from a single generic string with a few words > swapped in for placeholders. > > The new system is backwards-compatible, meaning the current way of > generating docstrings from a generic template continues to work if the > `distname_gen` class does not have its own docstring. So no extra work for > the people working on scipy.stats to change things around. > > The way it works is that you can use defaults for most elements of a > docstring, but add custom descriptions where necessary, add sections, links, > references, etc. An example: > > ??? """A Maxwell continuous random variable. > > ??? Methods > ??? ------- > ??? %(rvs)s > ??? pdf(x, loc=0, scale=1) > ??????? Probability density function. Given by > ??????? :math:`\sqrt(2/pi)*x^2 * exp(-x**2/2)` for ``x>0``. > ??? %(cdf)s > ??? %(sf)s > ??? %(ppf)s > ??? %(isf)s > ??? %(stats)s > ??? %(entropy)s > ??? %(fit)s > > ??? Notes > ??? ----- > ??? A special case of a `chi` distribution,? with ``df = 3``, ``loc = 0.0``, > ??? and given ``scale = 1.0 / sqrt(a)``, where a is the parameter used in > ??? the Mathworld description [1]_. > > ??? References > ??? ---------- > ??? .. [1] http://mathworld.wolfram.com/MaxwellDistribution.html > > ??? Examples > ??? -------- > ??? %(default_example)s > > ??? """ > > > The code can be found at http://github.com/rgommers/doctemple. The > oldstats.py file shows the basics of how docstring generation currently > works in scipy.stats.distributions, and newstats.py contains the new > functionality. > > Try this with: > In [1]: run newstats.py > In [2]: maxwell? > > Please let me know what you think. Just some quick comments (I will look more after Friday) I don't see a big break with the current system, but the templating system looks cleaner. One difference: standard help doesn't get the substitution in the docstring >>> help(maxwell) >>> maxwell.__doc__ >>> help(stats.distributions.norm) no difference in the docstring of the class: >>> print stats.distributions.norm_gen.__doc__ None >>> print distname_gen.__doc__ None >>> print maxwell_gen.__doc__ ... Is it possible to attach the docstring to the distribution (sub) class when it is instantiated, e.g. self.__class__.__doc__ = generated docs # I didn't try this (The current dist_gen classes are missing a __init__, so the class doesn't get any of the information.) So, I think overall it looks pretty good. Does it really make the life for the doceditor easier? The template substitution is still in the rvclass.__init__ Thanks, Josef > > Cheers, > Ralf > > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From ralf.gommers at googlemail.com Wed Oct 28 10:41:33 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Wed, 28 Oct 2009 15:41:33 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> Message-ID: Thanks for the feedback Josef. On Wed, Oct 28, 2009 at 2:52 PM, wrote: > Just some quick comments (I will look more after Friday) > > I don't see a big break with the current system, but the templating > system looks cleaner. > Well I did not want to be too radical, just make it a bit more flexible and better-looking:) > > One difference: standard help doesn't get the substitution in the docstring > >>> help(maxwell) > >>> maxwell.__doc__ > >>> help(stats.distributions.norm) > > > no difference in the docstring of the class: > >>> print stats.distributions.norm_gen.__doc__ > None > >>> print distname_gen.__doc__ > None > >>> print maxwell_gen.__doc__ > ... > > Is it possible to attach the docstring to the distribution (sub) class > when it is instantiated, e.g. > > self.__class__.__doc__ = generated docs # I didn't try this > That works, good point. Fixes the python help() as well. > > (The current dist_gen classes are missing a __init__, so the class > doesn't get any of the information.) > > So, I think overall it looks pretty good. > Does it really make the life for the doceditor easier? > It should do in the future, but right now that is just an added benefit. More important is that you just get better docstrings. > The template substitution is still in the rvclass.__init__ > This can disappear once all classes have their own docstring. I was being conservative by keeping the old system as a backup. If you would be happy if I gave each `_gen` class a default docstring based on the new system, and completely remove the templates in rv_continuous/rv_discrete, I will do that. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Wed Oct 28 11:13:10 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 28 Oct 2009 11:13:10 -0400 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> Message-ID: <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> On Wed, Oct 28, 2009 at 10:41 AM, Ralf Gommers wrote: > Thanks for the feedback Josef. > > On Wed, Oct 28, 2009 at 2:52 PM, wrote: >> >> Just some quick comments (I will look more after Friday) >> >> >> I don't see a big break with the current system, but the templating >> system looks cleaner. > > Well I did not want to be too radical, just make it a bit more flexible and > better-looking:) >> >> One difference: standard help doesn't get the substitution in the >> docstring >> >>> help(maxwell) >> >>> maxwell.__doc__ >> >>> help(stats.distributions.norm) >> >> >> no difference in the docstring of the class: >> >>> print stats.distributions.norm_gen.__doc__ >> None >> >>> print distname_gen.__doc__ >> None >> >>> print maxwell_gen.__doc__ >> ... >> >> Is it possible to attach the docstring to the distribution (sub) class >> when it is instantiated, e.g. >> >> self.__class__.__doc__ = generated docs # I didn't try this > > That works, good point. Fixes the python help() as well. > >> >> (The current dist_gen classes are missing a __init__, so the class >> doesn't get any of the information.) >> >> So, I think overall it looks pretty good. >> Does it really make the life for the doceditor easier? > > It should do in the future, but right now that is just an added benefit. > More important is that you just get better docstrings. > >> >> The template substitution is still in the rvclass.__init__ > > This can disappear once all classes have their own docstring. I was being > conservative by keeping the old system as a backup. > > If you would be happy if I gave each `_gen` class a default docstring based > on the new system, and completely remove the templates in > rv_continuous/rv_discrete, I will do that. No, I like the current system where we don't have any boiler plate in the individual distributions. Following your maxwell example, we would get a lot of repetition of template code in the individual distributions. I wouldn't mind adding some additional individualized docs similar to the extradoc to the distribution classes. Pierre thinks that we should keep it separate. If we can link to the full description of the distributions, which is currently stored in the tutorial, then we can have some basic docstring on the command line, and for additional information we can follow the link. One problem we still have with the examples in the docstrings is that they don't always work, since they are too generic and some values might not be feasible. Maintaining 90 individual distributions without a generic framework is too much work. However, with the increased flexibility with your changes, we could think about individual docstrings for the most popular distributions, normal, t, ..? Thanks for improving this. Josef > > Cheers, > Ralf > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From pav+sp at iki.fi Wed Oct 28 14:07:58 2009 From: pav+sp at iki.fi (Pauli Virtanen) Date: Wed, 28 Oct 2009 18:07:58 +0000 (UTC) Subject: [SciPy-dev] ndimage - docfiller and output_type References: <1e2af89e0910250927h3e8e123rf3cddd377ca873a4@mail.gmail.com> <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> Message-ID: Wed, 28 Oct 2009 11:13:10 -0400, josef.pktd wrote: [clip] > No, I like the current system where we don't have any boiler plate in > the individual distributions. Following your maxwell example, we would > get a lot of repetition of template code in the individual > distributions. Then, I'd suggest just adding %(info)s or so, that will be expanded to the full boilerplate information block. Then, only this would be repeated in each distribution docstring -- not too bad. It is clearer and cleaner if the distribution docstring is a template, resides in the class docstring, and the final product is assembled by substitution rather than by concatenating magically generated parts. (Think, for example -- why are HTML templating languages so popular, as opposed to just concatenating blocks of HTML code?) -- Pauli Virtanen From josef.pktd at gmail.com Wed Oct 28 15:34:58 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 28 Oct 2009 15:34:58 -0400 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> Message-ID: <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> On Wed, Oct 28, 2009 at 2:07 PM, Pauli Virtanen wrote: > Wed, 28 Oct 2009 11:13:10 -0400, josef.pktd wrote: > [clip] >> No, I like the current system where we don't have any boiler plate in >> the individual distributions. Following your maxwell example, we would >> get a lot of repetition of template code in the individual >> distributions. > > Then, I'd suggest just adding > > ? ? ? ?%(info)s > > or so, that will be expanded to the full boilerplate information block. > Then, only this would be repeated in each distribution docstring -- not > too bad. If you can do it with minimal code/text duplication, then any way is fine with me. > > It is clearer and cleaner if the distribution docstring is a template, > resides in the class docstring, and the final product is assembled by > substitution rather than by concatenating magically generated parts. > (Think, for example -- why are HTML templating languages so popular, as > opposed to just concatenating blocks of HTML code?) The last time I looked at mako, it allowed full inheritance and backward (?) substitution in the templates. I don't see how template inheritance would work with simple string substitution, but you are the experts for the numpy/scipy documentation system. Josef > > -- > Pauli Virtanen > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From ralf.gommers at googlemail.com Wed Oct 28 15:35:58 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Wed, 28 Oct 2009 20:35:58 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1e2af89e0910251220ue9398a5nf3df2f591fa3e582@mail.gmail.com> <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> Message-ID: On Wed, Oct 28, 2009 at 7:07 PM, Pauli Virtanen > wrote: > Wed, 28 Oct 2009 11:13:10 -0400, josef.pktd wrote: > [clip] > > No, I like the current system where we don't have any boiler plate in > > the individual distributions. Following your maxwell example, we would > > get a lot of repetition of template code in the individual > > distributions. > > Then, I'd suggest just adding > > %(info)s > > or so, that will be expanded to the full boilerplate information block. > Then, only this would be repeated in each distribution docstring -- not > too bad. > > It is clearer and cleaner if the distribution docstring is a template, > resides in the class docstring, and the final product is assembled by > substitution rather than by concatenating magically generated parts. > (Think, for example -- why are HTML templating languages so popular, as > opposed to just concatenating blocks of HTML code?) > Yes, this is what I was thinking. The default in each `_gen` class could for example be: """ The distribution name. %(rv_info)s Methods ----------- %(allmethods)s %(call_info)s Examples ------------- %(default_example)s """ If you really want to keep it short, you could even just do: """ The distribution name. %(default_contents)s """ Now imagine you want a References section. Here you just add it in at the right place in my first example. If the whole template is in the rv_continuous class, there is just no way to add this section. Additionally, this allows you to remove 3 keywords from the constructor (name, longname, extradoc). So the extra overhead is very small, and you gain a lot. Cheers, Ralf > -- > Pauli Virtanen > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Wed Oct 28 16:16:41 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Wed, 28 Oct 2009 21:16:41 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> References: <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> Message-ID: On Wed, Oct 28, 2009 at 8:34 PM, wrote: > On Wed, Oct 28, 2009 at 2:07 PM, Pauli Virtanen > > wrote: > > Wed, 28 Oct 2009 11:13:10 -0400, josef.pktd wrote: > > [clip] > >> No, I like the current system where we don't have any boiler plate in > >> the individual distributions. Following your maxwell example, we would > >> get a lot of repetition of template code in the individual > >> distributions. > > > > Then, I'd suggest just adding > > > > %(info)s > > > > or so, that will be expanded to the full boilerplate information block. > > Then, only this would be repeated in each distribution docstring -- not > > too bad. > > If you can do it with minimal code/text duplication, then any way is fine > with me. > > Okay, I'll come up with a patch with as little duplication as possible (one line per distribution). > > > > It is clearer and cleaner if the distribution docstring is a template, > > resides in the class docstring, and the final product is assembled by > > substitution rather than by concatenating magically generated parts. > > (Think, for example -- why are HTML templating languages so popular, as > > opposed to just concatenating blocks of HTML code?) > > The last time I looked at mako, it allowed full inheritance and backward > (?) > substitution in the templates. > > I don't see how template inheritance would work with simple string > substitution, but you are the experts for the numpy/scipy documentation > system. > Inheritance is the wrong word. The idea is to build up a docstring from smaller pieces, instead of starting with one big template and then trying to change it. In the default (simplest) case, you build up a string in the module scope, only substitute for each distribution, and assign it to __doc__. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Wed Oct 28 17:00:34 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 28 Oct 2009 17:00:34 -0400 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1cd32cbb0910251511x4b10ce85gf881997315ed6d05@mail.gmail.com> <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> Message-ID: <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> On Wed, Oct 28, 2009 at 4:16 PM, Ralf Gommers wrote: > > > On Wed, Oct 28, 2009 at 8:34 PM, wrote: >> >> On Wed, Oct 28, 2009 at 2:07 PM, Pauli Virtanen wrote: >> > Wed, 28 Oct 2009 11:13:10 -0400, josef.pktd wrote: >> > [clip] >> >> No, I like the current system where we don't have any boiler plate in >> >> the individual distributions. Following your maxwell example, we would >> >> get a lot of repetition of template code in the individual >> >> distributions. >> > >> > Then, I'd suggest just adding >> > >> > ? ? ? ?%(info)s >> > >> > or so, that will be expanded to the full boilerplate information block. >> > Then, only this would be repeated in each distribution docstring -- not >> > too bad. >> >> If you can do it with minimal code/text duplication, then any way is fine >> with me. >> > Okay, I'll come up with a patch with as little duplication as possible (one > line per distribution). > >> >> > >> > It is clearer and cleaner if the distribution docstring is a template, >> > resides in the class docstring, and the final product is assembled by >> > substitution rather than by concatenating magically generated parts. >> > (Think, for example -- why are HTML templating languages so popular, as >> > opposed to just concatenating blocks of HTML code?) >> >> The last time I looked at mako, it allowed full inheritance and backward >> (?) >> substitution in the templates. >> >> I don't see how template inheritance would work with simple string >> substitution, but you are the experts for the numpy/scipy documentation >> system. > > Inheritance is the wrong word. The idea is to build up a docstring from > smaller pieces, instead of starting with one big template and then trying to > change it. In the default (simplest) case, you build up a string in the > module scope, only substitute for each distribution, and assign it > to __doc__. > > Cheers, > Ralf I like your initially proposed solution, because it allows both ways, either full generic template (for distributions that nobody looks at), or following the maxwell pattern for the distributions, where someone would want to add individualized information. One other advantage of the generic docstring is that it is easy to create new classes outside of scipy by subclassing rv_continuous and have the docstring generated the same way as distribution in scipy. But even having your basic template is not really a problem, it's mainly the numpy docstring standard and the template is not something we would have to edit often. (It would still double the lines of code for many distributions). I still like to keep name and (maybe) longname since they are used in programs, if I remember correctly. Personally, I like extradoc, that's the only really interesting information about a distribution that I look at (besides dist.a and dist.b and dist.shape.) and I don't have to scroll through the boiler plate docstring to read it. (In general, I think the class should get the default constructor information in the __init__ method instead of in the instantiation.) Josef > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From josef.pktd at gmail.com Wed Oct 28 18:15:43 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 28 Oct 2009 18:15:43 -0400 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> References: <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> Message-ID: <1cd32cbb0910281515l2035fb0fjfb4ad8742095178e@mail.gmail.com> On Wed, Oct 28, 2009 at 5:00 PM, wrote: > On Wed, Oct 28, 2009 at 4:16 PM, Ralf Gommers > wrote: >> >> >> On Wed, Oct 28, 2009 at 8:34 PM, wrote: >>> >>> On Wed, Oct 28, 2009 at 2:07 PM, Pauli Virtanen wrote: >>> > Wed, 28 Oct 2009 11:13:10 -0400, josef.pktd wrote: >>> > [clip] >>> >> No, I like the current system where we don't have any boiler plate in >>> >> the individual distributions. Following your maxwell example, we would >>> >> get a lot of repetition of template code in the individual >>> >> distributions. >>> > >>> > Then, I'd suggest just adding >>> > >>> > ? ? ? ?%(info)s >>> > >>> > or so, that will be expanded to the full boilerplate information block. >>> > Then, only this would be repeated in each distribution docstring -- not >>> > too bad. >>> >>> If you can do it with minimal code/text duplication, then any way is fine >>> with me. >>> >> Okay, I'll come up with a patch with as little duplication as possible (one >> line per distribution). >> >>> >>> > >>> > It is clearer and cleaner if the distribution docstring is a template, >>> > resides in the class docstring, and the final product is assembled by >>> > substitution rather than by concatenating magically generated parts. >>> > (Think, for example -- why are HTML templating languages so popular, as >>> > opposed to just concatenating blocks of HTML code?) >>> >>> The last time I looked at mako, it allowed full inheritance and backward >>> (?) >>> substitution in the templates. >>> >>> I don't see how template inheritance would work with simple string >>> substitution, but you are the experts for the numpy/scipy documentation >>> system. >> >> Inheritance is the wrong word. The idea is to build up a docstring from >> smaller pieces, instead of starting with one big template and then trying to >> change it. In the default (simplest) case, you build up a string in the >> module scope, only substitute for each distribution, and assign it >> to __doc__. >> >> Cheers, >> Ralf > > I like your initially proposed solution, because it allows both ways, either > full generic template (for distributions that nobody looks at), or following > the maxwell pattern for the distributions, where someone would want to > add individualized information. > > One other advantage of the generic docstring is that it is easy to create new > classes outside of scipy by subclassing rv_continuous and have the docstring > generated the same way as distribution in scipy. > > But even having your basic template is not really a problem, it's mainly > the numpy docstring standard and the template is not something we would > have to edit often. > (It would still double the lines of code for many distributions). > > I still like to keep name and (maybe) longname since they are used in > programs, if I remember correctly. > Personally, I like extradoc, that's the only really interesting information > about a distribution that I look at (besides dist.a and dist.b and dist.shape.) > and I don't have to scroll through the boiler plate docstring to read it. > > (In general, I think the class should get the default constructor information > in the __init__ method instead of in the instantiation.) > > Josef > While I was looking at the docstrings for the distributions, I thought, that the list of parameters is a bit misleading. They are listed as if they were the parameters for the constructor or for calling the instance. Example http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.maxwell.html#scipy.stats.maxwell The class is already instantiated, and __call__ takes (, loc and scale) as parameters. Maybe that's the reason that most user examples, I have seen, use the frozen distribution. The distribution instance is documented as if it were a class. However it looks different in the doc editor http://docs.scipy.org/scipy/docs/scipy.stats.maxwell/#scipy-stats-maxwell In the current generated docs, I don't see the methods of the individual distributions listed, e.g. norm.cdf. Methods only have the generic docstring (?), but method doc inheritance doesn't work for some reason? (I'm starting to get too confused about which version of the docs, I'm looking at.) Just another thought: The instance/class docstrings are currently missing a link to the methods of the distributions. If the methods can be linked to from the distributions, then I think it would be nicer to have only a minimal (generic) descriptions of the methods in the "class"/instance docstring for the individual distributions and at see also to the generic distribution class which could have some more explicit information. If the "boiler plate" doc in the instance docstring shrinks, we could add over time some real, distributions specific examples, if we have the templating structure in place without overloading the docstring. Currently the only templated but correctly adjusted signature for the methods is in the class/instance docstring, so maybe this doesn't work. Josef (It's pretty tiring to figure out or keep track of how help(obj), print obj.__doc__, ipython, the doceditor and sphinx are generating and interpreting generated docstrings.) > > > >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > From nkilleen at unimelb.edu.au Wed Oct 28 20:02:19 2009 From: nkilleen at unimelb.edu.au (Neil Killeen) Date: Thu, 29 Oct 2009 11:02:19 +1100 Subject: [SciPy-dev] scipy.test() failures In-Reply-To: <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> References: <566F7560-BB31-49CD-921E-DAA95171885A@unimelb.edu.au> <7CA9907F-2C5F-4BAA-BF6D-7B729F09B6EA@cs.toronto.edu> Message-ID: Test post with digest back on From ralf.gommers at googlemail.com Thu Oct 29 05:04:55 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Thu, 29 Oct 2009 10:04:55 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> References: <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> Message-ID: On Wed, Oct 28, 2009 at 10:00 PM, wrote: > On Wed, Oct 28, 2009 at 4:16 PM, Ralf Gommers > wrote: > > I like your initially proposed solution, because it allows both ways, > either > full generic template (for distributions that nobody looks at), or > following > the maxwell pattern for the distributions, where someone would want to > add individualized information. > > One other advantage of the generic docstring is that it is easy to create > new > classes outside of scipy by subclassing rv_continuous and have the > docstring > generated the same way as distribution in scipy. > Ah, I had not considered that. I will make sure not to break anything for subclasses that do not have their own docstring. So will keep name, longname, and extradoc. > > But even having your basic template is not really a problem, it's mainly > the numpy docstring standard and the template is not something we would > have to edit often. > (It would still double the lines of code for many distributions). > It wouldn't. The simplest classes are now 18-20 lines. I can probably keep it at the same length, and definitely below 25 lines. > > I still like to keep name and (maybe) longname since they are used in > programs, if I remember correctly. > Sure, let's keep name. longname is not an attribute so can not be used anywhere. > Personally, I like extradoc, that's the only really interesting information > about a distribution that I look at (besides dist.a and dist.b and > dist.shape.) > and I don't have to scroll through the boiler plate docstring to read it. > I'll keep it working the same, but really it should not be used. The info in it ends up in the wrong place. > > (In general, I think the class should get the default constructor > information > in the __init__ method instead of in the instantiation.) > Not sure what you mean, but for normal classes in __init__ is no better than in the class docstring. For the stats.distributions case, the instances should not have the __init__ parameters visible at all, since they can not be instantiated anymore. One more thought: rv_continuous and rv_discrete should really have their own docstrings as well. Something like """A generic continuous random variable class meant for subclassing. These are my methods, and here is how you extend/override them: xxx Parameters: xxx Notes: more info on how to subclass, how to create a docstring, xxx Examples: an example subclass """ Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Thu Oct 29 05:28:44 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Thu, 29 Oct 2009 10:28:44 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910281515l2035fb0fjfb4ad8742095178e@mail.gmail.com> References: <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> <1cd32cbb0910281515l2035fb0fjfb4ad8742095178e@mail.gmail.com> Message-ID: On Wed, Oct 28, 2009 at 11:15 PM, wrote: > > While I was looking at the docstrings for the distributions, I > thought, that the list of > parameters is a bit misleading. They are listed as if they were the > parameters > for the constructor or for calling the instance. Example > > > http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.maxwell.html#scipy.stats.maxwell > > The class is already instantiated, and __call__ takes (, loc > and scale) as parameters. > Confusing right now, I agree. Maybe the heading should be changed to "Instance call parameters" or similar, or no heading at and just explain it in text. > Maybe that's the reason that most user examples, I have seen, use the > frozen distribution. The > distribution instance is documented as if it were a class. > > The whole explanation on frozen distributions, and RV objects, is confusing. My impression is that you would rarely need this (is that right?) so it should probably be moved to a Notes section. However it looks different in the doc editor > http://docs.scipy.org/scipy/docs/scipy.stats.maxwell/#scipy-stats-maxwell > > This are reordered a bit. They really should look the same. For now let's only look at the doc editor, and if there are differences consider it a bug in the doc build process. > > In the current generated docs, I don't see the methods of the > individual distributions listed, > e.g. norm.cdf. Methods only have the generic docstring (?), but method > doc inheritance > doesn't work for some reason? > Do you mean that you expect to see the longer docstring of rv_continuous.cdf in the Methods section of the instance docstring? It does not work like that for any class. For normal classes public methods would have their own docstring and the method name in the class docstring turns into a link. That should ideally be the case here as well, maxwell.cdf should link to http://docs.scipy.org/scipy/docs/scipy.stats.distributions.rv_continuous.cdf/ > (I'm starting to get too confused about which version of the docs, I'm > looking at.) > > Just another thought: > > The instance/class docstrings are currently missing a link to the > methods of the distributions. > If the methods can be linked to from the distributions, then I think > it would be nicer to have only a minimal (generic) descriptions of the > methods in the "class"/instance docstring for the individual > distributions and at see also to the generic distribution class which > could have some more explicit information. Ah yes, see what I typed above. > If the "boiler plate" doc > in the instance docstring shrinks, we could add over time some real, > distributions specific examples, if we have the templating structure > in place without overloading the docstring. > Exactly, this is what I'm hoping to achieve. > Currently the only templated but correctly adjusted signature for the > methods is in the class/instance docstring, so maybe this doesn't > work. > > A correct signature on a method page that is linked will not be possible I think. Method signature should remain in the instance docstring. Josef > > (It's pretty tiring to figure out or keep track of how help(obj), > print obj.__doc__, ipython, the doceditor and sphinx are generating > and interpreting generated docstrings.) > > With your name.__class__.__doc__ fix they all look pretty similar. That said, I consider html and ipython most important and help() a distant third. (How can you even live without readline?). Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Thu Oct 29 10:33:47 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Thu, 29 Oct 2009 10:33:47 -0400 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1cd32cbb0910280652o677b95d5g37d0e938bb82d7df@mail.gmail.com> <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> <1cd32cbb0910281515l2035fb0fjfb4ad8742095178e@mail.gmail.com> Message-ID: <1cd32cbb0910290733q1d034c03g3db530bb10bec564@mail.gmail.com> On Thu, Oct 29, 2009 at 5:28 AM, Ralf Gommers wrote: > > > On Wed, Oct 28, 2009 at 11:15 PM, wrote: >> >> While I was looking at the docstrings for the distributions, I >> thought, that the list of >> parameters is a bit misleading. They are listed as if they were the >> parameters >> for the constructor or for calling the instance. Example >> >> >> http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.maxwell.html#scipy.stats.maxwell >> >> The class is already instantiated, and __call__ takes (, loc >> and scale) as parameters. > > Confusing right now, I agree. Maybe the heading should be changed to > "Instance call parameters" or similar, or no heading at and just explain it > in text. > >> >> Maybe that's the reason that most user examples, I have seen, use the >> frozen distribution. The >> distribution instance is documented as if it were a class. >> > The whole explanation on frozen distributions, and RV objects, is > confusing.? My impression is that you would rarely need this (is that > right?) so it should probably be moved to a Notes section. > >> However it looks different in the doc editor >> http://docs.scipy.org/scipy/docs/scipy.stats.maxwell/#scipy-stats-maxwell >> > This are reordered a bit. They really should look the same. For now let's > only look at the doc editor, and if there are differences consider it a bug > in the doc build process. >> >> In the current generated docs, I don't see the methods of the >> individual distributions listed, >> e.g. norm.cdf. Methods only have the generic docstring (?), but method >> doc inheritance >> doesn't work for some reason? > > Do you mean that you expect to see the longer docstring of rv_continuous.cdf > in the Methods section of the instance docstring? It does not work like that > for any class. For normal classes public methods would have their own > docstring and the method name in the class docstring turns into a link. That > should ideally be the case here as well, maxwell.cdf should link to > http://docs.scipy.org/scipy/docs/scipy.stats.distributions.rv_continuous.cdf/ > > >> >> (I'm starting to get too confused about which version of the docs, I'm >> looking at.) >> >> Just another thought: >> >> The instance/class docstrings are currently missing a link to the >> methods of the distributions. >> If the methods can be linked to from the distributions, then I think >> it would be nicer to have only a minimal (generic) descriptions of the >> methods in the "class"/instance docstring for the individual >> distributions and at see also to the generic distribution class which >> could have some more explicit information. > > Ah yes, see what I typed above. > >> >> If the "boiler plate" doc >> in the instance docstring shrinks, we could add over time some real, >> distributions specific examples, if we have the templating structure >> in place without overloading the docstring. > > Exactly, this is what I'm hoping to achieve. > >> >> Currently the only templated but correctly adjusted signature for the >> methods is in the class/instance docstring, so maybe this doesn't >> work. >> > A correct signature on a method page that is linked will not be possible I > think. Method signature should remain in the instance docstring. > > >> Josef >> >> (It's pretty tiring to figure out or keep track of how help(obj), >> print ?obj.__doc__, ipython, the doceditor and sphinx are generating >> and interpreting generated docstrings.) >> > With your name.__class__.__doc__ fix they all look pretty similar. That > said, I consider html and ipython most important and help() a distant third. > (How can you even live without readline?). I have to look at this again next week when I have a bit more time. I don't mind the boiler plate code in the individual distributions anymore, if we can improve the docs for the distributions this way. You could try it out on a few distributions to avoid having to edit 90 templates if there are any changes. Just a quick summary (or wishlist), I agree with most/all of your proposals * more informative (developer) docstrings for generic rv_continuous rv_discrete * keep individual signature of methods in distribution class docstring and clarify role of parameters and __call__ for frozen distribution; get links to generic method description * two ways of creating doc string (?) - generically generated for distributions nobody wants to edit, including minimal templating for non-scipy subclasses. - docstrings with individualized parts, where someone can edit at least notes, example and reference sections * SeeAlso to generic distribution class description and link to distribution in tutorial (?) Can we have links to numpy docs for random, but this wouldn't work in a local html or htmlhelp with different files? numpy.random has already a lot more descriptions for the individual distributions. * attach docs to __class__ instead of instance * ...? Two more issues: How much boiler plate do I have to edit if I add a new method to the generic classes, rv_continuous, rv_discrete or edit an existing one? Maybe it would be good to add an attribute description for a, b, numargs and shape to the class docstrings. In the generic classes, I would also like to document other (and some private) methods. Currently starting with underline is not a good indication for whether the method is considered private. Several private methods are missing the underline, and many of the private methods are useful (with some restrictions). Longer term question: How much information do we want in the docs about individual distributions. e.g. compared to http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxcdf.htm with some duplication in http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxpdf.htm and http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxppf.htm and can we induce editors to edit the notes in the distribution reference (in tutorial) instead of blowing up the notes section in the actual docstring. (following Pierre's suggestions). Is there a page limit on the size of the docs? I will be very glad, when we get proper documentation for classes in scipy. Thanks, Josef (no readline for me, it's to unixy. I use htmlhelp for the docs that are up-to-date, doceditor for missing or messy docs, and Idle, Spyder, Eclipse, or a plain windows command shell, when I don't want any interference from magic or background processes of the editor.) > Cheers, > Ralf > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From ralf.gommers at googlemail.com Thu Oct 29 11:27:57 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Thu, 29 Oct 2009 16:27:57 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910290733q1d034c03g3db530bb10bec564@mail.gmail.com> References: <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> <1cd32cbb0910281515l2035fb0fjfb4ad8742095178e@mail.gmail.com> <1cd32cbb0910290733q1d034c03g3db530bb10bec564@mail.gmail.com> Message-ID: Hi Joseph, I now have a pretty complete implementation. http://github.com/rgommers/doctemple/blob/master/newstats.py Features: - backwards compatible (see olddefault) - new default way with only one word of "boilerplate" (see newdefault) - pdf method descriptions in extradoc ends up in the right place (see wald) - ability to keep defaults for most elements while adding new sections etc where wanted (see maxwell) - rv_continuous can now have a useful docstring for itself, to show how to subclass it On Thu, Oct 29, 2009 at 3:33 PM, wrote: > > I have to look at this again next week when I have a bit more time. > > I don't mind the boiler plate code in the individual distributions anymore, > if we can improve the docs for the distributions this way. You could try > it out on a few distributions to avoid having to edit 90 templates if there > are any changes. > Will do. Not too much boilerplate anyway. For the common case of a small class that has only a couple of lines in `extradoc` that describe the distname.pdf, the number of lines per class is pretty much unchanged. For an example, see `wald`. Both current trunk and the new version are 18 lines. > > Just a quick summary (or wishlist), I agree with most/all of your proposals > > * more informative (developer) docstrings for generic rv_continuous > rv_discrete > ready to go. * keep individual signature of methods in distribution class docstring > done > and clarify role of parameters and __call__ for frozen distribution; > to be done, straightforward. > get links to generic method description > will have a look later, but this is a doc build issue > * two ways of creating doc string (?) > - generically generated for distributions nobody wants to edit, > including minimal templating for non-scipy subclasses. > done, works with no docstring at all, or a one-liner > - docstrings with individualized parts, where someone can > edit at least notes, example and reference sections > done > * SeeAlso to generic distribution class description and link > to distribution in tutorial (?) > sure > Can we have links to numpy docs for random, but this wouldn't > work in a local html or htmlhelp with different files? > numpy.random has already a lot more descriptions for the > individual distributions. > intersphinx is enabled but doesn't work (i think). Pauli, can you clarify? > * attach docs to __class__ instead of instance > I was not paying attention there. Assigning to __class__.__doc__ only works for old-style classes. rv_generic inherits from object, so it does not work then. If this is important to you to fix help() you can always convert to old-style classes. I had a quick look, and I don't think any of the old-style / new-style differences matter for stats.distributions. * ...? > > Two more issues: > > How much boiler plate do I have to edit if I add a new method to the > generic classes, rv_continuous, rv_discrete > or edit an existing one? > > Not much. For a new method `xxx` you would add a variable `_doc_xxx`, then add that to `_doc_allmethods` and `_doc_afterpdf`. That's it for all classes that did not write a custom Methods section. For the classes with custom Methods, you would add "%(xxx)s" in there. If you edit an existing method, you do not have to do anything. Maybe it would be good to add an attribute description for a, b, > numargs and shape to the class docstrings. > sure, will do > In the generic classes, I would also like to document other (and some > private) methods. Currently starting with underline is not a good > indication for whether the method is considered private. Several > private methods are missing the underline, and > many of the private methods are useful (with some restrictions). > makes sense, but you do not want them to show up in the instance docstring right? maybe only in the rv_continuous and rv_discrete docstrings? Longer term question: > > How much information do we want in the docs about individual > distributions. e.g. compared to > > > http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxcdf.htm > with some duplication in > > http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxpdf.htmand > > http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxppf.htm > > That seems a bit too much right? One page for each distribution method = 90 dist * 6 methods = many hundreds of pages. > and can we induce editors to edit the notes in the distribution > reference (in tutorial) instead of blowing up the > notes section in the actual docstring. (following Pierre's > suggestions). Is there a page limit on the size of the docs? > > Do you mean this page: http://docs.scipy.org/scipy/docs/scipy-docs/tutorial/stats.rst/? There is no info there yet on specific distributions. I like the numpy.random pages, for example http://docs.scipy.org/numpy/docs/numpy.random.mtrand.RandomState.standard_t/ Maybe one or two paragraphs more, that should be enough for pretty much any distribution function, or not? No idea about a page limit, but if you plan on writing more than a page and a half per distribution, you have enough material for a book..... > I will be very glad, when we get proper documentation for classes in scipy. > > That will make two of us. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Thu Oct 29 12:12:48 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Thu, 29 Oct 2009 12:12:48 -0400 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: References: <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> <1cd32cbb0910281515l2035fb0fjfb4ad8742095178e@mail.gmail.com> <1cd32cbb0910290733q1d034c03g3db530bb10bec564@mail.gmail.com> Message-ID: <1cd32cbb0910290912m26ee9cbap59e67cd4bea20543@mail.gmail.com> On Thu, Oct 29, 2009 at 11:27 AM, Ralf Gommers wrote: > Hi Joseph, > > I now have a pretty complete implementation. > http://github.com/rgommers/doctemple/blob/master/newstats.py > > Features: > - backwards compatible (see olddefault) > - new default way with only one word of "boilerplate" (see newdefault) > - pdf method descriptions in extradoc ends up in the right place (see wald) > - ability to keep defaults for most elements while adding new sections etc > where wanted (see maxwell) > - rv_continuous can now have a useful docstring for itself, to show how to > subclass it Thanks, Sounds all very good. I won't have time to look at it though until the weekend. > > > On Thu, Oct 29, 2009 at 3:33 PM, wrote: >> >> I have to look at this again next week when I have a bit more time. >> >> I don't mind the boiler plate code in the individual distributions >> anymore, >> if we can improve the docs for the distributions this way. You could try >> it out on a few distributions to avoid having to edit 90 templates if >> there >> are any changes. > > Will do. Not too much boilerplate anyway. For the common case of a small > class that has only a couple of lines in `extradoc` that describe the > distname.pdf, the number of lines per class is pretty much unchanged. For an > example, see `wald`. Both current trunk and the new version are 18 lines. >> >> Just a quick summary (or wishlist), I agree with most/all of your >> proposals >> >> * more informative (developer) docstrings for generic rv_continuous >> ?rv_discrete > > ready to go. > >> * keep individual signature of methods in distribution class docstring > > done >> >> ?and clarify role of parameters and __call__ for frozen distribution; > > to be done, straightforward. >> >> ?get links to generic method description > > will have a look later, but this is a doc build issue > >> >> * two ways of creating doc string (?) >> ?- generically generated for distributions nobody wants to edit, >> ? ?including minimal templating for non-scipy subclasses. > > done, works with no docstring at all, or a one-liner >> >> ?- docstrings with individualized parts, where someone can >> ? ?edit at least notes, example and reference sections > > done > >> >> * SeeAlso to generic distribution class description and link >> ?to distribution in tutorial (?) > > sure >> >> ?Can we have links to numpy docs for random, but this wouldn't >> ?work in a local html or htmlhelp with different files? >> >> ?numpy.random has already a lot more descriptions for the >> ?individual distributions. > > intersphinx is enabled but doesn't work (i think). Pauli, can you clarify? > >> >> * attach docs to __class__ instead of instance > > I was not paying attention there. Assigning to __class__.__doc__ only works > for old-style classes. rv_generic inherits from object, so it does not work > then. If this is important to you to fix help() you can always convert to > old-style classes. I had a quick look, and I don't think any of the > old-style / new-style differences matter for stats.distributions. I thought that's what you did earlier. I will check again. > >> * ...? >> >> Two more issues: >> >> How much boiler plate do I have to edit if I add a new method to the >> generic classes, rv_continuous, rv_discrete >> or edit an existing one? >> > Not much. For a new method `xxx` you would add a variable `_doc_xxx`,? then > add that to `_doc_allmethods` and `_doc_afterpdf`. That's it for all classes > that did not write a custom Methods section. For the classes with custom > Methods, you would add "%(xxx)s" in there. > > If you edit an existing method, you do not have to do anything. That's very good, no boiler plate to edit. > > >> Maybe it would be good to add an attribute description for a, b, >> numargs and shape to the class docstrings. > > sure, will do > >> >> In the generic classes, I would also like to document other (and some >> private) methods. Currently starting with underline is not a good >> indication for whether the method is considered private. Several >> private methods are missing the underline, and >> many of the private methods are useful (with some restrictions). > > makes sense, but you do not want them to show up in the instance docstring > right? maybe only in the rv_continuous and rv_discrete docstrings? I think only in rv_continuous and rv_discrete docstrings, for those users that want to look closer, without confusing the user about the basic usage. > > >> Longer term question: >> >> How much information do we want in the docs about individual >> distributions. e.g. compared to >> >> >> http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxcdf.htm >> with some duplication in >> >> http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxpdf.htm >> and >> >> http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxppf.htm >> > That seems a bit too much right? One page for each distribution method = 90 > dist * 6 methods = many hundreds of pages. For us it should be more like one page description per distribution, makes 90 pages. We could add the scripts for the graphs, instead of the actual graphs, as it is done in the current class docstring. >> >> and can we induce editors to edit the notes in the distribution >> reference (in tutorial) instead of blowing up the >> notes section in the actual docstring. (following Pierre's >> suggestions). Is there a page limit on the size of the docs? >> > Do you mean this page: > http://docs.scipy.org/scipy/docs/scipy-docs/tutorial/stats.rst/? There is no > info there yet on specific distributions. The links are a bit difficult to see, in the second paragraph http://docs.scipy.org/scipy/docs/scipy-docs/tutorial/stats/continuous.rst/#continuous-random-variables http://docs.scipy.org/scipy/docs/scipy-docs/tutorial/stats/discrete.rst/#discrete-random-variables I think it was Gokhan Sever who converted the lynx docs of Travis to rst. The first is currently a huge page that takes a long time to load. It will eventually need to be broken up. > > I like the numpy.random pages, for example > http://docs.scipy.org/numpy/docs/numpy.random.mtrand.RandomState.standard_t/ > Maybe one or two paragraphs more, that should be enough for pretty much any > distribution function, or not? Yes, the notes in random are good, in scipy we have more formulas for the properties of the distributions, that can be calculated using the distribution methods. I don't know whether we want to copy or link to the description in numpy.random. > > No idea about a page limit, but if you plan on writing more than a page and > a half per distribution, you have enough material for a book..... Some packages, I have seen, have an almost book length description of the distributions. The one, I was reading more, was, however, distributed as a separate pdf file. > >> >> I will be very glad, when we get proper documentation for classes in >> scipy. >> > That will make two of us. (When you are ready to do the next group, the docs for classes in interpolate are bothering me for a long time, and several more once I see a clear pattern how classes and modules should be documented.) Thanks, Josef > > Cheers, > Ralf > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > From mellerf at netvision.net.il Thu Oct 29 13:04:16 2009 From: mellerf at netvision.net.il (Yosef Meller) Date: Thu, 29 Oct 2009 19:04:16 +0200 Subject: [SciPy-dev] Lambert W implementation Message-ID: <4AE9CB10.2090503@netvision.net.il> Hello, I searched through scipy.special and I saw no implementation of the Lambert W function. I'd like to add one, but before I do I thought I'd ask: 1) Did I just miss it? 2) Is there any implementation you know off that is suitable for wrapping or a good reference for an algorithm (I can find some, but if there's something well known...) Yours, Yosef. From dwf at cs.toronto.edu Thu Oct 29 14:34:59 2009 From: dwf at cs.toronto.edu (David Warde-Farley) Date: Thu, 29 Oct 2009 14:34:59 -0400 Subject: [SciPy-dev] Lambert W implementation In-Reply-To: <4AE9CB10.2090503@netvision.net.il> References: <4AE9CB10.2090503@netvision.net.il> Message-ID: <27437EC3-7E8D-4E11-BD57-FCD70C427443@cs.toronto.edu> On 29-Oct-09, at 1:04 PM, Yosef Meller wrote: > 1) Did I just miss it? Not as far as I can see. > 2) Is there any implementation you know off that is suitable for > wrapping or a good reference for an algorithm (I can find some, but if > there's something well known...) There is also the concern of licenses. i.e. there is one on netlib.org but it is part of the ACM collection which is not suitable for SciPy. David From jsseabold at gmail.com Thu Oct 29 14:54:31 2009 From: jsseabold at gmail.com (Skipper Seabold) Date: Thu, 29 Oct 2009 14:54:31 -0400 Subject: [SciPy-dev] Lambert W implementation In-Reply-To: <4AE9CB10.2090503@netvision.net.il> References: <4AE9CB10.2090503@netvision.net.il> Message-ID: On Thu, Oct 29, 2009 at 1:04 PM, Yosef Meller wrote: > Hello, > > I searched through scipy.special and I saw no implementation of the > Lambert W function. I'd like to add one, but before I do I thought I'd ask: > > 1) Did I just miss it? > 2) Is there any implementation you know off that is suitable for > wrapping or a good reference for an algorithm (I can find some, but if > there's something well known...) There's a Python example on the wikipedia page for Lambert W that works for W(x) where x >= -1/e. There's probably more information in the references for more general implementations and/or improved accuracy. From pav+sp at iki.fi Thu Oct 29 17:30:13 2009 From: pav+sp at iki.fi (Pauli Virtanen) Date: Thu, 29 Oct 2009 21:30:13 +0000 (UTC) Subject: [SciPy-dev] Lambert W implementation References: <4AE9CB10.2090503@netvision.net.il> Message-ID: Thu, 29 Oct 2009 14:54:31 -0400, Skipper Seabold wrote: > On Thu, Oct 29, 2009 at 1:04 PM, Yosef Meller > wrote: >> Hello, >> >> I searched through scipy.special and I saw no implementation of the >> Lambert W function. I'd like to add one, but before I do I thought I'd >> ask: >> >> 1) Did I just miss it? >> 2) Is there any implementation you know off that is suitable for >> wrapping or a good reference for an algorithm (I can find some, but if >> there's something well known...) > > There's a Python example on the wikipedia page for Lambert W that works > for W(x) where x >= -1/e. There's probably more information in the > references for more general implementations and/or improved accuracy. There's an implementation of lambertw in mpmath. It probably doesn't rely on arbitrary precision either, so might just work with floats. -- Pauli Virtanen From jakevdp at gmail.com Thu Oct 29 18:01:57 2009 From: jakevdp at gmail.com (Jake VanderPlas) Date: Thu, 29 Oct 2009 15:01:57 -0700 Subject: [SciPy-dev] Ball Tree class Message-ID: <58df6dc20910291501gdf4ea83k7817147616fdf5be@mail.gmail.com> Hello, I've been using scipy.spatial.KDTree for my research, and found it very useful. Recently, though, my datasets have been getting larger - upwards of 10,000 points in 1000-2000 dimensions. The KDTree starts getting slow with such a large dimensionality. I've addressed this by writing a C++ Ball Tree code, and wrapping it using swig and numpy.i. I've been wanting to begin contributing to the scipy project, and I think this would be a great place to start. I'd like to begin the process of adding this to the scipy.spatial package. A few questions: Is it preferable to have an implementation in C rather than C++? Cython, swig, or hand-wrapped code? cKDTree is written in Cython, with C. Should I stick to that convention to maintain uniformity in the scipy.spatial package? Please let me know what you think -Jake From robert.kern at gmail.com Thu Oct 29 18:10:42 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 29 Oct 2009 17:10:42 -0500 Subject: [SciPy-dev] Ball Tree class In-Reply-To: <58df6dc20910291501gdf4ea83k7817147616fdf5be@mail.gmail.com> References: <58df6dc20910291501gdf4ea83k7817147616fdf5be@mail.gmail.com> Message-ID: <3d375d730910291510o783f540fn89f9d486e512037a@mail.gmail.com> On Thu, Oct 29, 2009 at 17:01, Jake VanderPlas wrote: > Hello, > I've been using scipy.spatial.KDTree for my research, and found it > very useful. ?Recently, though, my datasets have been getting larger - > upwards of 10,000 points in 1000-2000 dimensions. > The KDTree starts getting slow with such a > large dimensionality. ?I've addressed this by writing a C++ Ball Tree > code, and wrapping it using swig and numpy.i. ?I've been wanting to > begin contributing to the scipy project, and I think this would be a > great place to start. ?I'd like to begin the process of adding > this to the scipy.spatial package. That would be fantastic! > A few questions: Is it preferable to have an > implementation in C rather than C++? I'd rather have well designed code and using a language that is appropriate for the design. I suspect either one will be okay for your task. >?Cython, swig, or hand-wrapped code? > cKDTree is written in Cython, with C. ?Should I stick to that > convention to maintain uniformity in the scipy.spatial package? I try to avoid SWIG these days for my usual wrapping tasks. I do recommend trying out Cython. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From david at ar.media.kyoto-u.ac.jp Fri Oct 30 01:58:40 2009 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 30 Oct 2009 14:58:40 +0900 Subject: [SciPy-dev] Ball Tree class In-Reply-To: <58df6dc20910291501gdf4ea83k7817147616fdf5be@mail.gmail.com> References: <58df6dc20910291501gdf4ea83k7817147616fdf5be@mail.gmail.com> Message-ID: <4AEA8090.4070400@ar.media.kyoto-u.ac.jp> Jake VanderPlas wrote: > Hello, > I've been using scipy.spatial.KDTree for my research, and found it > very useful. Recently, though, my datasets have been getting larger - > upwards of 10,000 points in 1000-2000 dimensions. > The KDTree starts getting slow with such a > large dimensionality. I've addressed this by writing a C++ Ball Tree > code, and wrapping it using swig and numpy.i. I've been wanting to > begin contributing to the scipy project, and I think this would be a > great place to start. I'd like to begin the process of adding > this to the scipy.spatial package. > > A few questions: Is it preferable to have an > implementation in C rather than C++? Cython, swig, or hand-wrapped code? Cython is nice to wrap code, but if your code is C++, you may have to create a C API around it. I know Cython developers want to improve support for C++, but I don't know the status of it. If you use C++, stick to the standardized STL: no boost whatsoever, no hash_map, etc... Avoid exception as much as possible as well. cheers, David From mellerf at netvision.net.il Fri Oct 30 04:54:40 2009 From: mellerf at netvision.net.il (Yosef Meller) Date: Fri, 30 Oct 2009 10:54:40 +0200 Subject: [SciPy-dev] Lambert W implementation In-Reply-To: References: <4AE9CB10.2090503@netvision.net.il> Message-ID: <4AEAA9D0.9050106@netvision.net.il> ????? Pauli Virtanen: > Thu, 29 Oct 2009 14:54:31 -0400, Skipper Seabold wrote: >> On Thu, Oct 29, 2009 at 1:04 PM, Yosef Meller >> wrote: >>> Hello, >>> >>> I searched through scipy.special and I saw no implementation of the >>> Lambert W function. I'd like to add one, but before I do I thought I'd >>> ask: >>> >>> 1) Did I just miss it? >>> 2) Is there any implementation you know off that is suitable for >>> wrapping or a good reference for an algorithm (I can find some, but if >>> there's something well known...) >> There's a Python example on the wikipedia page for Lambert W that works >> for W(x) where x >= -1/e. There's probably more information in the >> references for more general implementations and/or improved accuracy. > > There's an implementation of lambertw in mpmath. It probably doesn't rely > on arbitrary precision either, so might just work with floats. Ok, the mpmath implementation seems suitable, and they also have a reference and some test values. What I see as the way forward is to take all mpmath-related wrapping out and provide a simpler floating-point version. I think I'll do a Cython function for scalar argument, then count on the usual vectorization facilities in NumPy to handle the rest (np.vectorize). From pav+sp at iki.fi Fri Oct 30 05:28:24 2009 From: pav+sp at iki.fi (Pauli Virtanen) Date: Fri, 30 Oct 2009 09:28:24 +0000 (UTC) Subject: [SciPy-dev] Lambert W implementation References: <4AE9CB10.2090503@netvision.net.il> <4AEAA9D0.9050106@netvision.net.il> Message-ID: Fri, 30 Oct 2009 10:54:40 +0200, Yosef Meller wrote: [clip] > Ok, the mpmath implementation seems suitable, and they also have a > reference and some test values. What I see as the way forward is to take > all mpmath-related wrapping out and provide a simpler floating-point > version. I think I'll do a Cython function for scalar argument, then > count on the usual vectorization facilities in NumPy to handle the rest > (np.vectorize). It's faster to make it a Ufunc -- np.vectorize will likely sacrifice much of the speed gained from using Cython since it's going to box and unbox numbers in Python floats. There's an example how to do Ufuncs in Cython here: http://projects.scipy.org/scipy/browser/trunk/scipy/special/orthogonal_eval.pyx If you get it done, please post the result (preferably, create a ticket on http://projects.scipy.org/scipy/ and attach the file) and we'll integrate it in scipy.special. If you also write tests, that'd be great! Note that newer Cythons also have support for native complex numbers, so you can do cdef double complex a = ... The same algorithm probably works both for real and complex parameters. -- Pauli Virtanen From ralf.gommers at googlemail.com Fri Oct 30 06:55:07 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Fri, 30 Oct 2009 11:55:07 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910290912m26ee9cbap59e67cd4bea20543@mail.gmail.com> References: <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> <1cd32cbb0910281515l2035fb0fjfb4ad8742095178e@mail.gmail.com> <1cd32cbb0910290733q1d034c03g3db530bb10bec564@mail.gmail.com> <1cd32cbb0910290912m26ee9cbap59e67cd4bea20543@mail.gmail.com> Message-ID: On Thu, Oct 29, 2009 at 5:12 PM, wrote: > >> > >> * attach docs to __class__ instead of instance > > > > I was not paying attention there. Assigning to __class__.__doc__ only > works > > for old-style classes. rv_generic inherits from object, so it does not > work > > then. If this is important to you to fix help() you can always convert to > > old-style classes. I had a quick look, and I don't think any of the > > old-style / new-style differences matter for stats.distributions. > > I thought that's what you did earlier. I will check again. > > Yes but then I forgot I removed `object`. It seems fine, but it would be a little impolite from me to convert all classes to old-style without asking. Also, in Python 3 it will stop working because there are no old-style classes anymore. > > > > > > >> Longer term question: > >> > >> How much information do we want in the docs about individual > >> distributions. e.g. compared to > >> > >> > >> > http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxcdf.htm > >> with some duplication in > >> > >> > http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxpdf.htm > >> and > >> > >> > http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/maxppf.htm > >> > > That seems a bit too much right? One page for each distribution method = > 90 > > dist * 6 methods = many hundreds of pages. > > For us it should be more like one page description per distribution, > makes 90 pages. > We could add the scripts for the graphs, instead of the actual graphs, > as it is done in the current class docstring. > That seems reasonable. > > The links are a bit difficult to see, in the second paragraph > > > http://docs.scipy.org/scipy/docs/scipy-docs/tutorial/stats/continuous.rst/#continuous-random-variables > > http://docs.scipy.org/scipy/docs/scipy-docs/tutorial/stats/discrete.rst/#discrete-random-variables > > I think it was Gokhan Sever who converted the lynx docs of Travis to > rst. The first is currently a huge page that takes a long time to > load. It will eventually need to be broken up. > Ah very nice, I had not seen that page before. The docstrings should link there. It's mostly math anyway and would not work well in the docstrings themselves. > > > > > I like the numpy.random pages, for example > > > http://docs.scipy.org/numpy/docs/numpy.random.mtrand.RandomState.standard_t/ > > Maybe one or two paragraphs more, that should be enough for pretty much > any > > distribution function, or not? > > Yes, the notes in random are good, in scipy we have more formulas for > the properties of the distributions, that can be calculated using the > distribution methods. > I don't know whether we want to copy or link to the description in > numpy.random. > > Linking is better I think. > > > > No idea about a page limit, but if you plan on writing more than a page > and > > a half per distribution, you have enough material for a book..... > > Some packages, I have seen, have an almost book length description of > the distributions. The one, I was reading more, was, however, > distributed as a separate pdf file. > > It may indeed be a good idea to have a separate pdf of stats docs. The whole scipy reference guide pdf is already 710 pages and will eventually be a couple of times larger than that. > (When you are ready to do the next group, the docs for classes in > interpolate are bothering me for a long time, and several more once I > see a clear pattern how classes and modules should be documented.) > > Well the scipy doc editor says 5% worked on, 95% still untouched, so pretty much all of them still bother me:) Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Fri Oct 30 10:45:54 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Fri, 30 Oct 2009 15:45:54 +0100 Subject: [SciPy-dev] ndimage - docfiller and output_type In-Reply-To: <1cd32cbb0910290733q1d034c03g3db530bb10bec564@mail.gmail.com> References: <1cd32cbb0910280813t6c3655b7n68bdb8cbac57c9f0@mail.gmail.com> <1cd32cbb0910281234j540934d8kf53d253c6599e392@mail.gmail.com> <1cd32cbb0910281400p894d32cidac20aa450cf9391@mail.gmail.com> <1cd32cbb0910281515l2035fb0fjfb4ad8742095178e@mail.gmail.com> <1cd32cbb0910290733q1d034c03g3db530bb10bec564@mail.gmail.com> Message-ID: On Thu, Oct 29, 2009 at 3:33 PM, wrote: > I have to look at this again next week when I have a bit more time. > > I don't mind the boiler plate code in the individual distributions anymore, > if we can improve the docs for the distributions this way. You could try > it out on a few distributions to avoid having to edit 90 templates if there > are any changes. > > Josef, the changes are here: http://github.com/rgommers/scipy/tree/statsdocsFor examples of new docstrings, see maxwell, wald and nbinom. Do you use git, or would you prefer it if I opened a ticket and attached a diff? The things I was not sure about: 1. do you want the old-style classes with better help()? I actually consider this a bug: >>> help(wald) Help on wald_gen ..... # I asked for help on wald, not wald_gen 2. Where should doccer.py live? Now I do "from scipy.ndimage import doccer", but doccer is useful in multiple SciPy modules, and probably in NumPy as well. Could we move it to NumPy? Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Fri Oct 30 21:21:28 2009 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 30 Oct 2009 19:21:28 -0600 Subject: [SciPy-dev] module docstrings In-Reply-To: References: Message-ID: On Sun, Oct 25, 2009 at 5:58 AM, Ralf Gommers wrote: > Hi all, > > The doc standard does not say anything about module docstrings, and before > starting to clean those up it would be nice to agree on a format. > > Right now there are modules that: > - do not have any docstring (matrixlib, fromnumeric, ..) > - have license/author info in the docstring (scipy.interpolate.fitpack, ..) > - list their routines (linalg, core, lib, ..) > * some of these have a routines rst doc: routines.linalg.rst exists, > routines.lib.rst does not. > - have a single summary line (distutils, ..) > - have some basic explanation (ma, scimath, ..) > > > What do you all think about the following? : > > 1. every module needs a docstring with at least a summary line. > > 2. follow the doc standard where relevant, so typically: > - summary line > - extended summary paragraph(s) > - see also, if needed > - notes, if needed > - references, if needed > > 3. no routine listing in the docstring, each module gets a corresponding > routines.module.rst file > > I like routine listings. They may not add much to the processed documentation, but I find it helpful to have routines listed up top with some short notes when looking at the source. It is also an easy way to see what is in a module without having to browse through its directory listing. 4. license and author info can be in the source file, but not in the > docstring. (same principle for author info in reST docs by the way). > > Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at googlemail.com Sat Oct 31 10:00:58 2009 From: ralf.gommers at googlemail.com (Ralf Gommers) Date: Sat, 31 Oct 2009 15:00:58 +0100 Subject: [SciPy-dev] module docstrings In-Reply-To: References: Message-ID: On Sat, Oct 31, 2009 at 2:21 AM, Charles R Harris wrote: > > > On Sun, Oct 25, 2009 at 5:58 AM, Ralf Gommers > wrote: > >> Hi all, >> >> The doc standard does not say anything about module docstrings, and before >> starting to clean those up it would be nice to agree on a format. >> >> Right now there are modules that: >> - do not have any docstring (matrixlib, fromnumeric, ..) >> - have license/author info in the docstring (scipy.interpolate.fitpack, >> ..) >> - list their routines (linalg, core, lib, ..) >> * some of these have a routines rst doc: routines.linalg.rst exists, >> routines.lib.rst does not. >> - have a single summary line (distutils, ..) >> - have some basic explanation (ma, scimath, ..) >> >> >> What do you all think about the following? : >> >> 1. every module needs a docstring with at least a summary line. >> >> 2. follow the doc standard where relevant, so typically: >> - summary line >> - extended summary paragraph(s) >> - see also, if needed >> - notes, if needed >> - references, if needed >> >> 3. no routine listing in the docstring, each module gets a corresponding >> routines.module.rst file >> >> > I like routine listings. They may not add much to the processed > documentation, but I find it helpful to have routines listed up top with > some short notes when looking at the source. It is also an easy way to see > what is in a module without having to browse through its directory listing. > There are actually only 3 modules (+2 by mistake) in all of NumPy that have listings if I counted correctly. If you think those are useful maybe we can make it an optional section (i.e. they should not be deleted, but most modules don't need them)? In NumPy, the ones with listings are: - numerictypes - fftpack - arraysetops - random (accidental copy of random.info) - linalg (accidental copy of linalg.info) In addition there are a few large modules with listings in separate info files: - core - lib - random - linalg - fft In SciPy listings for modules (stats, interpolate, integrate, special, ...) also reside in info files. Cheers, Ralf > > 4. license and author info can be in the source file, but not in the >> docstring. (same principle for author info in reST docs by the way). >> >> > Chuck > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tpk at kraussfamily.org Sat Oct 31 16:02:35 2009 From: tpk at kraussfamily.org (Tom K.) Date: Sat, 31 Oct 2009 20:02:35 +0000 (UTC) Subject: [SciPy-dev] module docstrings References: Message-ID: Ralf Gommers googlemail.com> writes: > > > On Sat, Oct 31, 2009 at 2:21 AM, Charles R Harris gmail.com> wrote: > > On Sun, Oct 25, 2009 at 5:58 AM, Ralf Gommers googlemail.com> wrote: > I like routine listings.... > MATLAB has a default behavior for documenting a directory on the path when you don't have a Contents.m in that directory: it pulls the "H1" line (first line of the M-file's help) from each M-file and lists those for you. But the Contents.m usually was nicely formatted and grouped according to function, with a list of each M-file with a 1-line summary. FWIW. I kind of like a listing too, but not all functions in all modules warrant listing at the module level - there is an asymmetry problem. From josef.pktd at gmail.com Sat Oct 31 16:28:34 2009 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Sat, 31 Oct 2009 16:28:34 -0400 Subject: [SciPy-dev] module docstrings In-Reply-To: References: Message-ID: <1cd32cbb0910311328g731d5063n2587545e5a8f1fee@mail.gmail.com> On Sat, Oct 31, 2009 at 4:02 PM, Tom K. wrote: > Ralf Gommers googlemail.com> writes: > >> >> >> On Sat, Oct 31, 2009 at 2:21 AM, Charles R Harris > gmail.com> wrote: >> >> On Sun, Oct 25, 2009 at 5:58 AM, Ralf Gommers > googlemail.com> wrote: >> I like routine listings.... >> > MATLAB has a default behavior for documenting a directory on the path when > you don't have a Contents.m in that directory: it pulls the "H1" line (first > line of the M-file's help) from each M-file and lists those for you. > But the Contents.m usually was nicely formatted and grouped according > to function, with a list of each M-file with a 1-line summary. > FWIW. > I kind of like a listing too, but not all functions in all modules warrant > listing at the module level - there is an asymmetry problem. > > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > I'm not sure the info files in scipy are kept up to date. Since we moved the documentation to the rst files, I haven't looked at info.py anymore, except for those packages that have an automodule directive and load the info script. (If I'm not mistaken about the import mechanism for the docs.) I just rely on dir(modulename) to get the actual listing, or better in some cases modulename.__all__ for only the public functions matlab allows you to generate the contents.m file automatically or semi-automatically, which makes it easier to maintain. Josef