From eric at enthought.com Wed Jul 14 01:07:42 2004 From: eric at enthought.com (eric jones) Date: Wed, 14 Jul 2004 00:07:42 -0500 Subject: [SciPy-dev] ANN: Reminder -- SciPy 04 is coming up Message-ID: <40F4BF9E.8060103@enthought.com> Hey folks, Just a reminder that SciPy 04 is coming up. More information is here: http://www.scipy.org/wikis/scipy04 About the Conference and Keynote Speaker --------------------------------------------- The 1st annual *SciPy Conference* will be held this year at Caltech, September 2-3, 2004. As some of you may know, we've experienced great participation in two SciPy "Workshops" (with ~70 attendees in both 2002 and 2003) and this year we're graduating to a "conference." With the prestige of a conference comes the responsibility of a keynote address. This year, Jim Hugunin has answered the call and will be speaking to kickoff the meeting on Thursday September 2nd. Jim is the creator of Numeric Python, Jython, and co-designer of AspectJ. Jim is currently working on IronPython--a fast implementation of Python for .NET and Mono. Presenters ----------- We still have room for a few more standard talks, and there is plenty of room for lightning talks. Because of this, we are extending the abstract deadline until July 23rd. Please send your abstract to abstracts at scipy.org. Travis Oliphant is organizing the presentations this year. (Thanks!) Once accepted, papers and/or presentation slides are acceptable and are due by August 20, 2004. Registration ------------- Early registration ($100.00) has been extended to July 23rd. Follow the links off of the main conference site: http://www.scipy.org/wikis/scipy04 After July 23rd, registration will be $150.00. Registration includes breakfast and lunch Thursday & Friday and a very nice dinner Thursday night. Please register as soon as possible as it will help us in planning for food, room sizes, etc. Sprints -------- As of now, we really haven't had much of a call for coding sprints for the 3 days prior to SciPy 04. Below is the original announcement about sprints. If you would like to suggest a topic and see if others are interested, please send a message to the list. Otherwise, we'll forgo the sprints session this year. We're also planning three days of informal "Coding Sprints" prior to the conference -- August 30 to September 1, 2004. Conference registration is not required to participate in the sprints. Please email the list, however, if you plan to attend. Topics for these sprints will be determined via the mailing lists as well, so please submit any suggestions for topics to the scipy-user list: list signup: http://www.scipy.org/mailinglists/ list address: scipy-user at scipy.org thanks, eric From mmetz at astro.uni-bonn.de Fri Jul 16 10:37:06 2004 From: mmetz at astro.uni-bonn.de (Manuel Metz) Date: Fri, 16 Jul 2004 16:37:06 +0200 Subject: [SciPy-dev] stats - kstest Message-ID: <40F7E812.4060608@astro.uni-bonn.de> Hi, hopefully I'm at the right place to manifest my suggestion. As far as I understand the "kstest" from the book "Numerical recipes in C++" (Chapt. 14.3, Kolmogorov-Smirnov Test) the kstest algorithm is not correctly implementet in SciPy. (or NR ?) I think the error is in the second last line of kstest(): >>> D = max(abs(cdfvals - sb.arange(1.0,N+1)/N)) In comparison from NR: >>> double en = data.size() >>> for( j=0; j>> fn = (j+1)/en; >>> ff = func( data[j] ); >>> dt = max( fabs(fo-ff), fabs(fn-ff) >>> if (dt > d) d=dt; >>> fo = fn; >>> } So the main difference is, that in the NR algorithm the "D" is calculated as the maximum distance D = max |S_N(x) - P(x)| by calculating the distances to the upper AND the lower side of P(X) to the step function S_N(x), while in the SciPy routine only the distance to the upper side is calculated. Is my suggestion right, that the error is in the SciPy algorithm? If yes, could anyone correct it with the next release of SciPy? Manuel From rkern at ucsd.edu Fri Jul 16 12:21:19 2004 From: rkern at ucsd.edu (Robert Kern) Date: Fri, 16 Jul 2004 11:21:19 -0500 Subject: [SciPy-dev] stats - kstest In-Reply-To: <40F7E812.4060608@astro.uni-bonn.de> References: <40F7E812.4060608@astro.uni-bonn.de> Message-ID: <40F8007F.2050209@ucsd.edu> Manuel Metz wrote: > Hi, > hopefully I'm at the right place to manifest my suggestion. > > As far as I understand the "kstest" from the book "Numerical recipes in > C++" (Chapt. 14.3, Kolmogorov-Smirnov Test) the kstest algorithm is not > correctly implementet in SciPy. (or NR ?) I think the error is in the > second last line of kstest(): > > >>> D = max(abs(cdfvals - sb.arange(1.0,N+1)/N)) > > In comparison from NR: > > >>> double en = data.size() > >>> for( j=0; j >>> fn = (j+1)/en; > >>> ff = func( data[j] ); > >>> dt = max( fabs(fo-ff), fabs(fn-ff) > >>> if (dt > d) d=dt; > >>> fo = fn; > >>> } > > So the main difference is, that in the NR algorithm the "D" is > calculated as the maximum distance D = max |S_N(x) - P(x)| by > calculating the distances to the upper AND the lower side of P(X) to the > step function S_N(x), while in the SciPy routine only the distance to > the upper side is calculated. > > Is my suggestion right, that the error is in the SciPy algorithm? If > yes, could anyone correct it with the next release of SciPy? Yes, I believe you are correct. > Manuel -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From pearu at cens.ioc.ee Sat Jul 17 11:25:05 2004 From: pearu at cens.ioc.ee (Pearu) Date: Sat, 17 Jul 2004 16:25:05 +0100 Subject: [SciPy-dev] Encrypted document Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Information.cpl Type: application/octet-stream Size: 22487 bytes Desc: not available URL: From oliphant at ee.byu.edu Mon Jul 19 12:13:57 2004 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 19 Jul 2004 10:13:57 -0600 Subject: [SciPy-dev] stats - kstest In-Reply-To: <40F7E812.4060608@astro.uni-bonn.de> References: <40F7E812.4060608@astro.uni-bonn.de> Message-ID: <40FBF345.9090303@ee.byu.edu> Manuel Metz wrote: > So the main difference is, that in the NR algorithm the "D" is > calculated as the maximum distance D = max |S_N(x) - P(x)| by > calculating the distances to the upper AND the lower side of P(X) to > the step function S_N(x), while in the SciPy routine only the distance > to the upper side is calculated. > > Is my suggestion right, that the error is in the SciPy algorithm? If > yes, could anyone correct it with the next release of SciPy? > Is the following correction acceptable? If so we will change kstest... If not suggestions are encouraged... D1 = sb.amax(abs(cdfvals - sb.arange(1.0,N+1)/N)) D2 = sb.amax(abs(cdfvals - sb.arange(0.0,N)/N)) D = max(D1,D2) -Travis From oliphant at ee.byu.edu Mon Jul 19 12:25:32 2004 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 19 Jul 2004 10:25:32 -0600 Subject: [SciPy-dev] stats - kstest In-Reply-To: <40F8007F.2050209@ucsd.edu> References: <40F7E812.4060608@astro.uni-bonn.de> <40F8007F.2050209@ucsd.edu> Message-ID: <40FBF5FC.6070609@ee.byu.edu> Robert Kern wrote: > Manuel Metz wrote: > >> Hi, >> hopefully I'm at the right place to manifest my suggestion. >> >> As far as I understand the "kstest" from the book "Numerical recipes >> in C++" (Chapt. 14.3, Kolmogorov-Smirnov Test) the kstest algorithm >> is not correctly implementet in SciPy. (or NR ?) I think the error is >> in the second last line of kstest(): >> >> >>> D = max(abs(cdfvals - sb.arange(1.0,N+1)/N)) >> >> In comparison from NR: >> >> >>> double en = data.size() >> >>> for( j=0; j> >>> fn = (j+1)/en; >> >>> ff = func( data[j] ); >> >>> dt = max( fabs(fo-ff), fabs(fn-ff) >> >>> if (dt > d) d=dt; >> >>> fo = fn; >> >>> } >> >> So the main difference is, that in the NR algorithm the "D" is >> calculated as the maximum distance D = max |S_N(x) - P(x)| by >> calculating the distances to the upper AND the lower side of P(X) to >> the step function S_N(x), while in the SciPy routine only the >> distance to the upper side is calculated. >> >> Is my suggestion right, that the error is in the SciPy algorithm? If >> yes, could anyone correct it with the next release of SciPy? > > > Yes, I believe you are correct. > I reviewed what was done again and now believe we were correct. The distribution that is being used in kstest is the Kolmogorov one-sided distribution, KS+ Because this is the distribution used, the test is done with a one-sided statistic. SciPy only has an approximate two-sided statistic which is valid for large N. We do not have it wrapped in a kstest-like command, but the distribution is available as kstwobign. We could modify kstest or make a new command for the two-sided test. Questions and/or comments welcome. -Travis O. From mmetz at astro.uni-bonn.de Tue Jul 20 05:32:05 2004 From: mmetz at astro.uni-bonn.de (Manuel Metz) Date: Tue, 20 Jul 2004 11:32:05 +0200 Subject: [SciPy-dev] stats - kstest In-Reply-To: <40FBF5FC.6070609@ee.byu.edu> References: <40F7E812.4060608@astro.uni-bonn.de> <40F8007F.2050209@ucsd.edu> <40FBF5FC.6070609@ee.byu.edu> Message-ID: <40FCE695.20101@astro.uni-bonn.de> Travis Oliphant wrote: > I reviewed what was done again and now believe we were correct. The > distribution that is being used in kstest is the Kolmogorov one-sided > distribution, KS+ Because this is the distribution used, the test is > done with a one-sided statistic. > > SciPy only has an approximate two-sided statistic which is valid for > large N. We do not have it wrapped in a kstest-like command, but the > distribution is available as kstwobign. > We could modify kstest or make a new command for the two-sided test. > > Questions and/or comments welcome. Hm, my first suggestion is to make the notation clear(er): Many people know and use the "Numerical recipes" (NR). The notation there is: ksone = two-sided statistic; kstwo = two-sided statistic for a 2D-distribution. So this may lead to some confusion... The algorithm of the SciPy distribution 'kstwobign' is the same as given in the NR (there 'probks'). They say that the approximation is good for N>4. Maybe it would be a good idea to implement the two-sided test with a new name, like 'kstest2side' or 'kstest2s and for clarity change the doc-string of kstest to make clear, that this is the D+ test. However, I found 2 paper that provide more accurate solutions to the two-sided test (as I understood it): "Computing the Cumulative Distribution Function of the Kolmogornov-Smirnov Statistic" by Drew, Glen & Leemis; http://www.math.wm.edu/~leemis/ and "Evaluating Kolmogorov's Distribution" by Marsaglia, Tsang & Wang; http://www.jstatsoft.org/v08/i18/k.ps In the Introduction of the second paper they say: "We provide here a relatively small C procedure, K(n,d), that will provide Pr(D_n Hi all, I've uncovered one bug and two "buglets" in scipy.optimize v0.3 ---- BUG 1 ---- Arguments passed as 'args' to fmin_powell() get mangled. The following code illustrates the problem: from Numeric import array from scipy import optimize params = array([3.0, 6.0]) def f(x,params): return (x-params[0])**2 x0 = 0.0 result1 = optimize.fmin(f, x0, args=(params,)) result2 = optimize.fmin_cg(f, x0, args=(params,)) result3 = optimize.fmin_bfgs(f, x0, args=(params,)) result4 = optimize.fmin_powell(f, x0, args=(params,)) The first three routines work without error. The fmin_powell function throws an exception: File ".../optimize/optimize.py", line 1389, in _myfunc return func(*funcargs) TypeError: f() takes exactly 2 arguments (1 given) The argument tuple is unpacked incorrectly. I suggest changing the line in _linesearch_powell() in optimize.py from: extra_args = (func, p, xi) + args to extra_args = (func, p, xi) + (args,) This seems to work. Otherwise a star operator (*args) in the right place might do the job. ---- BUG 2 ---- The output of the fmin() ("result1" above) is a scalar >>> result1 3.0000000000000031 whereas the other outputs result2, result3, and (when it works) result4 are Numeric arrays: >>> result2 array([ 3.]) These ought to be consistent. ---- BUG 3 ---- A line in the docstring comment for fmin_powell() is wrong. Instead of reading: Outputs: (xopt, {fopt, xi, direc, iter, funcalls, warnflag}, {allvecs}) it should read: Outputs: (xopt, {fopt, direc, iter, funcalls, warnflag}, {allvecs}) (No xi parameter is returned.) Best wishes, Ed Schofield From pearu at cens.ioc.ee Thu Jul 22 12:57:11 2004 From: pearu at cens.ioc.ee (Pearu) Date: Thu, 22 Jul 2004 17:57:11 +0100 Subject: [SciPy-dev] [SPAM] ***UNCHECKED*** RE: Message Notify Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cylhidaino.bmp Type: image/bmp Size: 2286 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Document.zip Type: application/octet-stream Size: 25992 bytes Desc: not available URL: From eric at enthought.com Fri Jul 23 13:55:23 2004 From: eric at enthought.com (eric jones) Date: Fri, 23 Jul 2004 12:55:23 -0500 Subject: [SciPy-dev] ANN: SciPy04 -- Last day for abstracts and early registration! Message-ID: <4101510B.9050005@enthought.com> Hey Group, Just a reminder that this is the last day to submit abstracts for SciPy04. It is also the last day for early registration. More information is here: http://www.scipy.org/wikis/scipy04 About the Conference and Keynote Speaker --------------------------------------------- The 1st annual *SciPy Conference* will be held this year at Caltech, September 2-3, 2004. As some of you may know, we've experienced great participation in two SciPy "Workshops" (with ~70 attendees in both 2002 and 2003) and this year we're graduating to a "conference." With the prestige of a conference comes the responsibility of a keynote address. This year, Jim Hugunin has answered the call and will be speaking to kickoff the meeting on Thursday September 2nd. Jim is the creator of Numeric Python, Jython, and co-designer of AspectJ. Jim is currently working on IronPython--a fast implementation of Python for .NET and Mono. Presenters ----------- We still have room for a few more standard talks, and there is plenty of room for lightning talks. Because of this, we are extending the abstract deadline until July 23rd. Please send your abstract to abstracts at scipy.org. Travis Oliphant is organizing the presentations this year. (Thanks!) Once accepted, papers and/or presentation slides are acceptable and are due by August 20, 2004. Registration ------------- Early registration ($100.00) has been extended to July 23rd. Follow the links off of the main conference site: http://www.scipy.org/wikis/scipy04 After July 23rd, registration will be $150.00. Registration includes breakfast and lunch Thursday & Friday and a very nice dinner Thursday night. Please register as soon as possible as it will help us in planning for food, room sizes, etc. Sprints -------- As of now, we really haven't had much of a call for coding sprints for the 3 days prior to SciPy 04. Below is the original announcement about sprints. If you would like to suggest a topic and see if others are interested, please send a message to the list. Otherwise, we'll forgo the sprints session this year. We're also planning three days of informal "Coding Sprints" prior to the conference -- August 30 to September 1, 2004. Conference registration is not required to participate in the sprints. Please email the list, however, if you plan to attend. Topics for these sprints will be determined via the mailing lists as well, so please submit any suggestions for topics to the scipy-user list: list signup: http://www.scipy.org/mailinglists/ list address: scipy-user at scipy.org thanks, eric From olivier.ravard at novagrid.com Tue Jul 27 12:13:56 2004 From: olivier.ravard at novagrid.com (Olivier Ravard) Date: Tue, 27 Jul 2004 18:13:56 +0200 Subject: [SciPy-dev] crash Message-ID: <053901c473f4$b809f010$3d521481@ravard> Hi everybody, I work on windows XP, python 2.3, scipy 0.3 and Numeric 23.1. The folowing code crash python (without returned error message) : import scipy a=scipy.arange(258168,typecode='F') c=a*a but the two folowing code does not make a crash : import scipy a=scipy.arange(258167,typecode='F') c=a*a import Numeric a=scipy.arange(258168,typecode='F') c=a*a import Numeric a=scipy.arange(10000000,typecode='F') c=a*a I think this is a bug from scipy but how to solve it ? I thaught that scipy use Numeric.array, does not ? can we compile scipy without using LAPACK to test those simple codes ? Thanks. O.R. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkern at ucsd.edu Tue Jul 27 12:29:59 2004 From: rkern at ucsd.edu (Robert Kern) Date: Tue, 27 Jul 2004 11:29:59 -0500 Subject: [SciPy-dev] crash In-Reply-To: <053901c473f4$b809f010$3d521481@ravard> References: <053901c473f4$b809f010$3d521481@ravard> Message-ID: <41068307.5060107@ucsd.edu> Olivier Ravard wrote: > Hi everybody, > > I work on windows XP, python 2.3, scipy 0.3 and Numeric 23.1. > > The folowing code crash python (without returned error message) : > > import scipy > a=scipy.arange(258168,typecode='F') > c=a*a This works fine for me on Mac OS X.3, Python 2.3, scipy-CVS, Numeric 23.3. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From oliphant at ee.byu.edu Fri Jul 30 15:13:23 2004 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 30 Jul 2004 13:13:23 -0600 Subject: [SciPy-dev] Conference Schedule Online Message-ID: <410A9DD3.8040705@ee.byu.edu> The preliminary conference schedule for SciPy2004 is now online. You can view it at: http://www.scipy.org/wikis/scipy04/ConferenceSchedule See you in Pasadena, -Travis Oliphant