From swisher at enthought.com Mon Aug 1 11:23:12 2005 From: swisher at enthought.com (Janet M. Swisher) Date: Mon, 01 Aug 2005 10:23:12 -0500 Subject: [SciPy-dev] [Fwd: Bug in scipy_distutils-latest.tar.gz] Message-ID: <42EE3E60.2070709@enthought.com> This was sent to me and Pearu, probably because our email addresses are on the website. Pearu, have you had a chance to confirm this? --Janet -------- Original Message -------- Subject: Bug in scipy_distutils-latest.tar.gz Date: Mon, 01 Aug 2005 03:09:13 +0200 From: O G Ogunbade To: CC: I have just downloaded and installed both the F2Py-2- and scipy_distutils-latest versions from http://cens.ioc.ee/projects/f2py2e/. When I tried to run the test case given on the website and some other test cases I found some error report such as: ImportError: cannot import name process_file The coding error was found in the file build_src.py sitting in the directory /somedir/python2.4/site-packages/scipy_distutils/commands. I decided to change Lines 14 and 198 (I hope I'm write) on the file from process_file to process_str and the required results were obtained. I hope you'll attend to this fix as promptly as you can considering your tight schedule. Best regards, Prince Olusegun Ogunbade University of South Africa Department of Physics P O Box 392 UNISA Pretoria South Africa --------------------------------------------------------------------------- This message (and attachments) is subject to restrictions and a disclaimer. Please refer to http://www.unisa.ac.za/disclaimer for full details. --------------------------------------------------------------------------- <<<>>> <<<< gwavasig >>>> -- Janet Swisher --- Senior Technical Writer Enthought, Inc. http://www.enthought.com From jdhunter at ace.bsd.uchicago.edu Wed Aug 3 14:47:34 2005 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Wed, 03 Aug 2005 13:47:34 -0500 Subject: [SciPy-dev] problem with signal.butter Message-ID: <87hde66ent.fsf@peds-pc311.bsd.uchicago.edu> I am trying to design a bandpass butterworth filter and am getting an error import scipy.signal as sig Fs = 400. Nyq = Fs/2. # low pass stop freq, high pass corner freq, etc... lpsf = 5. lpcf = 7. hpcf = 12. hpsf = 18. Nyq = Fs/2. wp = [lpcf/Nyq, hpcf/Nyq] ws = [lpsf/Nyq, hpsf/Nyq] gpass = 3. gstop = 15. ord, Wn = sig.buttord(wp, ws, gpass, gstop) mybutt = sig.butter(ord, Wn, btype='bandpass') # pun intended With scipy version 0.3.2 I get the following the traceback Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 482, in butter return iirfilter(N, Wn, btype=btype, analog=analog, output=output, ftype='butter') File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 456, in iirfilter b, a = lp2bp(b,a,wo=wo,bw=bw) File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 253, in lp2bp aprime[Dp-j] = val TypeError: can't convert complex to float; use abs(z) Any ideas? Thanks, JDH From oliphant at ee.byu.edu Wed Aug 3 18:05:41 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 03 Aug 2005 16:05:41 -0600 Subject: [SciPy-dev] problem with signal.butter In-Reply-To: <87hde66ent.fsf@peds-pc311.bsd.uchicago.edu> References: <87hde66ent.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <42F13FB5.4070503@ee.byu.edu> John Hunter wrote: >I am trying to design a bandpass butterworth filter and am getting an >error > > import scipy.signal as sig > Fs = 400. > Nyq = Fs/2. > # low pass stop freq, high pass corner freq, etc... > lpsf = 5. > lpcf = 7. > hpcf = 12. > hpsf = 18. > Nyq = Fs/2. > wp = [lpcf/Nyq, hpcf/Nyq] > ws = [lpsf/Nyq, hpsf/Nyq] > gpass = 3. > gstop = 15. > ord, Wn = sig.buttord(wp, ws, gpass, gstop) > mybutt = sig.butter(ord, Wn, btype='bandpass') # pun intended > > > > >With scipy version 0.3.2 I get the following the traceback > >Traceback (most recent call last): > File "", line 1, in ? > File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 482, in butter > return iirfilter(N, Wn, btype=btype, analog=analog, output=output, ftype='butter') > File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 456, in iirfilter > b, a = lp2bp(b,a,wo=wo,bw=bw) > File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 253, in lp2bp > aprime[Dp-j] = val >TypeError: can't convert complex to float; use abs(z) > > val must be complex for some reason where it is not expected to be. Not sure what is going on. Which version of Numeric do you have installed? This is what I get. >>> import scipy >>> scipy.__version__ '0.3.3_304.4613' import scipy.signal as sig ord,Wn = sig.buttord([7/200.,12/200.],[5/200.0,18/200.0],3,15) >>> ord 2 >>> Wn array([ 0.034979417860466, 0.060035166350418]) >>> mybutt = sig.butter(ord,Wn,btype='bandpass') >>> mybutt (array([ 0.001466661479123, 0. , -0.002933322958246, 0. , 0.001466661479123]), array([ 1. , -3.848535133256346, 5.594309123067282, -3.640021384921099, 0.894652727053865])) -Travis From jdhunter at ace.bsd.uchicago.edu Wed Aug 3 18:18:11 2005 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Wed, 03 Aug 2005 17:18:11 -0500 Subject: [SciPy-dev] problem with signal.butter In-Reply-To: <42F13FB5.4070503@ee.byu.edu> (Travis Oliphant's message of "Wed, 03 Aug 2005 16:05:41 -0600") References: <87hde66ent.fsf@peds-pc311.bsd.uchicago.edu> <42F13FB5.4070503@ee.byu.edu> Message-ID: <87hde6r7fg.fsf@peds-pc311.bsd.uchicago.edu> >>>>> "Travis" == Travis Oliphant writes: Travis> val must be complex for some reason where it is not Travis> expected to be. Travis> Not sure what is going on. Which version of Numeric do Travis> you have installed? 23.7 Here is what I get for the same commands in a plain-ol-python-shell. > uname -a Linux peds-pc311 2.6.10-5-386 #1 Fri Jun 24 16:53:01 UTC 2005 i686 GNU/Linux > python Python 2.4.1 (#2, Mar 30 2005, 21:51:10) [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import Numeric as N >>> N.__version__ '23.7' >>> import scipy as s >>> s.__version__ '0.3.2' >>> import scipy.signal as sig >>> ord,Wn = sig.buttord([7/200.,12/200.],[5/200.0,18/200.0],3,15) >>> ord 2 >>> Wn array([ 0.03497942, 0.06003517]) >>> mybutt = sig.butter(ord,Wn,btype='bandpass') Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 482, in butter return iirfilter(N, Wn, btype=btype, analog=analog, output=output, ftype='butter') File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 456, in iirfilter b, a = lp2bp(b,a,wo=wo,bw=bw) File "/usr/lib/python2.4/site-packages/scipy/signal/filter_design.py", line 253, in lp2bp aprime[Dp-j] = val TypeError: can't convert complex to float; use abs(z) >>> Let me know if there is anything else you want me to try. I can always upgrade to scipy CVS and try that. Thanks! JDH From Fernando.Perez at colorado.edu Wed Aug 3 18:21:03 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Wed, 03 Aug 2005 16:21:03 -0600 Subject: [SciPy-dev] problem with signal.butter In-Reply-To: <87hde6r7fg.fsf@peds-pc311.bsd.uchicago.edu> References: <87hde66ent.fsf@peds-pc311.bsd.uchicago.edu> <42F13FB5.4070503@ee.byu.edu> <87hde6r7fg.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <42F1434F.10300@colorado.edu> John Hunter wrote: >>>>import scipy as s >>>>s.__version__ > > '0.3.2' [...] > Let me know if there is anything else you want me to try. I can > always upgrade to scipy CVS and try that. By all means do. 0.3.2 is quite out of date, so there's a good chance CVS will take care of things. Cheers, f From jdhunter at ace.bsd.uchicago.edu Thu Aug 4 08:56:20 2005 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Thu, 04 Aug 2005 07:56:20 -0500 Subject: [SciPy-dev] problem with signal.butter In-Reply-To: <42F1434F.10300@colorado.edu> (Fernando Perez's message of "Wed, 03 Aug 2005 16:21:03 -0600") References: <87hde66ent.fsf@peds-pc311.bsd.uchicago.edu> <42F13FB5.4070503@ee.byu.edu> <87hde6r7fg.fsf@peds-pc311.bsd.uchicago.edu> <42F1434F.10300@colorado.edu> Message-ID: <877jf1zwqz.fsf@peds-pc311.bsd.uchicago.edu> >>>>> "Fernando" == Fernando Perez writes: Fernando> By all means do. 0.3.2 is quite out of date, so there's Fernando> a good chance CVS will take care of things. That did the trick. Because I need this fix for software that I distribute to windoze users, I still have some interest in seeing if I can backport the fix the 0.3.2, since this is the current binary release and thus I suspect it will also be the release in the new enthought python. I'm posting the diff betwween scipy 0.3.3_309.4626 and 0.3.2 in the signal module, in case the person(s) who made those changes will have some insight into what is contributing to the fix. If not, it's no big deal because I can always try and build scipy CVS on windows (arggg) or just tell folks they can't use this feature. It would be much easier just to say, "drop this file in site-packages/scipy/signal" if it's just a one or two line fix. Thanks! JDH Only in scipy/Lib/signal/: CVS Only in scipy/Lib/signal/: docs diff -cr SciPy_complete-0.3.2/Lib/signal/filter_design.py scipy/Lib/signal/filter_design.py *** SciPy_complete-0.3.2/Lib/signal/filter_design.py 2004-02-26 02:55:12.000000000 -0600 --- scipy/Lib/signal/filter_design.py 2004-11-22 14:25:00.000000000 -0600 *************** *** 5,10 **** --- 5,11 ---- from scipy_base.fastumath import * from scipy_base import atleast_1d, poly, polyval, roots, imag, real, asarray,\ allclose, Float, resize, pi, concatenate, absolute, logspace, c_ + from scipy_base import mintypecode from scipy import comb, special, optimize, linalg import string, types *************** *** 228,236 **** a,b = map(atleast_1d,(a,b)) D = len(a) - 1 N = len(b) - 1 ! artype = b.typecode() ! if artype not in ['F','D','f','d']: ! artype = Num.Float ma = max([N,D]) Np = N + ma Dp = D + ma --- 229,235 ---- a,b = map(atleast_1d,(a,b)) D = len(a) - 1 N = len(b) - 1 ! artype = mintypecode((a,b)) ma = max([N,D]) Np = N + ma Dp = D + ma *************** *** 261,269 **** a,b = map(atleast_1d,(a,b)) D = len(a) - 1 N = len(b) - 1 ! artype = b.typecode() ! if artype not in ['F','D','f','d']: ! artype = Num.Float M = max([N,D]) Np = M + M Dp = M + M --- 260,266 ---- a,b = map(atleast_1d,(a,b)) D = len(a) - 1 N = len(b) - 1 ! artype = mintypecode((a,b)) M = max([N,D]) Np = M + M Dp = M + M diff -cr SciPy_complete-0.3.2/Lib/signal/firfilter.c scipy/Lib/signal/firfilter.c *** SciPy_complete-0.3.2/Lib/signal/firfilter.c 2003-07-08 20:52:20.000000000 -0500 --- scipy/Lib/signal/firfilter.c 2005-05-31 01:51:32.000000000 -0500 *************** *** 112,118 **** if ((sum = calloc(type_size,2))==NULL) return -3; /* No memory */ value = sum + type_size; ! if (outsize == FULL) {Os[0] = Ns[0]+Nwin[0]-1; Os[1] = Ns[0]+Nwin[1]-1;} else if (outsize == SAME) {Os[0] = Ns[0]; Os[1] = Ns[1];} else if (outsize == VALID) {Os[0] = Ns[0]-Nwin[0]+1; Os[1] = Ns[1]-Nwin[1]+1;} else return -1; /* Invalid output flag */ --- 112,118 ---- if ((sum = calloc(type_size,2))==NULL) return -3; /* No memory */ value = sum + type_size; ! if (outsize == FULL) {Os[0] = Ns[0]+Nwin[0]-1; Os[1] = Ns[1]+Nwin[1]-1;} else if (outsize == SAME) {Os[0] = Ns[0]; Os[1] = Ns[1];} else if (outsize == VALID) {Os[0] = Ns[0]-Nwin[0]+1; Os[1] = Ns[1]-Nwin[1]+1;} else return -1; /* Invalid output flag */ *************** *** 120,125 **** --- 120,128 ---- if ((boundary != PAD) && (boundary != REFLECT) && (boundary != CIRCULAR)) return -2; /* Invalid boundary flag */ + /* Speed this up by not doing any if statements in the for loop. Need 3*3*2=18 different + loops executed for different conditions */ + for (m=0; m < Os[0]; m++) { /* Reposition index into input image based on requested output size */ if (outsize == FULL) new_m = convolve ? m : (m-Nwin[0]+1); diff -cr SciPy_complete-0.3.2/Lib/signal/info_signal.py scipy/Lib/signal/info_signal.py *** SciPy_complete-0.3.2/Lib/signal/info_signal.py 2004-02-26 01:15:10.000000000 -0600 --- scipy/Lib/signal/info_signal.py 2005-05-31 01:51:32.000000000 -0500 *************** *** 6,15 **** --- 6,17 ---- convolve -- N-dimensional convolution. correlate -- N-dimensional correlation. + fftconvolve -- N-dimensional convolution using the FFT. convolve2d -- 2-dimensional convolution (more options). correlate2d -- 2-dimensional correlation (more options). sepfir2d -- Convolve with a 2-D separable FIR filter. + B-splines: bspline -- B-spline basis function of order n. *************** *** 45,52 **** freqs --- Analog filter frequency response freqz --- Digital filter frequency response ! ! matlab-style IIR filter design: butter (buttord) -- Butterworth cheby1 (cheb1ord) -- Chebyshev Type I --- 47,53 ---- freqs --- Analog filter frequency response freqz --- Digital filter frequency response ! Matlab-style IIR filter design: butter (buttord) -- Butterworth cheby1 (cheb1ord) -- Chebyshev Type I *************** *** 68,74 **** tf2ss -- transfer function to state-space. ss2tf -- state-pace to transfer function. zpk2ss -- zero-pole-gain to state-space. ! ss2zpk -- state-space to pole-zero-gain. """ postpone_import = 1 --- 69,88 ---- tf2ss -- transfer function to state-space. ss2tf -- state-pace to transfer function. zpk2ss -- zero-pole-gain to state-space. ! ss2zpk -- state-space to pole-zero-gain. ! ! Waveforms: ! ! sawtooth -- Periodic sawtooth ! square -- Square wave ! gausspulse -- Gaussian modulated sinusoid ! chirp -- Frequency swept cosine signal ! ! Wavelets: ! ! daub -- return low-pass filter for daubechies wavelets ! qmf -- return quadrature mirror filter from low-pass ! cascade -- compute scaling function and wavelet from coefficients """ postpone_import = 1 diff -cr SciPy_complete-0.3.2/Lib/signal/__init__.py scipy/Lib/signal/__init__.py *** SciPy_complete-0.3.2/Lib/signal/__init__.py 2004-04-13 16:23:51.000000000 -0500 --- scipy/Lib/signal/__init__.py 2005-05-31 01:51:32.000000000 -0500 *************** *** 10,13 **** --- 10,14 ---- from filter_design import * from ltisys import * from signaltools import * + from wavelets import * diff -cr SciPy_complete-0.3.2/Lib/signal/ltisys.py scipy/Lib/signal/ltisys.py *** SciPy_complete-0.3.2/Lib/signal/ltisys.py 2003-12-13 07:44:51.000000000 -0600 --- scipy/Lib/signal/ltisys.py 2005-02-24 04:42:24.000000000 -0600 *************** *** 223,230 **** self.__dict__['zeros'], self.__dict__['poles'], \ self.__dict__['gain'] = ss2zpk(*args) self.__dict__['num'], self.__dict__['den'] = ss2tf(*args) ! self.inputs = B.shape[-1] ! self.outputs = C.shape[0] else: raise ValueError, "Needs 2, 3, or 4 arguments." --- 223,230 ---- self.__dict__['zeros'], self.__dict__['poles'], \ self.__dict__['gain'] = ss2zpk(*args) self.__dict__['num'], self.__dict__['den'] = ss2tf(*args) ! self.inputs = self.B.shape[-1] ! self.outputs = self.C.shape[0] else: raise ValueError, "Needs 2, 3, or 4 arguments." diff -cr SciPy_complete-0.3.2/Lib/signal/setup_signal.py scipy/Lib/signal/setup_signal.py *** SciPy_complete-0.3.2/Lib/signal/setup_signal.py 2004-02-06 02:17:52.000000000 -0600 --- scipy/Lib/signal/setup_signal.py 2005-03-24 04:22:16.000000000 -0600 *************** *** 5,10 **** --- 5,11 ---- from scipy_distutils.misc_util import get_path, default_config_dict, dot_join def configuration(parent_package='',parent_path=None): + from scipy_distutils.system_info import get_info package = 'signal' local_path = get_path(__name__,parent_path) config = default_config_dict(package, parent_package) diff -cr SciPy_complete-0.3.2/Lib/signal/signaltools.py scipy/Lib/signal/signaltools.py *** SciPy_complete-0.3.2/Lib/signal/signaltools.py 2004-06-02 17:57:55.000000000 -0500 --- scipy/Lib/signal/signaltools.py 2005-01-17 18:53:50.000000000 -0600 *************** *** 10,17 **** import types import scipy from scipy.stats import mean ! import Numeric ! from Numeric import array, arange, where, sqrt, rank, zeros, NewAxis, argmax from scipy_base.fastumath import * _modedict = {'valid':0, 'same':1, 'full':2} --- 10,17 ---- import types import scipy from scipy.stats import mean ! import scipy_base as Numeric ! from scipy_base import array, arange, where, sqrt, rank, zeros, NewAxis, argmax, product from scipy_base.fastumath import * _modedict = {'valid':0, 'same':1, 'full':2} *************** *** 66,72 **** kernel = asarray(in2) if rank(volume) == rank(kernel) == 0: return volume*kernel ! if (Numeric.product(kernel.shape) > Numeric.product(volume.shape)): temp = kernel kernel = volume volume = temp --- 66,72 ---- kernel = asarray(in2) if rank(volume) == rank(kernel) == 0: return volume*kernel ! if (product(kernel.shape) > product(volume.shape)): temp = kernel kernel = volume volume = temp *************** *** 76,81 **** --- 76,117 ---- return sigtools._correlateND(volume, kernel, val) + def _centered(arr, newsize): + # Return the center newsize portion of the array. + newsize = asarray(newsize) + currsize = array(arr.shape) + startind = (currsize - newsize) / 2 + endind = startind + newsize + myslice = [slice(startind[k], endind[k]) for k in range(len(endind))] + return arr[tuple(myslice)] + + def fftconvolve(in1, in2, mode="full"): + """Convolve two N-dimensional arrays using FFT. SEE convolve + """ + s1 = array(in1.shape) + s2 = array(in2.shape) + if (s1.typecode() in ['D','F']) or (s2.typecode() in ['D', 'F']): + cmplx=1 + else: cmplx=0 + size = s1+s2-1 + IN1 = fftn(in1,size) + IN1 *= fftn(in2,size) + ret = ifftn(IN1) + del IN1 + if not cmplx: + ret = real(ret) + if mode == "full": + return ret + elif mode == "same": + if product(s1) > product(s2): + osize = s1 + else: + osize = s2 + return _centered(ret,osize) + elif mode == "valid": + return _centered(ret,abs(s2-s1)+1) + + def convolve(in1, in2, mode='full'): """Convolve two N-dimensional arrays. Only in scipy/Lib/signal/tests: CVS Only in scipy/Lib/signal/: wavelets.py From jmiller at stsci.edu Thu Aug 4 13:13:22 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu, 04 Aug 2005 13:13:22 -0400 Subject: [SciPy-dev] SciPy CVS to Subversion migration In-Reply-To: <42E98043.9030900@enthought.com> References: <42E98043.9030900@enthought.com> Message-ID: <1123175602.4467.121.camel@halloween.stsci.edu> Hi Joe, I never managed to get Scipy CVS access back prior to the switch to subversion. Has this switch gone through yet? On Thu, 2005-07-28 at 21:02, Joe Cooper wrote: > Hi all, > > The issues with converting our CVS repository to Subversion have been > resolved, and so I'd like to make the switch tomorrow (Friday) afternoon. > > If you have anything you'd like to check-in before the CVS repository is > gone forever, please do so before then. You will have to check out a > new working copy after the conversion occurs. > > If the timing of this outage will cause anyone discomfort, please let me > know, and I will re-schedule for another day. > > Just FYI: > > History will be maintained from the old CVS tree. > > Once the migration occurs, branching, merging, and reorganizing the tree > will become trivially simple and reliable (and anyone with commit access > can make those kinds of changes without system administrator > intervention). I know there is a contingent of SciPy developers that > would like to do some long-desired serious cleanups and purges within > the tree--the new Subversion repository will easily allow these kinds of > changes without having to throw out all of the old history and start > fresh. If, after the migration, you find you still want to start fresh, > let me know and I'll create a new Subversion repo for you to work in. > > Holler if you have any further questions or concerns. > > Thanks, > Joe > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev -- From ryorke at telkomsa.net Sat Aug 6 15:58:55 2005 From: ryorke at telkomsa.net (Rory Yorke) Date: Sat, 06 Aug 2005 21:58:55 +0200 Subject: [SciPy-dev] Scipy subversion repository URL Message-ID: <87ek96sups.fsf@localhost.localdomain.i-did-not-set--mail-host-address--so-tickle-me> Hi, I see that Scipy has relatively recently switched from CVS to Subversion; however, I can't seem to find what the URL for the SVN repository is. I browsed this: http://svn.enthought.com/svn/enthought/ but couldn't see anything that looked obviously like Scipy (it looks like Chaco); I tried http://svn.scipy.org/svn, also without luck -- I get a username/password prompt there. CVS checkout doesn't work either; the CVS login command given at http://www.scipy.org/cvs/ gives me a 'Connection refused' error -- I presume CVS access has been discontinued. Could someone in the know please post the URL for the repository? Thanks, Rory From joe at enthought.com Mon Aug 8 02:25:12 2005 From: joe at enthought.com (Joe Cooper) Date: Mon, 08 Aug 2005 01:25:12 -0500 Subject: [SciPy-dev] Subversion migration complete Message-ID: <42F6FAC8.4010709@enthought.com> Hi all, I've just completed the long-threatened migration of all of the scipy CVS repositories to Subversion on the new server. All future work will be done in the Subversion repositories. Here are the repository paths: http://svn.scipy.org/svn/scipy/ http://svn.scipy.org/svn/agg2/ http://svn.scipy.org/svn/chaco-1.0/ (Historical only. Do not use.) http://svn.scipy.org/svn/kiva-1.0/ (Historical only. Do not use.) http://svn.scipy.org/svn/fftw_win32/ http://svn.scipy.org/svn/freetype2/ http://svn.scipy.org/svn/PyCrust/ http://svn.scipy.org/svn/traits-1.0/ (Historical only. Do not use.) http://svn.scipy.org/svn/xml_installer/ All but the scipy repository have questionable lineage--i.e., either I don't know what they are used for or who is maintaining them (PyCrust, agg2, fftw_win32, freetype2) or I know that they are not being maintained and will never be maintained in their current form and shouldn't be used under any circumstance I can think of (chaco, kiva, traits which are part of the Enthought OSS tool suite and now live at http://code.enthought.com or xml_installer which is currently unmaintained, but some of it will find its way into the Enthon project before too long). The repositories existed on the old scipy.org, and so they came on over to the new scipy.org. I welcome input about what the repositories that are unknown to me actually are. freetype2 is a double mystery in that there were two freetype2 CVS repositories...if you are in a position to know whether I got the right one, I'd appreciate a note to that effect. Commit privileges will be restored in the next hour or so. If you find by tomorrow that you no longer have commit privileges to repositories that you work on, please let me know. I'm sure people will get missed during this process, as security is much tighter on the new server, and not all users on the old server are coming over. Please let me know if you see any problems in the migrated repositories. Documentation on the website will be updated shortly to reflect the changes. From rkern at ucsd.edu Mon Aug 8 02:38:35 2005 From: rkern at ucsd.edu (Robert Kern) Date: Sun, 07 Aug 2005 23:38:35 -0700 Subject: [SciPy-dev] Subversion migration complete In-Reply-To: <42F6FAC8.4010709@enthought.com> References: <42F6FAC8.4010709@enthought.com> Message-ID: <42F6FDEB.3010308@ucsd.edu> Joe Cooper wrote: > Hi all, > > I've just completed the long-threatened migration of all of the scipy > CVS repositories to Subversion on the new server. All future work will > be done in the Subversion repositories. > > Here are the repository paths: > > http://svn.scipy.org/svn/scipy/ > http://svn.scipy.org/svn/agg2/ > http://svn.scipy.org/svn/chaco-1.0/ (Historical only. Do not use.) > http://svn.scipy.org/svn/kiva-1.0/ (Historical only. Do not use.) > http://svn.scipy.org/svn/fftw_win32/ > http://svn.scipy.org/svn/freetype2/ > http://svn.scipy.org/svn/PyCrust/ > http://svn.scipy.org/svn/traits-1.0/ (Historical only. Do not use.) > http://svn.scipy.org/svn/xml_installer/ > > All but the scipy repository have questionable lineage--i.e., either I > don't know what they are used for or who is maintaining them (PyCrust, > agg2, fftw_win32, freetype2) or I know that they are not being > maintained and will never be maintained in their current form and > shouldn't be used under any circumstance I can think of (chaco, kiva, > traits which are part of the Enthought OSS tool suite and now live at > http://code.enthought.com or xml_installer which is currently > unmaintained, but some of it will find its way into the Enthon project > before too long). The repositories existed on the old scipy.org, and so > they came on over to the new scipy.org. I welcome input about what the > repositories that are unknown to me actually are. freetype2 is a double > mystery in that there were two freetype2 CVS repositories...if you are > in a position to know whether I got the right one, I'd appreciate a note > to that effect. I'm pretty sure that all of them except for scipy and possibly fftw_win32 and xml_installer can be dumped. agg2 and PyCrust are essentially obsolete copies of 3rd-party sources. freetype2 is now part of the enthought package. I'm guessing (without checking) that the other freetype2 repository was another one of those obsolete copies of the 3rd-party FreeType distribution. We might want to keep around the CVS history archived somewhere, but keeping the old stuff around is just going to confuse people. Having them in the CVS repositories certainly did. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From Alex.Perry at ge.com Tue Aug 9 14:55:45 2005 From: Alex.Perry at ge.com (Perry, Alexander (GE Infrastructure)) Date: Tue, 9 Aug 2005 14:55:45 -0400 Subject: [SciPy-dev] arithmetic on distributions Message-ID: <945F31304DDF6B46814E8422A6B1111D057D53CB@CINMLVEM11.e2k.ad.ge.com> Has anyone implemented, or considered implementing, the arithmetic methods for rv_continuous? First suggestion. When calling an object to freeze it, if the loc is already a distribution, we should merge its variance into the variance of the newly frozen distribution: > ten = norm ( 10, 3 ) > print norm ( ten, 4 ).stats() (10.0, 25.0) Currently, an error is triggered. Because of the difficulty of merging the standard deviation of the old value into the new desired standard deviation, I suspect that most people who encounter the error will take the lazy way out and simply discard the variance attached to the value: > try: value=value.stats()[0] > except: pass # the above is wrong! Second suggestion. When a binary operator such as __add__() is called on an object that derives from rv_continuous, currently an error is generated. There should be a function that checks whether the other parameter is a simple number and, if it is, modifies its own loc and scale. If both parameters of the binary operator are instances of rv_continuous, the function returns an instance of the new subclass rv_composite that simply stores those two parameters and the desired operator. The new subclass passes through all operations that can be directly decomposed (such as rvs and stats). This allows a real world distribution to be assembled as a single object, and that object directly supports both linear approximation and monte-carlo methods: > fifteen = norm ( 10, 3 ) + uniform( 3, 4 ) > # Linearized estimate > print "%7.5f" % norm ( fifteen, 0 ).cdf ( 20 ) 0.94008 > # Monte carlo estimate > print sum ( [ x<20 for x in fifteen.rvs(100000) ]) 94125 The composite distribution instance implements the remainder of the standard "rv" features through integration and differentiation of the underlying distributions. This is no different than what the users would have to do themselves, but saves the effort of explicitly calling those scipy features. > # Accurate integrated value > print "%7.5f" % fifteen.cdf(20) 0.94000 Third suggestion. The __cmp__ method should exist and can simply compare the mean since its python definition is that it returns a Boolean upon which flow of control can be decided. The more accurate comparison functions (such as __lt__) should exist and return probabilities because their python definition is that they return a data result with no flow control interpretation. > print "%7.5f" % ( fifteen < 20 ) 0.94000 Unlike the CDF, which assumes that the parameter does not itself have uncertainty, the comparison functions have to allow for the fact that both sides could be distributions. Consequently: > three = norm(3,0.3) > four = norm(4,0.4) > print three.cdf(4) 0.9996 > print four.sf(3) 0.9938 > print three < four 0.9772 def rv_continuous.__cmp__(self,other): try: other = other.stats()[0] except: pass return cmp ( self.stats()[0], other ) def rv_continuous.__lt__(self,other): return (self-other).cdf(0) Comments? Alex. PS. I don't use discrete distributions anything like as much as continuous, so I'm not going to comment on whether it's a good idea to extend the above concept to rv_discrete. It's possible to return any comparison result as a binomial distribution on a single trial, making it easier to pursue statistical conclusions on uncertain data, but I've no idea whether that would be helpful. Similarly, just like "norm+norm" can be optimized, so can "binomial**int" ... and similar cases. From oliphant at ee.byu.edu Wed Aug 10 02:18:41 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 10 Aug 2005 00:18:41 -0600 Subject: [SciPy-dev] Subversion migration complete In-Reply-To: <42F7F0C5.6010803@ucsd.edu> References: <42F6FAC8.4010709@enthought.com> <42F7012F.5060804@ucsd.edu> <42F79C7C.8000100@enthought.com> <42F7F0C5.6010803@ucsd.edu> Message-ID: <42F99C41.3090009@ee.byu.edu> Robert Kern wrote: > > Joe Cooper wrote: > >> >> Oops. It got overlooked. Fixed: >> >> http://svn.scipy.org/svn/scipy_core > > > We really need it to be inside scipy/trunk so that people can do one > checkout and be able to go at once. What's the best way to do this? I > don't think we really need it as a separate entity, so perhaps just > "svn copy"ing the tree over? Perhaps having scipy_core and scipy as separate checkouts is a good thing. Scipy core is supposed to replace Numeric and Numarray. Having a separate tree might help others realize that they can check out scipy_core separately from scipy and speed the adoption of the unified array. -Travis From oliphant at ee.byu.edu Wed Aug 10 02:29:11 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 10 Aug 2005 00:29:11 -0600 Subject: [SciPy-dev] SVN giving me a million files Message-ID: <42F99EB7.60108@ee.byu.edu> So, how do you check out scipy without getting the millions of files from branches and tags as well? -Travis From joe at enthought.com Wed Aug 10 02:51:09 2005 From: joe at enthought.com (Joe Cooper) Date: Wed, 10 Aug 2005 01:51:09 -0500 Subject: [SciPy-dev] SVN giving me a million files In-Reply-To: <42F99EB7.60108@ee.byu.edu> References: <42F99EB7.60108@ee.byu.edu> Message-ID: <42F9A3DD.6080402@enthought.com> Travis Oliphant wrote: > > So, how do you check out scipy without getting the millions of files > from branches and tags as well? svn co http://svn.scipy.org/svn/scipy/trunk i.e. just specify you want the trunk. It's almost certainly possible to make this the default on a checkout of the scipy module...If it would be more comfortable, I'm happy to work on getting it setup that way. From mmetz at astro.uni-bonn.de Wed Aug 10 04:38:47 2005 From: mmetz at astro.uni-bonn.de (Manuel Metz) Date: Wed, 10 Aug 2005 10:38:47 +0200 Subject: [SciPy-dev] SVN giving me a million files In-Reply-To: <42F9A3DD.6080402@enthought.com> References: <42F99EB7.60108@ee.byu.edu> <42F9A3DD.6080402@enthought.com> Message-ID: <42F9BD17.6080006@astro.uni-bonn.de> Hi, see also Appendix A "Subversion for CVS Users" in the subversion book "Version control with Subversion": http://svnbook.red-bean.com/ I found this very helpful ... Manuel Joe Cooper schrieb: > Travis Oliphant wrote: > >> >> So, how do you check out scipy without getting the millions of files >> from branches and tags as well? > > > svn co http://svn.scipy.org/svn/scipy/trunk > > i.e. just specify you want the trunk. It's almost certainly possible to > make this the default on a checkout of the scipy module...If it would be > more comfortable, I'm happy to work on getting it setup that way. > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev -- ------------------------------------- Manuel Metz Sternwarte der Universitaet Bonn Auf dem Huegel 71 (room 3.06) D - 53121 Bonn E-Mail: mmetz at astro.uni-bonn.de Phone: (+49) 228 / 73-3660 Fax: (+49) 228 / 73-3672 ------------------------------------- From travis at enthought.com Wed Aug 10 10:14:29 2005 From: travis at enthought.com (Travis N. Vaught) Date: Wed, 10 Aug 2005 09:14:29 -0500 Subject: [SciPy-dev] Re: scipy repository structure In-Reply-To: <42F9B817.8030404@enthought.com> References: <42F9B817.8030404@enthought.com> Message-ID: <42FA0BC5.6010406@enthought.com> CC'ing the dev list (please read below for context): So... It seems like we should first put things into: www.scipy.org/svn/scipy/trunk/src/lib/ scipy scipy_core (allowing one to do this: svn co https://svn.scipy.org/svn/scipy/trunk/ ...and get what's needed for a build) Next, get a release out of SciPy 0.3.3 using the existing relative locations of scipy, scipy_core, et al. Then, work toward the grand unification of scipy and Numeric3 (which is actually scipy_base). Constructing a world which will support a binary build of a stand_alone scipy_core (which will have scipy_base, f2py, scipy_distutils and scipy_test (or some approximation of those), and a binary build of the scipy "whole enchilada"--with proper consideration of the potential difficulties Pearu points out below. Any input/objections? Regards, Travis Joe Cooper wrote: > pearu at cens.ioc.ee wrote: > >> For installing Numeric3 and scipy we can use two separate setup.py files >> for each task but for checking out Numeric3 only (there might be users >> who are not interested in scipy but Numeric3 alone) from the above tree >> could be a problem. > > > You can checkout any path within a subversion repository, so putting > everything into one repository is entirely sane, if everything is > closely related. Some tools (trac, specifically) are designed around > a "one project, one repository" model, and so it does not deal with > many different packages in the same repository with anything > approaching sanity. This has shaped our thinking on the enthought > tree to some degree (not a large degree, but a little bit). > > I expect we will want to begin using trac on scipy, as well, and so it > may be worth considering the limitations of trac. > >>> I'd also like to leave behind any deprecated libraries in the "old >>> CVS" rather than cluttering the new repository. >> >> >> >> I agree. I think only scipy, scipy_core, ipython(?) from the "old CVS" >> need to be converted to svn. All other modules seem obsolute. > > > ipython was the first to move to subversion. Fernando has been > happily subverting for a couple of months. I'll remove all of the > obsoletes. > >>> Please let me know if your thoughts. I wanted to do any wholesale >>> changes now rather than later, since we're setting up the repository >>> anyway. If we should do this post 3.3 let me know how to structure >>> things to get us to that point. >> >> >> >> If we want to get 3.3 out as soon as possible then we should use the CVS >> tree. It may take time to get Numeric3-scipy and restructured >> svn-scipy to stable enough for the release. > > > There is no need to restructure the Subversion repo before a release, > or we can branch before the reorg, or the person doing the reorg can > do it in a working copy. Also, the subversion tree mimics the CVS > tree to a large degree, so there's probably no reason not to roll the > release from the SVN repo. I'd be shocked if you can't check out the > trunk of scipy_core and scipy and have them act exactly the same as > the CVS versions. > > Just a reminder about some of the reasons for migrating to subversion: > > Branching is cheap and safe. > Wholesale moves of directories and everything in them is cheap and safe. > Everything (almost) that one can do to a subversion repository can be > done in a working copy, so everything (almost) is cheap and safe. > > I know everyone who has worked with CVS has some old "Egads, don't > move things around...we don't know what could happen!" notions that > are hard to overcome, because I still have those feelings too > sometimes. So, it's worth mentioning that those fears are unjustified > with Subversion. -- ........................ Travis N. Vaught CEO Enthought, Inc. http://www.enthought.com ........................ From oliphant at ee.byu.edu Wed Aug 10 13:06:16 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 10 Aug 2005 11:06:16 -0600 Subject: [SciPy-dev] Re: scipy repository structure In-Reply-To: <42FA0BC5.6010406@enthought.com> References: <42F9B817.8030404@enthought.com> <42FA0BC5.6010406@enthought.com> Message-ID: <42FA3408.3070900@ee.byu.edu> Travis N. Vaught wrote: > CC'ing the dev list (please read below for context): > > So... > > It seems like we should first put things into: > > www.scipy.org/svn/scipy/trunk/src/lib/ > scipy > scipy_core > > (allowing one to do this: svn co https://svn.scipy.org/svn/scipy/trunk/ > ...and get what's needed for a build) Why the extra /src/lib/ portion and not just www.scipy.org/svn/scipy/trunk/ scipy scipy_core ?? -Travis O. From Alex.Perry at ge.com Wed Aug 10 14:57:24 2005 From: Alex.Perry at ge.com (Perry, Alexander (GE Infrastructure)) Date: Wed, 10 Aug 2005 14:57:24 -0400 Subject: [SciPy-dev] bugs in stats package Message-ID: <945F31304DDF6B46814E8422A6B1111D058FCFA5@CINMLVEM11.e2k.ad.ge.com> Debian has scipy version 0.3.2 and this is Bug Tracking System (BTS) 322454: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=322454 stats.py in nanmean() and nanstd() uses "N-sum" when "N.sum" is necessary. stats.py in nanstd() provides obviously wrong answers for data without NANs. distributions.py in est_loc_scale() only receives two items from self.stats(). distributions.py in est_loc_scale() refers to "stats" but was imported as "st". distributions.py in nnlf() wrongly passes its self twice to _nnlf(). distributions.py in nnlf() fails when everything that precedes "x" is consolidated into a numarray so it can only be used standalone. distributions.py in fit() doesn't comply with nnlf()'s argument API. From oliphant at ee.byu.edu Wed Aug 10 17:29:50 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 10 Aug 2005 15:29:50 -0600 Subject: [SciPy-dev] bugs in stats package In-Reply-To: <945F31304DDF6B46814E8422A6B1111D058FCFA5@CINMLVEM11.e2k.ad.ge.com> References: <945F31304DDF6B46814E8422A6B1111D058FCFA5@CINMLVEM11.e2k.ad.ge.com> Message-ID: <42FA71CE.5090009@ee.byu.edu> Perry, Alexander (GE Infrastructure) wrote: >Debian has scipy version 0.3.2 and this is Bug Tracking System (BTS) 322454: > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=322454 > >stats.py in nanmean() and nanstd() uses "N-sum" when "N.sum" is necessary. > >stats.py in nanstd() provides obviously wrong answers for data without NANs. > >distributions.py in est_loc_scale() only receives two items from self.stats(). > >distributions.py in est_loc_scale() refers to "stats" but was imported as "st". > >distributions.py in nnlf() wrongly passes its self twice to _nnlf(). > >distributions.py in nnlf() fails when everything that precedes "x" >is consolidated into a numarray so it can only be used standalone. > >distributions.py in fit() doesn't comply with nnlf()'s argument API. > > Thanks for reporting these... -Travis From travis at enthought.com Wed Aug 10 17:30:08 2005 From: travis at enthought.com (Travis N. Vaught) Date: Wed, 10 Aug 2005 16:30:08 -0500 Subject: [SciPy-dev] Re: scipy repository structure In-Reply-To: <42FA3408.3070900@ee.byu.edu> References: <42F9B817.8030404@enthought.com> <42FA0BC5.6010406@enthought.com> <42FA3408.3070900@ee.byu.edu> Message-ID: <42FA71E0.7030308@enthought.com> Travis Oliphant wrote: > Travis N. Vaught wrote: > >> CC'ing the dev list (please read below for context): >> >> So... >> >> It seems like we should first put things into: >> >> www.scipy.org/svn/scipy/trunk/src/lib/ >> scipy >> scipy_core >> >> (allowing one to do this: svn co https://svn.scipy.org/svn/scipy/trunk/ >> ...and get what's needed for a build) > > > > Why the extra /src/lib/ portion and not just > > www.scipy.org/svn/scipy/trunk/ > scipy > scipy_core > > ?? > > -Travis O. > We've done this for all our internal work and with the enthought open source packages in order to support an organization like: /svn/scipy/trunk/ doc/ src/ apps/ demo_app api_docs/ lib/ scipy scipy_core Perhaps all this isn't necessary for scipy, but we've found it to be a useful way to partition the repository. Travis V. From rkern at ucsd.edu Wed Aug 10 18:40:32 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 10 Aug 2005 15:40:32 -0700 Subject: [SciPy-dev] Subversion migration complete In-Reply-To: <42F99C41.3090009@ee.byu.edu> References: <42F6FAC8.4010709@enthought.com> <42F7012F.5060804@ucsd.edu> <42F79C7C.8000100@enthought.com> <42F7F0C5.6010803@ucsd.edu> <42F99C41.3090009@ee.byu.edu> Message-ID: <42FA8260.8090502@ucsd.edu> [This is somewhat old. I composed it on a train and only now have Internet, so ignore any bits about the SVN structure that have been already settled.] Travis Oliphant wrote: > Robert Kern wrote: > >> Joe Cooper wrote: >> >>> Oops. It got overlooked. Fixed: >>> >>> http://svn.scipy.org/svn/scipy_core >> >> We really need it to be inside scipy/trunk so that people can do one >> checkout and be able to go at once. What's the best way to do this? I >> don't think we really need it as a separate entity, so perhaps just >> "svn copy"ing the tree over? > > Perhaps having scipy_core and scipy as separate checkouts is a good thing. > > Scipy core is supposed to replace Numeric and Numarray. > > Having a separate tree might help others realize that they can check out > scipy_core separately from scipy and speed the adoption of the unified > array. Well, scipy.core is replacing Numeric. scipy_core is something else. We need something, be it a tag in the SVN repository or a real release, for people to easily build scipy *now* before the restructuring happens. The restructuring can happen in its own little branch and not even bother with the stuff that's currently there. You're right that scipy.core should probably have its own trunk/branches/etc. and be easily checked out alone. It can be dropped into the main scipy repository as an SVN external (sorry Joe). On a related note, I highly recommend looking at setuptools to complement scipy_distutils. It allows for creating "namespace packages" (e.g. scipy, enthought, twisted, zope) whose subpackages can be installed separately. They can even be made to depend on each other, either as a runtime dependency (like scipy.core) or a buildtime dependency (like scipy.distutils). I'll have some time in September to help out with this. We might even want to try scheduling a pre-SciPy-'05 sprint to do restructuring work. http://peak.telecommunity.com/DevCenter/setuptools -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From rkern at ucsd.edu Wed Aug 10 18:40:59 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 10 Aug 2005 15:40:59 -0700 Subject: [SciPy-dev] arithmetic on distributions In-Reply-To: <945F31304DDF6B46814E8422A6B1111D057D53CB@CINMLVEM11.e2k.ad.ge.com> References: <945F31304DDF6B46814E8422A6B1111D057D53CB@CINMLVEM11.e2k.ad.ge.com> Message-ID: <42FA827B.5010309@ucsd.edu> Perry, Alexander (GE Infrastructure) wrote: > Has anyone implemented, or considered implementing, the arithmetic methods for rv_continuous? > > First suggestion. When calling an object to freeze it, if the loc is already a distribution, we should merge its variance into the variance of the newly frozen distribution: > >>ten = norm ( 10, 3 ) >>print norm ( ten, 4 ).stats() > > (10.0, 25.0) > Currently, an error is triggered. Because of the difficulty of merging the standard deviation of the old value into the new desired standard deviation, I suspect that most people who encounter the error will take the lazy way out and simply discard the variance attached to the value: > >>try: value=value.stats()[0] >>except: pass # the above is wrong! I can't imagine why. I think most people would simply conclude that passing in a frozen distribution doesn't work (especially since it is certainly not documented to do so). > Second suggestion. When a binary operator such as __add__() is called on an object that derives from rv_continuous, currently an error is generated. There should be a function that checks whether the other parameter is a simple number and, if it is, modifies its own loc and scale. If both parameters of the binary operator are instances of rv_continuous, the function returns an instance of the new subclass rv_composite that simply stores those two parameters and the desired operator. The new subclass passes through all operations that can be directly decomposed (such as rvs and stats). This allows a real world distribution to be assembled as a single object, and that object directly supports both linear approximation and monte-carlo methods: > >>fifteen = norm ( 10, 3 ) + uniform( 3, 4 ) >># Linearized estimate >>print "%7.5f" % norm ( fifteen, 0 ).cdf ( 20 ) > > 0.94008 > >># Monte carlo estimate >>print sum ( [ x<20 for x in fifteen.rvs(100000) ]) > > 94125 > > The composite distribution instance implements the remainder of the standard "rv" features through integration and differentiation of the underlying distributions. This is no different than what the users would have to do themselves, but saves the effort of explicitly calling those scipy features. > >># Accurate integrated value >>print "%7.5f" % fifteen.cdf(20) > > 0.94000 I would suggest writing rv_composite first and implementing the appropriate algorithms and shortcuts (like norm(0, 3) + norm(0, 4) => norm(0, 5)). The general case for most operations is going to entail sampling which is going to require some more input from the user (e.g. the requested accuracy of the estimated parameters). Once the algorithms are implemented, and we know what information needs to flow where, then is the right time to think about implementing a nicer syntax. > Third suggestion. The __cmp__ method should exist and can simply compare the mean since its python definition is that it returns a Boolean upon which flow of control can be decided. The more accurate comparison functions (such as __lt__) should exist and return probabilities because their python definition is that they return a data result with no flow control interpretation. If the rich comparisons (__lt__ and __eq__, at least) are defined, then __cmp__ will never be called. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From Alex.Perry at ge.com Wed Aug 10 19:35:13 2005 From: Alex.Perry at ge.com (Perry, Alexander (GE Infrastructure)) Date: Wed, 10 Aug 2005 19:35:13 -0400 Subject: [SciPy-dev] arithmetic on distributions Message-ID: <945F31304DDF6B46814E8422A6B1111D058FD555@CINMLVEM11.e2k.ad.ge.com> > Perry, Alexander (GE Infrastructure) wrote: > > First suggestion. When calling an object to freeze it, > > if the loc is already a distribution, we should merge > > its variance into the variance of the newly frozen > > distribution [...] Currently, an error is triggered. From Robert Kern: > I think most people would simply conclude that > passing in a frozen distribution doesn't work > (especially since it is certainly not documented to do so). I _know_ it doesn't work and is nowhere documented as working. That's why it is a suggestion for improvement (not a bug report). > > Because of the difficulty of merging the standard deviation > > of the old value into the new desired standard deviation, You didn't comment on this assertion, but then disagreed below. The code inside the wrapper is not particularly clean or obvious, since norm() expects a stdev, yet stats() returns the variance. #! /usr/bin/python from scipy.stats.distributions import norm from math import sqrt # Proposal, written as a wrapper def normal(loc,scale,**args): try: loc,locvar = loc.stats() scale = sqrt ( scale*scale+locvar ) except: pass return norm(loc,scale,**args) # Two different early constants n1 = 10 n2 = norm(n1,3) # Lots of code later print normal(n1,4) .stats() print normal(n2,4) .stats() > > I suspect that most people who encounter the error will take the > > lazy way out and simply discard the variance attached to the value: > > try: value=value.stats()[0] > > except: pass # the above is wrong! > I can't imagine why. It's wrong because I quietly discarded the distribution. It was presumably present on that variable for a reason, and I only did it to make the subsequent setup call work. > > Third suggestion. The __cmp__ method should exist [...] > If the rich comparisons (__lt__ and __eq__, at least) are defined, > then __cmp__ will never be called. Yes, that's how my python version works, but I saw it documented the other way in some mailing list discussion archive. I'll look into it. From rkern at ucsd.edu Wed Aug 10 19:45:13 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 10 Aug 2005 16:45:13 -0700 Subject: [SciPy-dev] arithmetic on distributions In-Reply-To: <945F31304DDF6B46814E8422A6B1111D058FD555@CINMLVEM11.e2k.ad.ge.com> References: <945F31304DDF6B46814E8422A6B1111D058FD555@CINMLVEM11.e2k.ad.ge.com> Message-ID: <42FA9189.2060507@ucsd.edu> Perry, Alexander (GE Infrastructure) wrote: >>Perry, Alexander (GE Infrastructure) wrote: >> >>>First suggestion. When calling an object to freeze it, >>>if the loc is already a distribution, we should merge >>>its variance into the variance of the newly frozen >>>distribution [...] Currently, an error is triggered. > > From Robert Kern: > >>I think most people would simply conclude that >>passing in a frozen distribution doesn't work >>(especially since it is certainly not documented to do so). > > I _know_ it doesn't work and is nowhere documented as working. > That's why it is a suggestion for improvement (not a bug report). I was expanding on my comment (which you're responding to separately later) that I can't imagine why someone would bother doing a try: except: "to keep going" as it were when they should naturally conclude that it just doesn't work at all and shouldn't be putting frozen distributions as arguments. >>>Because of the difficulty of merging the standard deviation >>>of the old value into the new desired standard deviation, > > You didn't comment on this assertion, but then disagreed below. > The code inside the wrapper is not particularly clean or obvious, > since norm() expects a stdev, yet stats() returns the variance. > > #! /usr/bin/python > from scipy.stats.distributions import norm > from math import sqrt > # Proposal, written as a wrapper > def normal(loc,scale,**args): > try: > loc,locvar = loc.stats() > scale = sqrt ( scale*scale+locvar ) > except: > pass > return norm(loc,scale,**args) > # Two different early constants > n1 = 10 > n2 = norm(n1,3) > # Lots of code later > print normal(n1,4) .stats() > print normal(n2,4) .stats() > > >>>I suspect that most people who encounter the error will take the >>>lazy way out and simply discard the variance attached to the value: >>>try: value=value.stats()[0] >>>except: pass # the above is wrong! > >>I can't imagine why. > > It's wrong because I quietly discarded the distribution. > It was presumably present on that variable for a reason, > and I only did it to make the subsequent setup call work. My statements make a little more sense if they're kept in order. I can't imagine why someone would "take the lazy way out" when the code simply doesn't work. I'm afraid I don't understand any of this conversation anymore if I ever did. Can you try again from the beginning? -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From nwagner at mecha.uni-stuttgart.de Thu Aug 11 09:23:42 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Thu, 11 Aug 2005 15:23:42 +0200 Subject: [SciPy-dev] Scipy SVN updates mailing list Message-ID: <42FB515E.1000706@mecha.uni-stuttgart.de> Is it possible to set up a mailing list for SVN updates ? Nils From Alex.Perry at ge.com Thu Aug 11 13:25:07 2005 From: Alex.Perry at ge.com (Perry, Alexander (GE Infrastructure)) Date: Thu, 11 Aug 2005 13:25:07 -0400 Subject: [SciPy-dev] arithmetic on distributions Message-ID: <945F31304DDF6B46814E8422A6B1111D059A0F9D@CINMLVEM11.e2k.ad.ge.com> > My statements make a little more sense if they're kept in order. Sorry, I think that shows I didn't understand what you were getting at. To me, they seemed to make more sense the other way around. Oh well. > I'm afraid I don't understand any of this conversation anymore > if I ever did. Can you try again from the beginning? Hmm, no. If I'm having trouble explaining it to you, it isn't something we want to have to try and put in the documentation. Alternative suggestion, part 1: When rv_continuous is validating "loc" and "scale" in __fix_loc_scale it should check whether either of them is an instance of rv_continuous and, if it is, generate a situation-specific useful error message. The error message might refer the user to the new function below. Alternative suggestion, part 2: We should make it easy for people to add uncertainty to any variable. It needs to take into account the different distributions in use, taking into account the way loc and scale may be interacting, as well as accept non-distributed values and all the zero cases. The latter is important to allow people to write calculations where it is not locally obvious whether a given parameter is distributed. The code below is a first cut ... #! /usr/bin/python from scipy.stats.distributions import norm, uniform, rv_continuous from math import sqrt class undistributed_gen(rv_continuous): def _pdf(self,x): if x == 0: return 1 return 0 def _cdf(self,x): if x>0: return 1 if x<0: return 0 return 0.5 def _stats(self): return 0.0, 0.0, 0.0, 0.0 undistributed = undistributed_gen(name='undistributed', longname='Undistributed',extradoc=""" No distribution The loc keyword specifies the position of the delta function, The scale keyword is effectively ignored. """) def addStDev ( value, stdev=0 ): # Make sure we know how much uncertainty is added try: std, problem = stdev.stats() except: std, problem = stdev, 0 if problem > 0: raise ValueError, "addStDev does not allow uncertainty "+\ "in the amount of uncertainty being added to the value."+\ "Please rephrase." # Maybe this is an object that understands this? try: return value.addStDev(std) except: pass # If value has no real stats, create try: mean,variance = value.stats() if variance <= 0 and std > 0: value = mean raise ValueError except: if std > 0: return norm(value,std) else: return undistributed(value) # Maybe we're not actually adding any variance if std <= 0: return value s = sqrt ( 1 + std*std / variance ) # Worst case, must create a new rv_frozen d = value.dist a = value.args l = mean s *= a[1] l -= d ( 0, s, *a[2:], **value.kwds ) .stats()[0] return d ( l, s, *a[2:], **value.kwds ) for a in range(8): for ex in [ ( "Constant", 10 ), ( "Uniform4", uniform(10-2,4) ) ]: print "Added stdev %i to a %s distribution at 10 has CDF %5.3f at 11" %\ ( a, ex[0], addStDev(ex[1],a) .cdf(11) ) From alexandre.fayolle at logilab.fr Thu Aug 11 15:58:16 2005 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Thu, 11 Aug 2005 21:58:16 +0200 Subject: [SciPy-dev] semantics of stats.stats.nanstd Message-ID: <20050811195815.GE16319@logilab.fr> I'm working on the problem reported http://www.scipy.net/roundup/scipy/issue243, and I'm wondering if the patch I submitted implements the expected semantics for nanstd() The question is : shall the function simply discard the nan values, or should it consider that a population with nan values become a sample of the population, and change the value of the denominator. To put it otherwise, is it expected that nanstd([nan, 2, 3, 4,], bias=0) == std([2, 3, 4], bias=0) or std([2, 3, 4], bias=1) or some other value ? The patch I sent assumes the first option, but Alexander Perry pointed out that the second option may be what is expected. -- Alexandre Fayolle LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From travis at enthought.com Fri Aug 12 19:12:22 2005 From: travis at enthought.com (Travis N. Vaught) Date: Fri, 12 Aug 2005 18:12:22 -0500 Subject: [SciPy-dev] SciPy 2005 Conference: Preliminary Schedule of Presentations Message-ID: <42FD2CD6.9020904@enthought.com> All, I've posted a preliminary schedule of presentations at: http://www.scipy.org/wikis/scipy05/ConferenceSchedule Please feel free to provide comments or corrections--either to the list or directly to me. If you haven't registered, you may do so at: https://www.enthought.com/scipy05/ More information about the conference is at: http://www.scipy.org/wikis/scipy05/ Regards, Travis (Vaught) From nwagner at mecha.uni-stuttgart.de Tue Aug 16 09:07:44 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Tue, 16 Aug 2005 07:07:44 -0600 Subject: [SciPy-dev] scipy via svn Message-ID: <4301E520.2020100@mecha.uni-stuttgart.de> Hi all, Finally I have installed scipy via svn. svn co http://svn.scipy.org/svn/scipy/trunk scipy and in ~/scipy svn co http://svn.scipy.org/svn/scipy_core/trunk/ scipy_core Python 2.4 (#1, May 30 2005, 19:23:06) [GCC 3.3.5 20050117 (prerelease) (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy No module named __cvs_version__ ====================================================================== ERROR: check_integer (scipy.io.array_import.test_array_import.test_read_array) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/scipy/io/tests/test_array_import.py", line 57, in check_integer a = stats.randint.rvs(1,20,size=(3,4)) File "/usr/lib/python2.4/site-packages/scipy_base/ppimport.py", line 303, in __getattr__ module = self._ppimport_importer() File "/usr/lib/python2.4/site-packages/scipy_base/ppimport.py", line 258, in _ppimport_importer raise ImportError,self.__dict__.get('_ppimport_exc_info')[1] ImportError: No module named __cvs_version__ ---------------------------------------------------------------------- Ran 245 tests in 0.304s FAILED (errors=1) Some obsolote cvs related problems arise. Nils From joe at enthought.com Tue Aug 16 17:11:42 2005 From: joe at enthought.com (Joe Cooper) Date: Tue, 16 Aug 2005 16:11:42 -0500 Subject: [SciPy-dev] scipy via svn In-Reply-To: <4301E520.2020100@mecha.uni-stuttgart.de> References: <4301E520.2020100@mecha.uni-stuttgart.de> Message-ID: <4302568E.1020207@enthought.com> Hey Nils, I know what causes this, but I don't know how to go about fixing it off-hand. Pearu wrote some clever versioning code that used the CVS revision as a version number (this was to ease packaging via RPM among other things)...But that code couldn't come over from CVS, as the conversion broke with it in the repo. These bits need to be updated to support Subversion. I can re-add the files manually to the Subversion repository--but they obviously won't work as they should until being updated to with with Subversion version information. It might make the actual code work again though... Nils Wagner wrote: > Hi all, > > Finally I have installed scipy via svn. > > svn co http://svn.scipy.org/svn/scipy/trunk scipy > > and in ~/scipy > > svn co http://svn.scipy.org/svn/scipy_core/trunk/ scipy_core > > > Python 2.4 (#1, May 30 2005, 19:23:06) > [GCC 3.3.5 20050117 (prerelease) (SUSE Linux)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>>>import scipy > > No module named __cvs_version__ > > ====================================================================== > ERROR: check_integer > (scipy.io.array_import.test_array_import.test_read_array) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/usr/lib/python2.4/site-packages/scipy/io/tests/test_array_import.py", > line 57, in check_integer > a = stats.randint.rvs(1,20,size=(3,4)) > File "/usr/lib/python2.4/site-packages/scipy_base/ppimport.py", line > 303, in __getattr__ > module = self._ppimport_importer() > File "/usr/lib/python2.4/site-packages/scipy_base/ppimport.py", line > 258, in _ppimport_importer > raise ImportError,self.__dict__.get('_ppimport_exc_info')[1] > ImportError: No module named __cvs_version__ > > ---------------------------------------------------------------------- > Ran 245 tests in 0.304s > > FAILED (errors=1) > > > Some obsolote cvs related problems arise. > > Nils > > > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev From joe at enthought.com Tue Aug 16 19:39:32 2005 From: joe at enthought.com (Joe Cooper) Date: Tue, 16 Aug 2005 18:39:32 -0500 Subject: [SciPy-dev] SciPy SVN reorganization Message-ID: <43027934.1020002@enthought.com> Hi all, A concensus seems to have been reached on the re-organization of the SciPy repository, and it also has been agreed that a new release will be rolled immediately (or some approximation of immediately) based on the old layout. To that end I have tagged both scipy and scipy_core repositories with a "pre_org" tag: http://svn.scipy.org/svn/scipy_core/tags/pre_org/ http://svn.scipy.org/svn/scipy/tags/pre_org/ The immediate release should be built from this tag. If you need to make changes, please branch this tag, and work on the branch. I will not commit the re-organized repository until tomorrow, as I want folks to have a chance to be aware of the changes and make any commits they want to make before the re-organization. The changes are sweeping and /will/ certainly require a fresh checkout, as I'm moving scipy_core into scipy...so one repo will disappear. So, do whatever you have to do /now/, or let me know that you need more time. If anyone involved in the release wants me to re-tag in a few hours after some commits, I can probably do that too. If anyone needs assistance with using branches or tags in Subversion please let me know. From joe at enthought.com Tue Aug 16 23:03:34 2005 From: joe at enthought.com (Joe Cooper) Date: Tue, 16 Aug 2005 22:03:34 -0500 Subject: [SciPy-dev] scipy via svn In-Reply-To: <4301E520.2020100@mecha.uni-stuttgart.de> References: <4301E520.2020100@mecha.uni-stuttgart.de> Message-ID: <4302A906.3020406@enthought.com> Hey Nils, I've re-added the cvs_version files (just randomly pulled from an old CVS checkout I had laying around--as I mentioned, they have to be replaced with Subversion compatible code, so they don't really matter except that the lack of them causes problems). Anyway, if you update your tree, things might work again. Holler if it doesn't. Nils Wagner wrote: > Hi all, > > Finally I have installed scipy via svn. > > svn co http://svn.scipy.org/svn/scipy/trunk scipy > > and in ~/scipy > > svn co http://svn.scipy.org/svn/scipy_core/trunk/ scipy_core > > > Python 2.4 (#1, May 30 2005, 19:23:06) > [GCC 3.3.5 20050117 (prerelease) (SUSE Linux)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>>>import scipy > > No module named __cvs_version__ > > ====================================================================== > ERROR: check_integer > (scipy.io.array_import.test_array_import.test_read_array) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/usr/lib/python2.4/site-packages/scipy/io/tests/test_array_import.py", > line 57, in check_integer > a = stats.randint.rvs(1,20,size=(3,4)) > File "/usr/lib/python2.4/site-packages/scipy_base/ppimport.py", line > 303, in __getattr__ > module = self._ppimport_importer() > File "/usr/lib/python2.4/site-packages/scipy_base/ppimport.py", line > 258, in _ppimport_importer > raise ImportError,self.__dict__.get('_ppimport_exc_info')[1] > ImportError: No module named __cvs_version__ > > ---------------------------------------------------------------------- > Ran 245 tests in 0.304s > > FAILED (errors=1) > > > Some obsolote cvs related problems arise. > > Nils > > > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev From nwagner at mecha.uni-stuttgart.de Wed Aug 17 02:51:47 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 17 Aug 2005 08:51:47 +0200 Subject: [SciPy-dev] scipy via svn In-Reply-To: <4302A906.3020406@enthought.com> References: <4301E520.2020100@mecha.uni-stuttgart.de> <4302A906.3020406@enthought.com> Message-ID: <4302DE83.2040302@mecha.uni-stuttgart.de> Joe Cooper wrote: > Hey Nils, > > I've re-added the cvs_version files (just randomly pulled from an old > CVS checkout I had laying around--as I mentioned, they have to be > replaced with Subversion compatible code, so they don't really matter > except that the lack of them causes problems). Anyway, if you update > your tree, things might work again. Holler if it doesn't. > Hey Joe, Thank you very much for your workaround. BTW, is it possible to use Numeric3 instead of scipy_core ? And what is necessary to install scipy + Numeric3 ? Nils > Nils Wagner wrote: >> Hi all, >> >> Finally I have installed scipy via svn. >> >> svn co http://svn.scipy.org/svn/scipy/trunk scipy >> >> and in ~/scipy >> >> svn co http://svn.scipy.org/svn/scipy_core/trunk/ scipy_core >> >> >> Python 2.4 (#1, May 30 2005, 19:23:06) >> [GCC 3.3.5 20050117 (prerelease) (SUSE Linux)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >>>>> import scipy >> >> No module named __cvs_version__ >> >> ====================================================================== >> ERROR: check_integer >> (scipy.io.array_import.test_array_import.test_read_array) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "/usr/lib/python2.4/site-packages/scipy/io/tests/test_array_import.py", >> line 57, in check_integer >> a = stats.randint.rvs(1,20,size=(3,4)) >> File "/usr/lib/python2.4/site-packages/scipy_base/ppimport.py", line >> 303, in __getattr__ >> module = self._ppimport_importer() >> File "/usr/lib/python2.4/site-packages/scipy_base/ppimport.py", line >> 258, in _ppimport_importer >> raise ImportError,self.__dict__.get('_ppimport_exc_info')[1] >> ImportError: No module named __cvs_version__ >> >> ---------------------------------------------------------------------- >> Ran 245 tests in 0.304s >> >> FAILED (errors=1) >> >> >> Some obsolote cvs related problems arise. >> >> Nils >> >> >> >> >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.net >> http://www.scipy.net/mailman/listinfo/scipy-dev > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev From rkern at ucsd.edu Wed Aug 17 03:11:18 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 17 Aug 2005 00:11:18 -0700 Subject: [SciPy-dev] scipy via svn In-Reply-To: <4302DE83.2040302@mecha.uni-stuttgart.de> References: <4301E520.2020100@mecha.uni-stuttgart.de> <4302A906.3020406@enthought.com> <4302DE83.2040302@mecha.uni-stuttgart.de> Message-ID: <4302E316.8080703@ucsd.edu> Nils Wagner wrote: > Joe Cooper wrote: > >>Hey Nils, >> >>I've re-added the cvs_version files (just randomly pulled from an old >>CVS checkout I had laying around--as I mentioned, they have to be >>replaced with Subversion compatible code, so they don't really matter >>except that the lack of them causes problems). Anyway, if you update >>your tree, things might work again. Holler if it doesn't. > > Hey Joe, > > Thank you very much for your workaround. > BTW, is it possible to use Numeric3 instead of scipy_core ? > And what is necessary to install scipy + Numeric3 ? That will happen as part of the reorganization later. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From nwagner at mecha.uni-stuttgart.de Wed Aug 17 03:17:13 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 17 Aug 2005 09:17:13 +0200 Subject: [SciPy-dev] scipy via svn In-Reply-To: <4302E316.8080703@ucsd.edu> References: <4301E520.2020100@mecha.uni-stuttgart.de> <4302A906.3020406@enthought.com> <4302DE83.2040302@mecha.uni-stuttgart.de> <4302E316.8080703@ucsd.edu> Message-ID: <4302E479.9040302@mecha.uni-stuttgart.de> Robert Kern wrote: > Nils Wagner wrote: >> Joe Cooper wrote: >> >>> Hey Nils, >>> >>> I've re-added the cvs_version files (just randomly pulled from an old >>> CVS checkout I had laying around--as I mentioned, they have to be >>> replaced with Subversion compatible code, so they don't really matter >>> except that the lack of them causes problems). Anyway, if you update >>> your tree, things might work again. Holler if it doesn't. >> >> Hey Joe, >> >> Thank you very much for your workaround. >> BTW, is it possible to use Numeric3 instead of scipy_core ? >> And what is necessary to install scipy + Numeric3 ? > > That will happen as part of the reorganization later. > I thought that it is already possible to use Numeric3 within scipy (due to a previous mail by Travis 08/10/2005) >For anybody interested in the development of scipy.base. The repository is in a state that can be tested and played with. >I'm sure there are bugs, but I've removed the ones I've found. I'd be interested in help in tracking down other Nils From oliphant at ee.byu.edu Wed Aug 17 03:58:43 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 17 Aug 2005 01:58:43 -0600 Subject: [SciPy-dev] scipy via svn In-Reply-To: <4302E479.9040302@mecha.uni-stuttgart.de> References: <4301E520.2020100@mecha.uni-stuttgart.de> <4302A906.3020406@enthought.com> <4302DE83.2040302@mecha.uni-stuttgart.de> <4302E316.8080703@ucsd.edu> <4302E479.9040302@mecha.uni-stuttgart.de> Message-ID: <4302EE33.2020509@ee.byu.edu> Nils Wagner wrote: >I thought that it is already possible to use Numeric3 within scipy (due >to a previous mail by Travis 08/10/2005) > > > scipy.base is just Numeric + old scipy_base. The array object and ufuncs + methods are defined and working. We still need to construct a matrix sub-class, and add all the functionality of Numeric + MLab + old scipy_base. Then, scipy_core needs to be updated to include bare-bones, C-only versions of linalg, fft, and random. Don't expect to be able to just build scipy with the new array object for a few weeks at least. Getting a release of scipy out based on the old build is part of that process. I fixed a few bugs but could not commit because I didn't have password information (now I've forgotten it and can't find the email...) -Travis From nwagner at mecha.uni-stuttgart.de Wed Aug 17 04:06:55 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 17 Aug 2005 10:06:55 +0200 Subject: [SciPy-dev] scipy via svn In-Reply-To: <4302EE33.2020509@ee.byu.edu> References: <4301E520.2020100@mecha.uni-stuttgart.de> <4302A906.3020406@enthought.com> <4302DE83.2040302@mecha.uni-stuttgart.de> <4302E316.8080703@ucsd.edu> <4302E479.9040302@mecha.uni-stuttgart.de> <4302EE33.2020509@ee.byu.edu> Message-ID: <4302F01F.7000404@mecha.uni-stuttgart.de> Travis Oliphant wrote: > Nils Wagner wrote: > >> I thought that it is already possible to use Numeric3 within scipy (due >> to a previous mail by Travis 08/10/2005) >> >> >> > scipy.base is just Numeric + old scipy_base. The array object and > ufuncs + methods are defined and working. We still need to > construct a matrix sub-class, and add all the functionality of Numeric > + MLab + old scipy_base. Then, scipy_core needs to be updated to > include bare-bones, C-only versions of linalg, fft, and random. > > Don't expect to be able to just build scipy with the new array object > for a few weeks at least. > > Getting a release of scipy out based on the old build is part of that > process. I fixed a few bugs but could not commit because I didn't > have password information (now I've forgotten it and can't find the > email...) > > -Travis > So what is recommended in the meantime to get a working scipy version? May I use svn co http://svn.scipy.org/svn/scipy/trunk scipy cd scipy svn co http://svn.scipy.org/svn/scipy_core/trunk/ scipy_core python setup.py build python setup.py install Nils > > > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.net > http://www.scipy.net/mailman/listinfo/scipy-dev From novak at ucolick.org Wed Aug 17 15:49:11 2005 From: novak at ucolick.org (Greg Novak) Date: Wed, 17 Aug 2005 12:49:11 -0700 (PDT) Subject: [SciPy-dev] Problems with Scipy and Fedora Core 4 Message-ID: I'm having trouble getting Scipy to run on Fedora Core 4. The latest release (0.3.2) fails to compile with errors in the source, probably related to the switch to GCC 4.0. I'm not summarizing them here because the problems don't exist in the Subversion code. After checking out the latest code from svn, Scipy compiles but I get the following error: ImportError: /usr/lib/python2.4/site-packages/scipy/linalg/clapack.so: undefined symbol: clapack_sgesv I'm using Fedora Cora 4, with RPMS blas-3.0-29, lapack-3.0-29, and atlas-3.6.0-0.fdr.2 installed. I've also installed compat-gcc-33-g77 to get g77 and libg2c.a. This... ahem... would seem to have the effect of mixing GCC 3.3 and 4.0 code, which strikes me as dicey at best. Are my worries justified? I've attached a complete traceback and the output of scipy_distutils/system_info.py. Any tips are appreciated, including (if necessary) "It doesn't work right now." Thanks, Greg -------------- next part -------------- No module named __cvs_version__ --------------------------------------------------------------------------- exceptions.ImportError Traceback (most recent call last) /home/novak/Projects/Thesis/ /home/novak/Projects/Thesis/pn.py in ellipticity(original_rs, ms, scaled, verbose, **kw) 3696 rs_inside, ms_inside, id_inside = \ 3697 inside_ellipsoid(rotated_rs, ms, axes, **kw) -> 3698 eval, evect = linalg.eig( 3699 position_moment(rs_inside, ms_inside, 3700 spherical=position_moment_spherical, /usr/lib/python2.4/site-packages/scipy_base/ppimport.py in __getattr__(self, name) 301 module = self.__dict__['_ppimport_module'] 302 except KeyError: --> 303 module = self._ppimport_importer() 304 return getattr(module, name) 305 /usr/lib/python2.4/site-packages/scipy_base/ppimport.py in _ppimport_importer(self) 271 print 'Executing postponed import for %s' %(name) 272 try: --> 273 module = __import__(name,None,None,['*']) 274 except Exception,msg: # ImportError: 275 if DEBUG: /usr/lib/python2.4/site-packages/scipy/linalg/__init__.py 6 from linalg_version import linalg_version as __version__ 7 ----> 8 from basic import * 9 from decomp import * 10 from matfuncs import * /usr/lib/python2.4/site-packages/scipy/linalg/basic.py 10 11 #from blas import get_blas_funcs ---> 12 from lapack import get_lapack_funcs 13 from flinalg import get_flinalg_funcs 14 from scipy_base import asarray,zeros,sum,NewAxis,greater_equal,subtract,arange,\ /usr/lib/python2.4/site-packages/scipy/linalg/lapack.py 14 15 import flapack ---> 16 import clapack 17 _use_force_clapack = 1 18 if hasattr(clapack,'empty_module'): ImportError: /usr/lib/python2.4/site-packages/scipy/linalg/clapack.so: undefined symbol: clapack_sgesv -------------- next part -------------- [root at dionysus scipy_distutils]# python system_info.py _pkg_config_info: NOT AVAILABLE agg2_info: ( src_dirs = .:/usr/local/src ) NOT AVAILABLE atlas_blas_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/sse2 ) ( paths: /usr/lib/libf77blas.a ) ( paths: /usr/lib/libcblas.a ) ( paths: /usr/lib/libatlas.a ) ( include_dirs = /usr/local/include:/usr/include ) ( paths: /usr/include/atlas ) ( paths: /usr/include/atlas/cblas.h ) ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/sse2 ) FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib'] language = c include_dirs = ['/usr/include/atlas'] atlas_blas_threads_info: Setting PTATLAS=ATLAS ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/sse2 ) ( paths: /usr/lib/libptcblas.a ) ( paths: /usr/lib/libatlas.a ) NOT AVAILABLE atlas_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/sse2 ) ( paths: /usr/lib/libf77blas.a ) ( paths: /usr/lib/libcblas.a ) ( paths: /usr/lib/libatlas.a ) ( paths: /usr/lib/liblapack.so ) system_info.atlas_info ( include_dirs = /usr/local/include:/usr/include ) ( paths: /usr/include/atlas ) ( paths: /usr/include/atlas/cblas.h ) ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/sse2 ) FOUND: libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib'] language = f77 include_dirs = ['/usr/include/atlas'] atlas_threads_info: Setting PTATLAS=ATLAS ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/sse2 ) ( paths: /usr/lib/libptcblas.a ) ( paths: /usr/lib/libatlas.a ) system_info.atlas_threads_info NOT AVAILABLE blas_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/libblas.so ) ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['blas'] library_dirs = ['/usr/lib'] language = f77 blas_opt_info: running build_src applying backend (numeric) info to extensions numpy_info: FOUND: define_macros = [('NUMERIC_VERSION', '"\\"23.7\\""'), ('NUMERIC', None)] include_dirs = ['/usr/include/python2.4'] building extension "atlas_version" sources creating build creating build/src adding 'build/src/atlas_version_-0x5589b31.c' to sources. running build_ext customize UnixCCompiler customize UnixCCompiler using build_ext building 'atlas_version' extension compiling C sources gcc options: '-pthread -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -m32 -march=i386 -mtune=pentium4 -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fPIC' creating build/temp.linux-i686-2.4 creating build/temp.linux-i686-2.4/build creating build/temp.linux-i686-2.4/build/src compile options: '-DNUMERIC_VERSION="\"23.7\"" -DNUMERIC -I/usr/include/atlas -I/usr/include/python2.4 -I/usr/include/python2.4 -c' gcc: build/src/atlas_version_-0x5589b31.c gcc -pthread -shared build/temp.linux-i686-2.4/build/src/atlas_version_-0x5589b31.o -L/usr/lib -lf77blas -lcblas -latlas -o build/temp.linux-i686-2.4/atlas_version.so ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib'] language = c define_macros = [('ATLAS_INFO', '"\\"3.6.0\\""')] include_dirs = ['/usr/include/atlas'] blas_src_info: ( src_dirs = .:/usr/local/src ) NOT AVAILABLE boost_python_info: ( src_dirs = .:/usr/local/src ) NOT AVAILABLE dfftw_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( include_dirs = /usr/local/include:/usr/include ) dfftw not found NOT AVAILABLE dfftw_threads_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( include_dirs = /usr/local/include:/usr/include ) dfftw threads not found NOT AVAILABLE djbfft_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( include_dirs = /usr/local/include:/usr/include ) NOT AVAILABLE fftw_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( include_dirs = /usr/local/include:/usr/include ) fftw3 not found ( library_dirs = /usr/local/lib:/usr/lib ) ( include_dirs = /usr/local/include:/usr/include ) fftw2 not found NOT AVAILABLE fftw_threads_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( include_dirs = /usr/local/include:/usr/include ) fftw threads not found NOT AVAILABLE freetype2_info: ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['freetype', 'z'] define_macros = [('FREETYPE2_INFO', '"\\"9.7.3\\""'), ('FREETYPE2_VERSION_9_7_3', None)] include_dirs = ['/usr/include/freetype2'] gdk_2_info: ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['gdk-x11-2.0', 'gdk_pixbuf-2.0', 'm', 'pangoxft-1.0', 'pangox-1.0', 'pango-1.0', 'gobject-2.0', 'gmodule-2.0', 'dl', 'glib-2.0'] define_macros = [('GDK_2_INFO', '"\\"2.6.7\\""'), ('GDK_VERSION_2_6_7', None), ('XTHREADS', None), ('_REENTRANT', None), ('XUSE_MTSAFE_API', None)] include_dirs = ['/usr/include/gtk-2.0', '/usr/lib/gtk-2.0/include', '/usr/X11R6/include', '/usr/include/pango-1.0', '/usr/include/freetype2', '/usr/include/freetype2/config', '/usr/include/glib-2.0', '/usr/lib/glib-2.0/include'] gdk_info: NOT AVAILABLE gdk_pixbuf_2_info: ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['gdk_pixbuf-2.0', 'm', 'gobject-2.0', 'gmodule-2.0', 'dl', 'glib-2.0'] define_macros = [('GDK_PIXBUF_2_INFO', '"\\"2.6.7\\""'), ('GDK_PIXBUF_VERSION_2_6_7', None)] include_dirs = ['/usr/include/gtk-2.0', '/usr/include/glib-2.0', '/usr/lib/glib-2.0/include'] gdk_pixbuf_xlib_2_info: ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['gdk_pixbuf_xlib-2.0', 'gdk_pixbuf-2.0', 'm', 'gobject-2.0', 'gmodule-2.0', 'dl', 'glib-2.0'] extra_link_args = ['-Wl,--export-dynamic'] define_macros = [('GDK_PIXBUF_XLIB_2_INFO', '"\\"2.6.7\\""'), ('GDK_PIXBUF_XLIB_VERSION_2_6_7', None)] include_dirs = ['/usr/include/gtk-2.0', '/usr/include/glib-2.0', '/usr/lib/glib-2.0/include'] gdk_x11_2_info: ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['gdk-x11-2.0', 'gdk_pixbuf-2.0', 'm', 'pangoxft-1.0', 'pangox-1.0', 'pango-1.0', 'gobject-2.0', 'gmodule-2.0', 'dl', 'glib-2.0'] define_macros = [('GDK_X11_2_INFO', '"\\"2.6.7\\""'), ('GDK_X11_VERSION_2_6_7', None), ('XTHREADS', None), ('_REENTRANT', None), ('XUSE_MTSAFE_API', None)] include_dirs = ['/usr/include/gtk-2.0', '/usr/lib/gtk-2.0/include', '/usr/X11R6/include', '/usr/include/pango-1.0', '/usr/include/freetype2', '/usr/include/freetype2/config', '/usr/include/glib-2.0', '/usr/lib/glib-2.0/include'] gtkp_2_info: ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['gtk-x11-2.0', 'gdk-x11-2.0', 'atk-1.0', 'gdk_pixbuf-2.0', 'm', 'pangoxft-1.0', 'pangox-1.0', 'pango-1.0', 'gobject-2.0', 'gmodule-2.0', 'dl', 'glib-2.0'] define_macros = [('GTKP_2_INFO', '"\\"2.6.7\\""'), ('GTK_VERSION_2_6_7', None), ('XTHREADS', None), ('_REENTRANT', None), ('XUSE_MTSAFE_API', None)] include_dirs = ['/usr/include/gtk-2.0', '/usr/lib/gtk-2.0/include', '/usr/X11R6/include', '/usr/include/atk-1.0', '/usr/include/pango-1.0', '/usr/include/freetype2', '/usr/include/freetype2/config', '/usr/include/glib-2.0', '/usr/lib/glib-2.0/include'] gtkp_x11_2_info: ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['gtk-x11-2.0', 'gdk-x11-2.0', 'atk-1.0', 'gdk_pixbuf-2.0', 'm', 'pangoxft-1.0', 'pangox-1.0', 'pango-1.0', 'gobject-2.0', 'gmodule-2.0', 'dl', 'glib-2.0'] define_macros = [('GTKP_X11_2_INFO', '"\\"2.6.7\\""'), ('GTK_X11_VERSION_2_6_7', None), ('XTHREADS', None), ('_REENTRANT', None), ('XUSE_MTSAFE_API', None)] include_dirs = ['/usr/include/gtk-2.0', '/usr/lib/gtk-2.0/include', '/usr/X11R6/include', '/usr/include/atk-1.0', '/usr/include/pango-1.0', '/usr/include/freetype2', '/usr/include/freetype2/config', '/usr/include/glib-2.0', '/usr/lib/glib-2.0/include'] lapack_atlas_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/sse2 ) ( paths: /usr/lib/libf77blas.a ) ( paths: /usr/lib/libcblas.a ) ( paths: /usr/lib/libatlas.a ) system_info.lapack_atlas_info NOT AVAILABLE lapack_atlas_threads_info: Setting PTATLAS=ATLAS ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/sse2 ) ( paths: /usr/lib/libptcblas.a ) ( paths: /usr/lib/libatlas.a ) system_info.lapack_atlas_threads_info NOT AVAILABLE lapack_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/liblapack.so ) ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['lapack'] library_dirs = ['/usr/lib'] language = f77 lapack_opt_info: running build_src applying backend (numeric) info to extensions building extension "atlas_version" sources adding 'build/src/atlas_version_-0x70f57d4.c' to sources. running build_ext customize UnixCCompiler customize UnixCCompiler using build_ext customize GnuFCompiler customize GnuFCompiler customize GnuFCompiler using build_ext ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib'] language = f77 define_macros = [('ATLAS_INFO', '"\\"3.6.0\\""')] include_dirs = ['/usr/include/atlas'] lapack_src_info: ( src_dirs = .:/usr/local/src ) NOT AVAILABLE numarray_info: NOT AVAILABLE numpy_info: ( include_dirs = /usr/include/python2.4 ) ( paths: /usr/include/python2.4/Numeric/arrayobject.h ) ( library_dirs = ) FOUND: define_macros = [('NUMERIC_VERSION', '"\\"23.7\\""'), ('NUMERIC', None)] include_dirs = ['/usr/include/python2.4'] sfftw_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( include_dirs = /usr/local/include:/usr/include ) sfftw not found NOT AVAILABLE sfftw_threads_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( include_dirs = /usr/local/include:/usr/include ) sfftw threads not found NOT AVAILABLE wx_info: Could not locate executable wx-config File not found: wx-config. Cannot determine wx info. NOT AVAILABLE x11_info: ( library_dirs = /usr/X11R6/lib:/usr/lib ) ( include_dirs = /usr/X11R6/include:/usr/include ) ( paths: /usr/X11R6/lib/libX11.so ) ( paths: /usr/X11R6/include/X11/X.h ) ( library_dirs = /usr/X11R6/lib:/usr/lib ) FOUND: libraries = ['X11'] library_dirs = ['/usr/X11R6/lib'] include_dirs = ['/usr/X11R6/include'] xft_info: ( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['Xft', 'X11', 'freetype', 'Xrender', 'fontconfig'] library_dirs = ['/usr/X11R6/lib'] define_macros = [('XFT_INFO', '"\\"2.1.2.2\\""'), ('XFT_VERSION_2_1_2_2', None)] include_dirs = ['/usr/X11R6/include', '/usr/include/freetype2', '/usr/include/freetype2/config'] From ravi at ati.com Wed Aug 17 16:15:17 2005 From: ravi at ati.com (Ravikiran Rajagopal) Date: Wed, 17 Aug 2005 16:15:17 -0400 Subject: [SciPy-dev] Problems with Scipy and Fedora Core 4 In-Reply-To: References: Message-ID: <200508171615.17173.ravi@ati.com> Hi Greg, Attached is a patch to Scipy CVS from just before the subversion conversion that lets you compile with gcc 4.0 and gfortran (no need for compat-gcc*). Note, however, that there are problems with the special functions module which I guess are gfortran-related. Please make sure you have the gcc updates from a couple of weeks ago (at least version 4.0.1-4) since Hollerith constant support was absent in earlier versions of gfortran. Where did you get your ATLAS libraries? Did you build them with the same gcc and gfortran that you used to build scipy? To make ATLAS use gfortran, you will need to maually edit the top level Makefile after configuration; the Makefile should be called Makefile.arch. where 'arch' is a name like 'Linux_ATHLONSSE1'. Hope this helps. Ravi > After checking out the latest code from svn, Scipy compiles but I get the > following error: > ImportError: /usr/lib/python2.4/site-packages/scipy/linalg/clapack.so: > undefined symbol: clapack_sgesv > > I'm using Fedora Cora 4, with RPMS blas-3.0-29, lapack-3.0-29, and > atlas-3.6.0-0.fdr.2 installed. I've also installed compat-gcc-33-g77 to > get g77 and libg2c.a. This... ahem... would seem to have the effect of > mixing GCC 3.3 and 4.0 code, which strikes me as dicey at best. Are my > worries justified? > > I've attached a complete traceback and the output of > scipy_distutils/system_info.py. Any tips are appreciated, including (if > necessary) "It doesn't work right now." > > Thanks, > Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: s.diff Type: text/x-diff Size: 6072 bytes Desc: not available URL: From nwagner at mecha.uni-stuttgart.de Wed Aug 17 16:16:33 2005 From: nwagner at mecha.uni-stuttgart.de (Nils Wagner) Date: Wed, 17 Aug 2005 22:16:33 +0200 Subject: [SciPy-dev] Problems with Scipy and Fedora Core 4 In-Reply-To: References: Message-ID: On Wed, 17 Aug 2005 12:49:11 -0700 (PDT) Greg Novak wrote: > I'm having trouble getting Scipy to run on Fedora Core >4. The latest release (0.3.2) fails to compile with >errors in the source, probably related to the switch to >GCC 4.0. I'm not summarizing them here because the >problems don't exist in the Subversion code. > > After checking out the latest code from svn, Scipy >compiles but I get the following error: > ImportError: >/usr/lib/python2.4/site-packages/scipy/linalg/clapack.so: >undefined symbol: clapack_sgesv > Maybe you use an incomplete liblapack. http://math-atlas.sourceforge.net/errata.html#completelp Nils > I'm using Fedora Cora 4, with RPMS blas-3.0-29, >lapack-3.0-29, and atlas-3.6.0-0.fdr.2 installed. I've >also installed compat-gcc-33-g77 to get g77 and libg2c.a. > This... ahem... would seem to have the effect of mixing >GCC 3.3 and 4.0 code, which strikes me as dicey at best. > Are my worries justified? > > I've attached a complete traceback and the output of >scipy_distutils/system_info.py. Any tips are >appreciated, including (if necessary) "It doesn't work >right now." > > Thanks, > Greg From oliphant at ee.byu.edu Wed Aug 17 16:31:22 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 17 Aug 2005 14:31:22 -0600 Subject: [SciPy-dev] Restructuring of SciPy Message-ID: <43039E9A.5050905@ee.byu.edu> After the 0.3.2 release, scipy will be getting a face-lift. As part of that, I'm wondering about the restructuring process. For a long time, we've been operating under the "big-scipy-package" with capability for small packages to be released separately model. It has had limited success and I don't know of any small packages actually being released like that. With things like Enthon out there that act as the "batteries included super distribution" anyway. It makes me wonder if we shouldn't move scipy to the "scipy-modules" perspective directly and develop and think about scipy has a collection of modules that work together smoothly (i.e. naming conventions, etc.) scipy_core + available instructions could provide all the infrastructure someone needs to contribute packages to scipy. This would also allow people to develop (and maintain) their own modules (which they seem to like to do anyway). If we could just get them to play nice with the rest of scipy_core, it would all be for benefit. We already have the infrastructure in place for modules to play nicely with the overall package. I'm just wondering if we shouldn't emphasize more the modular approach by the way the svn tree is layed out (so individual modules can be checked out independently, for example). I've also spoken to people who find it hard to rely on their scipy in their code they pass on because you have to get the whole package, even if you only use a few parts. Matlab also uses modules effectively. I think we should take advantage of the restructuring process to think about this. Comments? -Travis From charles.harris at sdl.usu.edu Wed Aug 17 17:20:11 2005 From: charles.harris at sdl.usu.edu (Charles R Harris) Date: Wed, 17 Aug 2005 15:20:11 -0600 Subject: [SciPy-dev] Restructuring of SciPy In-Reply-To: <43039E9A.5050905@ee.byu.edu> References: <43039E9A.5050905@ee.byu.edu> Message-ID: <1124313611.4825.8.camel@E011704> On Wed, 2005-08-17 at 14:31 -0600, Travis Oliphant wrote: > After the 0.3.2 release, scipy will be getting a face-lift. > > As part of that, I'm wondering about the restructuring process. For a > long time, we've been operating under the "big-scipy-package" with > capability for small packages to be released separately model. It has > had limited success and I don't know of any small packages actually > being released like that. > > With things like Enthon out there that act as the "batteries included > super distribution" anyway. It makes me wonder if we shouldn't move > scipy to the "scipy-modules" perspective directly and develop and think > about scipy has a collection of modules that work together smoothly > (i.e. naming conventions, etc.) > > scipy_core + available instructions could provide all the infrastructure > someone needs to contribute packages to scipy. This would also allow > people to develop (and maintain) their own modules (which they seem to > like to do anyway). If we could just get them to play nice with the > rest of scipy_core, it would all be for benefit. > > We already have the infrastructure in place for modules to play nicely > with the overall package. I'm just wondering if we shouldn't emphasize > more the modular approach by the way the svn tree is layed out (so > individual modules can be checked out independently, for example). > > I've also spoken to people who find it hard to rely on their scipy in > their code they pass on because you have to get the whole package, even > if you only use a few parts. > > Matlab also uses modules effectively. > > I think we should take advantage of the restructuring process to think > about this. > > Comments? > > -Travis I think it is a good idea if: 1) There is good documentation so folks know which modules to get. For instance, as far as I know, the zero finders are still in Optimization and that is not the obvious place to look. Perhaps a package approach combined with modules would work better. Or at least some sort of index. 2) It is possible to deal with module dependences. The increasing number of programs also makes this desirable, as we can include more and worry less about suitability. -Chuck From aisaac at american.edu Wed Aug 17 22:05:11 2005 From: aisaac at american.edu (Alan G Isaac) Date: Wed, 17 Aug 2005 22:05:11 -0400 Subject: [SciPy-dev] Restructuring of SciPy In-Reply-To: <1124313611.4825.8.camel@E011704> References: <43039E9A.5050905@ee.byu.edu><1124313611.4825.8.camel@E011704> Message-ID: On Wed, 2005-08-17 at 14:31 -0600, Travis Oliphant wrote: > scipy_core + available instructions could provide all the > infrastructure someone needs to contribute packages to > scipy. This would also allow people to develop (and > maintain) their own modules (which they seem to like to do > anyway). If we could just get them to play nice with the > rest of scipy_core, it would all be for benefit. I think the user's perspective is: what we really need is a preconfigured package manager. From this I conclude that any restructuring will benefit by enabling easy access with a package manager such as PackMan or pypan. http://www.python.org/packman/ http://starship.python.net/crew/theller/pypan/ A user's perspective: as long as - installation is simple, and - the core of what I need is one install that is easy to update then I'll be happy. SciPy has largely met that demand for me, except that I had to flail about to learn how to best approach simple data driven graphics. Message from *that* experience: make smart choices for the new user! Do NOT just give me a bunch of options when I am starting out, as I will lack the information to choose among them. That may suggest a fairly large SciPy, since lots of initial choices should be made for the new users. (Of course it should also be easy to deviate from these if desired.) Question for the longer term: what structure will make it as easy as possible for people with very focused interests to add modules and submit patches that will be rapidly incorprated in SciPy? Surely some of the recent open source research literature must offer some help here? (But I do not know what it is.) I suspect the answer is: it must be possible for a person in charge of one module/package not to worry about what is happening with other modules/packages. Is this affected by the proposal? fwiw, Alan Isaac From Alex.Perry at ge.com Thu Aug 18 12:55:12 2005 From: Alex.Perry at ge.com (Perry, Alexander (GE Infrastructure)) Date: Thu, 18 Aug 2005 12:55:12 -0400 Subject: [SciPy-dev] stats._support and stats.anova ... SVN Message-ID: <945F31304DDF6B46814E8422A6B1111D05DEEE42@CINMLVEM11.e2k.ad.ge.com> I've just spent some time looking through the call heirarchy that underlies scipy.stats.anova() and there is considerable bitrot from years of neglect. Does anybody know when this was last known to work ? It looks like it would be easier to take an old working version and update it to the current python and scipy versions, than to fix the subtle interactions and side effects. Separately to that, can someone suggest a timeline on which the SVN tree will be up and able to build itself again? With the project repository broken, maybe I should fork (it'd be a branch, if we had a repository), fix up the stuff I need using local CVS, wait for the SVN to become stable, migrate my fork to become a branch, then merge the branch. Anybody disagree? From perry at stsci.edu Thu Aug 18 18:01:05 2005 From: perry at stsci.edu (Perry Greenfield) Date: Thu, 18 Aug 2005 18:01:05 -0400 Subject: [SciPy-dev] Restructuring of SciPy In-Reply-To: <43039E9A.5050905@ee.byu.edu> References: <43039E9A.5050905@ee.byu.edu> Message-ID: On Aug 17, 2005, at 4:31 PM, Travis Oliphant wrote: > > After the 0.3.2 release, scipy will be getting a face-lift. > > As part of that, I'm wondering about the restructuring process. For a > long time, we've been operating under the "big-scipy-package" with > capability for small packages to be released separately model. It has > had limited success and I don't know of any small packages actually > being released like that. > > With things like Enthon out there that act as the "batteries included > super distribution" anyway. It makes me wonder if we shouldn't move > scipy to the "scipy-modules" perspective directly and develop and > think about scipy has a collection of modules that work together > smoothly (i.e. naming conventions, etc.) > > scipy_core + available instructions could provide all the > infrastructure someone needs to contribute packages to scipy. This > would also allow people to develop (and maintain) their own modules > (which they seem to like to do anyway). If we could just get them to > play nice with the rest of scipy_core, it would all be for benefit. > > We already have the infrastructure in place for modules to play nicely > with the overall package. I'm just wondering if we shouldn't > emphasize more the modular approach by the way the svn tree is layed > out (so individual modules can be checked out independently, for > example). > > I've also spoken to people who find it hard to rely on their scipy in > their code they pass on because you have to get the whole package, > even if you only use a few parts. > Matlab also uses modules effectively. > I think we should take advantage of the restructuring process to think > about this. > > Comments? > > -Travis It's probably already obvious what I think but I don't see any harm in reiterating it. I think the biggest obstacle to scipy in its current form is that many users find it hard to install. Yes, some of you may have no problems, but I've run into many that have (and some of them are pretty experienced at doing installations). It is essential that whatever the core is, it be an easy install for the vast majority of users. I think Travis is on the right track here. My only variance is that I wouldn't object to including more popular modules in the core distribution *so long as they are easy installs on all popular platforms*. Anything that isn't shouldn't be in the core. But there are other good reasons for splitting them that Travis alludes to. Some module may have a faster pace of development and don't want to be synced to scipy.core releases. Not everything will be at the same level of maturity, testing, or documentation. The core should have higher standards than what optional models have in that respect. Also, nothing prevents larger binary bundles (assuming someone is willing to put the work into making them). But there is no denying that doing this introduces dependency issues that must be managed as well. Perry From oliphant at ee.byu.edu Sat Aug 20 14:32:44 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat, 20 Aug 2005 12:32:44 -0600 Subject: [SciPy-dev] Restructuring of SciPy In-Reply-To: References: <43039E9A.5050905@ee.byu.edu> Message-ID: <4307774C.4070405@ee.byu.edu> > I think the biggest obstacle to scipy in its current form is that many > users find it hard to install. Yes, some of you may have no problems, > but I've run into many that have (and some of them are pretty > experienced at doing installations). It is essential that whatever the > core is, it be an easy install for the vast majority of users. I think > Travis is on the right track here. My only variance is that I wouldn't > object to including more popular modules in the core distribution *so > long as they are easy installs on all popular platforms*. Anything > that isn't shouldn't be in the core. Yes, scipy_core will be easy to install (right now that translates to no Fortran code). That is a very important goal. I won't be happy if people do not find scipy_core to be a useful and better replacement for Numeric and numarray. -Travis From jh at oobleck.astro.cornell.edu Mon Aug 22 16:26:00 2005 From: jh at oobleck.astro.cornell.edu (Joe Harrington) Date: Mon, 22 Aug 2005 16:26:00 -0400 Subject: [SciPy-dev] Restructuring of SciPy In-Reply-To: <20050818155541.1A6C73EB6B@www.scipy.com> (scipy-dev-request@scipy.net) References: <20050818155541.1A6C73EB6B@www.scipy.com> Message-ID: <200508222026.j7MKQ0cD005736@oobleck.astro.cornell.edu> On this thread from last week, I thought we had already decided to move in the direction of modules and well-written intro docs that point you to a specific graphics package (while letting you know that others exist and pointing you to more info). The concept was laid out in the ASP doc and ensuing discussion, for which see the wiki. We also introduced the idea of umbrella packages, which are (nearly-)empty packages that depend on (and therefore cause your package manager to download and install) useful sets of modules. For example, we could have a scipy-astro-all that got scipy-core, scipy-docs, scipy-astro, scipy-matplotlib, and so on, but not scipy-bio-molecules or scipy-em-whatever. For that matter, we could (should) have a scipy-all, that gets everything. --jh-- From oliphant at ee.byu.edu Mon Aug 22 16:35:46 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 22 Aug 2005 14:35:46 -0600 Subject: [SciPy-dev] Restructuring of SciPy In-Reply-To: <200508222026.j7MKQ0cD005736@oobleck.astro.cornell.edu> References: <20050818155541.1A6C73EB6B@www.scipy.com> <200508222026.j7MKQ0cD005736@oobleck.astro.cornell.edu> Message-ID: <430A3722.1090806@ee.byu.edu> Joe Harrington wrote: >On this thread from last week, I thought we had already decided to >move in the direction of modules and well-written intro docs that >point you to a specific graphics package (while letting you know that >others exist and pointing you to more info). The concept was laid out >in the ASP doc and ensuing discussion, for which see the wiki. We >also introduced the idea of umbrella packages, which are >(nearly-)empty packages that depend on (and therefore cause your >package manager to download and install) useful sets of modules. For >example, we could have a scipy-astro-all that got scipy-core, >scipy-docs, scipy-astro, scipy-matplotlib, and so on, but not >scipy-bio-molecules or scipy-em-whatever. For that matter, we could >(should) have a scipy-all, that gets everything. > >--jh-- > > O.K., then from your perspective, my question has to do with should what is currently under the umbrella of the scipy source tree (i.e. what is in svn) be more fully restructured into scipy-optimize, scipy-integrate, scipy-linalg, etc. with this restructuring also reflected in the svn layout (so that one can check out just a portion of the source). Do certain choices in svn structure make package management more or less easy. I don't have the answer, I was just pointing out that we ought to think about it. I'm gathering that nobody really knows... -Travis From timcera at earthlink.net Fri Aug 26 01:20:19 2005 From: timcera at earthlink.net (Tim Cera) Date: Fri, 26 Aug 2005 01:20:19 -0400 Subject: [SciPy-dev] Utility to pad a matrix Message-ID: <200508260120.19808.timcera@earthlink.net> Hello, Please find at http://www.scipy.org/Members/timcera/pad-0.1.tar.gz/download a utility to pad an n-dimensional matrix. You can pad with the mean, median, maximum, or minimum of each vector along each axis or just use a portion of the vector near the edges. You can also assign a constant to the pad values, or ramp from the edge value to a constant, or mirror along the edge values, or pad with values wrapped from the other end of the vector. I had an earlier version that I was using for rank-1 matrices, but I have extended the padding functions to n-dimensions. Now that pad can work with n-dimensional matrics, I have hopes that pad (or something like it) could be a utility used by FFT, convolution, and filter functions. Some of these functions support some forms of padding internally, most do not. Kindest regards, Tim Cera From aisaac at american.edu Mon Aug 29 13:53:00 2005 From: aisaac at american.edu (Alan G Isaac) Date: Mon, 29 Aug 2005 13:53:00 -0400 Subject: [SciPy-dev] natural neighber interpolation Message-ID: Just noticed http://dilbert.engr.ucdavis.edu/~suku/nem/thesis.html#CODE which provides public domain Fortran code. Cheers, Alan Isaac From harkal at sylphis3d.com Wed Aug 31 03:55:24 2005 From: harkal at sylphis3d.com (harkal at sylphis3d.com) Date: Wed, 31 Aug 2005 09:55:24 +0200 (CEST) Subject: [SciPy-dev] Submitting Patches Message-ID: <4569.195.251.195.65.1125474924.squirrel@webmail1.b-one.nu> Hello, what is the appropriate procedure to submit patches? I sended to the list a patch for RQ decomposition a month ago but got no responce. Will is be commited? Will it not? Best regards, Harry From timcera at earthlink.net Wed Aug 31 20:12:02 2005 From: timcera at earthlink.net (Tim Cera) Date: Wed, 31 Aug 2005 20:12:02 -0400 Subject: [SciPy-dev] Suggest Trac instead of Plone Message-ID: <200508312012.03503.timcera@earthlink.net> Hello, As I am just starting to use sci-py in a serious way, the topic of how a casual developer could help out with patches or features has been a concern of mine. I put this comment on the Plone wiki, which has gone unanswered. I have made other suggestions on the wiki, hopeful for some discussion, but nothing. In looking around at wikis for our intranet (for various reasons I chose moinmoin) I saw an intriguing system seemingly ideal for software development. It is called Trac (http://www.edgewall.com/trac/). Since the recent conversion to Subversion, it might be an ideal solution. Trac is also written in Python, and has an intergrated wiki, issue management, and Subversion viewer. I would also encourage moving to a forum type communication rather than mailing lists. Kindest regards, Tim Cera From rkern at ucsd.edu Wed Aug 31 20:30:36 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 31 Aug 2005 17:30:36 -0700 Subject: [SciPy-dev] Suggest Trac instead of Plone In-Reply-To: <200508312012.03503.timcera@earthlink.net> References: <200508312012.03503.timcera@earthlink.net> Message-ID: <43164BAC.9080907@ucsd.edu> Tim Cera wrote: > Hello, > > As I am just starting to use sci-py in a serious way, the topic of how a > casual developer could help out with patches or features has been a concern > of mine. I put this comment on the Plone wiki, which has gone unanswered. > I have made other suggestions on the wiki, hopeful for some discussion, but > nothing. By and large, active discussion happens here while semi-permanent (but still fluid) information goes up on the Wiki. > In looking around at wikis for our intranet (for various reasons I chose > moinmoin) I saw an intriguing system seemingly ideal for software > development. It is called Trac (http://www.edgewall.com/trac/). Since the > recent conversion to Subversion, it might be an ideal solution. Enthought is using Trac for the rest of its projects. I'm sure that a Trac instance for Scipy is on its way. I think there are still some kinks in the way multiple projects are being hosted, but they're being worked out using ipython as the guinea pig. > Trac is also written in Python, and has an intergrated wiki, issue management, > and Subversion viewer. > > I would also encourage moving to a forum type communication rather than > mailing lists. I grew up with USENET and mailing lists, so personally, I hate web forums. Fortunately, with GMane the people who hate mailing lists can get a web interface if they so prefer. Does that satisfy your desire for "forum type communication?" http://dir.gmane.org/gmane.comp.python.scientific.devel http://dir.gmane.org/gmane.comp.python.scientific.user -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From Fernando.Perez at colorado.edu Wed Aug 31 20:42:01 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Wed, 31 Aug 2005 18:42:01 -0600 Subject: [SciPy-dev] Suggest Trac instead of Plone In-Reply-To: <43164BAC.9080907@ucsd.edu> References: <200508312012.03503.timcera@earthlink.net> <43164BAC.9080907@ucsd.edu> Message-ID: <43164E59.3090905@colorado.edu> Robert Kern wrote: > Tim Cera wrote: > >>Hello, >> >>As I am just starting to use sci-py in a serious way, the topic of how a >>casual developer could help out with patches or features has been a concern >>of mine. I put this comment on the Plone wiki, which has gone unanswered. >>I have made other suggestions on the wiki, hopeful for some discussion, but >>nothing. > > > By and large, active discussion happens here while semi-permanent (but > still fluid) information goes up on the Wiki. From recently using Trac extensively for all new ipython development (http://projects.scipy.org/ipython/ipython for the curious), I can offer at least my take on this issue, which aligns with Robert's comment. I've found that back-and-forth productive discussion is much better handled on mailing lists. Part of the problem is that a discussion in a wiki requires that you remember to go and check that wiki, and I already have way too many things to track as it is. What wikis are _great_ for, IMO, is tracking ideas which have 'gelled' a bit. This requires that somebody does the extra work of summarizing a discussion and putting it up on the wiki, but it's excellent as a reference point once a certain consensus has been reached on a given topic. The wiki still allows ideas to evolve, but it really doesn't lend itself well to the kind of active argument that is well supported by a mailing list. Another area where wikis shine is for keeping community-based information centers: 'cookbooks' like mayavi's and matplotlib's , the scipy TopicalSoftware wiki, etc, are good examples of this. Anyone can contribute new information, and as time goes by the wiki improves and becomes a useful resource which is easier to find than a maling list archive. I guess it's just a case of 'right tool for the job'. Wikis are great and I _love_ Trac, but they are not the end-all be-all of communications. For some things, a plain mailing list (or its usenet equivalent thanks to gmane, which can be used via a browser or a news client) just works better. At least that's been my experience. Cheers, f From novak at ucolick.org Wed Aug 31 21:01:16 2005 From: novak at ucolick.org (Greg Novak) Date: Wed, 31 Aug 2005 18:01:16 -0700 (PDT) Subject: [SciPy-dev] Re: Scipy-dev Digest, Vol 22, Issue 8 In-Reply-To: <20050831234349.A6DB93EB97@www.scipy.com> References: <20050831234349.A6DB93EB97@www.scipy.com> Message-ID: >> With things like Enthon out there that act as the "batteries included >> super distribution" anyway. It makes me wonder if we shouldn't move >> scipy to the "scipy-modules" perspective directly and develop and >> think about scipy has a collection of modules that work together >> smoothly (i.e. naming conventions, etc.) Everyone agrees that Scipy should be easy to install, so I'm sure that whatever happens, installation will at least continue to get easier. However, I would like to point out that (at least as far as I can see) Enthon is Windows only, so (IMO) it shouldn't be advanced as a possible solution for users who want all the extras. At least not until (if) it's available on more platforms. I recently suffered a hard disk crash, and I can say that getting Scipy reinstalled was quite nerve racking for me, in spite of long experience with software installations in general. My problems surrounded undefined symbols in Fortran code. Granted, the situation is made worse by the simultaneous switch to GCC 4.0 that most major distributions seem to be making at the same time. In any case, the problems were severe enough that I switched from Fedora to Debian because there were pre-packaged Scipy binaries, so I could finally start using the code that I'd written once more. Cheers, Greg