From faltet at carabos.com Sun Oct 1 07:49:31 2006 From: faltet at carabos.com (Francesc Altet) Date: Sun, 01 Oct 2006 13:49:31 +0200 Subject: [Numpy-discussion] Non-writeable default for numpy.ndarray In-Reply-To: <451D9DBF.1030303@ee.byu.edu> References: <200609291723.26896.faltet@carabos.com> <451D45D9.6050005@ieee.org> <200609291844.07928.faltet@carabos.com> <451D9DBF.1030303@ee.byu.edu> Message-ID: <1159703372.3950.29.camel@localhost.localdomain> El dv 29 de 09 del 2006 a les 16:27 -0600, en/na Travis Oliphant va escriure: > Francesc Altet wrote: > > >I see. Thanks for the explanation. > > > > > You deserve the thanks for the great testing of less-traveled corners of > NumPy. It's exactly the kind of thing needed to get NumPy ready for > release. You are welcome, but actually the merit is more of the pytables test suite than mine :-) Travis, I've expressed publicly many times my gratitude to you, but I'm going to do so once more(so, be ready :-): I was absolutely thrilled that each and everyone of my bug reports were either solved or declared features in a matter of 1 or 2 days as maximum, and this is just *amazing* in a complex software like NumPy. I don't need to say it again (but will do): You have made of NumPy *the* reference of python array packages, and better that this, you have made this only in one year and a half! Definitely, I have come to the conclusion that you are not human, but rather a MONSTER! :-) Also thanks goes to Charles Harris, for implementing very fine-tuned sorting algorithms in both numpy and numarray packages. His work is very useful for us. Finally, and although not directly related with numpy, I'd like to express my gratitude and acknowledgement to David Cooke (and Tim Hochberg) for it's work on numexpr: it's a perfect complement to numpy in that it allows to accelerate even more many of the basic operations with arrays. All of you, and the rest of the numpy community are a first-class team, and this is why it makes me so happy to help you! -- >0,0< Francesc Altet http://www.carabos.com/ V V C?rabos Coop. V. Enjoy Data "-" From gael.varoquaux at normalesup.org Sun Oct 1 15:42:43 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 1 Oct 2006 21:42:43 +0200 Subject: [Numpy-discussion] Adding an assert to ctypeslib.py Message-ID: <20061001194242.GC17427@clipper.ens.fr> Hi, I just lost a bit of time because I had an old version of ctypes (the one that comes with ubuntu dapper). It doesn't work with ctypeslib.py. I think we should add a warning just after the import. Cheers, Ga?l -------------- next part -------------- 22d21 < assert ctypes.__version__ >= '0.9.9.6' From kferrio at cox.net Sun Oct 1 22:32:15 2006 From: kferrio at cox.net (Kyle Ferrio) Date: Sun, 01 Oct 2006 19:32:15 -0700 Subject: [Numpy-discussion] Problem building from svn under cygwin Message-ID: <45207A2F.6020409@cox.net> You're correct. I built python with cygwin. Thanks for the clue. Robert Kern wrote: Coatimundi wrote: > I have the latest cygwin environment and am using gcc 3.4.4. to build numpy. > > Configure works: > > python setup.py configure --compiler=mingw32 > > Build fails: > > python setup.py build --compiler=mingw32 What Python are you using? It looks like you built Python 2.5 using cygwin (that is, the UNIX emulation layer). You cannot build your extensions with mingw32 in that case. Don't use --compiler at all and you will be fine. If you are using the regular Python binaries, then you should be using --compiler=mingw32, but you're getting the wrong Python headers in that case. -- Robert Kern From olivetti at itc.it Mon Oct 2 04:44:51 2006 From: olivetti at itc.it (Emanuele Olivetti) Date: Mon, 02 Oct 2006 10:44:51 +0200 Subject: [Numpy-discussion] bincount on multidimensional array (missing 'axis' parameter) Message-ID: <4520D183.4030003@itc.it> Hi, I have to perform bincount on each column of a matrix so I was wondering why there is no 'axis=?' parameter in bincount function. Currently I'm looping through columns using a python loop but I'd appreciate to have the loop embedded in the bincount function (and possibily using a C loop, since I'm doing this stuff thousands times in a case of many columns). Is there a good alternative solution to extending bincount? Suggestions? Thanks in advance, Emanuele From perhamyhavv at intel.net Mon Oct 2 13:21:46 2006 From: perhamyhavv at intel.net (Jeno Kingsley) Date: Mon, 2 Oct 2006 18:21:46 +0100 Subject: [Numpy-discussion] Hi Message-ID: <01c6e63e$dadd39d0$7801a8c0@zuccoli> Hi, ClAxLIS VlAxGRA AMBxlEN VALxlUM Economize over 50% http://www.defunjeriondsafun.com _____ farms. The only really puzzling thing about the 3D map was what looked enthusiastically. My medication was beginning to override the pain and that I could see-except in the close vicinity of the device. I moved -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mailer-Daemon at lists.sourceforge.net Mon Oct 2 12:52:49 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Mon, 02 Oct 2006 09:52:49 -0700 Subject: [Numpy-discussion] Warning: message 1GUFPK-0001wW-5k delayed 10 hours Message-ID: This message was created automatically by mail delivery software. A message that you sent has not yet been delivered to one or more of its recipients after more than 10 hours on the queue on externalmx-1.sourceforge.net. The message identifier is: 1GUFPK-0001wW-5k The date of the message is: Mon, 02 Oct 2006 09:32:13 -0200 The subject of the message is: doctor, at the revolt is located treating the planet he gold; The address to which the message has not yet been delivered is: numpy-discussion at lists.sourceforge.net Delay reason: SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is No action is required on your part. Delivery attempts will continue for some time, and this warning may be repeated at intervals if the message remains undelivered. Eventually the mail delivery software will give up, and when that happens, the message will be returned to you. From rowen at cesmail.net Mon Oct 2 14:08:11 2006 From: rowen at cesmail.net (Russell E. Owen) Date: Mon, 02 Oct 2006 11:08:11 -0700 Subject: [Numpy-discussion] numarray and/or Numeric with python 2.5? Message-ID: Has anyone tried to build numarray or Numeric with python 2.5? Since the type of array index was changed, I'm wondering if it works (and if so, if there are any gotchas to worry about). (And yes, I do plan to switch to numpy, but meanwhile I'm still using numarray and Numeric.) Regards, -- Russell From jmiller at stsci.edu Mon Oct 2 15:02:32 2006 From: jmiller at stsci.edu (Todd Miller) Date: Mon, 02 Oct 2006 15:02:32 -0400 Subject: [Numpy-discussion] numarray and/or Numeric with python 2.5? In-Reply-To: References: Message-ID: <45216248.60905@stsci.edu> Russell E. Owen wrote: > Has anyone tried to build numarray or Numeric with python 2.5? Since the > numarray builds and runs fine for Python-2.5, but only as 32-bit. > type of array index was changed, I'm wondering if it works (and if so, > NOTE: numarray does not exploit the new ssize_t index in Python-2.5 so numarray is not 64-bit enabled. Regards, Todd From lars.bittrich at googlemail.com Mon Oct 2 03:26:56 2006 From: lars.bittrich at googlemail.com (Lars Bittrich) Date: Mon, 2 Oct 2006 09:26:56 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) Message-ID: <200610020926.56965.lars.bittrich@googlemail.com> Hi all, recently I tried to compile numpy/scipy with Python 2.5 and Intel C++ compiler 9.1. At first everything was fine, but the scipy test produced a few errors. The reason was a little difference: numpy svn(1.0.dev3233) with Intel C compiler (Python 2.5): ----------------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:x[i] Out[4]:1.0 numpy svn(1.0.dev3233) with GCC 3.3 (Python 2.3): -------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:print x[i] Out[4]:array([1]) The Intel version gives me a scalar whereas the gcc version an array. Maybe Python 2.5 is causing this problem but my first guess was the compiler. A typical error message from scipy.test(10): ====================================================================== ERROR: check_chebyc (scipy.special.tests.test_basic.test_chebyc) ---------------------------------------------------------------------- Traceback (most recent call last): File "[...]/lib/python2.5/site-packages/scipy/special/tests/test_basic.py", line 667, in check_chebyc C1 = chebyc(1) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 461, in chebyc p = orthopoly1d(x,w,hn,kn,wfunc=lambda x: 1.0/sqrt(1-x*x/4.0),limits=(-2,2),monic=monic) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 78, in __init__ equiv_weights = [weights[k] / wfunc(roots[k]) for k in range(len(roots))] TypeError: object of type 'numpy.complex128' has no len() Does anyone know how to resolve the problem? Best regards, Lars From oliphant at ee.byu.edu Mon Oct 2 17:53:08 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 02 Oct 2006 15:53:08 -0600 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) In-Reply-To: <200610020926.56965.lars.bittrich@googlemail.com> References: <200610020926.56965.lars.bittrich@googlemail.com> Message-ID: <45218A44.3070101@ee.byu.edu> Lars Bittrich wrote: >Hi all, > >recently I tried to compile numpy/scipy with Python 2.5 and Intel C++ compiler >9.1. At first everything was fine, but the scipy test produced a few errors. >The reason was a little difference: > >numpy svn(1.0.dev3233) with Intel C compiler (Python 2.5): >----------------------------------------------------------- >In [1]:from numpy import ones, zeros, integer > >In [2]: > >In [2]:x = ones(1) > >In [3]:i = zeros(1, integer) > >In [4]:x[i] >Out[4]:1.0 > >numpy svn(1.0.dev3233) with GCC 3.3 (Python 2.3): >-------------------------------------------------- >In [1]:from numpy import ones, zeros, integer > >In [2]: > >In [2]:x = ones(1) > >In [3]:i = zeros(1, integer) > >In [4]:print x[i] >Out[4]:array([1]) > >The Intel version gives me a scalar whereas the gcc version an array. Maybe >Python 2.5 is causing this problem but my first guess was the compiler. > > > This is a Python 2.5 issue (the new __index__ method) was incorrectly implemented and allowing a 1-d array to be interpreted as an index. This should be fixed in SVN. -Travis From lars.bittrich at googlemail.com Sun Oct 1 08:35:06 2006 From: lars.bittrich at googlemail.com (Lars Bittrich) Date: Sun, 1 Oct 2006 14:35:06 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) Message-ID: <200610011435.06426.lars.bittrich@googlemail.com> Hi all, recently I tried to compile numpy/scipy with Python 2.5 and Intel C++ compiler 9.1. At first everything was fine, but the scipy test produced a few errors. The reason was a little difference: numpy svn(1.0.dev3233) with Intel C compiler (Python 2.5): ----------------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:x[i] Out[4]:1.0 numpy svn(1.0.dev3233) with GCC 3.3 (Python 2.3): -------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:print x[i] Out[4]:array([1]) The Intel version gives me a scalar whereas the gcc version an array. Maybe Python 2.5 is causing this problem but my first guess was the compiler. A typical error message from scipy.test(10): ====================================================================== ERROR: check_chebyc (scipy.special.tests.test_basic.test_chebyc) ---------------------------------------------------------------------- Traceback (most recent call last): File "[...]/lib/python2.5/site-packages/scipy/special/tests/test_basic.py", line 667, in check_chebyc C1 = chebyc(1) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 461, in chebyc p = orthopoly1d(x,w,hn,kn,wfunc=lambda x: 1.0/sqrt(1-x*x/4.0),limits=(-2,2),monic=monic) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 78, in __init__ equiv_weights = [weights[k] / wfunc(roots[k]) for k in range(len(roots))] TypeError: object of type 'numpy.complex128' has no len() Does anyone know how to resolve the problem? Best regards, Lars From lars.bittrich at googlemail.com Sun Oct 1 08:10:34 2006 From: lars.bittrich at googlemail.com (Lars Bittrich) Date: Sun, 1 Oct 2006 14:10:34 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) Message-ID: <200610011410.34552.lars.bittrich@googlemail.com> Hi all, recently I tried to compile numpy/scipy with Python 2.5 and Intel C++ compiler 9.1. At first everything was fine, but the scipy test produced a few errors. The reason was a little difference: numpy svn(1.0.dev3233) with Intel C compiler (Python 2.5): ----------------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:x[i] Out[4]:1.0 numpy svn(1.0.dev3233) with GCC 3.3 (Python 2.3): -------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:print x[i] Out[4]:array([1]) The Intel version gives me a scalar whereas the gcc version an array. Maybe Python 2.5 is causing this problem but my first guess was the compiler. A typical error message from scipy.test(10): ====================================================================== ERROR: check_chebyc (scipy.special.tests.test_basic.test_chebyc) ---------------------------------------------------------------------- Traceback (most recent call last): File "[...]/lib/python2.5/site-packages/scipy/special/tests/test_basic.py", line 667, in check_chebyc C1 = chebyc(1) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 461, in chebyc p = orthopoly1d(x,w,hn,kn,wfunc=lambda x: 1.0/sqrt(1-x*x/4.0),limits=(-2,2),monic=monic) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 78, in __init__ equiv_weights = [weights[k] / wfunc(roots[k]) for k in range(len(roots))] TypeError: object of type 'numpy.complex128' has no len() Does anyone know how to resolve the problem? Best regards, Lars From 13640887 at sun.ac.za Mon Oct 2 18:10:33 2006 From: 13640887 at sun.ac.za (Albert Strasheim) Date: Tue, 3 Oct 2006 00:10:33 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that Message-ID: Hello all I recently started looking at David Cournapeau's PyEM package, specifically his implementation of the K-Means algorithm. He implemented part of this algorithm with in pure Python version and also provided a Pyrex alternative that is significantly faster (about 10 times with the data I tested with). I tried to figure out why this is so. The job of this part of the algorithm is pretty simple: from a bunch of cluster means (the codebook) find the nearest cluster mean for each data point. The first attempt at implementing this algorithm might use two for loops, one over the data points and one over the cluster means, computing a Euclidean distance at each step. Slightly better is to use one loop and some broadcasting. By randomly trying some code, I arrived at the following one-liner: N.sum((data[...,N.newaxis,...]-code)**2, 2).argmin(axis=1) where data = N.random.randn(2000, 39) nclusters = 64 code = data[0:nclusters,:] This code was still slow, so I mangled it a bit so that I could profile it using cProfile under Python 2.5 (use profile if you have an older version of Python): def kmean(): data = N.random.randn(2000, 39) nclusters = 64 code = data[0:nclusters,:] for i in xrange(10): def a(): return data[...,N.newaxis,...] z = a() def b(): return z-code z = b() def c(): return z**2 z = c() def d(): return N.sum(z, 2) z = d() def e(): return z.argmin(axis=1) z = e() if __name__ == '__main__': import cProfile cProfile.run("kmean()") Profiler output: In [570]: %run -t -e kmean2.py 84 function calls in 8.567 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 8.567 8.567 :1() 10 0.000 0.000 0.835 0.084 fromnumeric.py:375(sum) 10 0.000 0.000 0.000 0.000 kmean2.py:10(a) 10 6.837 0.684 6.837 0.684 kmean2.py:12(b) 10 0.623 0.062 0.623 0.062 kmean2.py:14(c) 10 0.000 0.000 0.835 0.084 kmean2.py:16(d) 10 0.080 0.008 0.080 0.008 kmean2.py:18(e) 1 0.180 0.180 8.567 8.567 kmean2.py:5(kmean2) 10 0.000 0.000 0.000 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 1 0.012 0.012 0.012 0.012 {method 'randn' of 'mtrand.RandomState' objects} 10 0.835 0.083 0.835 0.083 {method 'sum' of 'numpy.ndarray' objects} It seems the b part of the computation is taking most of the time. I tried to simulate this separately: In [571]: x1 = N.random.randn(2000,39) In [572]: y1 = N.random.randn(64,39) In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms per loop In [575]: z1.shape Out[575]: (2000, 64, 39) As far as I can figure, this operation is doing 2000*64*39 subtractions. Doing this straight up yields the following: In [576]: x2 = N.random.randn(2000,64,39) In [577]: y2 = N.random.randn(2000,64,39) In [578]: %timeit z2 = x2-y2 10 loops, best of 3: 108 ms per loop Does anybody have any ideas on why this is so much faster? Hopefully I didn't mess up somewhere... Thanks! Regards, Albert P.S. I'm used NumPy 1.0rc1 with Python 2.5 on Windows for these tests. I get similar results with 1.0.dev3239 with Python 2.4.3 on Linux. P.P.S. SourceForge and GMail *really* don't like each other right now. From oliphant at ee.byu.edu Mon Oct 2 19:03:50 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 02 Oct 2006 17:03:50 -0600 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: References: Message-ID: <45219AD6.5020203@ee.byu.edu> Albert Strasheim wrote: >In [571]: x1 = N.random.randn(2000,39) > >In [572]: y1 = N.random.randn(64,39) > >In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms >per loop > >In [575]: z1.shape >Out[575]: (2000, 64, 39) > >As far as I can figure, this operation is doing 2000*64*39 subtractions. >Doing this straight up yields the following: > >In [576]: x2 = N.random.randn(2000,64,39) > >In [577]: y2 = N.random.randn(2000,64,39) > >In [578]: %timeit z2 = x2-y2 >10 loops, best of 3: 108 ms per loop > >Does anybody have any ideas on why this is so much faster? Hopefully I >didn't mess up somewhere... > > I suspect I know why, although the difference seems rather large. There is code optimization that is being taken advantage of in the second case. If you have contiguous arrays (no broadcasting needed), then 1 C-loop is used for the subtraction (your second case). In the first case you are using broadcasting to generate the larger array. This requires more complicated looping constructs under the covers which causes your overhead. Bascially, you will have 64*39 1-d loops of 2000 elements each in the first example with a bit of calculation over-head to reset the pointers before each loop. In the ufunc code, compare the ONE_UFUNCLOOP case with the NOBUFFER_UFUNCLOOP case. If you want to be sure what is running un-comment the fprintf statements so you can tell. I'm surprised the overhead of adjusting pointers is so high, but then again you are probably getting a lot of cache misses in the first case so there is more to it than that, the loops may run more slowly too. -Travis From tim.hochberg at ieee.org Mon Oct 2 19:49:52 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 02 Oct 2006 16:49:52 -0700 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45219AD6.5020203@ee.byu.edu> References: <45219AD6.5020203@ee.byu.edu> Message-ID: <4521A5A0.8040902@ieee.org> Travis Oliphant wrote: > Albert Strasheim wrote: > > >> In [571]: x1 = N.random.randn(2000,39) >> >> In [572]: y1 = N.random.randn(64,39) >> >> In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms >> per loop >> >> In [575]: z1.shape >> Out[575]: (2000, 64, 39) >> >> As far as I can figure, this operation is doing 2000*64*39 subtractions. >> Doing this straight up yields the following: >> >> In [576]: x2 = N.random.randn(2000,64,39) >> >> In [577]: y2 = N.random.randn(2000,64,39) >> >> In [578]: %timeit z2 = x2-y2 >> 10 loops, best of 3: 108 ms per loop >> >> Does anybody have any ideas on why this is so much faster? Hopefully I >> didn't mess up somewhere... >> >> >> > > I suspect I know why, although the difference seems rather large. There > is code optimization that is being taken advantage of in the second > case. If you have contiguous arrays (no broadcasting needed), then 1 > C-loop is used for the subtraction (your second case). > > In the first case you are using broadcasting to generate the larger > array. This requires more complicated looping constructs under the > covers which causes your overhead. Bascially, you will have 64*39 1-d > loops of 2000 elements each in the first example with a bit of > calculation over-head to reset the pointers before each loop. > > In the ufunc code, compare the ONE_UFUNCLOOP case with the > NOBUFFER_UFUNCLOOP case. If you want to be sure what is running > un-comment the fprintf statements so you can tell. > > I'm surprised the overhead of adjusting pointers is so high, but then > again you are probably getting a lot of cache misses in the first case > so there is more to it than that, the loops may run more slowly too. > > I suspect that Travis is basically right about why your simple subtraction runs much faster than your test case. However, that doesn't mean you can't do better than at present. When dealing with large, multidimensional arrays my experience has been that unrolling all of the for loops is frequently counterproductive. I chalk this up to two factors: first that you tend to end up generating large temporary arrays and this in turn leads to cache misses. Second, you loose flexibility in how you perform the calculation, which in turns limits other possible explanations. I just spent a while playing with this, and assuming I've correctly translated your original intent I've come up with two alternative, looping versions that run, respectively 2 and 3 times faster. I've a feeling that kmean3, the fastest one, still has a little more room to be sped up, but I'm out of time now. Code is below -tim import numpy as N data = N.random.randn(2000, 39) def kmean0(data): nclusters = 64 code = data[0:nclusters,:] return N.sum((data[...,N.newaxis,...]-code)**2, 2).argmin(axis=1) def kmean1(data): nclusters = 64 code = data[0:nclusters,:] z = data[:,N.newaxis,:] z = z-code z = z**2 z = N.sum(z, 2) return z.argmin(axis=1) def kmean2(data): nclusters = 64 naxes = data.shape[-1] code = data[0:nclusters,:] data = data[:, N.newaxis] allz = N.zeros([len(data)]) for i, x in enumerate(data): z = (x - code) z **= 2 allz[i] = z.sum(-1).argmin(0) return allz def kmean3(data): nclusters = 64 naxes = data.shape[-1] code = data[0:nclusters] totals = N.zeros([nclusters, len(data)], float) transdata = data.transpose().copy() for cluster, tot in zip(code, totals): for di, ci in zip(transdata, cluster): delta = di - ci delta **=2 tot += delta return totals.argmin(axis=0) if __name__ == '__main__': assert N.alltrue(kmean0(data) == kmean1(data)) assert N.alltrue(kmean0(data) == kmean2(data)) assert N.alltrue(kmean0(data) == kmean3(data)) from timeit import Timer print Timer('kmean0(data)', 'from scratch import kmean0, data').timeit(3) print Timer('kmean1(data)', 'from scratch import kmean1, data').timeit(3) print Timer('kmean2(data)', 'from scratch import kmean2, data').timeit(3) print Timer('kmean3(data)', 'from scratch import kmean3, data').timeit(3) From tim.hochberg at ieee.org Mon Oct 2 20:17:46 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 02 Oct 2006 17:17:46 -0700 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521A5A0.8040902@ieee.org> References: <45219AD6.5020203@ee.byu.edu> <4521A5A0.8040902@ieee.org> Message-ID: <4521AC2A.5050101@ieee.org> Tim Hochberg wrote: > Travis Oliphant wrote: > >> Albert Strasheim wrote: >> >> >> >>> In [571]: x1 = N.random.randn(2000,39) >>> >>> In [572]: y1 = N.random.randn(64,39) >>> >>> In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms >>> per loop >>> >>> In [575]: z1.shape >>> Out[575]: (2000, 64, 39) >>> >>> As far as I can figure, this operation is doing 2000*64*39 subtractions. >>> Doing this straight up yields the following: >>> >>> In [576]: x2 = N.random.randn(2000,64,39) >>> >>> In [577]: y2 = N.random.randn(2000,64,39) >>> >>> In [578]: %timeit z2 = x2-y2 >>> 10 loops, best of 3: 108 ms per loop >>> >>> Does anybody have any ideas on why this is so much faster? Hopefully I >>> didn't mess up somewhere... >>> >>> >>> [SNIP] > > > I just spent a while playing with this, and assuming I've correctly > translated your original intent I've come up with two alternative, > looping versions that run, respectively 2 and 3 times faster. I've a > feeling that kmean3, the fastest one, still has a little more room to be > sped up, but I'm out of time now. Code is below > > -tim > > > One more iterations, this time a little algorithmic improvement, and were up to 4x as fast as the original code. Here we take advantage of the fact that the term di**2 is constant across the axis we are minimizing on, computation of ci**2 can be hoisted out of the loop and the -2*di*ci term can be rejiggered to di*ci by appropriate rescaling. This reduces the computation in the inner loop from a subtract and a multiply to just a multiply. def kmean4(data): nclusters = 64 naxes = data.shape[-1] code = data[:nclusters] transdata = data.transpose().copy() totals = N.empty([nclusters, len(data)], float) code2 = (code**2).sum(-1) code2 *= -0.5 totals[:] = code2[:, N.newaxis] for cluster, tot in zip(code, totals): for di, ci in zip(transdata, cluster): tot += di*ci return totals.argmax(axis=0) [SNIP CODE] From oliphant at ee.byu.edu Mon Oct 2 20:32:26 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 02 Oct 2006 18:32:26 -0600 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45219AD6.5020203@ee.byu.edu> References: <45219AD6.5020203@ee.byu.edu> Message-ID: <4521AF9A.50701@ee.byu.edu> Travis Oliphant wrote: > >I suspect I know why, although the difference seems rather large. > [snip] >I'm surprised the overhead of adjusting pointers is so high, but then >again you are probably getting a lot of cache misses in the first case >so there is more to it than that, the loops may run more slowly too. > > I'm personally bothered that this example runs so much more slowly. I don't think it should. Perhaps it is unavoidable because of the memory-layout issues. It is just hard to believe that the overhead for calling into the loop and adjusting the pointers is so much higher. But, that isn't the problem, here. Notice the following: x3 = N.random.rand(39,2000) x4 = N.random.rand(39,64,1) %timeit z3 = x3[:,None,:] - x4 10 loops, best of 3: 76.4 ms per loop Hmm... It looks like cache misses are a lot more important than making sure the inner loop is taken over the largest number of variables (that's the current way ufuncs decide which axis ought to be used as the 1-d loop). Perhaps those inner 1-d loops could be optimized (using prefetch or something) to reduce the number of cache misses on the inner computation, and the concept of looping over the largest dimension (instead of the last dimension) should be re-considered. Ideas, -Travis From 13640887 at sun.ac.za Mon Oct 2 20:42:24 2006 From: 13640887 at sun.ac.za (Albert Strasheim) Date: Tue, 3 Oct 2006 02:42:24 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521AF9A.50701@ee.byu.edu> Message-ID: Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant > Sent: 03 October 2006 02:32 > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] Vectorizing code, for loops, and all that > > Travis Oliphant wrote: > > > > >I suspect I know why, although the difference seems rather large. > > > [snip] > > >I'm surprised the overhead of adjusting pointers is so high, but then > >again you are probably getting a lot of cache misses in the first case > >so there is more to it than that, the loops may run more slowly too. > > > > > > I'm personally bothered that this example runs so much more slowly. I > don't think it should. Perhaps it is unavoidable because of the > memory-layout issues. It is just hard to believe that the overhead for > calling into the loop and adjusting the pointers is so much higher. Firstly, thanks to Tim... I'll try his functions tomorrow. Meanwhile, I can confirm that the NOBUFFER_UFUNCLOOP case in PyUFunc_GenericFunction is getting exercised in the slower case. Here's some info on what's happening, courtesy of Rational Quantify: case NOBUFFER_UFUNCLOOP: while (loop->index < loop->size) { for (i=0; inargs; i++) loop->bufptr[i] = loop->iters[i]->dataptr; [1] loop->function((char **)loop->bufptr, &(loop->bufcnt), loop->steps, loop->funcdata); [2] UFUNC_CHECK_ERROR(loop); for (i=0; inargs; i++) { PyArray_ITER_NEXT(loop->iters[i]); [3] } loop->index++; } break; [1] 12.97% of function time [2] 8.65% of functiont ime [3] 62.14% of function time If statistics from elsewhere in the code would be helpful, let me know, and I'll see if I can convince Quantify to cough it up. > Thanks for all code and suggestions. Cheers, Albert From oliphant at ee.byu.edu Mon Oct 2 21:50:35 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 02 Oct 2006 19:50:35 -0600 Subject: [Numpy-discussion] ***[Possible UCE]*** Re: Vectorizing code, for loops, and all that In-Reply-To: References: Message-ID: <4521C1EB.30205@ee.byu.edu> Albert Strasheim wrote: >Hello all > > > >>-----Original Message----- >>From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- >>discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant >>Sent: 03 October 2006 02:32 >>To: Discussion of Numerical Python >>Subject: Re: [Numpy-discussion] Vectorizing code, for loops, and all that >> >>Travis Oliphant wrote: >> >> >> >>>I suspect I know why, although the difference seems rather large. >>> >>> >>> >>[snip] >> >> >> >>>I'm surprised the overhead of adjusting pointers is so high, but then >>>again you are probably getting a lot of cache misses in the first case >>>so there is more to it than that, the loops may run more slowly too. >>> >>> >>> >>> >>I'm personally bothered that this example runs so much more slowly. I >>don't think it should. Perhaps it is unavoidable because of the >>memory-layout issues. It is just hard to believe that the overhead for >>calling into the loop and adjusting the pointers is so much higher. >> >> > >Firstly, thanks to Tim... I'll try his functions tomorrow. > >Meanwhile, I can confirm that the NOBUFFER_UFUNCLOOP case in >PyUFunc_GenericFunction is getting exercised in the slower case. Here's some >info on what's happening, courtesy of Rational Quantify: > >case NOBUFFER_UFUNCLOOP: >while (loop->index < loop->size) { > for (i=0; inargs; i++) > loop->bufptr[i] = loop->iters[i]->dataptr; [1] > > loop->function((char **)loop->bufptr, &(loop->bufcnt), > loop->steps, loop->funcdata); [2] > UFUNC_CHECK_ERROR(loop); > > for (i=0; inargs; i++) { > PyArray_ITER_NEXT(loop->iters[i]); [3] > } > loop->index++; >} >break; > >[1] 12.97% of function time >[2] 8.65% of functiont ime >[3] 62.14% of function time > >If statistics from elsewhere in the code would be helpful, let me know, and >I'll see if I can convince Quantify to cough it up. > > > Please run the same test but using x1 = N.random.rand(39,2000) x2 = N.random.rand(39,64,1) z1 = x1[:,N.newaxis,:] - x2 Thanks, -Travis From Mailer-Daemon at lists.sourceforge.net Tue Oct 3 02:06:57 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Mon, 02 Oct 2006 23:06:57 -0700 Subject: [Numpy-discussion] Warning: message 1GUFPK-0001wW-5k delayed 24 hours Message-ID: This message was created automatically by mail delivery software. A message that you sent has not yet been delivered to one or more of its recipients after more than 24 hours on the queue on externalmx-1.sourceforge.net. The message identifier is: 1GUFPK-0001wW-5k The date of the message is: Mon, 02 Oct 2006 09:32:13 -0200 The subject of the message is: doctor, at the revolt is located treating the planet he gold; The address to which the message has not yet been delivered is: numpy-discussion at lists.sourceforge.net Delay reason: SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is No action is required on your part. Delivery attempts will continue for some time, and this warning may be repeated at intervals if the message remains undelivered. Eventually the mail delivery software will give up, and when that happens, the message will be returned to you. From david at ar.media.kyoto-u.ac.jp Tue Oct 3 03:33:01 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 03 Oct 2006 16:33:01 +0900 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: References: Message-ID: <4522122D.9070407@ar.media.kyoto-u.ac.jp> Albert Strasheim wrote: > Hello all > > I recently started looking at David Cournapeau's PyEM package, specifically > his implementation of the K-Means algorithm. He implemented part of this > algorithm with in pure Python version and also provided a Pyrex alternative > that is significantly faster (about 10 times with the data I tested with). I > tried to figure out why this is so. > > For what it worths, PyEM has been quietly updated in the past weeks: most responses I got initially were people reporting errors almost always related to change in numpy API changes, so I decided to keep quiet for a while, waiting for an API freeze on numpy side (last versions use vq from scipy.cluster, for example, the plotting have been much improved, a fast Ctype version for diagonal gaussian kernel enables to run EM on hundred of thousand frames of several dozens dimensions with several dozens of gaussian in the mixture in reasonable times) > The job of this part of the algorithm is pretty simple: from a bunch of > cluster means (the codebook) find the nearest cluster mean for each data > point. The first attempt at implementing this algorithm might use two for > loops, one over the data points and one over the cluster means, computing a > Euclidean distance at each step. Slightly better is to use one loop and some > broadcasting. > > By randomly trying some code, I arrived at the following one-liner: > > N.sum((data[...,N.newaxis,...]-code)**2, 2).argmin(axis=1) > > where > > data = N.random.randn(2000, 39) > nclusters = 64 > code = data[0:nclusters,:] > > This code was still slow, so I mangled it a bit so that I could profile it > using cProfile under Python 2.5 (use profile if you have an older version of > Python): > > def kmean(): > data = N.random.randn(2000, 39) > nclusters = 64 > code = data[0:nclusters,:] > for i in xrange(10): > def a(): return data[...,N.newaxis,...] > z = a() > def b(): return z-code > z = b() > def c(): return z**2 > z = c() > def d(): return N.sum(z, 2) > z = d() > def e(): return z.argmin(axis=1) > z = e() > > if __name__ == '__main__': > import cProfile > cProfile.run("kmean()") > > Profiler output: > I got exactly the same kind of weird behaviour in other places of PyEM (apparently simple substractions taking a lot of time compared to other parts although it should theoratically be negligeable: eg X - mu where X is (n, d) and mu (d, 1) takeing almost as much time as exp( X**2) !). It would be great to at least know the origin of this non-intuitive result, David From 13640887 at sun.ac.za Tue Oct 3 05:12:36 2006 From: 13640887 at sun.ac.za (Albert Strasheim) Date: Tue, 3 Oct 2006 11:12:36 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521C1EB.30205@ee.byu.edu> Message-ID: Hello all > -----Original Message----- > From: Travis Oliphant [mailto:oliphant at ee.byu.edu] > Sent: 03 October 2006 03:51 > To: Discussion of Numerical Python; Albert Strasheim > Subject: Re: ***[Possible UCE]*** Re: [Numpy-discussion] Vectorizing > code, for loops, and all that > > > >Meanwhile, I can confirm that the NOBUFFER_UFUNCLOOP case in > >PyUFunc_GenericFunction is getting exercised in the slower case. > >Here's > some > >info on what's happening, courtesy of Rational Quantify: > > > >case NOBUFFER_UFUNCLOOP: > >while (loop->index < loop->size) { > > for (i=0; inargs; i++) > > loop->bufptr[i] = loop->iters[i]->dataptr; [1] > > > > loop->function((char **)loop->bufptr, &(loop->bufcnt), > > loop->steps, loop->funcdata); [2] > > UFUNC_CHECK_ERROR(loop); > > > > for (i=0; inargs; i++) { > > PyArray_ITER_NEXT(loop->iters[i]); [3] > > } > > loop->index++; > >} > >break; > > > >[1] 12.97% of function time > >[2] 8.65% of functiont ime > >[3] 62.14% of function time > > > >If statistics from elsewhere in the code would be helpful, let me > >know, > and > >I'll see if I can convince Quantify to cough it up. > > > Please run the same test but using > > x1 = N.random.rand(39,2000) > x2 = N.random.rand(39,64,1) > > z1 = x1[:,N.newaxis,:] - x2 Very similar results to what I had previously: [1] 10.88% [2] 7.25% [3] 68.25% Cheers, Albert From david at ar.media.kyoto-u.ac.jp Tue Oct 3 06:55:31 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 03 Oct 2006 19:55:31 +0900 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) Message-ID: <452241A3.3000702@ar.media.kyoto-u.ac.jp> Hi, I was wondering if there was any way to speed up the following code: y = N.zeros((n, K)) for i in range(K): y[:, i] = gauss_den(data, mu[i, :], va[i, :]) Where K is of order 1e1, n of order 1e5. Normally, gauss_den is a quite expensive function, but the profiler tells me that the indexing y[:,i] takes almost as much time as the gauss_den computation (which computes n exp !). To see if the profiler is "right", i replaces with the (non valid) following function: y = N.zeros((n, K)) for i in range(K): yt = gauss_den(data, mu[i, :], va[i, :]) return y Where more than 99% of the code is spent inside gauss_den. I guess the problem is coming from the fact that y being C order, y[:, i] needs accessing data in a non 'linear' way. Is there a way to speed this up ? I did something like this: y = N.zeros((K, n)) for i in range(K): y[i] = gauss_den(data, mu[i, :], va[i, :]) return y.T which works, but I don't like it very much. Isn't there any other way ? David From oliphant.travis at ieee.org Tue Oct 3 10:59:18 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 03 Oct 2006 08:59:18 -0600 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: References: Message-ID: <45227AC6.4010801@ieee.org> Albert Strasheim wrote: >>> [1] 12.97% of function time >>> [2] 8.65% of functiont ime >>> [3] 62.14% of function time >>> >>> If statistics from elsewhere in the code would be helpful, let me >>> know, >>> >> and >> >>> I'll see if I can convince Quantify to cough it up. >>> >>> >> Please run the same test but using >> >> x1 = N.random.rand(39,2000) >> x2 = N.random.rand(39,64,1) >> >> z1 = x1[:,N.newaxis,:] - x2 >> > > Very similar results to what I had previously: > > [1] 10.88% > [2] 7.25% > [3] 68.25% > > Thanks, I've got some ideas about how to speed this up by eliminating some of the unnecessary calculations going on outside of the function loop, but there will still be some speed issues depending on how the array is traversed once you get above a certain size. I'm not sure there anyway around that, ultimately, due to memory access being slow on most hardware. If anyone has any ideas, I'd love to hear them. I won't be able to get to implementing my ideas until at least Friday (also when rc2 will be released). -Travis From oliphant.travis at ieee.org Tue Oct 3 11:04:33 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 03 Oct 2006 09:04:33 -0600 Subject: [Numpy-discussion] RC2 to be released Friday Message-ID: <45227C01.6080109@ieee.org> I'm going to have to put off release of rc2 for Friday. I'm just too busy right now. That might help us get some speed-ups into the NOBUFFER_UFUNCLOOP code as well. My speed-up ideas are: 1) Only keep track of 1 set of coordinates instead of self->nargs sets (1 for each iterator). 2) Keep track of the dataptr for each iterator in the bufptr array (so a copy isn't necessary) 3) Not increment the index on each iterator separately. All of these changes will be made directly in the NOBUFFER_UFUNCLOOP code. More generally, it would be nice to take these ideas and push them into other areas of the code --- perhaps through the multi-iterator that is already present. Probably, this will have to wait until 1.0.1 though. -Travis From kortmann at ideaworks.com Tue Oct 3 12:03:25 2006 From: kortmann at ideaworks.com (kortmann at ideaworks.com) Date: Tue, 3 Oct 2006 09:03:25 -0700 (PDT) Subject: [Numpy-discussion] ValueError: object too deep for desired array In-Reply-To: References: Message-ID: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\Documents and Settings\kenny>cd c:\lameness C:\lameness>c:\Python24\python.exe templatewindow.py a = [[ 1.00013175e+00 2.63483019e-05 1.00006740e+00 5.22246363e-05 1.00008735e+00 -2.77694969e-07 -1.30756273e-07 1.03202730e-06] [ 2.63483019e-05 6.95644927e-05 -7.15426839e-07 1.99534228e-05 5.29400631e-05 -3.07638369e-09 -5.52729618e-06 -7.61431767e-06] [ 1.00006740e+00 -7.15426839e-07 1.00011917e+00 2.50407751e-05 1.00006219e+00 -2.77757947e-07 -1.30856101e-07 1.23058301e-07] [ 5.22246363e-05 1.99534228e-05 2.50407751e-05 8.21505582e-05 5.26966037e-05 -6.13563429e-09 -2.76420755e-06 -3.80791858e-06] [ 1.00008735e+00 5.29400631e-05 1.00006219e+00 5.26966037e-05 1.00020132e+00 5.01389982e-05 3.45959412e-05 3.17129503e-05] [ -2.77694969e-07 -3.07638369e-09 -2.77757947e-07 -6.13563429e-09 5.01389982e-05 3.45959412e-05 3.17129503e-05 3.27035490e-05] [ -1.30756273e-07 -5.52729618e-06 -1.30856101e-07 -2.76420755e-06 3.45959412e-05 3.17129503e-05 3.27035490e-05 3.59732704e-05] [ 1.03202730e-06 -7.61431767e-06 1.23058301e-07 -3.80791858e-06 3.17129503e-05 3.27035490e-05 3.59732704e-05 4.12184645e-05]] F = [[ -1.44231014e+03] [ -7.54006917e-02] [ -1.44227222e+03] [ -7.49199956e-02] [ -1.44242446e+03] [ -4.24684780e-02] [ -2.49566072e-02] [ -2.16978637e-02]] Traceback (most recent call last): File "C:\lameness\PlotPanel.py", line 476, in OnNo self.parent.peaktime() File "C:\lameness\PlotPanel.py", line 64, in peaktime self._replot() File "C:\lameness\PlotPanel.py", line 159, in _replot self.drawpeak() File "C:\lameness\PlotPanel.py", line 233, in drawpeak COE[:,j1-1] = linalg.solve(A,F) ValueError: object too deep for desired array Anyone know what this error could be coming from? Is it because the floating point numbers are too large, memory wise? -Kenny From tim.hochberg at ieee.org Tue Oct 3 12:12:37 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 09:12:37 -0700 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45227AC6.4010801@ieee.org> References: <45227AC6.4010801@ieee.org> Message-ID: <45228BF5.1070604@ieee.org> Travis Oliphant wrote: > Albert Strasheim wrote: > >>>> [1] 12.97% of function time >>>> [2] 8.65% of functiont ime >>>> [3] 62.14% of function time >>>> >>>> If statistics from elsewhere in the code would be helpful, let me >>>> know, >>>> >>>> >>> and >>> >>> >>>> I'll see if I can convince Quantify to cough it up. >>>> >>>> >>>> >>> Please run the same test but using >>> >>> x1 = N.random.rand(39,2000) >>> x2 = N.random.rand(39,64,1) >>> >>> z1 = x1[:,N.newaxis,:] - x2 >>> >>> >> Very similar results to what I had previously: >> >> [1] 10.88% >> [2] 7.25% >> [3] 68.25% >> >> >> > Thanks, > > I've got some ideas about how to speed this up by eliminating some of > the unnecessary calculations going on outside of the function loop, but > there will still be some speed issues depending on how the array is > traversed once you get above a certain size. I'm not sure there anyway > around that, ultimately, due to memory access being slow on most hardware. > > If anyone has any ideas, I'd love to hear them. I won't be able to > get to implementing my ideas until at least Friday (also when rc2 will > be released). > > I had an idea regarding which axis to operate on first. Rather than operate on strictly the longest axis or strictly the innermost axis, a hybrid approach could be used. We would operate on the longest axis that would not result in the inner loop overflowing the cache. The idea is minimize the loop overhead as we do now by choosing the largest axis, while at the same time attempting to maintain cache friendliness. Let's suppose that it's safe to stuff 2000 array elements into the cache[1], and lets consider two arrays where the lengths of the axes[2], from innermost to outermost are [10,100,1000] and [100,10,1000]. Under this scheme, we would loop over the length 100 axis in both cases. Looping over the length-1000 axis pull 1e6 elements into the cache. If the innermost axis alone exceeded our cache imposed limit, we would of course just loop over that axis first. -tim [1] This would really be specified in bytes and thus the number of allowable elements would vary by data type. It might also need to be tuned based on architecture. [2] I believe that there's some consolidation of adjacent axes that goes on where possible, let's assume that's already happened at this point. From oliphant at ee.byu.edu Tue Oct 3 12:14:32 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 03 Oct 2006 10:14:32 -0600 Subject: [Numpy-discussion] ValueError: object too deep for desired array In-Reply-To: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> References: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> Message-ID: <45228C68.5030302@ee.byu.edu> kortmann at ideaworks.com wrote: >Microsoft Windows XP [Version 5.1.2600] >(C) Copyright 1985-2001 Microsoft Corp. > >C:\Documents and Settings\kenny>cd c:\lameness > >C:\lameness>c:\Python24\python.exe templatewindow.py >a = [[ 1.00013175e+00 2.63483019e-05 1.00006740e+00 5.22246363e-05 > 1.00008735e+00 -2.77694969e-07 -1.30756273e-07 1.03202730e-06] > [ 2.63483019e-05 6.95644927e-05 -7.15426839e-07 1.99534228e-05 > 5.29400631e-05 -3.07638369e-09 -5.52729618e-06 -7.61431767e-06] > [ 1.00006740e+00 -7.15426839e-07 1.00011917e+00 2.50407751e-05 > 1.00006219e+00 -2.77757947e-07 -1.30856101e-07 1.23058301e-07] > [ 5.22246363e-05 1.99534228e-05 2.50407751e-05 8.21505582e-05 > 5.26966037e-05 -6.13563429e-09 -2.76420755e-06 -3.80791858e-06] > [ 1.00008735e+00 5.29400631e-05 1.00006219e+00 5.26966037e-05 > 1.00020132e+00 5.01389982e-05 3.45959412e-05 3.17129503e-05] > [ -2.77694969e-07 -3.07638369e-09 -2.77757947e-07 -6.13563429e-09 > 5.01389982e-05 3.45959412e-05 3.17129503e-05 3.27035490e-05] > [ -1.30756273e-07 -5.52729618e-06 -1.30856101e-07 -2.76420755e-06 > 3.45959412e-05 3.17129503e-05 3.27035490e-05 3.59732704e-05] > [ 1.03202730e-06 -7.61431767e-06 1.23058301e-07 -3.80791858e-06 > 3.17129503e-05 3.27035490e-05 3.59732704e-05 4.12184645e-05]] >F = [[ -1.44231014e+03] > [ -7.54006917e-02] > [ -1.44227222e+03] > [ -7.49199956e-02] > [ -1.44242446e+03] > [ -4.24684780e-02] > [ -2.49566072e-02] > [ -2.16978637e-02]] >Traceback (most recent call last): > File "C:\lameness\PlotPanel.py", line 476, in OnNo > self.parent.peaktime() > File "C:\lameness\PlotPanel.py", line 64, in peaktime > self._replot() > File "C:\lameness\PlotPanel.py", line 159, in _replot > self.drawpeak() > File "C:\lameness\PlotPanel.py", line 233, in drawpeak > COE[:,j1-1] = linalg.solve(A,F) >ValueError: object too deep for desired array > >Anyone know what this error could be coming from? Is it because the >floating point numbers are too large, memory wise? > > No, it's because you are trying to fit a 2-d array into a 1-d position. linalg.solve(A,F) produces a 2-d array in this instance (because F is a 2-d array). COE[:,j1-1:] =linalg.solve(A,F) should work or squeezing the result of the solution to a 1-d array would also work. -Travis From tim.hochberg at ieee.org Tue Oct 3 12:16:02 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 09:16:02 -0700 Subject: [Numpy-discussion] ValueError: object too deep for desired array In-Reply-To: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> References: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> Message-ID: <45228CC2.1010101@ieee.org> kortmann at ideaworks.com wrote: > Microsoft Windows XP [Version 5.1.2600] > (C) Copyright 1985-2001 Microsoft Corp. > > C:\Documents and Settings\kenny>cd c:\lameness > > C:\lameness>c:\Python24\python.exe templatewindow.py > a = [[ 1.00013175e+00 2.63483019e-05 1.00006740e+00 5.22246363e-05 > 1.00008735e+00 -2.77694969e-07 -1.30756273e-07 1.03202730e-06] > [ 2.63483019e-05 6.95644927e-05 -7.15426839e-07 1.99534228e-05 > 5.29400631e-05 -3.07638369e-09 -5.52729618e-06 -7.61431767e-06] > [ 1.00006740e+00 -7.15426839e-07 1.00011917e+00 2.50407751e-05 > 1.00006219e+00 -2.77757947e-07 -1.30856101e-07 1.23058301e-07] > [ 5.22246363e-05 1.99534228e-05 2.50407751e-05 8.21505582e-05 > 5.26966037e-05 -6.13563429e-09 -2.76420755e-06 -3.80791858e-06] > [ 1.00008735e+00 5.29400631e-05 1.00006219e+00 5.26966037e-05 > 1.00020132e+00 5.01389982e-05 3.45959412e-05 3.17129503e-05] > [ -2.77694969e-07 -3.07638369e-09 -2.77757947e-07 -6.13563429e-09 > 5.01389982e-05 3.45959412e-05 3.17129503e-05 3.27035490e-05] > [ -1.30756273e-07 -5.52729618e-06 -1.30856101e-07 -2.76420755e-06 > 3.45959412e-05 3.17129503e-05 3.27035490e-05 3.59732704e-05] > [ 1.03202730e-06 -7.61431767e-06 1.23058301e-07 -3.80791858e-06 > 3.17129503e-05 3.27035490e-05 3.59732704e-05 4.12184645e-05]] > F = [[ -1.44231014e+03] > [ -7.54006917e-02] > [ -1.44227222e+03] > [ -7.49199956e-02] > [ -1.44242446e+03] > [ -4.24684780e-02] > [ -2.49566072e-02] > [ -2.16978637e-02]] > Traceback (most recent call last): > File "C:\lameness\PlotPanel.py", line 476, in OnNo > self.parent.peaktime() > File "C:\lameness\PlotPanel.py", line 64, in peaktime > self._replot() > File "C:\lameness\PlotPanel.py", line 159, in _replot > self.drawpeak() > File "C:\lameness\PlotPanel.py", line 233, in drawpeak > COE[:,j1-1] = linalg.solve(A,F) > ValueError: object too deep for desired array > > Anyone know what this error could be coming from? Is it because the > floating point numbers are too large, memory wise?\ > No, this error has to do with the number of dimensions being wrong somewhere. Try printing out the shapes of COE[:,j-1] and linalg.solve(A,F); you'll probably see that they don't match. -tim From tim.hochberg at ieee.org Tue Oct 3 12:24:10 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 09:24:10 -0700 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: <452241A3.3000702@ar.media.kyoto-u.ac.jp> References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> Message-ID: <45228EAA.9040301@ieee.org> David Cournapeau wrote: > Hi, > > I was wondering if there was any way to speed up the following code: > > y = N.zeros((n, K)) > for i in range(K): > y[:, i] = gauss_den(data, mu[i, :], va[i, :]) > > Where K is of order 1e1, n of order 1e5. Normally, gauss_den is a quite > expensive function, but the profiler tells me that the indexing y[:,i] > takes almost as much time as the gauss_den computation (which computes n > exp !). To see if the profiler is "right", i replaces with the (non > valid) following function: > > y = N.zeros((n, K)) > for i in range(K): > yt = gauss_den(data, mu[i, :], va[i, :]) > return y > > Where more than 99% of the code is spent inside gauss_den. > > I guess the problem is coming from the fact that y being C order, y[:, > i] needs accessing data in a non 'linear' way. Is there a way to speed > this up ? I did something like this: > > y = N.zeros((K, n)) > for i in range(K): > y[i] = gauss_den(data, mu[i, :], va[i, :]) > return y.T > > which works, but I don't like it very much. Why not? > Isn't there any other way That depends on the details of gauss_den. A general note: for this kind of microoptimization puzzle, it's much easier to help if you can post a self contained example, preferably something fairly simple that still illustrates the speed issue, that we can experiment with. -tim From faltet at carabos.com Tue Oct 3 12:55:31 2006 From: faltet at carabos.com (Francesc Altet) Date: Tue, 3 Oct 2006 18:55:31 +0200 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? Message-ID: <200610031855.32056.faltet@carabos.com> Hi, I thought that numpy.isscalar was a good way of distinguising a numpy scalar from a python scalar, but it seems not: >>> numpy.isscalar(numpy.string_('3')) True >>> numpy.isscalar('3') True Is there an easy (and fast, if possible) way to check whether an object is a numpy scalar or a python one? Thanks, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From kortmann at ideaworks.com Tue Oct 3 13:02:51 2006 From: kortmann at ideaworks.com (Kenny Ortmann) Date: Tue, 3 Oct 2006 12:02:51 -0500 Subject: [Numpy-discussion] ValueError: object too deep for References: Message-ID: <00fd01c6e70d$c33af590$0700a8c0@yairlap> Yeah thanks guys, I printed them out the size and realized what it was doing then used a for loop to put it into the 1D array, but thanks Travis the way you said makes it easier than the for loop. -Kenny From haase at msg.ucsf.edu Tue Oct 3 13:24:28 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue, 3 Oct 2006 10:24:28 -0700 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? In-Reply-To: <200610031855.32056.faltet@carabos.com> References: <200610031855.32056.faltet@carabos.com> Message-ID: <200610031024.28188.haase@msg.ucsf.edu> Hi, a noticed the underscore in "numpy.string_" ... I thought the underscores were removed in favour of handling the "from numpy import *" separately via the __all__ variable. Is this done only for *some* members of numpy ? (sorry for hijacking your thread ...) -Sebastian Haase On Tuesday 03 October 2006 09:55, Francesc Altet wrote: > Hi, > > I thought that numpy.isscalar was a good way of distinguising a numpy > scalar > > from a python scalar, but it seems not: > >>> numpy.isscalar(numpy.string_('3')) > > True > > >>> numpy.isscalar('3') > > True > > Is there an easy (and fast, if possible) way to check whether an object is > a numpy scalar or a python one? > > Thanks, From tim.hochberg at ieee.org Tue Oct 3 13:27:13 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 10:27:13 -0700 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? In-Reply-To: <200610031855.32056.faltet@carabos.com> References: <200610031855.32056.faltet@carabos.com> Message-ID: <45229D71.3010800@ieee.org> Francesc Altet wrote: > Hi, > > I thought that numpy.isscalar was a good way of distinguising a numpy scalar > from a python scalar, but it seems not: > > >>>> numpy.isscalar(numpy.string_('3')) >>>> > True > >>>> numpy.isscalar('3') >>>> > True > > Is there an easy (and fast, if possible) way to check whether an object is a > numpy scalar or a python one? > > It looks like isinstance(x, numpy.generic) works, but I didn't test it extensively. -tim From rowen at cesmail.net Tue Oct 3 13:31:26 2006 From: rowen at cesmail.net (Russell E. Owen) Date: Tue, 03 Oct 2006 10:31:26 -0700 Subject: [Numpy-discussion] numarray and/or Numeric with python 2.5? References: <45216248.60905@stsci.edu> Message-ID: In article <45216248.60905 at stsci.edu>, Todd Miller wrote: > Russell E. Owen wrote: > > Has anyone tried to build numarray or Numeric with python 2.5? Since the > > > numarray builds and runs fine for Python-2.5, but only as 32-bit. > > type of array index was changed, I'm wondering if it works (and if so, > > > NOTE: numarray does not exploit the new ssize_t index in Python-2.5 so > numarray is not 64-bit enabled. Thank you very much. That's great. It means my current code is compatible with Python 2.5 (assuming Numeric works the same way, which seems likely). I am really looking forward to switching to numpy, but now is just a bit too early. -- Russell From oliphant at ee.byu.edu Tue Oct 3 13:47:05 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 03 Oct 2006 11:47:05 -0600 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? In-Reply-To: <200610031024.28188.haase@msg.ucsf.edu> References: <200610031855.32056.faltet@carabos.com> <200610031024.28188.haase@msg.ucsf.edu> Message-ID: <4522A219.504@ee.byu.edu> Sebastian Haase wrote: >Hi, >a noticed the underscore in "numpy.string_" ... >I thought the underscores were removed in favour of handling the >"from numpy import *" separately via the __all__ variable. >Is this done only for *some* members of numpy ? > > Yeah. The underscores are still on the conflicting type names. -Travis From oliphant at ee.byu.edu Tue Oct 3 13:47:55 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 03 Oct 2006 11:47:55 -0600 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? In-Reply-To: <45229D71.3010800@ieee.org> References: <200610031855.32056.faltet@carabos.com> <45229D71.3010800@ieee.org> Message-ID: <4522A24B.5010205@ee.byu.edu> Tim Hochberg wrote: >Francesc Altet wrote: > > >>Hi, >> >>I thought that numpy.isscalar was a good way of distinguising a numpy scalar >>from a python scalar, but it seems not: >> >> >> >> >>>>>numpy.isscalar(numpy.string_('3')) >>>>> >>>>> >>>>> >>True >> >> >> >>>>>numpy.isscalar('3') >>>>> >>>>> >>>>> >>True >> >>Is there an easy (and fast, if possible) way to check whether an object is a >>numpy scalar or a python one? >> >> >> >> >It looks like isinstance(x, numpy.generic) works, but I didn't test it >extensively. > > > That should definitely work. All the array scalars are in a hierarchy inheriting from numpy.generic. There are also sub-levels (numpy.integer, numpy.inexact, etc...) -Travis From kortmann at ideaworks.com Tue Oct 3 16:01:48 2006 From: kortmann at ideaworks.com (Kenny Ortmann) Date: Tue, 3 Oct 2006 15:01:48 -0500 Subject: [Numpy-discussion] complex numbers References: Message-ID: <005f01c6e726$c3317650$0700a8c0@yairlap> excuse my laziness for not looking this up, I googled it but could not find a solution. matlab has a isreal(array) where if the array is full of real numbers the value returned is 1. I'm translating matlab code and ran across if ~isreal(array) array = abs(array) Is there a way to check to see if a number is real or complex? and if so is there a way to extract the(a + ib) because the absolute value of a complex number is like the pythagorean therom on a and b? thanks for your help, Kenny From David.L.Goldsmith at noaa.gov Tue Oct 3 16:22:38 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Tue, 03 Oct 2006 13:22:38 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <005f01c6e726$c3317650$0700a8c0@yairlap> References: <005f01c6e726$c3317650$0700a8c0@yairlap> Message-ID: <4522C68E.4030101@noaa.gov> numpy.isreal(a) is a "top level" function (i.e., not a class member function) in numpy; here's its help doc: Help on function isreal in module numpy.lib.type_check: isreal(x) Return a boolean array where elements are True if that element is real (has zero imaginary part) For scalars, return a boolean. As for checking for pure imaginaries (i.e., real part == 0), one could use: x.imag == -j*x or numpy.isreal(i*x) or simply (x.real == 0.0) and (x.imag != 0.0) DG Kenny Ortmann wrote: > excuse my laziness for not looking this up, I googled it but could not find > a solution. > matlab has a > isreal(array) > where if the array is full of real numbers the value returned is 1. > I'm translating matlab code and ran across > > if ~isreal(array) > array = abs(array) > > Is there a way to check to see if a number is real or complex? and if so is > there a way to extract the(a + ib) because the absolute value of a complex > number is like the pythagorean therom on a and b? > > thanks for your help, > Kenny > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From David.L.Goldsmith at noaa.gov Tue Oct 3 16:33:19 2006 From: David.L.Goldsmith at noaa.gov (David L Goldsmith) Date: Tue, 03 Oct 2006 13:33:19 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <005f01c6e726$c3317650$0700a8c0@yairlap> References: <005f01c6e726$c3317650$0700a8c0@yairlap> Message-ID: <4522C90F.9050206@noaa.gov> PS: The Python built in function (i.e., you don't even need numpy for this) abs(x) is "vectorized" (i.e., accepts a (nested) sequence, incl. numpy array, argument) and overloaded to give the modulus (i.e., Pythagorean "length") of a complex number when such is its argument. DG Kenny Ortmann wrote: > excuse my laziness for not looking this up, I googled it but could not find > a solution. > matlab has a > isreal(array) > where if the array is full of real numbers the value returned is 1. > I'm translating matlab code and ran across > > if ~isreal(array) > array = abs(array) > > Is there a way to check to see if a number is real or complex? and if so is > there a way to extract the(a + ib) because the absolute value of a complex > number is like the pythagorean therom on a and b? > > thanks for your help, > Kenny > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- HMRD/ORR/NOS/NOAA From tim.hochberg at ieee.org Tue Oct 3 16:34:54 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 13:34:54 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <005f01c6e726$c3317650$0700a8c0@yairlap> References: <005f01c6e726$c3317650$0700a8c0@yairlap> Message-ID: <4522C96E.2010401@ieee.org> Kenny Ortmann wrote: > excuse my laziness for not looking this up, I googled it but could not find > a solution. > matlab has a > isreal(array) > where if the array is full of real numbers the value returned is 1. > I'm translating matlab code and ran across > > if ~isreal(array) > array = abs(array) > > Is there a way to check to see if a number is real or complex? There may be a better way, but:: alltrue(isreal(x)) Would work. As would: not sometrue(x.imag) In the above test you are already negating the test, so you could just drop the not. > and if so is > there a way to extract the(a + ib) because the absolute value of a complex > number is like the pythagorean therom on a and b? > I'm not entirely sure what you are looking for here, but x.imag and x.real will give you the real and imaginary parts. abs(x) will return the magnitude of x whether x is real or complex. x.conj() will return complex conjugate. I'm somewhat suspicious of that matlab code. The code given is discontinuous as you cross the negative real axis. Does the result subsequently get squared or something? I'm guessing that either the matlab code is doing extra work, or there are some hidden assumptions (all values are in the positive real half-plane). Or some such. In either case, you'd probably be OK just skipping the check for realness and always taking the absolute of array. I can't say for sure without more context though. -tim From tim.hochberg at ieee.org Tue Oct 3 16:38:15 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 13:38:15 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <4522C90F.9050206@noaa.gov> References: <005f01c6e726$c3317650$0700a8c0@yairlap> <4522C90F.9050206@noaa.gov> Message-ID: <4522CA37.9060108@ieee.org> David L Goldsmith wrote: > PS: The Python built in function (i.e., you don't even need numpy for > this) abs(x) is "vectorized" (i.e., accepts a (nested) sequence, incl. > numpy array, argument) and overloaded to give the modulus (i.e., > Pythagorean "length") of a complex number when such is its argument. > This isn't quite right. The built in abs function looks for the special method __abs__. So, abs(x) is equivalent to x.__abs__(). Arrays supply an appropriate __abs__ method, lists do not. For example: >>> l = [1,2,3,4] >>> abs(l) Traceback (most recent call last): File "", line 1, in ? TypeError: bad operand type for abs() >>> a = numpy.arange(5) >>> abs(a) array([0, 1, 2, 3, 4]) -tim > DG > > Kenny Ortmann wrote: > >> excuse my laziness for not looking this up, I googled it but could not find >> a solution. >> matlab has a >> isreal(array) >> where if the array is full of real numbers the value returned is 1. >> I'm translating matlab code and ran across >> >> if ~isreal(array) >> array = abs(array) >> >> Is there a way to check to see if a number is real or complex? and if so is >> there a way to extract the(a + ib) because the absolute value of a complex >> number is like the pythagorean therom on a and b? >> >> thanks for your help, >> Kenny >> >> >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share your >> opinions on IT & business topics through brief surveys -- and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> > > > From David.L.Goldsmith at noaa.gov Tue Oct 3 17:21:55 2006 From: David.L.Goldsmith at noaa.gov (David L Goldsmith) Date: Tue, 03 Oct 2006 14:21:55 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <4522CA37.9060108@ieee.org> References: <005f01c6e726$c3317650$0700a8c0@yairlap> <4522C90F.9050206@noaa.gov> <4522CA37.9060108@ieee.org> Message-ID: <4522D473.8020506@noaa.gov> Thanks Tim, DG Tim Hochberg wrote: > David L Goldsmith wrote: > >> PS: The Python built in function (i.e., you don't even need numpy for >> this) abs(x) is "vectorized" (i.e., accepts a (nested) sequence, incl. >> numpy array, argument) and overloaded to give the modulus (i.e., >> Pythagorean "length") of a complex number when such is its argument. >> >> > This isn't quite right. The built in abs function looks for the special > method __abs__. So, abs(x) is equivalent to x.__abs__(). Arrays supply > an appropriate __abs__ method, lists do not. For example: > > >>> l = [1,2,3,4] > >>> abs(l) > Traceback (most recent call last): > File "", line 1, in ? > TypeError: bad operand type for abs() > >>> a = numpy.arange(5) > >>> abs(a) > array([0, 1, 2, 3, 4]) > > > -tim > >> DG >> >> Kenny Ortmann wrote: >> >> >>> excuse my laziness for not looking this up, I googled it but could not find >>> a solution. >>> matlab has a >>> isreal(array) >>> where if the array is full of real numbers the value returned is 1. >>> I'm translating matlab code and ran across >>> >>> if ~isreal(array) >>> array = abs(array) >>> >>> Is there a way to check to see if a number is real or complex? and if so is >>> there a way to extract the(a + ib) because the absolute value of a complex >>> number is like the pythagorean therom on a and b? >>> >>> thanks for your help, >>> Kenny >>> >>> >>> >>> ------------------------------------------------------------------------- >>> Take Surveys. Earn Cash. Influence the Future of IT >>> Join SourceForge.net's Techsay panel and you'll get the chance to share your >>> opinions on IT & business topics through brief surveys -- and earn cash >>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >>> _______________________________________________ >>> Numpy-discussion mailing list >>> Numpy-discussion at lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >>> >>> >>> >> >> > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- HMRD/ORR/NOS/NOAA From lfqfbhj at sofresh.com Tue Oct 3 17:34:39 2006 From: lfqfbhj at sofresh.com (Alerts. Terms) Date: Tue, 03 Oct 2006 21:34:39 -0000 Subject: [Numpy-discussion] News Maps more Message-ID: <000e01c389f6$2576ab00$15f9347d@expvcianpc> VideoNew News Maps more Maps more raquo Sign in Web fewer your on can try Answers for expert different keywords. general fewer your on can fewer your on raquo Advanced Search Members: Alerts Create new group About raquo Advanced Search Members: users: Join Alerts Create groups Your search did not match any Make sure words About Searched all groups Your search did not match any any Make sure words are spelled Try different Search Members: latest the latest messages emailed search. Get the latest Try different -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: in.gif Type: image/gif Size: 9782 bytes Desc: not available URL: From kortmann at ideaworks.com Tue Oct 3 17:49:29 2006 From: kortmann at ideaworks.com (Kenny Ortmann) Date: Tue, 3 Oct 2006 16:49:29 -0500 Subject: [Numpy-discussion] Complex numbers response References: Message-ID: <007d01c6e735$cebe2a90$0700a8c0@yairlap> > There may be a better way, but:: > > alltrue(isreal(x)) > > Would work. As would: > > not sometrue(x.imag) > > In the above test you are already negating the test, so you could just > drop the not. >> and if so is >> there a way to extract the(a + ib) because the absolute value of a >> complex >> number is like the pythagorean therom on a and b? what i was looking for in the above is that when I looked up the abs(complexnum) i saw that the abs() of a complex num is (a^2 + b^2)^.5 so i was asking for a way to do this because i was assuming that i was going to have to parse through the array in a for loop and if i found a complex number then get the a and b to do the pythagorean theorom but it seems i will not have to do this, as for the matlab code see below > I'm not entirely sure what you are looking for here, but x.imag and > x.real will give you the real and imaginary parts. abs(x) will return > the magnitude of x whether x is real or complex. x.conj() will return > complex conjugate. > > I'm somewhat suspicious of that matlab code. The code given is > discontinuous as you cross the negative real axis. Does the result > subsequently get squared or something? I'm guessing that either the > matlab code is doing extra work, or there are some hidden assumptions > (all values are in the positive real half-plane). Or some such. In > either case, you'd probably be OK just skipping the check for realness > and always taking the absolute of array. I can't say for sure without > more context though. > > -tim this is taken out of a program i am converting from matlab to python, so I can not explain all of it, it would take forever, BUT COEmix, the input, is equal to a different arrays 0 and 2 rows added together. So with that being said, it can not be empty, and must be actual numbers. The reason I am "scared" to remove the isreal statement that i asked about is because the array before hand has values that are taken from square roots. I do not know if these could ever end up being complex and I am meeting with the head of this project tomorrow to discuss it, but you asked so i thought i would share. function [pospeakind,negpeakind]=peakdetect(COEmix) if ~nargin COEmix=input('enter COEmix vector or return for empty outputs\n'); if isempty(COEmix) pospeakind=[]; negpeakind=[]; return end end sizsig=size(COEmix); while isempty(COEmix)|~isnumeric(COEmix)|~all(all(isfinite(COEmix)))... |length(sizsig)>2|min(sizsig)~=1 COEmix=input(['COEmix is empty, nonnumeric, nonfinite, or nonvector:\nenter '... 'finite vector or return for empty outputs\n']); if isempty(COEmix) pospeakind=[]; negpeakind=[]; return end sizsig=size(COEmix); end if ~isreal(COEmix) COEmix=abs(COEmix); end if ~any(COEmix-COEmix(1)) pospeakind=[]; negpeakind=[]; disp('constant COEmix graph suppressed') return end From tim.hochberg at ieee.org Tue Oct 3 18:59:57 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 15:59:57 -0700 Subject: [Numpy-discussion] Complex numbers response In-Reply-To: <007d01c6e735$cebe2a90$0700a8c0@yairlap> References: <007d01c6e735$cebe2a90$0700a8c0@yairlap> Message-ID: <4522EB6D.5010604@ieee.org> Kenny Ortmann wrote: >> There may be a better way, but:: >> >> alltrue(isreal(x)) >> >> Would work. As would: >> >> not sometrue(x.imag) >> >> In the above test you are already negating the test, so you could just >> drop the not. >> >>> and if so is >>> there a way to extract the(a + ib) because the absolute value of a >>> complex >>> number is like the pythagorean therom on a and b? >>> > > what i was looking for in the above is that when I looked up the > abs(complexnum) i saw that the abs() of a complex num is (a^2 + b^2)^.5 so i > was asking for a way to do this because i was assuming that i was going to > have to parse through the array in a for loop and if i found a complex > number then get the a and b to do the pythagorean theorom but it seems i > will not have to do this, as for the matlab code see below > OK. Yeah, you shouldn't have to do element by element calculations here. >> I'm not entirely sure what you are looking for here, but x.imag and >> x.real will give you the real and imaginary parts. abs(x) will return >> the magnitude of x whether x is real or complex. x.conj() will return >> complex conjugate. >> >> I'm somewhat suspicious of that matlab code. The code given is >> discontinuous as you cross the negative real axis. Does the result >> subsequently get squared or something? I'm guessing that either the >> matlab code is doing extra work, or there are some hidden assumptions >> (all values are in the positive real half-plane). Or some such. In >> either case, you'd probably be OK just skipping the check for realness >> and always taking the absolute of array. I can't say for sure without >> more context though. >> >> -tim >> > > this is taken out of a program i am converting from matlab to python, so I > can not explain all of it, it would take forever, BUT > COEmix, the input, is equal to a different arrays 0 and 2 rows added > together. So with that being said, it can not be empty, and must be actual > numbers. The reason I am "scared" to remove the isreal statement that i > asked about is because the array before hand has values that are taken from > square roots. I do not know if these could ever end up being complex and I > am meeting with the head of this project tomorrow to discuss it, but you > asked so i thought i would share. > Just to beat a deceased equine a bit more: what I'm concerned about here is that the code as it stands will result in a positive number for (-x+jy), not matter how small y is, while resulting in a negative number for -x. That discontinuity seems unlikely to be what is desired. Actually, it's worse than that, if there are any complex numbers in the array, all the numbers will be converted to positive real numbers. So, for example: [-1, 1, -2, 2, 5] => [-1, 1, -2, 2, 5] while: [-1, 1, -2, 2, 3+4j] => [1, 1, 2, 2, 5] Now it's quite possible that the way COEmix is generated constrains it's value in some way. It's possible that it's constrained to the positive half plane, or perhaps more likely that it's ideally a positive real number that may stray slightly off the real axis, so to clean it up the absolute value is being taken. Since, I can't think of a case where you actually want the discontinuity described above, I'm guessing that the code would be just fine as COEmix = abs(COEmix) That is, unconditionally take the absolute value. However, that *is* just a guess. The answer lies in how COEmix is originally generated and perhaps in how it is used as well. Regards, -tim [... CODE...] From peridot.faceted at gmail.com Tue Oct 3 02:06:42 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Tue, 3 Oct 2006 02:06:42 -0400 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521AF9A.50701@ee.byu.edu> References: <45219AD6.5020203@ee.byu.edu> <4521AF9A.50701@ee.byu.edu> Message-ID: On 02/10/06, Travis Oliphant wrote: > Perhaps those inner 1-d loops could be optimized (using prefetch or > something) to reduce the number of cache misses on the inner > computation, and the concept of looping over the largest dimension > (instead of the last dimension) should be re-considered. Cache control seems to be the main factor deciding the speed of many algorithms. Prefectching could make a huge difference, particularly on NUMA machines (like a dual opteron). I think GCC has a moderately portable way to request it (though it may be only in beta versions as yet). More generally, all the tricks that ATLAS uses to accelerate BLAS routines would (in principle) be applicable here. The implementation would be extremely difficult, though, even if all the basic loops could be expressed in a few primitives. A. M. Archibald From lars.bittrich at googlemail.com Tue Oct 3 05:53:22 2006 From: lars.bittrich at googlemail.com (Lars Bittrich) Date: Tue, 3 Oct 2006 11:53:22 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) In-Reply-To: <45218A44.3070101@ee.byu.edu> References: <200610020926.56965.lars.bittrich@googlemail.com> <45218A44.3070101@ee.byu.edu> Message-ID: <200610031153.22422.lars.bittrich@googlemail.com> On Monday 02 October 2006 23:53, Travis Oliphant wrote: > This is a Python 2.5 issue (the new __index__ method) was incorrectly > implemented and allowing a 1-d array to be interpreted as an index. > > This should be fixed in SVN. Ok, thank you. I tried that but the version of python-svn is 2.6 and there are much more complications like: File "[...]/lib/python2.6/site-packages/numpy/core/numeric.py", line 284 as = a.shape ^ SyntaxError: invalid syntax File "[...]/lib/python2.6/site-packages/numpy/f2py/crackfortran.py", line 1647 as=b['args'] ^ SyntaxError: invalid syntax I should try Python 2.4 for more stable results. P.S. Sorry for spaming you. I had problems with my mail address settings. I did not expect the mails to arrive after a few days. I thought that they were rejected. From gael.varoquaux at normalesup.org Tue Oct 3 22:11:51 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Wed, 4 Oct 2006 04:11:51 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) In-Reply-To: <200610031153.22422.lars.bittrich@googlemail.com> References: <200610020926.56965.lars.bittrich@googlemail.com> <45218A44.3070101@ee.byu.edu> <200610031153.22422.lars.bittrich@googlemail.com> Message-ID: <20061004021151.GA12486@clipper.ens.fr> On Tue, Oct 03, 2006 at 11:53:22AM +0200, Lars Bittrich wrote: > On Monday 02 October 2006 23:53, Travis Oliphant wrote: > > This is a Python 2.5 issue (the new __index__ method) was incorrectly > > implemented and allowing a 1-d array to be interpreted as an index. > > This should be fixed in SVN. > Ok, thank you. I tried that but the version of python-svn is 2.6 No, Travis was talking of the numpy svn, not the python svn. Gael From robert.kern at gmail.com Tue Oct 3 23:01:16 2006 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 03 Oct 2006 22:01:16 -0500 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python Message-ID: Has anyone implemented an easier or more efficient way to broadcast arrays to a common shape at the Python level? I was hoping that the broadcast iterator would actually provide the broadcasted arrays, but it does not. I've attached my best pure-Python effort. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -------------- next part -------------- A non-text attachment was scrubbed... Name: broadcast_array.py Type: application/x-python Size: 1280 bytes Desc: not available URL: From peridot.faceted at gmail.com Tue Oct 3 12:38:55 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Tue, 3 Oct 2006 12:38:55 -0400 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45228BF5.1070604@ieee.org> References: <45227AC6.4010801@ieee.org> <45228BF5.1070604@ieee.org> Message-ID: On 03/10/06, Tim Hochberg wrote: > I had an idea regarding which axis to operate on first. Rather than > operate on strictly the longest axis or strictly the innermost axis, a > hybrid approach could be used. We would operate on the longest axis that > would not result in the inner loop overflowing the cache. The idea is > minimize the loop overhead as we do now by choosing the largest axis, > while at the same time attempting to maintain cache friendliness. If elements are smaller than cache lines (usually at least eight bytes, I think), we might end up pulling many times as many bytes into the cache as we actually need if we don't loop along axes with small strides first. Can BLAS be used for some of these operations? A. M. Archibald A. M. Archibald From fullung at gmail.com Mon Oct 2 18:00:10 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 3 Oct 2006 00:00:10 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that Message-ID: <5eec5f300610021500g6034c106h11f4c32d77e642ac@mail.gmail.com> Hello all I recently started looking at David Cournapeau's PyEM package, specifically his implementation of the K-Means algorithm. He implemented part of this algorithm with in pure Python version and also provided a Pyrex alternative that is significantly faster (about 10 times with the data I tested with). I tried to figure out why this is so. The job of this part of the algorithm is pretty simple: from a bunch of cluster means (the codebook) find the nearest cluster mean for each data point. The first attempt at implementing this algorithm might use two for loops, one over the data points and one over the cluster means, computing a Euclidean distance at each step. Slightly better is to use one loop and some broadcasting. By randomly trying some code, I arrived at the following one-liner: N.sum((data[...,N.newaxis,...]-code)**2, 2).argmin(axis=1) where data = N.random.randn(2000, 39) nclusters = 64 code = data[0:nclusters,:] This code was still slow, so I mangled it a bit so that I could profile it using cProfile under Python 2.5 (use profile if you have an older version of Python): def kmean(): data = N.random.randn(2000, 39) nclusters = 64 code = data[0:nclusters,:] for i in xrange(10): def a(): return data[...,N.newaxis,...] z = a() def b(): return z-code z = b() def c(): return z**2 z = c() def d(): return N.sum(z, 2) z = d() def e(): return z.argmin(axis=1) z = e() if __name__ == '__main__': import cProfile cProfile.run("kmean()") Profiler output: In [570]: %run -t -e kmean2.py 84 function calls in 8.567 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 8.567 8.567 :1() 10 0.000 0.000 0.835 0.084 fromnumeric.py:375(sum) 10 0.000 0.000 0.000 0.000 kmean2.py:10(a) 10 6.837 0.684 6.837 0.684 kmean2.py:12(b) 10 0.623 0.062 0.623 0.062 kmean2.py:14(c) 10 0.000 0.000 0.835 0.084 kmean2.py:16(d) 10 0.080 0.008 0.080 0.008 kmean2.py:18(e) 1 0.180 0.180 8.567 8.567 kmean2.py:5(kmean2) 10 0.000 0.000 0.000 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 1 0.012 0.012 0.012 0.012 {method 'randn' of 'mtrand.RandomState' objects} 10 0.835 0.083 0.835 0.083 {method 'sum' of 'numpy.ndarray' objects} It seems the b part of the computation is taking most of the time. I tried to simulate this separately: In [571]: x1 = N.random.randn(2000,39) In [572]: y1 = N.random.randn(64,39) In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms per loop In [575]: z1.shape Out[575]: (2000, 64, 39) As far as I can figure, this operation is doing 2000*64*39 subtractions. Doing this straight up yields the following: In [576]: x2 = N.random.randn(2000,64,39) In [577]: y2 = N.random.randn(2000,64,39) In [578]: %timeit z2 = x2-y2 10 loops, best of 3: 108 ms per loop Does anybody have any ideas on why this is so much faster? Hopefully I didn't mess up somewhere... Thanks! Regards, Albert P.S. I'm used NumPy 1.0rc1 with Python 2.5 on Windows for these tests. I get similar results with 1.0.dev3239 with Python 2.4.3 on Linux. From peridot.faceted at gmail.com Wed Oct 4 01:20:44 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 4 Oct 2006 01:20:44 -0400 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python In-Reply-To: References: Message-ID: On 03/10/06, Robert Kern wrote: > Has anyone implemented an easier or more efficient way to broadcast arrays to a > common shape at the Python level? I was hoping that the broadcast iterator would > actually provide the broadcasted arrays, but it does not. How about vectorize(lambda *args: args)? Almost works, only it sometimes segfaults with more than two arguments... but that's clearly a numpy bug. A. M. Archibald From wbaxter at gmail.com Tue Oct 3 05:15:25 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Tue, 3 Oct 2006 18:15:25 +0900 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4522122D.9070407@ar.media.kyoto-u.ac.jp> References: <4522122D.9070407@ar.media.kyoto-u.ac.jp> Message-ID: Er, is this the PyEM in question? It doesn't have much of a web presence... http://www.ar.media.kyoto-u.ac.jp/members/david/ --bb On 10/3/06, David Cournapeau wrote: > Albert Strasheim wrote: > > Hello all > > > > I recently started looking at David Cournapeau's PyEM package, specifically > > his implementation of the K-Means algorithm. He implemented part of this > > algorithm with in pure Python version and also provided a Pyrex alternative > > that is significantly faster (about 10 times with the data I tested with). I > > tried to figure out why this is so. From fullung at gmail.com Tue Oct 3 05:12:02 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 3 Oct 2006 11:12:02 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521C1EB.30205@ee.byu.edu> Message-ID: Hello all > -----Original Message----- > From: Travis Oliphant [mailto:oliphant at ee.byu.edu] > Sent: 03 October 2006 03:51 > To: Discussion of Numerical Python; Albert Strasheim > Subject: Re: ***[Possible UCE]*** Re: [Numpy-discussion] Vectorizing code, > for loops, and all that > > > >Meanwhile, I can confirm that the NOBUFFER_UFUNCLOOP case in > >PyUFunc_GenericFunction is getting exercised in the slower case. Here's > some > >info on what's happening, courtesy of Rational Quantify: > > > >case NOBUFFER_UFUNCLOOP: > >while (loop->index < loop->size) { > > for (i=0; inargs; i++) > > loop->bufptr[i] = loop->iters[i]->dataptr; [1] > > > > loop->function((char **)loop->bufptr, &(loop->bufcnt), > > loop->steps, loop->funcdata); [2] > > UFUNC_CHECK_ERROR(loop); > > > > for (i=0; inargs; i++) { > > PyArray_ITER_NEXT(loop->iters[i]); [3] > > } > > loop->index++; > >} > >break; > > > >[1] 12.97% of function time > >[2] 8.65% of functiont ime > >[3] 62.14% of function time > > > >If statistics from elsewhere in the code would be helpful, let me know, > and > >I'll see if I can convince Quantify to cough it up. > > > Please run the same test but using > > x1 = N.random.rand(39,2000) > x2 = N.random.rand(39,64,1) > > z1 = x1[:,N.newaxis,:] - x2 Very similar results to what I had previously: [1] 10.88% [2] 7.25% [3] 68.25% Cheers, Albert From robert.kern at gmail.com Wed Oct 4 01:33:05 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 04 Oct 2006 00:33:05 -0500 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python In-Reply-To: References: Message-ID: A. M. Archibald wrote: > On 03/10/06, Robert Kern wrote: >> Has anyone implemented an easier or more efficient way to broadcast arrays to a >> common shape at the Python level? I was hoping that the broadcast iterator would >> actually provide the broadcasted arrays, but it does not. > > How about vectorize(lambda *args: args)? Almost works, only it > sometimes segfaults with more than two arguments... but that's clearly > a numpy bug. Yeah, a segfault would be problematic. Otherwise, it works and is in fact faster than what I wrote. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From peridot.faceted at gmail.com Wed Oct 4 01:37:55 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 4 Oct 2006 01:37:55 -0400 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python In-Reply-To: References: Message-ID: On 04/10/06, Robert Kern wrote: > Yeah, a segfault would be problematic. Otherwise, it works and is in fact faster > than what I wrote. It's a bit tricky to trigger but I think it was fixed in 1.0rc1 (in changeset 3125, in fact: http://projects.scipy.org/scipy/numpy/changeset/3125 ) Would it be useful for me to contribute the tiny script I wrote to trigger it as a regression test? A. M. Archibald -------------- next part -------------- A non-text attachment was scrubbed... Name: bug.py Type: text/x-python Size: 248 bytes Desc: not available URL: From Mailer-Daemon at lists.sourceforge.net Wed Oct 4 01:57:41 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Tue, 03 Oct 2006 22:57:41 -0700 Subject: [Numpy-discussion] Warning: message 1GUFPK-0001wW-5k delayed 48 hours Message-ID: This message was created automatically by mail delivery software. A message that you sent has not yet been delivered to one or more of its recipients after more than 48 hours on the queue on externalmx-1.sourceforge.net. The message identifier is: 1GUFPK-0001wW-5k The date of the message is: Mon, 02 Oct 2006 09:32:13 -0200 The subject of the message is: doctor, at the revolt is located treating the planet he gold; The address to which the message has not yet been delivered is: numpy-discussion at lists.sourceforge.net Delay reason: SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is No action is required on your part. Delivery attempts will continue for some time, and this warning may be repeated at intervals if the message remains undelivered. Eventually the mail delivery software will give up, and when that happens, the message will be returned to you. From david at ar.media.kyoto-u.ac.jp Wed Oct 4 06:58:25 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 04 Oct 2006 19:58:25 +0900 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: <45228EAA.9040301@ieee.org> References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> <45228EAA.9040301@ieee.org> Message-ID: <452393D1.2020308@ar.media.kyoto-u.ac.jp> Tim Hochberg wrote: >> >> >> I guess the problem is coming from the fact that y being C order, y[:, >> i] needs accessing data in a non 'linear' way. Is there a way to speed >> this up ? I did something like this: >> >> y = N.zeros((K, n)) >> for i in range(K): >> y[i] = gauss_den(data, mu[i, :], va[i, :]) >> return y.T >> >> which works, but I don't like it very much. > Why not? > Mainly because using those transpose do not really reflect the intention, and this does not seem natural. >> Isn't there any other way > That depends on the details of gauss_den. > > A general note: for this kind of microoptimization puzzle, it's much > easier to help if you can post a self contained example, preferably > something fairly simple that still illustrates the speed issue, that we > can experiment with. > Here we are (the difference may not seem that much between the two multiple_ga, but in reality, _diag_gauss_den is an internal function which is done in C, and is much faster... By writing this example, I've just realized that the function _diag_gauss_den may be slow for exactly the same reasons): #! /usr/bin/env python # Last Change: Wed Oct 04 07:00 PM 2006 J import numpy as N from numpy.random import randn def _diag_gauss_den(x, mu, va): """ This function is the actual implementation of gaussian pdf in scalar case. It assumes all args are conformant, so it should not be used directly Call gauss_den instead""" # Diagonal matrix case d = mu.size inva = 1/va[0] fac = (2*N.pi) ** (-d/2.0) * N.sqrt(inva) y = (x[:,0] - mu[0]) ** 2 * inva * -0.5 for i in range(1, d): inva = 1/va[i] fac *= N.sqrt(inva) y += (x[:,i] - mu[i]) ** 2 * inva * -0.5 y = fac * N.exp(y) def multiple_gauss_den1(data, mu, va): """Helper function to generate several Gaussian pdf (different parameters) from the same data: unoptimized version""" K = mu.shape[0] n = data.shape[0] d = data.shape[1] y = N.zeros((n, K)) for i in range(K): y[:, i] = _diag_gauss_den(data, mu[i, :], va[i, :]) return y def multiple_gauss_den2(data, mu, va): """Helper function to generate several Gaussian pdf (different parameters) from the same data: optimized version""" K = mu.shape[0] n = data.shape[0] d = data.shape[1] y = N.zeros((K, n)) for i in range(K): y[i] = _diag_gauss_den(data, mu[i, :], va[i, :]) return y.T def bench(): #=========================================== # GMM of 30 comp, 15 dimension, 1e4 frames #=========================================== d = 15 k = 30 nframes = 1e4 niter = 10 mode = 'diag' mu = randn(k, d) va = randn(k, d) ** 2 X = randn(nframes, d) print "=============================================================" print "(%d dim, %d components) GMM with %d iterations, for %d frames" \ % (d, k, niter, nframes) for i in range(niter): y1 = multiple_gauss_den1(X, mu, va) for i in range(niter): y2 = multiple_gauss_den2(X, mu, va) se = N.sum(y1-y2) print se if __name__ == '__main__': import hotshot, hotshot.stats profile_file = 'foo.prof' prof = hotshot.Profile(profile_file, lineevents=1) prof.runcall(bench) p = hotshot.stats.load(profile_file) print p.sort_stats('cumulative').print_stats(20) prof.close() I am a bit puzzled by all those C vs F storage, though. In Matlab, where the storage was always F as far as I know, I have never encountered such differences (eg between y(:, i) and y(:, i)); I don't know if this is because I am doing it badly, or because matlab is much more clever than numpy at handling those cases, or if that is the price to pay for the added flexibility of numpy... David From david at ar.media.kyoto-u.ac.jp Wed Oct 4 07:13:22 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 04 Oct 2006 20:13:22 +0900 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: <452393D1.2020308@ar.media.kyoto-u.ac.jp> References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> <45228EAA.9040301@ieee.org> <452393D1.2020308@ar.media.kyoto-u.ac.jp> Message-ID: <45239752.2030401@ar.media.kyoto-u.ac.jp> David Cournapeau wrote: > Here we are (the difference may not seem that much between the two > multiple_ga, but in reality, _diag_gauss_den is an internal function > which is done in C, and is much faster... By writing this example, I've > just realized that the function _diag_gauss_den may be slow for exactly > the same reasons): > > I checked my assumption about the _diag_gauss_den function being slow because of the same problem, and this is indeed the case: If I replace X = randn(nframes, d) by X = randn(d, nframes); X = X.T, the function is mode than twice as fast ! This seems way to much of a difference to me.... David From ivilata at carabos.com Wed Oct 4 07:23:52 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Wed, 04 Oct 2006 13:23:52 +0200 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays Message-ID: <452399C8.7050805@carabos.com> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or so, but I have come across some alignment or striding problems which can be seen with the following code:: import numpy import numexpr array_length = 10 array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] array = numpy.empty((array_length,), dtype=array_descr) for i in xrange(array_length): array['c1'][i] = i array['c2'][i] = 0xaaaa print numexpr.evaluate('c1', {'c1': array['c1']}) print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 (unmodified) this gives the following result:: [ 0 109226 -1431699456 2 240298 -1431699456 4 371370 8 633514] [0 1 2 3 4 5 6 7 8 9] The test works right when ``evaluate()`` is used with 'c2' instead of 'c1', and also when 'c2' also measures 32 bits and fields are aligned. Maybe the ``memsteps`` value is not getting used somewhere. Any ideas on this? :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: From bsouthey at gmail.com Wed Oct 4 09:26:53 2006 From: bsouthey at gmail.com (Bruce Southey) Date: Wed, 4 Oct 2006 08:26:53 -0500 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <5eec5f300610021500g6034c106h11f4c32d77e642ac@mail.gmail.com> References: <5eec5f300610021500g6034c106h11f4c32d77e642ac@mail.gmail.com> Message-ID: Hi, On 10/2/06, Albert Strasheim wrote: > I recently started looking at David Cournapeau's PyEM package, > specifically his implementation of the K-Means algorithm. He > implemented part of this algorithm with in pure Python version and > also provided a Pyrex alternative that is significantly faster (about > 10 times with the data I tested with). I tried to figure out why this > is so. Just of of curosity, have you seen the implementation at BioPython (http://biopython.org)? Michiel de Hoon and other wrote 'The C Clustering Library', see the following document for more details (linked from documentation page - http://biopython.org/wiki/Documentation): http://biopython.org/DIST/docs/cluster/cluster.pdf Regards Bruce From bsouthey at gmail.com Wed Oct 4 11:36:09 2006 From: bsouthey at gmail.com (Bruce Southey) Date: Wed, 4 Oct 2006 10:36:09 -0500 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: <452393D1.2020308@ar.media.kyoto-u.ac.jp> References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> <45228EAA.9040301@ieee.org> <452393D1.2020308@ar.media.kyoto-u.ac.jp> Message-ID: Hi, I think what you are after is the multivariate normal distribution. Assuming I have it correctly, it is clearer to see (and probably more accurate to compute) in the log form as: -(N/2)*log(2*PI) - 0.5*log(determinant of V) - 0.5*(transpose of (x-mu))*inverse(V)*(x-mu) where N is the number of observations, PI is math constant, V is the known variance co-variance matrix, x is vector of values, mu is the known mean. If so, then you can vectorize the density calculation. Of course if V has a simple structure (it is unclear from your code on this) and can be factored out, then the only 'hard' part to compute is the product of the vector transpose with the vector. If mu and va are scalars for each calculation, then you can factor them out for so the only multidimensional calculations are squares and sums of each frame. Regards Bruce On 10/4/06, David Cournapeau wrote: > Tim Hochberg wrote: > >> > >> > >> I guess the problem is coming from the fact that y being C order, y[:, > >> i] needs accessing data in a non 'linear' way. Is there a way to speed > >> this up ? I did something like this: > >> > >> y = N.zeros((K, n)) > >> for i in range(K): > >> y[i] = gauss_den(data, mu[i, :], va[i, :]) > >> return y.T > >> > >> which works, but I don't like it very much. > > Why not? > > > Mainly because using those transpose do not really reflect the > intention, and this does not seem natural. > >> Isn't there any other way > > That depends on the details of gauss_den. > > > > A general note: for this kind of microoptimization puzzle, it's much > > easier to help if you can post a self contained example, preferably > > something fairly simple that still illustrates the speed issue, that we > > can experiment with. > > > Here we are (the difference may not seem that much between the two > multiple_ga, but in reality, _diag_gauss_den is an internal function > which is done in C, and is much faster... By writing this example, I've > just realized that the function _diag_gauss_den may be slow for exactly > the same reasons): > > > #! /usr/bin/env python > # Last Change: Wed Oct 04 07:00 PM 2006 J > > import numpy as N > from numpy.random import randn > > def _diag_gauss_den(x, mu, va): > """ This function is the actual implementation > of gaussian pdf in scalar case. It assumes all args > are conformant, so it should not be used directly > > Call gauss_den instead""" > # Diagonal matrix case > d = mu.size > inva = 1/va[0] > fac = (2*N.pi) ** (-d/2.0) * N.sqrt(inva) > y = (x[:,0] - mu[0]) ** 2 * inva * -0.5 > for i in range(1, d): > inva = 1/va[i] > fac *= N.sqrt(inva) > y += (x[:,i] - mu[i]) ** 2 * inva * -0.5 > y = fac * N.exp(y) > > def multiple_gauss_den1(data, mu, va): > """Helper function to generate several Gaussian > pdf (different parameters) from the same data: unoptimized version""" > K = mu.shape[0] > n = data.shape[0] > d = data.shape[1] > > y = N.zeros((n, K)) > for i in range(K): > y[:, i] = _diag_gauss_den(data, mu[i, :], va[i, :]) > > return y > > def multiple_gauss_den2(data, mu, va): > """Helper function to generate several Gaussian > pdf (different parameters) from the same data: optimized version""" > K = mu.shape[0] > n = data.shape[0] > d = data.shape[1] > > y = N.zeros((K, n)) > for i in range(K): > y[i] = _diag_gauss_den(data, mu[i, :], va[i, :]) > > return y.T > > def bench(): > #=========================================== > # GMM of 30 comp, 15 dimension, 1e4 frames > #=========================================== > d = 15 > k = 30 > nframes = 1e4 > niter = 10 > mode = 'diag' > > mu = randn(k, d) > va = randn(k, d) ** 2 > X = randn(nframes, d) > > print "=============================================================" > print "(%d dim, %d components) GMM with %d iterations, for %d frames" \ > % (d, k, niter, nframes) > > for i in range(niter): > y1 = multiple_gauss_den1(X, mu, va) > > for i in range(niter): > y2 = multiple_gauss_den2(X, mu, va) > > se = N.sum(y1-y2) > > print se > > if __name__ == '__main__': > import hotshot, hotshot.stats > profile_file = 'foo.prof' > prof = hotshot.Profile(profile_file, lineevents=1) > prof.runcall(bench) > p = hotshot.stats.load(profile_file) > print p.sort_stats('cumulative').print_stats(20) > prof.close() > > I am a bit puzzled by all those C vs F storage, though. In Matlab, where > the storage was always F as far as I know, I have never encountered such > differences (eg between y(:, i) and y(:, i)); I don't know if this is > because I am doing it badly, or because matlab is much more clever than > numpy at handling those cases, or if that is the price to pay for the > added flexibility of numpy... > > David > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From tim.hochberg at ieee.org Wed Oct 4 12:32:24 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 04 Oct 2006 09:32:24 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <452399C8.7050805@carabos.com> References: <452399C8.7050805@carabos.com> Message-ID: <4523E218.1030204@ieee.org> Ivan Vilata i Balaguer wrote: > It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > so, but I have come across some alignment or striding problems which can > be seen with the following code:: > > import numpy > import numexpr > > array_length = 10 > array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] > > array = numpy.empty((array_length,), dtype=array_descr) > for i in xrange(array_length): > array['c1'][i] = i > array['c2'][i] = 0xaaaa > > print numexpr.evaluate('c1', {'c1': array['c1']}) > print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) > > Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 > (unmodified) this gives the following result:: > > [ 0 109226 -1431699456 2 240298 -1431699456 > 4 371370 8 633514] > [0 1 2 3 4 5 6 7 8 9] > > The test works right when ``evaluate()`` is used with 'c2' instead of > 'c1', and also when 'c2' also measures 32 bits and fields are aligned. > Maybe the ``memsteps`` value is not getting used somewhere. Any ideas > on this? > I suspect that there are some assumptions that the element separation is an integral multiple of the element size. I certainly didn't have record arrays in mind when I was working on the striding stuff, so it wouldn't surprise me. This should be fixed: preferably to do the right thing and at a minimum to cleanly raise an exception rather than spitting out garbage. I don't know that I'll have time to mess with it soon though. -tim From torgil.svensson at gmail.com Wed Oct 4 12:40:01 2006 From: torgil.svensson at gmail.com (Torgil Svensson) Date: Wed, 4 Oct 2006 18:40:01 +0200 Subject: [Numpy-discussion] Experience with Visit? In-Reply-To: References: Message-ID: Sourceforge mailing list doesn't like me. Here's another try. I've managed to use visit through it's cli and giving it data through pyvtk so that this code if __name__=='__main__': from numpy import * from numpy.random import randn n=10000 t=linspace(0,1,n)+randn(n)*0.01 a=t+t*(1+randn(n)*0.1) phi=randn(n)*2*pi*0.05 scatter4( a*cos(2.0*2*pi*t+phi) , a*sin(1.5*2*pi*t+phi) , t ,1.0/sqrt(1+t**2) ) gives this result: http://www.torgil.org/visit/scatter4.png You can grab the source using git: (see http://git.or.cz/ for details) git clone http://torgil.org/pyvisit.git I'm using the pty module in python to communicate with the visit cli which unfortunately is only available on Unix. If you do any updates, like an usefule API or windows support, please share (that's why I put it in git due to it's excellent distributed development capabilities). This was more of a proof of concept for me. > On 9/15/06, Rob Hetland wrote: > > > > I have looked at it, and I even know somebody who is using python to > > put in a data reader for FVCOM (a finite volume ocean model). He has > > a prototype working. I have no experience with the python bindings > > personally. > > > > I looked into compiling it from source (for an intel mac), but it was > > daunting so I gave up. > > > > I also contacted them to try to put in lower level python bindings, > > so that we could create 3D graphics from the command line, but they > > didn't seem too interested in that. > > > > Otherwise, it is a very nice workable app. > > > > -Rob > > > > > > On Sep 15, 2006, at 11:23 AM, O'Keefe, Michael wrote: > > > > > I haven't tried VisIT before but thanks for the link. I also > > > downloaded and am checking it out. > > > > > > Along this same line of discussion, has anyone tried OOF2 which is > > > an FEA package that also has some strong python connections? > > > > > > http://www.ctcms.nist.gov/oof/oof2.html > > > > > > I'm working on a Windows machine and it the current code-base > > > doesn't seem to support Windows out of the box (if at all). Looks > > > like you can put it together for *nix or Mac OS X, though... > > > > > > --Michael > > > > > >> -----Original Message----- > > >> From: numpy-discussion-bounces at lists.sourceforge.net > > >> [mailto:numpy-discussion-bounces at lists.sourceforge.net] On > > >> Behalf Of Robert Cimrman > > >> Sent: Friday, September 15, 2006 8:29 > > >> To: Discussion of Numerical Python > > >> Subject: Re: [Numpy-discussion] Experience with Visit? > > >> > > >> Travis Oliphant wrote: > > >>> Has anybody had any experience with the 3-D visualization software > > >>> VISIT? It has Python bindings and seems to be pretty > > >> sophisticated. > > >>> I'm wondering why I haven't heard more about it. > > >>> > > >>> http://www.llnl.gov/visit/ > > >> > > >> No reaction up to now, so... > > >> > > >> I have just tried the 'getting started' part and was quite impressed, > > >> thanks for posting the link! Up to now I have used ParaView > > >> and was very > > >> satisfied, but the Python bindings of VisIt are a great lure. > > >> > > >> r. > > >> > > >> -------------------------------------------------------------- > > >> ----------- > > >> Using Tomcat but need to do more? Need to support web > > >> services, security? > > >> Get stuff done quickly with pre-integrated technology to make > > >> your job easier > > >> Download IBM WebSphere Application Server v.1.0.1 based on > > >> Apache Geronimo > > >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& > > >> dat=121642 > > >> _______________________________________________ > > >> Numpy-discussion mailing list > > >> Numpy-discussion at lists.sourceforge.net > > >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > >> > > > > > > ---------------------------------------------------------------------- > > > --- > > > Using Tomcat but need to do more? Need to support web services, > > > security? > > > Get stuff done quickly with pre-integrated technology to make your > > > job easier > > > Download IBM WebSphere Application Server v.1.0.1 based on Apache > > > Geronimo > > > http://sel.as-us.falkag.net/sel? > > > cmd=lnk&kid=120709&bid=263057&dat=121642 > > > _______________________________________________ > > > Numpy-discussion mailing list > > > Numpy-discussion at lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ---- > > Rob Hetland, Associate Professor > > Dept. of Oceanography, Texas A&M University > > http://pong.tamu.edu/~rob > > phone: 979-458-0096, fax: 979-845-6331 > > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, security? > > Get stuff done quickly with pre-integrated technology to make your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > From haase at msg.ucsf.edu Wed Oct 4 12:47:52 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Wed, 4 Oct 2006 09:47:52 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523E218.1030204@ieee.org> References: <452399C8.7050805@carabos.com> <4523E218.1030204@ieee.org> Message-ID: <200610040947.52533.haase@msg.ucsf.edu> Quick question hopefully somewhat related to this: Does numexpr fully support float32 arrays ? -Sebastian On Wednesday 04 October 2006 09:32, Tim Hochberg wrote: > Ivan Vilata i Balaguer wrote: > > It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > > so, but I have come across some alignment or striding problems which can > > be seen with the following code:: > > > > import numpy > > import numexpr > > > > array_length = 10 > > array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] > > > > array = numpy.empty((array_length,), dtype=array_descr) > > for i in xrange(array_length): > > array['c1'][i] = i > > array['c2'][i] = 0xaaaa > > > > print numexpr.evaluate('c1', {'c1': array['c1']}) > > print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) > > > > Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 > > (unmodified) this gives the following result:: > > > > [ 0 109226 -1431699456 2 240298 -1431699456 > > 4 371370 8 633514] > > [0 1 2 3 4 5 6 7 8 9] > > > > The test works right when ``evaluate()`` is used with 'c2' instead of > > 'c1', and also when 'c2' also measures 32 bits and fields are aligned. > > Maybe the ``memsteps`` value is not getting used somewhere. Any ideas > > on this? > > I suspect that there are some assumptions that the element separation > is an integral multiple of the element size. I certainly didn't have > record arrays in mind when I was working on the striding stuff, so it > wouldn't surprise me. This should be fixed: preferably to do the right > thing and at a minimum to cleanly raise an exception rather than > spitting out garbage. I don't know that I'll have time to mess with it > soon though. > > -tim > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your opinions on IT & business topics through brief surveys -- and earn > cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Wed Oct 4 13:13:10 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 04 Oct 2006 10:13:10 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <200610040947.52533.haase@msg.ucsf.edu> References: <452399C8.7050805@carabos.com> <4523E218.1030204@ieee.org> <200610040947.52533.haase@msg.ucsf.edu> Message-ID: <4523EBA6.8070107@ieee.org> Sebastian Haase wrote: > Quick question hopefully somewhat related to this: > Does numexpr fully support float32 arrays ? > I don't recall. At one point there was a tentative plan to support float32 by casting them a block at a time to float64, operating on them and them casting them back. That's to limit the number of bytecodes that we need to support and keep the switch statement at a manageable size. However, it doesn't look like that ever got implemented, so the answer is probably no. -tim > -Sebastian > > > On Wednesday 04 October 2006 09:32, Tim Hochberg wrote: > >> Ivan Vilata i Balaguer wrote: >> >>> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >>> so, but I have come across some alignment or striding problems which can >>> be seen with the following code:: >>> >>> import numpy >>> import numexpr >>> >>> array_length = 10 >>> array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] >>> >>> array = numpy.empty((array_length,), dtype=array_descr) >>> for i in xrange(array_length): >>> array['c1'][i] = i >>> array['c2'][i] = 0xaaaa >>> >>> print numexpr.evaluate('c1', {'c1': array['c1']}) >>> print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) >>> >>> Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 >>> (unmodified) this gives the following result:: >>> >>> [ 0 109226 -1431699456 2 240298 -1431699456 >>> 4 371370 8 633514] >>> [0 1 2 3 4 5 6 7 8 9] >>> >>> The test works right when ``evaluate()`` is used with 'c2' instead of >>> 'c1', and also when 'c2' also measures 32 bits and fields are aligned. >>> Maybe the ``memsteps`` value is not getting used somewhere. Any ideas >>> on this? >>> >> I suspect that there are some assumptions that the element separation >> is an integral multiple of the element size. I certainly didn't have >> record arrays in mind when I was working on the striding stuff, so it >> wouldn't surprise me. This should be fixed: preferably to do the right >> thing and at a minimum to cleanly raise an exception rather than >> spitting out garbage. I don't know that I'll have time to mess with it >> soon though. >> >> -tim >> >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share >> your opinions on IT & business topics through brief surveys -- and earn >> cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > From tim.hochberg at ieee.org Wed Oct 4 13:19:08 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 04 Oct 2006 10:19:08 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <452399C8.7050805@carabos.com> References: <452399C8.7050805@carabos.com> Message-ID: <4523ED0C.4070306@ieee.org> Ivan Vilata i Balaguer wrote: > It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > so, but I have come across some alignment or striding problems which can > be seen with the following code:: I looked at this just a little bit and clearly this bit from interp_body cannot work in the presence of recor arrays: //.... intp sf1 = sb1 / sizeof(double); \ //... #define f1 ((double *)x1)[j*sf1] There are clearly some assumptions that sb1 is evenly divisible by sizeof(double). Blech!. This is likely my fault, and I expect it won't be too horrible to fix, but I don't know that I'll have time immediately. -tim From pfdubois at gmail.com Wed Oct 4 13:19:53 2006 From: pfdubois at gmail.com (Paul Dubois) Date: Wed, 4 Oct 2006 10:19:53 -0700 Subject: [Numpy-discussion] Experience with Visit? In-Reply-To: References: Message-ID: Sorry I didn't post earlier. VisIt is used extensively at LLNL for our 3D apps. I will try to get someone who uses it to comment. On 04 Oct 2006 09:42:20 -0700, Torgil Svensson wrote: > > Sourceforge mailing list doesn't like me. Here's another try. > > I've managed to use visit through it's cli and giving it data through > pyvtk so that this code > > if __name__=='__main__': > from numpy import * > from numpy.random import randn > n=10000 > t=linspace(0,1,n)+randn(n)*0.01 > a=t+t*(1+randn(n)*0.1) > phi=randn(n)*2*pi*0.05 > scatter4( a*cos(2.0*2*pi*t+phi) , a*sin(1.5*2*pi*t+phi) , t > ,1.0/sqrt(1+t**2) ) > > gives this result: > http://www.torgil.org/visit/scatter4.png > > You can grab the source using git: (see http://git.or.cz/ for details) > git clone http://torgil.org/pyvisit.git > > I'm using the pty module in python to communicate with the visit cli > which unfortunately is only available on Unix. If you do any updates, > like an usefule API or windows support, please share (that's why I put > it in git due to it's excellent distributed development capabilities). > This was more of a proof of concept for me. > > > On 9/15/06, Rob Hetland wrote: > > > > > > I have looked at it, and I even know somebody who is using python to > > > put in a data reader for FVCOM (a finite volume ocean model). He has > > > a prototype working. I have no experience with the python bindings > > > personally. > > > > > > I looked into compiling it from source (for an intel mac), but it was > > > daunting so I gave up. > > > > > > I also contacted them to try to put in lower level python bindings, > > > so that we could create 3D graphics from the command line, but they > > > didn't seem too interested in that. > > > > > > Otherwise, it is a very nice workable app. > > > > > > -Rob > > > > > > > > > On Sep 15, 2006, at 11:23 AM, O'Keefe, Michael wrote: > > > > > > > I haven't tried VisIT before but thanks for the link. I also > > > > downloaded and am checking it out. > > > > > > > > Along this same line of discussion, has anyone tried OOF2 which is > > > > an FEA package that also has some strong python connections? > > > > > > > > http://www.ctcms.nist.gov/oof/oof2.html > > > > > > > > I'm working on a Windows machine and it the current code-base > > > > doesn't seem to support Windows out of the box (if at all). Looks > > > > like you can put it together for *nix or Mac OS X, though... > > > > > > > > --Michael > > > > > > > >> -----Original Message----- > > > >> From: numpy-discussion-bounces at lists.sourceforge.net > > > >> [mailto:numpy-discussion-bounces at lists.sourceforge.net] On > > > >> Behalf Of Robert Cimrman > > > >> Sent: Friday, September 15, 2006 8:29 > > > >> To: Discussion of Numerical Python > > > >> Subject: Re: [Numpy-discussion] Experience with Visit? > > > >> > > > >> Travis Oliphant wrote: > > > >>> Has anybody had any experience with the 3-D visualization software > > > >>> VISIT? It has Python bindings and seems to be pretty > > > >> sophisticated. > > > >>> I'm wondering why I haven't heard more about it. > > > >>> > > > >>> http://www.llnl.gov/visit/ > > > >> > > > >> No reaction up to now, so... > > > >> > > > >> I have just tried the 'getting started' part and was quite > impressed, > > > >> thanks for posting the link! Up to now I have used ParaView > > > >> and was very > > > >> satisfied, but the Python bindings of VisIt are a great lure. > > > >> > > > >> r. > > > >> > > > >> -------------------------------------------------------------- > > > >> ----------- > > > >> Using Tomcat but need to do more? Need to support web > > > >> services, security? > > > >> Get stuff done quickly with pre-integrated technology to make > > > >> your job easier > > > >> Download IBM WebSphere Application Server v.1.0.1 based on > > > >> Apache Geronimo > > > >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& > > > >> dat=121642 > > > >> _______________________________________________ > > > >> Numpy-discussion mailing list > > > >> Numpy-discussion at lists.sourceforge.net > > > >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > >> > > > > > > > > > ---------------------------------------------------------------------- > > > > --- > > > > Using Tomcat but need to do more? Need to support web services, > > > > security? > > > > Get stuff done quickly with pre-integrated technology to make your > > > > job easier > > > > Download IBM WebSphere Application Server v.1.0.1 based on Apache > > > > Geronimo > > > > http://sel.as-us.falkag.net/sel? > > > > cmd=lnk&kid=120709&bid=263057&dat=121642 > > > > _______________________________________________ > > > > Numpy-discussion mailing list > > > > Numpy-discussion at lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > ---- > > > Rob Hetland, Associate Professor > > > Dept. of Oceanography, Texas A&M University > > > http://pong.tamu.edu/~rob > > > phone: 979-458-0096, fax: 979-845-6331 > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > Using Tomcat but need to do more? Need to support web services, > security? > > > Get stuff done quickly with pre-integrated technology to make your job > easier > > > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > > _______________________________________________ > > > Numpy-discussion mailing list > > > Numpy-discussion at lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From haase at msg.ucsf.edu Wed Oct 4 13:23:25 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Wed, 4 Oct 2006 10:23:25 -0700 Subject: [Numpy-discussion] =?iso-8859-1?q?Problems_with_Numexpr_and_disco?= =?iso-8859-1?q?ntiguous=09arrays?= In-Reply-To: <4523EBA6.8070107@ieee.org> References: <452399C8.7050805@carabos.com> <200610040947.52533.haase@msg.ucsf.edu> <4523EBA6.8070107@ieee.org> Message-ID: <200610041023.25343.haase@msg.ucsf.edu> On Wednesday 04 October 2006 10:13, Tim Hochberg wrote: > Sebastian Haase wrote: > > Quick question hopefully somewhat related to this: > > Does numexpr fully support float32 arrays ? > > I don't recall. At one point there was a tentative plan to support > float32 by casting them a block at a time to float64, operating on them > and them casting them back. That's to limit the number of bytecodes that > we need to support and keep the switch statement at a manageable size. > However, it doesn't look like that ever got implemented, so the answer > is probably no. > > -tim Does that mean its considered "impratical" to ever add native float32 support ? Is the switch-statement you mention written by hand or is that automatically generated ? -Sebastian > > > -Sebastian > > > > On Wednesday 04 October 2006 09:32, Tim Hochberg wrote: > >> Ivan Vilata i Balaguer wrote: > >>> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > >>> so, but I have come across some alignment or striding problems which > >>> can be seen with the following code:: > >>> > >>> import numpy > >>> import numexpr > >>> > >>> array_length = 10 > >>> array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] > >>> > >>> array = numpy.empty((array_length,), dtype=array_descr) > >>> for i in xrange(array_length): > >>> array['c1'][i] = i > >>> array['c2'][i] = 0xaaaa > >>> > >>> print numexpr.evaluate('c1', {'c1': array['c1']}) > >>> print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) > >>> > >>> Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 > >>> (unmodified) this gives the following result:: > >>> > >>> [ 0 109226 -1431699456 2 240298 > >>> -1431699456 4 371370 8 633514] > >>> [0 1 2 3 4 5 6 7 8 9] > >>> > >>> The test works right when ``evaluate()`` is used with 'c2' instead of > >>> 'c1', and also when 'c2' also measures 32 bits and fields are aligned. > >>> Maybe the ``memsteps`` value is not getting used somewhere. Any ideas > >>> on this? > >> > >> I suspect that there are some assumptions that the element separation > >> is an integral multiple of the element size. I certainly didn't have > >> record arrays in mind when I was working on the striding stuff, so it > >> wouldn't surprise me. This should be fixed: preferably to do the right > >> thing and at a minimum to cleanly raise an exception rather than > >> spitting out garbage. I don't know that I'll have time to mess with it > >> soon though. > >> > >> -tim > >> > >> > >> ------------------------------------------------------------------------ > >>- Take Surveys. Earn Cash. Influence the Future of IT > >> Join SourceForge.net's Techsay panel and you'll get the chance to share > >> your opinions on IT & business topics through brief surveys -- and earn > >> cash > >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE > >>V _______________________________________________ > >> Numpy-discussion mailing list > >> Numpy-discussion at lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share > > your opinions on IT & business topics through brief surveys -- and earn > > cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your opinions on IT & business topics through brief surveys -- and earn > cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cookedm at physics.mcmaster.ca Wed Oct 4 13:27:36 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Wed, 4 Oct 2006 13:27:36 -0400 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523ED0C.4070306@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> Message-ID: <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> On Wed, 04 Oct 2006 10:19:08 -0700 Tim Hochberg wrote: > Ivan Vilata i Balaguer wrote: > > It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > > so, but I have come across some alignment or striding problems which can > > be seen with the following code:: > I looked at this just a little bit and clearly this bit from interp_body > cannot work in the presence of recor arrays: > > //.... > intp sf1 = sb1 / sizeof(double); \ > //... > #define f1 ((double *)x1)[j*sf1] > > > There are clearly some assumptions that sb1 is evenly divisible by > sizeof(double). Blech!. This is likely my fault, and I expect it won't > be too horrible to fix, but I don't know that I'll have time immediately. My thinking is that this should be handled by a copy, so that the opcodes always work on contiguous data. The copy can be another opcode. One advantage of operating on contiguous data is that it's easier to use the processor's vector instructions, if applicable. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From tim.hochberg at ieee.org Wed Oct 4 13:42:33 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 04 Oct 2006 10:42:33 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> Message-ID: <4523F289.901@ieee.org> David M. Cooke wrote: > On Wed, 04 Oct 2006 10:19:08 -0700 > Tim Hochberg wrote: > > >> Ivan Vilata i Balaguer wrote: >> >>> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >>> so, but I have come across some alignment or striding problems which can >>> be seen with the following code:: >>> >> I looked at this just a little bit and clearly this bit from interp_body >> cannot work in the presence of recor arrays: >> >> //.... >> intp sf1 = sb1 / sizeof(double); \ >> //... >> #define f1 ((double *)x1)[j*sf1] >> >> >> There are clearly some assumptions that sb1 is evenly divisible by >> sizeof(double). Blech!. This is likely my fault, and I expect it won't >> be too horrible to fix, but I don't know that I'll have time immediately. >> > > My thinking is that this should be handled by a copy, so that the opcodes > always work on contiguous data. The copy can be another opcode. One advantage > of operating on contiguous data is that it's easier to use the processor's > vector instructions, if applicable. > That would be easy to do. Right now the opcodes should work correctly on data that is spaced in multiples of the itemsize on the last axis. Other arrays are copied (no opcode required, it's embedded at the top of interp_body lines 64-80). The record array case apparently slips through the cracks when we're checking whether an array is suitable to be used correctly (interpreter.c 1086-1103). It would certainly not be any harder to only allow contiguous arrays than to correctly deal with record arrays. Only question I have is whether the extra copy will overwhelm the savings of that operating on contiguous data gives. The thing to do is probably try it and see what happens. -tim From cookedm at physics.mcmaster.ca Wed Oct 4 13:44:53 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Wed, 4 Oct 2006 13:44:53 -0400 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <200610041023.25343.haase@msg.ucsf.edu> References: <452399C8.7050805@carabos.com> <200610040947.52533.haase@msg.ucsf.edu> <4523EBA6.8070107@ieee.org> <200610041023.25343.haase@msg.ucsf.edu> Message-ID: <20061004134453.754f5bb6@arbutus.physics.mcmaster.ca> On Wed, 4 Oct 2006 10:23:25 -0700 Sebastian Haase wrote: > On Wednesday 04 October 2006 10:13, Tim Hochberg wrote: > > Sebastian Haase wrote: > > > Quick question hopefully somewhat related to this: > > > Does numexpr fully support float32 arrays ? > > > > I don't recall. At one point there was a tentative plan to support > > float32 by casting them a block at a time to float64, operating on them > > and them casting them back. That's to limit the number of bytecodes that > > we need to support and keep the switch statement at a manageable size. > > However, it doesn't look like that ever got implemented, so the answer > > is probably no. > > > > -tim > > Does that mean its considered "impratical" to ever add native float32 > support ? Is the switch-statement you mention written by hand or is that > automatically generated ? > -Sebastian > Currently by hand. I've got a rewrite lying around that generates the C code for it using a description in Python, but I haven't finished it yet. It should make it much easier to add different types, along with different methods of calculating and switching (switch vs. gcc's label pointers, for instance). Probably, if float32 is added, there will be two (internal) implementations: one that uses float64 mainly, and coerces float32 to float64, and one that does the reverse (this would be invisible to the user, of course). The same would handle int32 and int64. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From oliphant at ee.byu.edu Wed Oct 4 14:21:43 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 04 Oct 2006 12:21:43 -0600 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523F289.901@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> Message-ID: <4523FBB7.1060500@ee.byu.edu> Tim Hochberg wrote: >David M. Cooke wrote: > > >>On Wed, 04 Oct 2006 10:19:08 -0700 >>Tim Hochberg wrote: >> >> >> >> >>>Ivan Vilata i Balaguer wrote: >>> >>> >>> >>>>It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >>>>so, but I have come across some alignment or striding problems which can >>>>be seen with the following code:: >>>> >>>> >>>> >>>I looked at this just a little bit and clearly this bit from interp_body >>>cannot work in the presence of recor arrays: >>> >>>//.... >>> intp sf1 = sb1 / sizeof(double); \ >>>//... >>> #define f1 ((double *)x1)[j*sf1] >>> >>> >>>There are clearly some assumptions that sb1 is evenly divisible by >>>sizeof(double). Blech!. This is likely my fault, and I expect it won't >>>be too horrible to fix, but I don't know that I'll have time immediately. >>> >>> >>> >>My thinking is that this should be handled by a copy, so that the opcodes >>always work on contiguous data. The copy can be another opcode. One advantage >>of operating on contiguous data is that it's easier to use the processor's >>vector instructions, if applicable. >> >> >> > >That would be easy to do. Right now the opcodes should work correctly on >data that is spaced in multiples of the itemsize on the last axis. Other >arrays are copied (no opcode required, it's embedded at the top of >interp_body lines 64-80). The record array case apparently slips through >the cracks when we're checking whether an array is suitable to be used >correctly (interpreter.c 1086-1103). It would certainly not be any >harder to only allow contiguous arrays than to correctly deal with >record arrays. Only question I have is whether the extra copy will >overwhelm the savings of that operating on contiguous data gives. > With record arrays you have to worry about alignment issues. The most complicated part of the ufunc code is to handle that. The usual approach is to copy (and possibly byte-swap at least the axis you are working on) over to a buffer (the copyswapn functions will do that using a pretty fast approach for each data-type). This is ultimately how the ufuncs work (though the buffer-size is fixed so the data is copied and operated on in chunks). -Travis From haase at msg.ucsf.edu Wed Oct 4 19:28:27 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Wed, 4 Oct 2006 16:28:27 -0700 Subject: [Numpy-discussion] OSX 10.4 vs 10.3 - long double dynlib problem Message-ID: <200610041628.27926.haase@msg.ucsf.edu> Hi, I just compiled CVS numpy on OSX 10.4 (tiger). Runs fine. I was surprised when I tried to run the same mode on 10.3 (panther?) and got a importError (dyld:...) related to missing acoshl (long double) functions: ImportError: Failure linking new module: : dyld: /Library/Frameworks/Python.framework/Versions/2.4/Resources/Python.app/Contents/MacOS/Python Undefined symbols: /Volumes/haase/PrMacNPPC/numpy/core/umath.so undefined reference to _acoshl$LDBL128 expected to be defined in /usr/lib/libmx.A.dylib /Volumes/haase/PrMacNPPC/numpy/core/umath.so undefined reference to _acosl$LDBL128 expected to be defined in /usr/lib/libmx.A.dylib /Volumes/haase/PrMacNPPC/numpy/core/umath.so undefined reference to _asinhl$LDBL128 expected to be After asking around in my lab I was told that "long double" might pose a problem when going backwards from 10.4 to 10.3. (I used gfortran gcc 4.x) Can I tell setup.py to skip "long double" support ? Thanks, Sebastian Haase From gregwillden at gmail.com Wed Oct 4 23:05:31 2006 From: gregwillden at gmail.com (Greg Willden) Date: Wed, 4 Oct 2006 22:05:31 -0500 Subject: [Numpy-discussion] Hello and my first patch Message-ID: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> Hello All, I introduced myself on the Scipy list and I have a feeling that most of the subscribers here are also on Scipy-devel. Anyway I just submitted my first patch to numpy (ticket #316). I added the blackman-harris, Nuttall and Flat Top windowing functions and added "See also" sections to the docstrings for all the window functions. I probably didn't do the patch right though because I just did an "svn diff >patch". If there is a way you'd rather see it please let me know and I'll fix it. I look forward to contributing a lot more in the future. Have a nice day. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From oliphant at ee.byu.edu Thu Oct 5 00:15:50 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 04 Oct 2006 22:15:50 -0600 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> Message-ID: <452486F6.3060508@ee.byu.edu> Greg Willden wrote: > Hello All, > I introduced myself on the Scipy list and I have a feeling that most > of the subscribers here are also on Scipy-devel. Anyway I just > submitted my first patch to numpy (ticket #316). I added the > blackman-harris, Nuttall and Flat Top windowing functions and added > "See also" sections to the docstrings for all the window functions. Great contribution. Thanks a bunch. I think this will probably go into the scipy package, though. There is already a lot of windows available in the scipy.signal.window function. The window functions that are in NumPy are there only for historical purposes only (i.e. compatibility with old MLab). On the other hand, the other thought to consider is that since we have window functions in NumPy already. Let's just move them all from scipy.signal into NumPy. -Travis From haase at msg.ucsf.edu Thu Oct 5 00:35:49 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Wed, 04 Oct 2006 21:35:49 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452486F6.3060508@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> Message-ID: <45248BA5.6010709@msg.ucsf.edu> Travis Oliphant wrote: > Greg Willden wrote: > >> Hello All, >> I introduced myself on the Scipy list and I have a feeling that most >> of the subscribers here are also on Scipy-devel. Anyway I just >> submitted my first patch to numpy (ticket #316). I added the >> blackman-harris, Nuttall and Flat Top windowing functions and added >> "See also" sections to the docstrings for all the window functions. > > Great contribution. Thanks a bunch. I think this will probably go into > the scipy package, though. There is already a lot of windows available > in the scipy.signal.window function. > > The window functions that are in NumPy are there only for historical > purposes only (i.e. compatibility with old MLab). > > On the other hand, the other thought to consider is that since we have > window functions in NumPy already. Let's just move them all from > scipy.signal into NumPy. If scipy is going to be installable as separate sub-packages maybe all window functions can be moved to scipy ;-) In other words, if the ones in numpy are there only for "historical reasons" maybe they should be cleaned out before the 1.0 release. All arguments seem similar to ndimage (which was is numarray and is now in scipy) -Sebastian > > -Travis > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 5 00:54:51 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 04 Oct 2006 22:54:51 -0600 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <45248BA5.6010709@msg.ucsf.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <45248BA5.6010709@msg.ucsf.edu> Message-ID: <4524901B.3090505@ee.byu.edu> Sebastian Haase wrote: >If scipy is going to be installable as separate sub-packages maybe >all window functions can be moved to scipy ;-) > >In other words, if the ones in numpy are there only for "historical >reasons" maybe they should be cleaned out before the 1.0 release. >All arguments seem similar to ndimage (which was is numarray and is now >in scipy) > > Not really, because these functions were in *both* Numeric and numarray. That's the trouble. And the multiple scipy packages situation needs more discussion.... We are all ears... -Travis From david at ar.media.kyoto-u.ac.jp Thu Oct 5 02:08:33 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 05 Oct 2006 15:08:33 +0900 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> <45228EAA.9040301@ieee.org> <452393D1.2020308@ar.media.kyoto-u.ac.jp> Message-ID: <4524A161.8060708@ar.media.kyoto-u.ac.jp> Bruce Southey wrote: > Hi, > I think what you are after is the multivariate normal distribution. > Indeed > Assuming I have it correctly, it is clearer to see (and probably more > accurate to compute) in the log form as: > > -(N/2)*log(2*PI) - 0.5*log(determinant of V) - 0.5*(transpose of > (x-mu))*inverse(V)*(x-mu) > > where N is the number of observations, PI is math constant, V is the > known variance co-variance matrix, x is vector of values, mu is the > known mean. > Sure, but I need the exponential form at the end (actually, in the real code, you can choose between log form and 'standard' form, but I removed this to make the example as simple as possible). > If so, then you can vectorize the density calculation. I am not sure to understand what you mean: the computation is already vectorized in _diag_gauss_den; there is no loop there, and the function expects x to be of shape (n, d), where d is the dimension and n the number of samples. The loop in multiple_gaussian in not on samples, but on densities, that is I need to compute the normal multivariate densities on the same data but with different (vector) means and (diagonal ) variances, for which I don't see any easy way to vectorize without huge memory usage (using rank 3 arrays). Anyway, the problem I try to understand here is not related to gaussian kernel computation, but rather on cost difference for accessing row and columns depending on the underlying storage (C or Fortran). Don't try to find flaws in _diag_gauss_den, as it is just a toy example to make my point clearer. David From Mailer-Daemon at lists.sourceforge.net Thu Oct 5 02:46:13 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Wed, 04 Oct 2006 23:46:13 -0700 Subject: [Numpy-discussion] Warning: message 1GUFPK-0001wW-5k delayed 72 hours Message-ID: This message was created automatically by mail delivery software. A message that you sent has not yet been delivered to one or more of its recipients after more than 72 hours on the queue on externalmx-1.sourceforge.net. The message identifier is: 1GUFPK-0001wW-5k The date of the message is: Mon, 02 Oct 2006 09:32:13 -0200 The subject of the message is: doctor, at the revolt is located treating the planet he gold; The address to which the message has not yet been delivered is: numpy-discussion at lists.sourceforge.net Delay reason: SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is No action is required on your part. Delivery attempts will continue for some time, and this warning may be repeated at intervals if the message remains undelivered. Eventually the mail delivery software will give up, and when that happens, the message will be returned to you. From ckkart at hoc.net Thu Oct 5 04:41:25 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Thu, 5 Oct 2006 08:41:25 +0000 (UTC) Subject: [Numpy-discussion] compatibility of extension modules Message-ID: Hi, i've got problems running a numpy/scipy extension module (scipy.sandbox.odr) built with cygwin/mingw32 on XP on other machines (windows 2000). I get those very informative 'windows encountered a problem' messages when calling the extension module - importing seems to work. Is there any optimization done in the build process which creates incompatibilities between different systems? And if yes, how can I turn them of? Regards, Christian From ivilata at carabos.com Thu Oct 5 04:42:05 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 05 Oct 2006 10:42:05 +0200 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523ED0C.4070306@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> Message-ID: <4524C55D.6050001@carabos.com> En/na Tim Hochberg ha escrit:: > Ivan Vilata i Balaguer wrote: >> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >> so, but I have come across some alignment or striding problems which can >> be seen with the following code:: > I looked at this just a little bit and clearly this bit from interp_body > cannot work in the presence of recor arrays: > > //.... > intp sf1 = sb1 / sizeof(double); \ > //... > #define f1 ((double *)x1)[j*sf1] > > There are clearly some assumptions that sb1 is evenly divisible by > sizeof(double). [...] I noticed something strange in those statements when implementing support for strings, and I must confess that I didn't grasp their meaning, so I implemented it a little differently for strings:: #define s1 ((char *)x1 + j*params.memsteps[arg1]) That seemed to work, but it might not be right (though I tested a bit), and certainly it may not be efficient enough. Here you have my previous patches if you want to have a look at how I (try to) do it: 1.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg01551.html 2.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg02261.html 3.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg02644.html :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: From gregwillden at gmail.com Thu Oct 5 09:24:35 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 08:24:35 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <4524901B.3090505@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <45248BA5.6010709@msg.ucsf.edu> <4524901B.3090505@ee.byu.edu> Message-ID: <903323ff0610050624n7e6949d6gb566030590fc2271@mail.gmail.com> On 10/4/06, Travis Oliphant wrote: > > Not really, because these functions were in *both* Numeric and > numarray. That's the trouble. > > And the multiple scipy packages situation needs more discussion.... We > are all ears... Well I started here because I have been using the Matplotlib package and it uses these compatibility functions. So I wanted more window functions available for calls to specgram. This does bring up a frustration for users new to scipy/numpy/matplotlib. In short, that there are three packages. I know the historical reasons why but I hope that the ultimate destination is a tight integration among these packages. I hope to help out toward that goal. Thanks Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.hochberg at ieee.org Thu Oct 5 09:24:42 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 05 Oct 2006 06:24:42 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523F289.901@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> Message-ID: <4525079A.4080801@ieee.org> Tim Hochberg wrote: > David M. Cooke wrote: >> On Wed, 04 Oct 2006 10:19:08 -0700 >> Tim Hochberg wrote: >> >> >>> Ivan Vilata i Balaguer wrote: >>> >>>> It seemed that discontiguous arrays worked OK in Numexpr since >>>> r1977 or >>>> so, but I have come across some alignment or striding problems >>>> which can >>>> be seen with the following code:: >>>> >>> I looked at this just a little bit and clearly this bit from >>> interp_body cannot work in the presence of recor arrays: >>> >>> //.... >>> intp sf1 = sb1 / sizeof(double); \ >>> //... >>> #define f1 ((double *)x1)[j*sf1] >>> >>> >>> There are clearly some assumptions that sb1 is evenly divisible by >>> sizeof(double). Blech!. This is likely my fault, and I expect it >>> won't be too horrible to fix, but I don't know that I'll have time >>> immediately. >>> >> >> My thinking is that this should be handled by a copy, so that the >> opcodes >> always work on contiguous data. The copy can be another opcode. One >> advantage >> of operating on contiguous data is that it's easier to use the >> processor's >> vector instructions, if applicable. >> > > That would be easy to do. Right now the opcodes should work correctly > on data that is spaced in multiples of the itemsize on the last axis. > Other arrays are copied (no opcode required, it's embedded at the top > of interp_body lines 64-80). The record array case apparently slips > through the cracks when we're checking whether an array is suitable to > be used correctly (interpreter.c 1086-1103). It would certainly not be > any harder to only allow contiguous arrays than to correctly deal with > record arrays. Only question I have is whether the extra copy will > overwhelm the savings of that operating on contiguous data gives. The > thing to do is probably try it and see what happens. OK, I've checked in a fix for this that makes a copy when the array is not strided in an even multiple of the itemsize. I first tried copying for all discontiguous array, but this resulted in a large speed hit for vanilla strided arrays (a=arange(10)[::2], etc.), so I was more frugal with my copying. I'm not entirely certain that I caught all of the problematic cases, so let me know if you run into any more issues like this. -tim From oliphant at ee.byu.edu Thu Oct 5 09:34:45 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 07:34:45 -0600 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45227AC6.4010801@ieee.org> References: <45227AC6.4010801@ieee.org> Message-ID: <452509F5.4050702@ee.byu.edu> Travis Oliphant wrote: >Albert Strasheim wrote: > > >>>>[1] 12.97% of function time >>>>[2] 8.65% of functiont ime >>>>[3] 62.14% of function time >>>> >>>>If statistics from elsewhere in the code would be helpful, let me >>>>know, >>>> >>>> >>>> >>>and >>> >>> >>> >>>>I'll see if I can convince Quantify to cough it up. >>>> >>>> >>>> >>>> >>>Please run the same test but using >>> >>>x1 = N.random.rand(39,2000) >>>x2 = N.random.rand(39,64,1) >>> >>>z1 = x1[:,N.newaxis,:] - x2 >>> >>> >>> >>Very similar results to what I had previously: >> >>[1] 10.88% >>[2] 7.25% >>[3] 68.25% >> >> >> >> >Thanks, > >I've got some ideas about how to speed this up by eliminating some of >the unnecessary calculations going on outside of the function loop, but >there will still be some speed issues depending on how the array is >traversed once you get above a certain size. I'm not sure there anyway >around that, ultimately, due to memory access being slow on most hardware. > > Well, I tried out my ideas and didn't get much improvement (8-10%). Then, I finally realized more fully that the slowness was due to the loop taking place over an axis which had a very large stride so that the memory access was taking a long time. Thus, instead of picking the loop axis to correspond to the axis with the longest dimension, I've picked the loop axis to be one with the smallest sum of strides. In this particular example, the speed-up is about 6-7 times... -Travis From david at ar.media.kyoto-u.ac.jp Thu Oct 5 09:37:33 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 05 Oct 2006 22:37:33 +0900 Subject: [Numpy-discussion] On loop and broadcasting (again) Message-ID: <45250A9D.5040403@ar.media.kyoto-u.ac.jp> Hi, The email from Albert made me look again on some surprising results I got a few months ago when starting my first "serious" numpy project. I noticed that when computing multivariate gaussian densities, centering the data was more expensive than everything else, including exponentiation. Now that I have some experience with numpy, and following the previous discussion, I tried the following script: import numpy from numpy.random import randn import numpy as N def center_broadcast(X, mu): return X - mu def center_broadcast2(X, mu): return N.transpose(X.T - mu.T) def center_repmat(X, mu): n = X.shape[0] return X - N.repmat(mu, n, 1) def center_outer(X, mu): n = X.shape[0] return X - N.outer(N.ones(n), mu) def center_dot(X, mu): n = X.shape[0] return X - N.dot(N.ones((n, 1)), mu[N.newaxis, :]) def center_manual(X, mu): return X - mu def bench(): n = 1e5 d = 30 niter = 10 X = randn(n, d) mu = randn(d) va = randn(d) ** 2 mur = N.repmat(mu, n, 1) for i in range(niter): y1 = center_broadcast(X, mu) y2 = center_repmat(X, mu) y3 = center_dot(X, mu) y4 = center_outer(X, mu) y5 = center_manual(X, mur) y6 = center_broadcast2(X, mur) if __name__ == '__main__': import hotshot, hotshot.stats profile_file = 'storage.prof' prof = hotshot.Profile(profile_file, lineevents=1) prof.runcall(bench) p = hotshot.stats.load(profile_file) print p.sort_stats('cumulative').print_stats(20) prof.close() If X is an array containing n samples of d dimension, and mu an array with d elements, I want to compute x - mu, for each row x of X. The idea is to compare timing of "manual broadcasting" using repmat vs broadcasting, and storage influence: - center_broadcast expect mu to be a d elements array, and uses the numpy broadcast rules - center_broadcast2 expect same args, but tranpose to force a C-like storage. - center_repmat uses repmat instead of the automatic broadcast to make mu a (n, d) array with n times the same row - center_outer and center_dot implements manually the repmat using matrix product (under matlab, this was much faster on large matrices) - center_manual expects mu to be already in a (n, d) form with n times the same row: this is to see the cost of the substraction itself. The results are the following: 10 4.204 0.420 4.204 0.420 storage_email.py:8(center_broadcast) 10 0.475 0.048 3.449 0.345 storage_email.py:18(center_outer) 10 2.964 0.296 2.964 0.296 /usr/lib/python2.4/site-packages/numpy/core/numeric.py:227(outer) 10 2.767 0.277 2.768 0.277 storage_email.py:11(center_broadcast2) 10 0.485 0.049 1.288 0.129 storage_email.py:14(center_repmat) 10 1.217 0.122 1.227 0.123 storage_email.py:22(center_dot) 11 0.883 0.080 0.883 0.080 /usr/lib/python2.4/site-packages/numpy/lib/shape_base.py:522(repmat) 10 0.457 0.046 0.457 0.046 storage_email.py:26(center_manual) 5 0.137 0.027 0.137 0.027 /usr/lib/python2.4/site-packages/numpy/core/fromnumeric.py:375(sum) 20 0.020 0.001 0.020 0.001 /usr/lib/python2.4/site-packages/numpy/core/numeric.py:527(ones) 31 0.000 0.000 0.000 0.000 /usr/lib/python2.4/site-packages/numpy/core/numeric.py:125(asarray) 10 0.000 0.000 0.000 0.000 /usr/lib/python2.4/site-packages/numpy/core/fromnumeric.py:116(transpose) As you can see, there is a quite a difference between implementations: using repmat is the fastest, with repmat taking twice as much time as the substraction itself. Replacing repmat with dot does not give any advantage (I guess the underlying implementation uses the same core C functions ?). Is it expected than automatic broadcast is so much expensive ? And why using tranpose gives a 40% speed improvement ? What makes automatic broadcast so expensive compared to repmat ? For what it worths, the substraction itself is as fast as in matlab, whereas the repmat is twice slower (this quick script is not meant as a comparison against matlab of course, but I just wanted to check how matlab behaves in those situations), David From oliphant at ee.byu.edu Thu Oct 5 09:38:34 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 07:38:34 -0600 Subject: [Numpy-discussion] On loop and broadcasting (again) In-Reply-To: <45250A9D.5040403@ar.media.kyoto-u.ac.jp> References: <45250A9D.5040403@ar.media.kyoto-u.ac.jp> Message-ID: <45250ADA.105@ee.byu.edu> David Cournapeau wrote: >Hi, > > The email from Albert made me look again on some surprising results I >got a few months ago when starting my first "serious" numpy project. I >noticed that when computing multivariate gaussian densities, centering >the data was more expensive than everything else, including >exponentiation. Now that I have some experience with numpy, and >following the previous discussion, I tried the following script: > > Try it again with the new code in SVN. -Travis From oliphant at ee.byu.edu Thu Oct 5 09:39:53 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 07:39:53 -0600 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4525079A.4080801@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> <4525079A.4080801@ieee.org> Message-ID: <45250B29.4040001@ee.byu.edu> Tim Hochberg wrote: >>> >>> >>> >>That would be easy to do. Right now the opcodes should work correctly >>on data that is spaced in multiples of the itemsize on the last axis. >>Other arrays are copied (no opcode required, it's embedded at the top >>of interp_body lines 64-80). The record array case apparently slips >>through the cracks when we're checking whether an array is suitable to >>be used correctly (interpreter.c 1086-1103). It would certainly not be >>any harder to only allow contiguous arrays than to correctly deal with >>record arrays. Only question I have is whether the extra copy will >>overwhelm the savings of that operating on contiguous data gives. The >>thing to do is probably try it and see what happens. >> >> > >OK, I've checked in a fix for this that makes a copy when the array is >not strided in an even multiple of the itemsize. I first tried copying >for all discontiguous array, but this resulted in a large speed hit for >vanilla strided arrays (a=arange(10)[::2], etc.), so I was more frugal >with my copying. I'm not entirely certain that I caught all of the >problematic cases, so let me know if you run into any more issues like this. > > > There is an ElementStrides check and similar requirement flag you can use to make sure that you have an array whose strides are multiples of it's itemsize. -Travis From gregwillden at gmail.com Thu Oct 5 10:02:25 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 09:02:25 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452486F6.3060508@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> Message-ID: <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> On 10/4/06, Travis Oliphant wrote: > > Great contribution. Thanks a bunch. I think this will probably go into > the scipy package, though. There is already a lot of windows available > in the scipy.signal.window function. > BTW. Is there some sort of clear statement about what goes in Scipy versus what goes in Numpy? It's a bit confusing for this newbie. Thanks Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 5 11:34:13 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 08:34:13 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> Message-ID: Greg Willden wrote: > On 10/4/06, *Travis Oliphant* > wrote: > > Great contribution. Thanks a bunch. I think this will probably go into > the scipy package, though. There is already a lot of windows available > in the scipy.signal.window function. > > BTW. Is there some sort of clear statement about what goes in Scipy > versus what goes in Numpy? It's a bit confusing for this newbie. IMO, I'd rather see this and similar functions go into scipy. New functions that apply semantics to arrays (in this case, treating them as time series), I think should go into scipy. New functions that treat arrays simply as arrays and are generally useful can probably go into numpy. There's some grey area in this scheme, of course. New non-uniform random number generators might need to go into numpy, for now, simply due to technical reasons. Fleshing out the masked array and matrix classes would be similar, I imagine. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From gregwillden at gmail.com Thu Oct 5 11:58:17 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 10:58:17 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> Message-ID: <903323ff0610050858l57681adbmeeb75bf874043dcf@mail.gmail.com> On 10/5/06, Robert Kern wrote: > > IMO, I'd rather see this and similar functions go into scipy. New > functions that > apply semantics to arrays (in this case, treating them as time series), I > think > should go into scipy. New functions that treat arrays simply as arrays and > are > generally useful can probably go into numpy. > Okay I'll take a look at the Scipy parts tonight. So do you cancel that ticket I created or do I attach a new patch against scipy or what? Thanks Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivilata at carabos.com Thu Oct 5 11:59:45 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 05 Oct 2006 17:59:45 +0200 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4525079A.4080801@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> <4525079A.4080801@ieee.org> Message-ID: <45252BF1.8070705@carabos.com> En/na Tim Hochberg ha escrit:: > Tim Hochberg wrote: >>>> Ivan Vilata i Balaguer wrote: >>>>> It seemed that discontiguous arrays worked OK in Numexpr since >>>>> r1977 or >>>>> so, but I have come across some alignment or striding problems >>>>> which can >>>>> be seen with the following code:: > OK, I've checked in a fix for this that makes a copy when the array is > not strided in an even multiple of the itemsize. I first tried copying > for all discontiguous array, but this resulted in a large speed hit for > vanilla strided arrays (a=arange(10)[::2], etc.), so I was more frugal > with my copying. I'm not entirely certain that I caught all of the > problematic cases, so let me know if you run into any more issues like this. Great! For the moment I just can say that the problems I had with that have disappeared, but I will keep you up to date if I find something else. Thank you very much! :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: From jdhunter at ace.bsd.uchicago.edu Thu Oct 5 12:33:22 2006 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Thu, 05 Oct 2006 11:33:22 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: (Robert Kern's message of "Thu, 05 Oct 2006 08:34:13 -0700") References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> Message-ID: <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> >>>>> "Robert" == Robert Kern writes: Robert> IMO, I'd rather see this and similar functions go into Robert> scipy. New functions that apply semantics to arrays (in Robert> this case, treating them as time series), I think should Robert> go into scipy. New functions that treat arrays simply as Robert> arrays and are generally useful can probably go into Robert> numpy. I prefer Perry's longstanding suggestion: things that do not add to distribution complexity should go into numpy. If it compiles as easily as numpy itself, it should go into numpy where sensible. It remains a fact of life that numpy gets a wider distribution than scipy, and some packages are hesitant to require scipy as a prereq because of the additional complexity or building fortran, etc. I would be nice to have as much as possible in the most widely distributed package IMO. JDH From tim.hochberg at ieee.org Thu Oct 5 12:52:14 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 05 Oct 2006 09:52:14 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4524C55D.6050001@carabos.com> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <4524C55D.6050001@carabos.com> Message-ID: <4525383E.2090209@ieee.org> Ivan Vilata i Balaguer wrote: > En/na Tim Hochberg ha escrit:: > > >> Ivan Vilata i Balaguer wrote: >> >>> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >>> so, but I have come across some alignment or striding problems which can >>> be seen with the following code:: >>> >> I looked at this just a little bit and clearly this bit from interp_body >> cannot work in the presence of recor arrays: >> >> //.... >> intp sf1 = sb1 / sizeof(double); \ >> //... >> #define f1 ((double *)x1)[j*sf1] >> >> There are clearly some assumptions that sb1 is evenly divisible by >> sizeof(double). [...] >> > > I noticed something strange in those statements when implementing > support for strings, and I must confess that I didn't grasp their > meaning, so I implemented it a little differently for strings:: > > #define s1 ((char *)x1 + j*params.memsteps[arg1]) > I believe that these approaches are the same as long as memstep is a multiple of itemsize. I chose the indexing rather than the pointer foo version[1] because there's a rumor that compilers will sometimes generate faster code for it. One additional potential slowdown in the above is the compiler may not be able to tell that memsteps[arg1]) is constant and may do that lookup repeatedly. Or maybe not, I try not to second guess compilers too much. -tim [1] I'm pretty sure I used the pointer foo version at least for a while. and may have gone back and forth several times. > That seemed to work, but it might not be right (though I tested a bit), > and certainly it may not be efficient enough. Here you have my previous > patches if you want to have a look at how I (try to) do it: > > 1.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg01551.html > 2.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg02261.html > 3.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg02644.html > > :: > > Ivan Vilata i Balaguer >qo< http://www.carabos.com/ > C?rabos Coop. V. V V Enjoy Data > "" > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From bhendrix at enthought.com Thu Oct 5 13:34:24 2006 From: bhendrix at enthought.com (Bryce Hendrix) Date: Thu, 05 Oct 2006 12:34:24 -0500 Subject: [Numpy-discussion] zip safe egg? Message-ID: <45254220.2010001@enthought.com> Is anyone using the numpy egg compressed? Recent versions of setuptools seem to think it is zip safe, but this causes builds to fail which compile with the numpy headers. Can we change the setup.py to default to not zip safe (by adding zip_safe=False as a keyword arg to the setup function call)? Bryce From robert.kern at gmail.com Thu Oct 5 13:41:53 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 10:41:53 -0700 Subject: [Numpy-discussion] zip safe egg? In-Reply-To: <45254220.2010001@enthought.com> References: <45254220.2010001@enthought.com> Message-ID: Bryce Hendrix wrote: > Is anyone using the numpy egg compressed? Recent versions of setuptools > seem to think it is zip safe, but this causes builds to fail which > compile with the numpy headers. Can we change the setup.py to default to > not zip safe (by adding zip_safe=False as a keyword arg to the setup > function call)? It is not zip-safe if you want to compile against the headers. That keyword can't be added to the setup() call in the trunk's setup.py because numpy cannot depend on setuptools, at the moment. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From bhendrix at enthought.com Thu Oct 5 13:52:27 2006 From: bhendrix at enthought.com (Bryce Hendrix) Date: Thu, 05 Oct 2006 12:52:27 -0500 Subject: [Numpy-discussion] zip safe egg? In-Reply-To: References: <45254220.2010001@enthought.com> Message-ID: <4525465B.9050608@enthought.com> Robert Kern wrote: > It is not zip-safe if you want to compile against the headers. That keyword > can't be added to the setup() call in the trunk's setup.py because numpy cannot > depend on setuptools, at the moment. > > Adding the keyword does not break builds not using setuptools, the build just prints a warning that its not a valid keyword. I just discovered this is also an issue for scipy eggs when building with weave. Perhaps the best solution is to complain to the setuptools list, I'm just looking for a quick fix for now. Bryce From robert.kern at gmail.com Thu Oct 5 14:25:55 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 11:25:55 -0700 Subject: [Numpy-discussion] zip safe egg? In-Reply-To: <4525465B.9050608@enthought.com> References: <45254220.2010001@enthought.com> <4525465B.9050608@enthought.com> Message-ID: Bryce Hendrix wrote: > Robert Kern wrote: >> It is not zip-safe if you want to compile against the headers. That keyword >> can't be added to the setup() call in the trunk's setup.py because numpy cannot >> depend on setuptools, at the moment. >> > Adding the keyword does not break builds not using setuptools, the build > just prints a warning that its not a valid keyword. I just discovered > this is also an issue for scipy eggs when building with weave. A warning is not really acceptable in the trunk, either. We've found that warnings during the build process tend to make people think that something went wrong. > Perhaps the best solution is to complain to the setuptools list, I'm > just looking for a quick fix for now. Patch setup.py in our build system, I would think. What revision of numpy and what version of setuptools are you using? setuptools 0.7a1 and numpy r3261 correctly recognizes numpy as not zip-safe. zip_safe flag not set; analyzing archive contents... numpy._import_tools: module references __file__ numpy._import_tools: module references __path__ numpy.version: module references __file__ numpy.core.generate_array_api: module references __file__ numpy.core.setup: module references __file__ numpy.distutils.exec_command: module references __file__ numpy.distutils.misc_util: module references __file__ numpy.distutils.system_info: module references __file__ numpy.distutils.command.build_src: module references __file__ numpy.f2py.diagnose: module references __file__ numpy.f2py.f2py2e: module references __file__ numpy.f2py.setup: module references __file__ numpy.f2py.lib.wrapper_base: module references __file__ numpy.lib.utils: module MAY be using inspect.getsource numpy.lib.utils: module MAY be using inspect.getsourcefile numpy.numarray.util: module references __file__ numpy.testing.numpytest: module references __file__ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From bhendrix at enthought.com Thu Oct 5 14:51:52 2006 From: bhendrix at enthought.com (Bryce Hendrix) Date: Thu, 05 Oct 2006 13:51:52 -0500 Subject: [Numpy-discussion] zip safe egg? In-Reply-To: References: <45254220.2010001@enthought.com> <4525465B.9050608@enthought.com> Message-ID: <45255448.9010100@enthought.com> Robert Kern wrote: >> Perhaps the best solution is to complain to the setuptools list, I'm >> just looking for a quick fix for now. >> > > Patch setup.py in our build system, I would think. > > Thats what I did, but patching working copies of files has been troublesome in the past when svn conflicts have wiggled their way into shipped builds. > What revision of numpy and what version of setuptools are you using? setuptools > 0.7a1 and numpy r3261 correctly recognizes numpy as not zip-safe. > > 0.6c3. I think 0.6c1 did _not_ tag it as zip safe either. Guess I'll just update setuptools on our build system. Bryce -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregwillden at gmail.com Thu Oct 5 15:25:16 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 14:25:16 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> On 10/5/06, John Hunter wrote: > > I prefer Perry's longstanding suggestion: things that do not add to > distribution complexity should go into numpy. If it compiles as > easily as numpy itself, it should go into numpy where sensible. It > remains a fact of life that numpy gets a wider distribution than > scipy, and some packages are hesitant to require scipy as a prereq > because of the additional complexity or building fortran, etc. I > would be nice to have as much as possible in the most widely > distributed package IMO. That is a much better policy in my view. I (gently) encourage this group (Travis?) to make this the policy for Numpy/Scipy. >From my view as a newbie to numpy/scipy/matplotlib it isn't clear where I should look for what functionality. Matplotlib plots the spectrogram but it only supports two or three window functions. Numpy supports 4 or 5 window functions and Scipy apparently supports more but Matplotlib doesn't support Scipy. Of course this is a minor example and I could just write the window function myself and then use it in Matplotlib but I want to give back so that the project can grow. I'd really like to be able to leave Matlab behind and encourage everyone else to do the same but there are still these annoyances that need to be worked out. Thanks Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 5 15:39:35 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 12:39:35 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> Message-ID: Greg Willden wrote: > From my view as a newbie to numpy/scipy/matplotlib it isn't clear where > I should look for what functionality. Matplotlib plots the spectrogram > but it only supports two or three window functions. Numpy supports 4 or > 5 window functions and Scipy apparently supports more but Matplotlib > doesn't support Scipy. That's not true. specgram() takes a windowing function. It doesn't matter where that function comes from. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From peridot.faceted at gmail.com Thu Oct 5 15:46:48 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Thu, 5 Oct 2006 15:46:48 -0400 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> Message-ID: On 05/10/06, Greg Willden wrote: > That is a much better policy in my view. > > I (gently) encourage this group (Travis?) to make this the policy for > Numpy/Scipy. > > From my view as a newbie to numpy/scipy/matplotlib it isn't clear where I > should look for what functionality. Matplotlib plots the spectrogram but it > only supports two or three window functions. Numpy supports 4 or 5 window > functions and Scipy apparently supports more but Matplotlib doesn't support > Scipy. Of course this is a minor example and I could just write the window > function myself and then use it in Matplotlib but I want to give back so > that the project can grow. I'd really like to be able to leave Matlab > behind and encourage everyone else to do the same but there are still these > annoyances that need to be worked out. Unfortunately, that policy (put it in numpy if it doesn't make the build dependencies any worse) makes it even harder for the user to figure out what is where. Say I want a fast matrix product. Do I look in numpy or scipy? It'll run faster if it uses a tuned BLAS, so it ought to have external requirements, so I'd look in scipy, but maybe there's a simple non-tuned implementation in numpy instead... Frankly, I tend to prefer the other approach to solving all these issues: distribute numpy, scipy, and matplotlib as one bundle. The requirements for scipy are not particularly onerous, particularly if it comes as part of your distribution. There are currently some problems successfully finding optimized versions of LAPACK and BLAS, but to me the benefits of bundling the packages together outweigh the difficulties: * routines and objects can be in the package in which they make the most semantic sense, rather than the one with the correct external dependencies (how is a user supposed to know whether convolution uses an external library or not?) * documentation can be cross-referenced between packages (so the Matrix class can tell people to look in scipy.linalg for inverses, for example) * users can more easily find what's available rather than rewriting from scratch * derived packages (ipython, IDEs, MATLAB-alikes) have many fewer possibilities to support I'm not arguing that the development processes need to be connected, just that making the primary distributed object a package containing all the components will make life easier for everyone involved. A. M. Archibald From oliphant at ee.byu.edu Thu Oct 5 15:53:58 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 13:53:58 -0600 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <452562D6.7010605@ee.byu.edu> John Hunter wrote: >>>>>>"Robert" == Robert Kern writes: >>>>>> >>>>>> > > Robert> IMO, I'd rather see this and similar functions go into > Robert> scipy. New functions that apply semantics to arrays (in > Robert> this case, treating them as time series), I think should > Robert> go into scipy. New functions that treat arrays simply as > Robert> arrays and are generally useful can probably go into > Robert> numpy. > >I prefer Perry's longstanding suggestion: things that do not add to >distribution complexity should go into numpy. If it compiles as >easily as numpy itself, it should go into numpy where sensible. > I don't think this is as feasible as it sounds at first. Some people complain that NumPy is too big already. SciPy is very easy to install on Windows (there is a binary available). The only major platform that still gives some trouble is Intel Mac (due to the fortran compiler situation). But, all you need is one person who can build it and distribute a binary. I think a better long-term solution is to understand how to package things better by working with people at Enthought so that when you advertise to the ex-Matlab user you point him to a "super-package" that installs a bunch of other small packages. This is a more maintainable solution as long as we set standards for 1) documentation 2) tests 3) some kind of problem-domain hierarchy The idea of just lumping more an more things into NumPy itself is not a good idea. What most users want is something that installs easily (like Enthon). How it is packaged is not as important. What developers need is a sane multi-namespace system that can be maintained separately if needed. I think we decided a while ago, that the package approach should contain indicators as to whether or not a fortran compiler was needed to build the system so that dependency on those things could be eliminated if needed. Do we want to pull scipy apart into two components: one that needs Fortran to build and another that doesn't? Perhaps that is the best way to move forward along with the work on a "pylab" super-package. -Travis From gregwillden at gmail.com Thu Oct 5 15:56:38 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 14:56:38 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> Message-ID: <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> On 10/5/06, Robert Kern wrote: > > Greg Willden wrote: > > > From my view as a newbie to numpy/scipy/matplotlib it isn't clear where > > I should look for what functionality. Matplotlib plots the spectrogram > > but it only supports two or three window functions. Numpy supports 4 or > > 5 window functions and Scipy apparently supports more but Matplotlib > > doesn't support Scipy. > > That's not true. specgram() takes a windowing function. It doesn't matter > where > that function comes from. The next sentence (that you snipped) afirms what you said. Of course this is a minor example and I could just write the window function myself and then use it in Matplotlib The details of my off-the-cuff example would probably be better addressed by the Matplotlib folks. (i.e. why they don't have builtin functions for more windows). You can see how it's all very confusing to someone new. "Matplotlib can plot a spectrogram but I need to use a window function from SciPy because Matplotlib only supports NumPy and NumPy doesn't have the one I want?" You get the idea. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregwillden at gmail.com Thu Oct 5 16:04:57 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 15:04:57 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452562D6.7010605@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> Message-ID: <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> On 10/5/06, Travis Oliphant wrote: > > Perhaps that is the best way to move forward along with the work on a > "pylab" super-package. That is exactly what I want. In the end I want a nice collection of functions, logically organized, that let me analyze/filter/plot etc. etc. etc. The key for me is "logically organized". And right now that means "So a newbie can find the function I need and the function I need is there" I'm not criticising. I'd like to help get there. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 5 16:02:52 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 13:02:52 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> Message-ID: Greg Willden wrote: > On 10/5/06, *Robert Kern* > wrote: > > Greg Willden wrote: > > > From my view as a newbie to numpy/scipy/matplotlib it isn't > clear where > > I should look for what functionality. Matplotlib plots the > spectrogram > > but it only supports two or three window functions. Numpy > supports 4 or > > 5 window functions and Scipy apparently supports more but Matplotlib > > doesn't support Scipy. > > That's not true. specgram() takes a windowing function. It doesn't > matter where > that function comes from. > > The next sentence (that you snipped) afirms what you said. > > Of course this is a minor example and I could just write the window > function myself and then use it in Matplotlib > > > The details of my off-the-cuff example would probably be better > addressed by the Matplotlib folks. (i.e. why they don't have builtin > functions for more windows). > > You can see how it's all very confusing to someone new. > "Matplotlib can plot a spectrogram but I need to use a window function > from SciPy because Matplotlib only supports NumPy and NumPy doesn't have > the one I want?" > > You get the idea. No, I'm afraid I don't. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From aisaac at american.edu Thu Oct 5 16:12:44 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 5 Oct 2006 16:12:44 -0400 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com><452486F6.3060508@ee.byu.edu><903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com><87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> Message-ID: > On 10/5/06, John Hunter wrote: >> I would be nice to have as much as possible in the most >> widely distributed package IMO. On Thu, 5 Oct 2006, Greg Willden apparently wrote: > That is a much better policy in my view. A user's perspective: Well yes, all else equal, I'd like to have as much as possible in the easiest to install package, BUT my top priority is that numpy be completely bulletproof. Do these goals conflict? Cheers, Alan Isaac From peridot.faceted at gmail.com Thu Oct 5 16:13:13 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Thu, 5 Oct 2006 16:13:13 -0400 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> Message-ID: On 05/10/06, Greg Willden wrote: > On 10/5/06, Travis Oliphant wrote: > > Perhaps that is the best way to move forward along with the work on a > > "pylab" super-package. > > That is exactly what I want. What is unsatisfactory about installing numpy+scipy+matplotlib? I've found they're generally pretty complete (except where no decent python alternative exists). > In the end I want a nice collection of functions, logically organized, that > let me analyze/filter/plot etc. etc. etc. > > The key for me is "logically organized". For the most part, the organization is pretty logical: * Basic array and matrix operations in numpy * linear algebra, differential equation, interpolation, etc. tools are in scipy, each in their own subpackage * weave is mysteriously in scipy * plotting tools are in matplotlib There are a few historical quirks, like window functions in numpy (they really belong in scipy), and some of the less-used scipy subpackages are a bit of a mess internally (scipy.interpolate for example), but otherwise I'm not sure what you want to be different. > And right now that means "So a newbie can find the function I need and the > function I need is there" > > I'm not criticising. I'd like to help get there. Install all three major packages. Use the window functions from scipy in matplotlib. Task-oriented documentation is so far a bit scant, although the scipy cookbook (http://www.scipy.org/Cookbook ) and the numpy examples list (http://www.scipy.org/Numpy_Example_List ) are a good start. A. M. Archibald From martin.wiechert at gmx.de Thu Oct 5 16:09:03 2006 From: martin.wiechert at gmx.de (Martin Wiechert) Date: Thu, 5 Oct 2006 22:09:03 +0200 Subject: [Numpy-discussion] flat indexing of object arrays Message-ID: <200610052209.04341.martin.wiechert@gmx.de> Hi list, when I try to assign a sequence as an element of an object array via flat indexing only the first element of the sequence is assigned: >>> import numpy >>> numpy.version.version '1.0rc1.dev3171' >>> from numpy import * >>> a = ndarray ((2,2), object) >>> a.flat [2] = (1, 2, 3) >>> a.flat [2] 1 >>> a array([[None, None], [1, None]], dtype=object) Is this a feature? Wouldn't a naive user like me expect a.flat [2] == (1, 2, 3)? Thanks in advance, Martin Wiechert From fullung at gmail.com Thu Oct 5 16:19:33 2006 From: fullung at gmail.com (Albert Strasheim) Date: Thu, 5 Oct 2006 22:19:33 +0200 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452562D6.7010605@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> Message-ID: <20061005201932.GA31182@dogbert.sdsl.sun.ac.za> Hello all Some comments from a Windows user's perspective. On Thu, 05 Oct 2006, Travis Oliphant wrote: > John Hunter wrote: > > >>>>>>"Robert" == Robert Kern writes: > > > > Robert> IMO, I'd rather see this and similar functions go into > > Robert> scipy. New functions that apply semantics to arrays (in > > Robert> this case, treating them as time series), I think should > > Robert> go into scipy. New functions that treat arrays simply as > > Robert> arrays and are generally useful can probably go into > > Robert> numpy. > > > >I prefer Perry's longstanding suggestion: things that do not add to > >distribution complexity should go into numpy. If it compiles as > >easily as numpy itself, it should go into numpy where sensible. > > I don't think this is as feasible as it sounds at first. Some people > complain that NumPy is too big already. I agree here. Focus NumPy on doing array library things. > SciPy is very easy to install on Windows (there is a binary available). > The only major platform that still gives some trouble is Intel Mac (due > to the fortran compiler situation). But, all you need is one person > who can build it and distribute a binary. So far, I've been shying away from SciPy, because, if I encounter a problem, I have no easy way of building from SVN on Windows. I don't think I'm the only one: few Windows users have a proper Fortran compiler. Sure, there's MinGW, but that breaks all my other tools, most notably the Visual Studio debugger and other useful things like profilers (e.g. IBM Rational Quantify). That being said, Enthought's nightly builds obviate the need of most Windows users to build from source. (Enthought rocks.) Two feature requests at this point, which would make NumPy/SciPy/Matplotlib dead easy to use on Windows, even if you're trying to stay close to trunk: 1. Please consider setting up a buildbot(*) that builds and runs the tests on every checkin. I've set up a buildbot for NumPy on my own machine; it takes a matter of minutes. Probably they already have something like this in place. (*) http://buildbot.sourceforge.net/ 2. Please consider doing separate builds per CPU with ATLAS 3.7.11, Intel MKL and ACML. By all means, make a generic build available that runs everywhere. This will require some reading of the MKL license agreement, but I think Enthought should be able to distribute Windows builds based on MKL without problems. Why go to this trouble? MATLAB R2006b uses Intel MKL on my CPU, and it is much faster than NumPy with ATLAS 3.6.0. Core Duo users also have the option of enabling OpenMP, to spread calculations to multiple cores. > I think a better long-term solution is to understand how to package > things better by working with people at Enthought so that when you > advertise to the ex-Matlab user you point him to a "super-package" that > installs a bunch of other small packages. This is a more maintainable > solution as long as we set standards for > > 1) documentation > 2) tests > 3) some kind of problem-domain hierarchy Agreed. If Enthought is willing to provide the resources, Enthon could be the perfect solution to many of the issues that we currently encounter to get decent builds on Windows. > The idea of just lumping more an more things into NumPy itself is not a > good idea. What most users want is something that installs easily (like > Enthon). How it is packaged is not as important. What developers need > is a sane multi-namespace system that can be maintained separately if > needed. > > I think we decided a while ago, that the package approach should contain > indicators as to whether or not a fortran compiler was needed to build > the system so that dependency on those things could be eliminated if > needed. > > Do we want to pull scipy apart into two components: one that needs > Fortran to build and another that doesn't? Maybe. Maybe not. On Linux this doesn't make much difference to me if I check out 3 projects or 10 -- builds are easy. On Windows, getting the support libraries, build tools and configuration right is much harder. Hard enough, that I don't think anybody wants to do it regularly. > Perhaps that is the best way to move forward along with the work on a > "pylab" super-package. Yes, please. Regards, Albert From tim.hochberg at ieee.org Thu Oct 5 16:13:53 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 05 Oct 2006 13:13:53 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <45250B29.4040001@ee.byu.edu> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> <4525079A.4080801@ieee.org> <45250B29.4040001@ee.byu.edu> Message-ID: <45256781.9090401@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > > >>>> >>>> >>>> >>>> >>> That would be easy to do. Right now the opcodes should work correctly >>> on data that is spaced in multiples of the itemsize on the last axis. >>> Other arrays are copied (no opcode required, it's embedded at the top >>> of interp_body lines 64-80). The record array case apparently slips >>> through the cracks when we're checking whether an array is suitable to >>> be used correctly (interpreter.c 1086-1103). It would certainly not be >>> any harder to only allow contiguous arrays than to correctly deal with >>> record arrays. Only question I have is whether the extra copy will >>> overwhelm the savings of that operating on contiguous data gives. The >>> thing to do is probably try it and see what happens. >>> >>> >>> >> OK, I've checked in a fix for this that makes a copy when the array is >> not strided in an even multiple of the itemsize. I first tried copying >> for all discontiguous array, but this resulted in a large speed hit for >> vanilla strided arrays (a=arange(10)[::2], etc.), so I was more frugal >> with my copying. I'm not entirely certain that I caught all of the >> problematic cases, so let me know if you run into any more issues like this. >> >> >> >> > There is an ElementStrides check and similar requirement flag you can > use to make sure that you have an array whose strides are multiples of > it's itemsize. > > Thanks Travis, I'll make a note; next time I look at this code I'll see if that can be used to simplify the code in question. -tim From Chris.Barker at noaa.gov Thu Oct 5 16:22:52 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 05 Oct 2006 13:22:52 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> Message-ID: <4525699C.1050707@noaa.gov> The situation is confusing, we all know that, and we all want to move toward a better way. Key to that is that SciPy needs to be easier to build and install -- that's happening, but I don't know that it's there yet. Maybe it can be built on Fedora Core 4 now, but last I tried it couldn't be. Anyway, few thoughts on comments made here: > Matplotlib plots the spectrogram Here's a key problem -- matplotlib includes WAY too much. There are reasons, and there is history, but as a goal, I think matplotlib should be just what it says it is -- a plotting library. I don't know that MPL has been declared the "official" plotting package for SciPy, but it would be nice if it were. SciPy has suffered for a very long time without a full-featured, cross-platform, "official" plotting package. AS far as I know, MPL comes the closest (except it doesn't do 3-d -- darn!) A. M. Archibald wrote: > Frankly, I tend to prefer the other approach to solving all these > issues: distribute numpy, scipy, and matplotlib as one bundle. This is really the only way to set things up for someone that want what could be used as a "matlab replacement". If we ever get settools working just right, we should be able to do: easy_install scipy and have it all! woo hoo! However, as we work to that goal, i do think it makes sense that numpy, and matplotlib be packages unto themselves -- developed separately, etc. In fact, SciPy should be a collection of distinct packages as well. I think there is a real benefit to be able to install just what you need. Not very user of numpy and/or MPL is looking for a full scientific software package. I'm a big advocate of people using numpy arrays for all sorts of thinks that fit well into an n-d array, that have nothing to do with Scientific programming. Matplotlib is also very useful for people who need a quick plot for a web site or something. These people don't want to install the entirety of SciPy. > * routines and objects can be in the package in which they make the > most semantic sense, exactly. If it's plotting (but not computing stuff to plot) put it in MPL, if it's generic computation (if you can't understand it with high school math, it's not generic), put it in numpy. Of course, these aren't clear definitions, but can still be useful as guidelines. > * documentation can be cross-referenced between packages (so the > Matrix class can tell people to look in scipy.linalg for inverses, for > example) If it were me, I'd probably have the Matrix package depend on a linear algebra package, and have only one of those. Travis Oliphant wrote: > 3) some kind of problem-domain hierarchy +1 > Do we want to pull scipy apart into two components: one that needs > Fortran to build and another that doesn't? yup -- I don't like it, but the state of Fortran compilers really gives little choice. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From matthew.brett at gmail.com Thu Oct 5 17:30:16 2006 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu, 5 Oct 2006 22:30:16 +0100 Subject: [Numpy-discussion] flat indexing of object arrays In-Reply-To: <200610052209.04341.martin.wiechert@gmx.de> References: <200610052209.04341.martin.wiechert@gmx.de> Message-ID: <1e2af89e0610051430t1ed6da8ayd6471313ccd6002d@mail.gmail.com> Hi, On 10/5/06, Martin Wiechert wrote: > Hi list, > > when I try to assign a sequence as an element of an object array via flat > indexing only the first element of the sequence is assigned: I've also been having trouble with flat on object arrays. Is this intended? In [1]: from numpy import * In [2]: a = arange(2) In [3]: a[1] Out[3]: 1 In [4]: a.flat[1] Out[4]: 1 In [5]: b = array([a], dtype=object) In [6]: b[1] --------------------------------------------------------------------------- exceptions.IndexError Traceback (most recent call last) /home/mb312/devel_trees/scipy/Lib/io/ IndexError: index out of bounds In [7]: b.flat[1] Out[7]: 1 Best, Matthew From oliphant at ee.byu.edu Thu Oct 5 18:11:34 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 16:11:34 -0600 Subject: [Numpy-discussion] flat indexing of object arrays In-Reply-To: <200610052209.04341.martin.wiechert@gmx.de> References: <200610052209.04341.martin.wiechert@gmx.de> Message-ID: <45258316.60803@ee.byu.edu> Martin Wiechert wrote: >Hi list, > >when I try to assign a sequence as an element of an object array via flat >indexing only the first element of the sequence is assigned: > > > >>>>import numpy >>>>numpy.version.version >>>> >>>> >'1.0rc1.dev3171' > > >>>>from numpy import * >>>>a = ndarray ((2,2), object) >>>>a.flat [2] = (1, 2, 3) >>>>a.flat [2] >>>> >>>> >1 > > >>>>a >>>> >>>> >array([[None, None], > [1, None]], dtype=object) > >Is this a feature? Wouldn't a naive user like me expect >a.flat [2] == (1, 2, 3)? > > > You are probably right. This should be changed. -Travis From oliphant at ee.byu.edu Thu Oct 5 18:19:26 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 16:19:26 -0600 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> Message-ID: <452584EE.70106@ee.byu.edu> A. M. Archibald wrote: >On 05/10/06, Greg Willden wrote: > > >>On 10/5/06, Travis Oliphant wrote: >> >> >>>Perhaps that is the best way to move forward along with the work on a >>>"pylab" super-package. >>> >>> >>That is exactly what I want. >> >> > >What is unsatisfactory about installing numpy+scipy+matplotlib? I've >found they're generally pretty complete (except where no decent python >alternative exists). > > > >>In the end I want a nice collection of functions, logically organized, that >>let me analyze/filter/plot etc. etc. etc. >> >>The key for me is "logically organized". >> >> > > > There is a structure to it, but it's more organic because of the multiple contributors. weave should be in NumPy but nobody was willing to step up to maintain it a year ago. I may be willing to step up at this point. I would like to see weave in NumPy (maybe not the blitz libraries though...) I think a hybrid for weave / f2py / ctypes that allows "inlining in multiple languages" as well as automatic extension module generation for "already-written" code is in order. -Travis From oliphant at ee.byu.edu Thu Oct 5 18:42:01 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 16:42:01 -0600 Subject: [Numpy-discussion] flat indexing of object arrays In-Reply-To: <1e2af89e0610051430t1ed6da8ayd6471313ccd6002d@mail.gmail.com> References: <200610052209.04341.martin.wiechert@gmx.de> <1e2af89e0610051430t1ed6da8ayd6471313ccd6002d@mail.gmail.com> Message-ID: <45258A39.6080201@ee.byu.edu> Matthew Brett wrote: >Hi, > >On 10/5/06, Martin Wiechert wrote: > > >>Hi list, >> >>when I try to assign a sequence as an element of an object array via flat >>indexing only the first element of the sequence is assigned: >> >> > >I've also been having trouble with flat on object arrays. > >Is this intended? > >In [1]: from numpy import * > >In [2]: a = arange(2) > >In [3]: a[1] >Out[3]: 1 > >In [4]: a.flat[1] >Out[4]: 1 > >In [5]: b = array([a], dtype=object) > >In [6]: b[1] >--------------------------------------------------------------------------- >exceptions.IndexError Traceback (most >recent call last) > >/home/mb312/devel_trees/scipy/Lib/io/ > >IndexError: index out of bounds > >In [7]: b.flat[1] >Out[7]: 1 > > > This is correct behavior. Look at the shape of b. It is being indexed correctly. The problem is that it is ambiguous as to what is wanted when you write b = array([a], dtype=object). We have gone through the rounds on this one and the current behavior is our best compromise. -Travis From peridot.faceted at gmail.com Thu Oct 5 19:02:03 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Thu, 5 Oct 2006 19:02:03 -0400 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452584EE.70106@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> <452584EE.70106@ee.byu.edu> Message-ID: On 05/10/06, Travis Oliphant wrote: > I think a hybrid for weave / f2py / ctypes that allows "inlining in > multiple languages" as well as automatic extension module generation for > "already-written" code is in order. It might make sense to also include SWIG (since that seems to be a popular choice for wrapping "already-written" C and C++ code). A. M. Archibald From vjetly at i3.com Thu Oct 5 19:27:52 2006 From: vjetly at i3.com (Vikalpa Jetly) Date: Thu, 5 Oct 2006 17:27:52 -0600 Subject: [Numpy-discussion] Memory errors In-Reply-To: Message-ID: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> I am reading a very large array (~9000,11000) of 1 byte image values. I need to change values in the array that meet a certain condition so I am running something like: b = numpy.where(a>200,0,1) to create a new array with the changed values. However, I get a "MemoryError" everytime I try this. I have over 3gb of RAM on my machine (most of which is available). The process runs fine on smaller datasets. Is there a maximum array size that numpy handles? Any alternatives/workarounds? Thanks. Vikalpa -----Original Message----- From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy-discussion-bounces at lists.sourceforge.net] On Behalf Of A. M. Archibald Sent: Thursday, October 05, 2006 5:02 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Hello and my first patch On 05/10/06, Travis Oliphant wrote: > I think a hybrid for weave / f2py / ctypes that allows "inlining in > multiple languages" as well as automatic extension module generation for > "already-written" code is in order. It might make sense to also include SWIG (since that seems to be a popular choice for wrapping "already-written" C and C++ code). A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 5 19:44:05 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 17:44:05 -0600 Subject: [Numpy-discussion] Memory errors In-Reply-To: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> References: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> Message-ID: <452598C5.3060004@ee.byu.edu> Vikalpa Jetly wrote: >I am reading a very large array (~9000,11000) of 1 byte image values. I need >to change values in the array that meet a certain condition so I am running >something like: > >b = numpy.where(a>200,0,1) > >to create a new array with the changed values. However, I get a >"MemoryError" everytime I try this. I have over 3gb of RAM on my machine >(most of which is available). The process runs fine on smaller datasets. Is >there a maximum array size that numpy handles? Any alternatives/workarounds? > > > The MemoryError is a direct result when system malloc fails. Rather than use where with two scalars (you're resulting array will be int32 and therefore 4-times larger). Use b = zeros_like(a) b[a>200] = 1 which will consume less memory. -Travis From robert.kern at gmail.com Thu Oct 5 19:46:15 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 16:46:15 -0700 Subject: [Numpy-discussion] Memory errors In-Reply-To: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> References: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> Message-ID: If you are going to start a new topic, please start a new thread as well, instead of replying to a message. Thanks. Vikalpa Jetly wrote: > I am reading a very large array (~9000,11000) of 1 byte image values. I need > to change values in the array that meet a certain condition so I am running > something like: > > b = numpy.where(a>200,0,1) > > to create a new array with the changed values. However, I get a > "MemoryError" everytime I try this. I have over 3gb of RAM on my machine > (most of which is available). The process runs fine on smaller datasets. Is > there a maximum array size that numpy handles? Any alternatives/workarounds? There is no predefined limit on the array size, just your memory. However, note that what you are doing here is creating an int32 (or int64 if you are on a 64-bit machine) array since you are using Python integers in your where() function. You can save quite a bit of memory by using the array from (a>200) and simply casting it to the uint8 type. That way, you only have two arrays in memory at any time, a and b. b = (a > 200).astype(numpy.uint8) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From wbaxter at gmail.com Thu Oct 5 21:30:43 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 6 Oct 2006 10:30:43 +0900 Subject: [Numpy-discussion] repmat Message-ID: [There seem to have been some gmail delivery problems that prevented my previous mail on this subject from being delivered] I've proposed that we fix repmat handle arbitrary dimensions before 1.0. http://projects.scipy.org/scipy/numpy/ticket/292 I don't think this is particularly controversial, just I'm guessing no-one's found the time to look at my proposed fixes. And gmail/sourceforge issues haven't helped either. In short, repmat(A, M,N) is an oddball. It only deals with 2D arrays. We should do some combination of: 1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability to it, while maintaining backwards compatibility. 2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability to it, and bring it into line with the "numpy standard" of shapes being passed as one tuple arg. 3) add a new method like reparray(A, shape) that has the multidim ability, and matches numpy conventions. 4) if 3), move repmat out of the top-level namespace to indicate deprecation The tracker item has all the code necessary to implement these fixes, it's just a matter of deciding which one to go with. My personal preference would be a combination of 1 and 3. Add reparray(A,shape) , and change repmat to repmat(A, *shape). Then just implement repmat as a call to reparray. If someone will decide which route to go, I'd be happy to provide an actual patch against the latest SVN to implement that decision. If this is somehow controversial for some reason then let's discuss it. But so far the only response has been "copying data is a bad idea", which is really a separate issue. --bb From tim.hochberg at ieee.org Thu Oct 5 21:51:50 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 05 Oct 2006 18:51:50 -0700 Subject: [Numpy-discussion] repmat In-Reply-To: References: Message-ID: <4525B6B6.7050104@ieee.org> Bill Baxter wrote: > [There seem to have been some gmail delivery problems that prevented > my previous mail on this subject from being delivered] > > I've proposed that we fix repmat handle arbitrary dimensions before 1.0. > > http://projects.scipy.org/scipy/numpy/ticket/292 > > I don't think this is particularly controversial, just I'm guessing > no-one's found the time to look at my proposed fixes. And > gmail/sourceforge issues haven't helped either. > > In short, repmat(A, M,N) is an oddball. It only deals with 2D arrays. > > We should do some combination of: > 1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability > to it, while maintaining backwards compatibility. > Doesn't repmat imply matrix and thus 2d arrays? It is pretty horrible though. > 2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability > to it, and bring it into line with the "numpy standard" of shapes > being passed as one tuple arg. > I presume shape actually means number of repetitions, and not shape. If so, shape is the wrong name for the second argument. Anyway as long as your breaking backward compatibility you might as well improve the name. > 3) add a new method like reparray(A, shape) that has the multidim > ability, and matches numpy conventions. > Same comment on the name shape. > 4) if 3), move repmat out of the top-level namespace to indicate deprecation > I'm not completely sold on the whole reparray thing: does repmat have any use outside of the linear algebra convention of squashing a 3D array into a 2D matrix (or however that goes)? If not, perhaps it should just get left alone and exciled to matrix specific functions namespace. So: +1 on deprecating, moving or otherwise getting rid of repmat. Neutral on whether to replace it with something else, but if it is replaced #3 looks like the correct route, but not with a different argument name than 'shape'. Perhaps 'reps'. -tim > The tracker item has all the code necessary to implement these fixes, > it's just a matter of deciding which one to go with. > > My personal preference would be a combination of 1 and 3. Add > reparray(A,shape) , and change repmat to repmat(A, *shape). Then just > implement repmat as a call to reparray. > > If someone will decide which route to go, I'd be happy to provide an > actual patch against the latest SVN to implement that decision. > > If this is somehow controversial for some reason then let's discuss > it. But so far the only response has been "copying data is a bad > idea", which is really a separate issue. > > --bb > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > From wbaxter at gmail.com Thu Oct 5 22:18:22 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 6 Oct 2006 11:18:22 +0900 Subject: [Numpy-discussion] repmat In-Reply-To: <4525B6B6.7050104@ieee.org> References: <4525B6B6.7050104@ieee.org> Message-ID: On 10/6/06, Tim Hochberg wrote: > Bill Baxter wrote: > > In short, repmat(A, M,N) is an oddball. It only deals with 2D arrays. > > > > We should do some combination of: > > 1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability > > to it, while maintaining backwards compatibility. > > > Doesn't repmat imply matrix and thus 2d arrays? It is pretty horrible > though. It does kind of imply that (which is why I think something like 'reparray' would be a better name) but even matlab's repmat() function doesn't restrict you to 2D arrays, and I'm pretty sure numpy.repmat was intended to be a clone of that. > > 2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability > > to it, and bring it into line with the "numpy standard" of shapes > > being passed as one tuple arg. > > > I presume shape actually means number of repetitions, and not shape. If > so, shape is the wrong name for the second argument. Anyway as long as > your breaking backward compatibility you might as well improve the name. Yeh, that did occur to me. Maybe 'reps' would be a better name for the argument. > > 3) add a new method like reparray(A, shape) that has the multidim > > ability, and matches numpy conventions. > > > Same comment on the name shape. > > > > 4) if 3), move repmat out of the top-level namespace to indicate deprecation > > > I'm not completely sold on the whole reparray thing: does repmat have > any use outside of the linear algebra convention of squashing a 3D array > into a 2D matrix (or however that goes)? If not, perhaps it should just > get left alone and exciled to matrix specific functions namespace. So: Well, I'm not sure about N-d, but I've definitely had a need for 1-d version of it. That's what prompted me to look into this. Also I think I read a comment in the other thread about pyEM that repmat() turned out to be one of the faster ways to implement something-or-other (?). At the very least you could think of tiling 3D image data as one application, in fact 'tile()' is probably a better name for the function than 'reparray'. I was initially suprised that 'repeat()' featured so prominently in numpy. I initially assumed it meant "repmat", and was surprised when it didn't do that. I thought "why on earth would you want to do that?". But it does turn out to have lots of uses (like implementing "repmat" ;-)) > +1 on deprecating, moving or otherwise getting rid of repmat. > Neutral on whether to replace it with something else, but if it is > replaced #3 looks like the correct route, but not with a different > argument name than 'shape'. Perhaps 'reps'. Hey! Looks like we agree on the name of the argument, apparently. :-) I do think there is a strong connection with "shape" though, that 'reps' hides. Maybe 'repshape' or 'tileshape'? I'd be happy with reps, though. Agreed that 'shape' is bad. Makes it sound like you should specify the actual .shape of the output. --bb From fperez.net at gmail.com Thu Oct 5 22:26:54 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 5 Oct 2006 20:26:54 -0600 Subject: [Numpy-discussion] [OT] Interesting interview with Pearu (f2py and others...) Message-ID: Hi all, since this is not about me, I think it's fair to plug an interview :) http://www.pythonthreads.com/articles/interviews/once-i-learned-about-python,-i-stopped-trying-out-different-languages.html contains a very nice interview with Pearu, where in fact I learned a number of interesting things not only about him but also about what's happening with f2py. Good job, Pearu :) Best, f From pfdubois at gmail.com Thu Oct 5 23:00:55 2006 From: pfdubois at gmail.com (Paul Dubois) Date: Thu, 5 Oct 2006 20:00:55 -0700 Subject: [Numpy-discussion] contribution about history of Numeric Message-ID: I was reading the 'History of SciPy' page and noticed the discussion about Numeric. Here's the true story about why the various names for the original: numpy, Numeric, Numerical. At the time Source Forge was pretty young, and I decided to put the project there. We all said 'numpy' informally not Numerical Python but the module name was Numeric. I created the project as numpy. I have no memory of why I didn't call it Numeric, but if it wasn't a conflict, probably I was focused on making it clear it was for Python and/or easy to type. (the FTP's etc. all had to go through a long directory path that involved the name). The documentation for the CVS stuff was confusing, and I made a mistake with my first submit of 'Numeric' (I think it was ending up with everything in Numeric/Numeric) and then discovered I could not delete it; you had to ask the Source Forge staff. Impatient, I did a second submit as Numerical. In short, all my fault, but then again, SF was so security-minded that it was hard to do anything. That's why I soon gave up on their website and hosted it at my own site for so long. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Oct 6 00:08:28 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 5 Oct 2006 22:08:28 -0600 Subject: [Numpy-discussion] contribution about history of Numeric In-Reply-To: References: Message-ID: On 10/5/06, Paul Dubois wrote: > I was reading the 'History of SciPy' page and noticed the discussion about > Numeric. Here's the true story about why the various names for the original: > numpy, Numeric, Numerical. I added this to the wiki page (http://www.scipy.org/History_of_SciPy), though I don't know how to make a block quote in wiki markup, so I left it without any visual clue. Thanks for the info! Best, f From david at ar.media.kyoto-u.ac.jp Fri Oct 6 02:04:44 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 06 Oct 2006 15:04:44 +0900 Subject: [Numpy-discussion] repmat In-Reply-To: References: <4525B6B6.7050104@ieee.org> Message-ID: <4525F1FC.8070908@ar.media.kyoto-u.ac.jp> Bill Baxter wrote: > >> I'm not completely sold on the whole reparray thing: does repmat have >> any use outside of the linear algebra convention of squashing a 3D array >> into a 2D matrix (or however that goes)? If not, perhaps it should just >> get left alone and exciled to matrix specific functions namespace. So: >> > > Well, I'm not sure about N-d, but I've definitely had a need for 1-d > version of it. That's what prompted me to look into this. Also I > think I read a comment in the other thread about pyEM that repmat() > turned out to be one of the faster ways to implement > something-or-other (?). It was in a very specific case, and it could have been replaced by an other function in this case without any loss of performance. The following is my experience with matlab/repmat: generally, repmat was necessary in many cases in matlab either for efficiency (big scalar matrix were slow to generate using ones and zeros, for example) and because matlab had almost no broadcasting rules, repmat was almost mandatory for matlab. In numpy, the syntax at least does not require repmat (at least, I never *had* to use it in numpy, and I used it a lot with matlab). Moreoever, the slowness of some broadcasting seem solved in recent subversion (see my answer on the specific thread), David > At the very least you could think of tiling > 3D image data as one application, in fact 'tile()' is probably a > better name for the function than 'reparray'. > > I was initially suprised that 'repeat()' featured so prominently in > numpy. I initially assumed it meant "repmat", and was surprised when > it didn't do that. I thought "why on earth would you want to do > that?". But it does turn out to have lots of uses (like implementing > "repmat" ;-)) > > >> +1 on deprecating, moving or otherwise getting rid of repmat. >> Neutral on whether to replace it with something else, but if it is >> replaced #3 looks like the correct route, but not with a different >> argument name than 'shape'. Perhaps 'reps'. >> > > Hey! Looks like we agree on the name of the argument, apparently. :-) > I do think there is a strong connection with "shape" though, that > 'reps' hides. Maybe 'repshape' or 'tileshape'? I'd be happy with > reps, though. Agreed that 'shape' is bad. Makes it sound like you > should specify the actual .shape of the output. > > --bb > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > From david at ar.media.kyoto-u.ac.jp Fri Oct 6 02:33:11 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 06 Oct 2006 15:33:11 +0900 Subject: [Numpy-discussion] On loop and broadcasting (again) In-Reply-To: <45250ADA.105@ee.byu.edu> References: <45250A9D.5040403@ar.media.kyoto-u.ac.jp> <45250ADA.105@ee.byu.edu> Message-ID: <4525F8A7.6030306@ar.media.kyoto-u.ac.jp> Travis Oliphant wrote: > David Cournapeau wrote: > >> Hi, >> >> The email from Albert made me look again on some surprising results I >> got a few months ago when starting my first "serious" numpy project. I >> noticed that when computing multivariate gaussian densities, centering >> the data was more expensive than everything else, including >> exponentiation. Now that I have some experience with numpy, and >> following the previous discussion, I tried the following script: >> >> > > Try it again with the new code in SVN. > It looks like your modification solved this particular issue !: 10 0.645 0.065 0.645 0.065 storage_email.py:8(center_broadcast) 10 0.625 0.062 0.625 0.062 storage_email.py:26(center_manual) 1 0.333 0.333 0.333 0.333 /usr/lib/python2.4/site-packages/numpy/lib/shape_base.py:530(repmat) Now, using broadcast is as fast as doing the substraction itself (which does not include the necessary repmat). I tried it on my laptop, where I can safely use beta code (python 2.4 + SVN numpy), which explains the timing differences compared to my previous email; as the memory is limited on the laptop, I only benchmarked the brodcasting. For smaller arrays, I couldn't see major differences in relative timing for the other implementations. Thank you very much ! David From stefan at sun.ac.za Fri Oct 6 04:46:01 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 6 Oct 2006 10:46:01 +0200 Subject: [Numpy-discussion] repmat In-Reply-To: References: Message-ID: <20061006084601.GJ20486@mentat.za.net> On Fri, Oct 06, 2006 at 10:30:43AM +0900, Bill Baxter wrote: > If this is somehow controversial for some reason then let's discuss > it. But so far the only response has been "copying data is a bad > idea", which is really a separate issue. An interesting issue, though. I've often wondered about an array view where data is visible in more than one place. For example, you have an array x = N.arange((100)) on which you'd like to apply a filter, which, say, finds the moving variance of the 5 closest elements. Here, a new representation of x would be useful: In [23]: indices = N.arange(5) + N.arange(96).reshape((96,1)) In [24]: indices Out[24]: array([[ 0, 1, 2, 3, 4], [ 1, 2, 3, 4, 5], ... [95, 96, 97, 98, 99]]) In [25]: y = x.index_view(indices) # returns array with same values as # y = x[indices] # except that no values are copied In [29]: y Out[29]: array([[ 0, 1, 2, 3, 4], [ 1, 2, 3, 4, 5], [ 2, 3, 4, 5, 6], ... after which you would be able to do In [30]: y.var(axis=1) Out[30]: array([ 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., While there would be a lot of jumping around in indices, at the loops can be done in C, which should speed up the process. Of course, with small arrays I can simply copy the data, but it becomes troublesome with images and other such large signals. I am not sure if providing the indices is the best way to initialise such an array (otherwise maybe a rule for index calculation?) or how feasible it would be to implement, so I'd be glad to hear anyone's thoughts on the topic. Regards St?fan From Mailer-Daemon at lists.sourceforge.net Fri Oct 6 04:47:03 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Fri, 06 Oct 2006 01:47:03 -0700 Subject: [Numpy-discussion] Mail delivery failed: returning message to sender Message-ID: This message was created automatically by mail delivery software. A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed: numpy-discussion at lists.sourceforge.net SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is : retry timeout exceeded ------ This is a copy of the message, including all the headers. ------ From knfpts at allwrite.be Mon Oct 2 07:32:13 2006 From: knfpts at allwrite.be (Daniel Davis) Date: Mon, 02 Oct 2006 09:32:13 -0200 Subject: doctor, at the revolt is located treating the planet he gold; Message-ID: <560583873365.2006100209321346524648@allwrite.be> HOT STOCK ALERT - THIS ONE IS STILL CLIMBING THE STOCK CHARTS ALERT -- BREAKING MARKET NEWS REPORT ---- PGCN.OB Company: Pingchuan Pharmaceutical Inc. Ticker: PGCN.OB Current Price: $.26 5-Day Target: $1.5 - $2 Rating: 10(10) Recommendation: S T R 0 N G - B U Y !! This quick rising stock is a good long term winner. This stock is going high due to superb business solutions and creative partnerships in the business world. Could this company be the next PFIZER?? Pingchaun Pharmaceuticals disributes its medications to over 50 percent of CHINA and has a tremendous presence via exports to U.S.A, Russia, Japan and South-Eastern Asia. They have partnerships with the biggest drugstores, equivalent in the US to SAV-ON, CVS, RITE-AID, etc. With BIG NEWS anticipated this coming WEEK we strongly advise you to get in EARLY and issue a 10/10 Alert. About Pingchuan Pharmaceutical Inc. Pingchuan Pharmaceutical Inc. ("PINGCHUAN") is a modernized pharmaceutical manufacturer with first-class medical R&D ability, pioneered medicine products, and well-established marketing network. Since its establishment, PINGCHUAN has focused its businesses on diabetes medicine and its medical products. The products of PINGCHUAN include health care products, varieties of medicine, as well as medical apparatus. Kang Da Glycosuria Capsule is a successful product of PINGCHUAN for treating diabetes, which was first developed in 1998 and introduced to the market in 2001. PINGCHUAN has well-established marketing network and sale branches, which include about 180,000 retail pharmacies /drugstores across China, agency network in major cities like Beijing, Shanghai, Guangzhou, and Xi'an, special counter sale at drug chain stores in key regions, residence community clinic sale and promotion, as well as internet marketing through the company's website. The marketing network of PINGCHUAN covers more than 50 percent districts of China and exports to US, Japan, Russia, and south-eastern Asia. Dont miss the boat, this is a new issue, is thinly traded and could move up quickly. We anticipate that shares of PGCN will be much higher in the short-term. ACT NOW! Remember the gains from our recent recommendations... Do not miss out, Put PGCN.OB on your radar's now. Good luck and trade out at the top!! From Jonathan.Peirce at nottingham.ac.uk Fri Oct 6 06:55:18 2006 From: Jonathan.Peirce at nottingham.ac.uk (Jon Peirce) Date: Fri, 06 Oct 2006 11:55:18 +0100 Subject: [Numpy-discussion] super-packages [was "Hello and my first patch"] In-Reply-To: References: Message-ID: <45263616.7090809@psychology.nottingham.ac.uk> > Perhaps that is the best way to move forward along with the work on a > "pylab" super-package. A few years ago it seemed that scipy was developing to BE the super-package. It was to have plotting and some numpy flavour and all sorts of goodies for scientists. The name still implies that. People are now talking about a super-package that includes scipy as one of three packages. Definitely it seems that pylab is a bad name since it's already used by matplotlib and the last thing we want is another confusing name convention that becomes hard to google. But why not scipy? It seems the right name for a super-package. Jon This message has been checked for viruses but the contents of an attachment may still contain software viruses, which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From vjetly at i3.com Fri Oct 6 09:30:51 2006 From: vjetly at i3.com (Vikalpa Jetly) Date: Fri, 6 Oct 2006 07:30:51 -0600 Subject: [Numpy-discussion] Memory errors In-Reply-To: <452598C5.3060004@ee.byu.edu> Message-ID: <003401c6e94b$a4a23270$1b28c80a@i3.local> Thanks Travis and Robert. I am just getting my feet wet in numpy. Both approaches i.e: b = zeros_like(a) b[a>200] = 1 or b = (a > 200).astype(numpy.uint8) avoid the memory error. Related question is that I need to test for multiple conditions on the same array and set values to 1 or 0. I see that statements like b[a>200 or a<50] = 1 do not work. So is the way to do this simply to run a separate statement in the form b[condition]= 1 for each test? Also since my output has to be a binary array, can the new array be defined as binary type or nibble, potentially reducing memory overhead? Thanks. Vikalpa -----Original Message----- From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy-discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant Sent: Thursday, October 05, 2006 5:44 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Memory errors > The MemoryError is a direct result when system malloc fails. Rather than use where with two scalars (you're resulting array will be int32 and therefore 4-times larger). Use b = zeros_like(a) b[a>200] = 1 which will consume less memory. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From faltet at carabos.com Fri Oct 6 09:46:43 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 6 Oct 2006 15:46:43 +0200 Subject: [Numpy-discussion] Memory errors In-Reply-To: <003401c6e94b$a4a23270$1b28c80a@i3.local> References: <003401c6e94b$a4a23270$1b28c80a@i3.local> Message-ID: <200610061546.44463.faltet@carabos.com> A Divendres 06 Octubre 2006 15:30, Vikalpa Jetly va escriure: > avoid the memory error. Related question is that I need to test for > multiple conditions on the same array and set values to 1 or 0. I see that > statements like b[a>200 or a<50] = 1 do not work. So is the way to do this > simply to run a separate statement in the form b[condition]= 1 for each > test? For this, you have to use the binary operators instead of logical ones. For example: In [43]: a=numpy.arange(10) In [44]: b=numpy.arange(10) In [45]: b[(a < 2) | (a > 8)] = 1 In [46]: b Out[46]: array([1, 1, 2, 3, 4, 5, 6, 7, 8, 1]) [be sure to use parentesizes appropriately] Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From emsellem at obs.univ-lyon1.fr Fri Oct 6 10:31:04 2006 From: emsellem at obs.univ-lyon1.fr (Eric Emsellem) Date: Fri, 06 Oct 2006 16:31:04 +0200 Subject: [Numpy-discussion] ide for python/numpy/scipy/mpl development ? Message-ID: <452668A8.7000904@obs.univ-lyon1.fr> Hi, I am looking for an IDE to develop python programs and I am not sure what to take. The two critical items for me are 1/ a good debugger (simple and efficient) 2/ something simple to manage the files. I would also very much like to keep some basic things such as (if possible): - editing with gvim - running things with Ipython -pylab So the question is: does such an IDE exists? I have looked through a number of them, but they look either very complex or too simple. Any advice here is very welcome! thanks, and cheers Eric From robert.kern at gmail.com Fri Oct 6 10:41:29 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 06 Oct 2006 07:41:29 -0700 Subject: [Numpy-discussion] ide for python/numpy/scipy/mpl development ? In-Reply-To: <452668A8.7000904@obs.univ-lyon1.fr> References: <452668A8.7000904@obs.univ-lyon1.fr> Message-ID: Eric Emsellem wrote: > Hi, > > I am looking for an IDE to develop python programs and I am not sure > what to take. > The two critical items for me are 1/ a good debugger (simple and > efficient) 2/ something simple to manage the files. > > I would also very much like to keep some basic things such as (if possible): > - editing with gvim This probably is the most limiting factor. I use pida because it embeds gvim into a PyGTK frame with all of the IDE goodies around it. http://pida.berlios.de/ I believe it can use one of the PyGTK debugger GUIs, but I've never used it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From gnurser at googlemail.com Fri Oct 6 10:53:40 2006 From: gnurser at googlemail.com (George Nurser) Date: Fri, 6 Oct 2006 15:53:40 +0100 Subject: [Numpy-discussion] [OT] Interesting interview with Pearu (f2py and others...) In-Reply-To: References: Message-ID: <1d1e6ea70610060753j6c954232xf9d4d63a4cc01231@mail.gmail.com> On 06/10/06, Fernando Perez wrote: > Hi all, > > since this is not about me, I think it's fair to plug an interview :) > > http://www.pythonthreads.com/articles/interviews/once-i-learned-about-python,-i-stopped-trying-out-different-languages.html > > contains a very nice interview with Pearu, where in fact I learned a > number of interesting things not only about him but also about what's > happening with f2py. > > Good job, Pearu :) Great to hear that f2py is being developed so actively. Feature suggestion: Fortran 90 has interface blocks that allow the calling routine to pick up whichever of a set of routines has the appropriate datatype. So one has foo_real32,foo_real64 etc held within the interface block, but calling foo will automatically choose the correct routine. I don't know whether it's possible to make this work with f2py, but it would be very useful if it could be. --George Nurser. From peridot.faceted at gmail.com Fri Oct 6 11:01:28 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 6 Oct 2006 11:01:28 -0400 Subject: [Numpy-discussion] super-packages [was "Hello and my first patch"] In-Reply-To: <45263616.7090809@psychology.nottingham.ac.uk> References: <45263616.7090809@psychology.nottingham.ac.uk> Message-ID: On 06/10/06, Jon Peirce wrote: > > Perhaps that is the best way to move forward along with the work on a > > "pylab" super-package. > > [...] > > But why not scipy? It seems the right name for a super-package. It does, particularly if (as is being discussed) the current content of scipy might get subdivided into separately buildable modules (weave, fortran, non-fortran, say). scipy would then refer to the lot - numpy, matplotlib, weave, scipy-fortran, scipy-nonfortran, and any others that seem essential. Is setting out to create a "MATLAB replacement" a good goal? I think so. Really the goal should be to create a useful tool for scientists, but I think MATLAB is clearly already that, so (at first) it can serve as a definite objective. Of course, anyone who wants to make scipy *better* than MATLAB should be warmly encouraged... A. M. Archibald From faltet at carabos.com Fri Oct 6 13:08:20 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 6 Oct 2006 19:08:20 +0200 Subject: [Numpy-discussion] PyTables passes all tests with NumPy at its core Message-ID: <200610061908.21964.faltet@carabos.com> Hi, I'm very happy to announce that, after several weeks of effort, PyTables is able to run its complete test suite (made of more than 3000 test units) using *NumPy* at its core (now numarray is just an optional package :-). However, and before the new PyTables will hit the public (most probably released as PyTables 2.0), we should still find more time to add more tests (specially for numarray, which we plan to continue supporting), update manual, and making sure that the new package deploys well on every supported plaform. So there is a lot of work to be done yet, but things are in good shape. I have to say that although NumPy was an already well tested package for homogeneous data containers, PyTables has strong requirements for heterogeneous arrays as well, and most of my efforts lately have been to locate the flaws in that area and reporting them properly to the NumPy issue tracker. Fortunately, at the NumPy end there was a formidable team (and in particular, a formidable Travis Oliphant) ready for eat all bugs that I was able to provide. As a result, I'd say that NumPy is quite mature for dealing with arrays of records as well, which is great news :-) Cheers! -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From stefan at sun.ac.za Fri Oct 6 13:33:40 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 6 Oct 2006 19:33:40 +0200 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python In-Reply-To: References: Message-ID: <20061006173340.GA1929@mentat.za.net> On Wed, Oct 04, 2006 at 01:37:55AM -0400, A. M. Archibald wrote: > Would it be useful for me to contribute the tiny script I wrote to > trigger it as a regression test? > > A. M. Archibald > > from numpy import vectorize, zeros > > vt = vectorize(lambda *args: args) > # Removing either of the following lines cures the segfault > vt(zeros((1,2,1)), zeros((2,1,1)), zeros((1,1,2))) > vt(zeros((1,2,1)), zeros((2,1,1)), zeros((1,1,2)), zeros((2,2))) I also get a segfault when running this, but the strange thing is that I can't catch it with valgrind (which also segfaults). I've filed a ticket at http://projects.scipy.org/scipy/numpy/ticket/325 Regards St?fan From Chris.Barker at noaa.gov Fri Oct 6 14:08:25 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Fri, 06 Oct 2006 11:08:25 -0700 Subject: super-packages [was "Hello and my first patch"] In-Reply-To: References: <45263616.7090809@psychology.nottingham.ac.uk> Message-ID: <45269B99.2000104@noaa.gov> A. M. Archibald wrote: > Is setting out to create a "MATLAB replacement" a good goal? No. > the goal should be to create a useful tool for scientists. Yes. > but I think MATLAB is clearly already that, so (at first) it can serve > as a definite objective. Of course, anyone who wants to make scipy > *better* than MATLAB should be warmly encouraged... It already *is* better! I say this as a former, committed MATLAB user. I personally think Python is a much better foundation language, and numpy a better array implementation. MATLAB two advantages (and they are big ones!): integrated plotting and a large and broad library of ready-to-use functions for a wide variety of computing. MPL has almost solved one, and SciPy is getting better an better -- so we're really on the right track! -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From peridot.faceted at gmail.com Fri Oct 6 16:35:34 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 6 Oct 2006 16:35:34 -0400 Subject: super-packages [was "Hello and my first patch"] In-Reply-To: <45269B99.2000104@noaa.gov> References: <45263616.7090809@psychology.nottingham.ac.uk> <45269B99.2000104@noaa.gov> Message-ID: On 06/10/06, Christopher Barker wrote: > A. M. Archibald wrote: > > Is setting out to create a "MATLAB replacement" a good goal? > > No. I see what you mean, but I still think that using MATLAB as a yardstick ("Can users do everything with the python superpackage that they could with MATLAB?" "Is it of comparable or lesser difficulty?") is a good idea, at least until we've clearly outstripped it in all categories. > MATLAB two advantages (and they are big ones!): integrated plotting and > a large and broad library of ready-to-use functions for a wide variety > of computing. This is what I mean about using it as a yardstick. As a MATLAB survivor, would you be interested in going to http://www.scipy.org/NumPy_for_Matlab_Users and adding numerical tools that MATLAB has, so that we can use it as a checklist for what tools we'd like to have (first)? I put a short list there, but it's years since I last used MATLAB. > MPL has almost solved one, and SciPy is getting better an better -- so > we're really on the right track! I agree. A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant at ee.byu.edu Fri Oct 6 21:52:55 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 06 Oct 2006 19:52:55 -0600 Subject: repmat In-Reply-To: References: Message-ID: <45270877.7030903@ee.byu.edu> Bill Baxter wrote: >[There seem to have been some gmail delivery problems that prevented >my previous mail on this subject from being delivered] > >I've proposed that we fix repmat handle arbitrary dimensions before 1.0. > > http://projects.scipy.org/scipy/numpy/ticket/292 > >I don't think this is particularly controversial, just I'm guessing >no-one's found the time to look at my proposed fixes. And >gmail/sourceforge issues haven't helped either. > > Thanks for reminding us again. I don't think this is too bad of a deal. I propose to move repmat(A, M,N) to matlib.py and replace it in numpy with a function named tile(a, reps) That is more general-purpose. If this is not acceptable, please speak up. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Sat Oct 7 00:16:26 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 6 Oct 2006 22:16:26 -0600 Subject: ide for python/numpy/scipy/mpl development ? In-Reply-To: References: <452668A8.7000904@obs.univ-lyon1.fr> Message-ID: On 10/6/06, Robert Kern wrote: > > Eric Emsellem wrote: > > Hi, > > > > I am looking for an IDE to develop python programs and I am not sure > > what to take. > > The two critical items for me are 1/ a good debugger (simple and > > efficient) 2/ something simple to manage the files. > > > > I would also very much like to keep some basic things such as (if > possible): > > - editing with gvim > > This probably is the most limiting factor. I use pida because it embeds > gvim > into a PyGTK frame with all of the IDE goodies around it. > > http://pida.berlios.de/ > > I believe it can use one of the PyGTK debugger GUIs, but I've never used > it. Pida looks nice, thanks for the pointer. Unfortunately, the mercurial support needs editing to make it work at a basic level. To make pida really work right for distributed repositories looks trickier because distributed version control doesn't necessarily have a central repository while the current pida interface looks to be based around that concept. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From boae at kobtv.com Sat Oct 7 03:26:29 2006 From: boae at kobtv.com (Calvagh Moldenhauer) Date: Sat, 7 Oct 2006 00:26:29 -0700 Subject: Hi Message-ID: <000001c6e9e1$e83293e0$88faa8c0@dzkyk> Hi, VrAGRA for less http://www.ruishinimdesin.com _____ time-what had it done for my personal poisonous deadline! I groped for He was gray-haired, stern-faced, dressed in the same combat fatigues ask the right question. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From karol.langner at kn.pl Sat Oct 7 08:06:57 2006 From: karol.langner at kn.pl (Karol Langner) Date: Sat, 7 Oct 2006 14:06:57 +0200 Subject: problem with lapack_lite / BLAS Message-ID: <200610071406.57647.karol.langner@kn.pl> Dear list, I'm trying to compile ATLAS 3.6.0 and LAPACK 3.0 (+ update) and use them with the latest numpy (checked out today). Installation seems to be fine, all the libraries are detected. When I import numpy, however, I get this: Python 2.5 (r25:51908, Sep 27 2006, 17:40:29) [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "", line 1, in File "/home/langner/apps/python/lib/python2.5/site-packages/numpy/__init__.py", line 40, in import linalg File "/home/langner/apps/python/lib/python2.5/site-packages/numpy/linalg/__init__.py", line 4, in from linalg import * File "/home/langner/apps/python/lib/python2.5/site-packages/numpy/linalg/linalg.py", line 25, in from numpy.linalg import lapack_lite ImportError: /home/langner/apps/python/lib/python2.5/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: xerbla_ Does anyone know where this comes from? Cheers, Karol -- written by Karol Langner Sat Oct 7 14:02:46 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From matthew.brett at gmail.com Sat Oct 7 08:15:36 2006 From: matthew.brett at gmail.com (Matthew Brett) Date: Sat, 7 Oct 2006 13:15:36 +0100 Subject: Numpy types and object comparison Message-ID: <1e2af89e0610070515v10fec160saa352335c86c53f6@mail.gmail.com> Hi, I am sorry, I am still struggling with object arrays - and here - numpy data types. Is there any easy way to explain why comparisons with numpy types give a different output from comparisons with non-numpy types: In [108]:oa = zeros((1,), dtype=object) In [109]:oa == int Out[109]:array([False], dtype=bool) In [110]:oa == int32 Out[110]:False Many thanks for your patience, Matthew ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From fullung at gmail.com Sat Oct 7 08:20:06 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sat, 7 Oct 2006 14:20:06 +0200 Subject: problem with lapack_lite / BLAS In-Reply-To: <200610071406.57647.karol.langner@kn.pl> References: <200610071406.57647.karol.langner@kn.pl> Message-ID: Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Karol Langner > Sent: 07 October 2006 14:07 > To: NumPy List > Subject: [Numpy-discussion] problem with lapack_lite / BLAS > > Dear list, > > I'm trying to compile ATLAS 3.6.0 and LAPACK 3.0 (+ update) and use them > with > the latest numpy (checked out today). Installation seems to be fine, all > the > libraries are detected. When I import numpy, however, I get this: > > > ImportError: /home/langner/apps/python/lib/python2.5/site- > packages/numpy/linalg/lapack_lite.so: > undefined symbol: xerbla_ > > Does anyone know where this comes from? xerbla is an error handler for the LAPACK routines. Make sure you included xerbla.f when building your LAPACK library. Check the symbols included in the library with strings or objdump. Hope this helps. Cheers, Albert ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From yuki at yahoo.co.jp Sat Oct 7 08:41:59 2006 From: yuki at yahoo.co.jp (=?iso-2022-jp?B?eXVraQ==?=) Date: Sat, 7 Oct 2006 05:41:59 -0700 (PDT) Subject: (no subject) Message-ID: :?? INFORMATION ?????????????????????????: ?????????????????????? ???????????? http://love-match.bz/pc/01 :??????????????????????????????????: *????*:.?. .?.:*????*:.?..?:*????*:.?..?:**????* ??????????????????????????????????? ??? ???????????????????Love?Match? ?----------------------------------------------------------------- ??????????????????????? ??????????????????????? ??????????????????????? ??????????????????????? ??????????????????????? ??????????????????????? ?----------------------------------------------------------------- ????????????????http://love-match.bz/pc/01 ??????????????????????????????????? ??? ?????????????????????? ?----------------------------------------------------------------- ???????????????????????????? ?----------------------------------------------------------------- ????????????????????????????? ??????????????????????????????? ?http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????????????????????????????? ?----------------------------------------------------------------- ???????????????????????????????? ????????????????????? ?http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????????????????? ?----------------------------------------------------------------- ???????????????????????? ?????????????????????????????????? ?http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????????????????????????? ?----------------------------------------------------------------- ??????????????????????????? ????????????????????????????????? ?http://love-match.bz/pc/01 ?----------------------------------------------------------------- ????????????????????????? ?----------------------------------------------------------------- ????????????????????????? ????????????????????????????????? ?http://love-match.bz/pc/01 ??????????????????????????????????? ??? ??500???????????????? ?----------------------------------------------------------------- ???????/???? ???????????????????? ????????????????????????????????? ???????????????????????????????? ?????????????????????????? ?????????????????????????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????/?????? ?????????????????????????????????? ??????????????????????????????????? ?????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????/????? ?????????????????????????????????? ???????????????????????????????? ?????????????????????????(^^) ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????/???? ??????????????????????????????? ?????????????????????????????? ?????????????????????????????? ???????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ????????/??? ???????????????1??? ????????????????????????? ????????????????????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????/??????? ????18?????????????????????????? ????????????????????????????? ????????????????????????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???`????/??? ????????????????????? ?????????????????????? ?????????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????????????????? ?????????????????????????????????? ????????????? ??------------------------------------------------------------- ???????????????????????????????? ??[??????????]?http://love-match.jp/pc/?1 ??------------------------------------------------------------- ????????????????????? ??????????????????????????? ??????????????????? ??????????????????????????????? ??[??????????]?http://love-match.bz/pc/01 ?????????????????????????????????? ??????????3-6-4-533 ?????? 139-3668-7892 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From karol.langner at kn.pl Sat Oct 7 09:00:08 2006 From: karol.langner at kn.pl (Karol Langner) Date: Sat, 7 Oct 2006 15:00:08 +0200 Subject: problem with lapack_lite / BLAS In-Reply-To: References: Message-ID: <200610071500.08199.karol.langner@kn.pl> On Saturday 07 of October 2006 14:20, Albert Strasheim wrote: > Hello all > > > -----Original Message----- > > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > > discussion-bounces at lists.sourceforge.net] On Behalf Of Karol Langner > > Sent: 07 October 2006 14:07 > > To: NumPy List > > Subject: [Numpy-discussion] problem with lapack_lite / BLAS > > > > Dear list, > > > > I'm trying to compile ATLAS 3.6.0 and LAPACK 3.0 (+ update) and use them > > with > > the latest numpy (checked out today). Installation seems to be fine, all > > the > > libraries are detected. When I import numpy, however, I get this: > > > > > > ImportError: /home/langner/apps/python/lib/python2.5/site- > > packages/numpy/linalg/lapack_lite.so: > > undefined symbol: xerbla_ > > > > Does anyone know where this comes from? > > xerbla is an error handler for the LAPACK routines. Make sure you included > xerbla.f when building your LAPACK library. Check the symbols included in > the library with strings or objdump. > > Hope this helps. > > Cheers, > > Albert Yes, I know, and I checked this, and xerbla.o is in my lapack library, ad provides "xerbla_": langner@~/apps/linalg/lib: objdump -x liblapack.a | grep "xerbla.o" -A 100 xerbla.o: file format elf32-i386 rw-r--r-- 1000/1000 1380 Oct 6 17:51 2006 xerbla.o architecture: i386, flags 0x00000011: HAS_RELOC, HAS_SYMS start address 0x00000000 Sections: Idx Name Size VMA LMA File off Algn 0 .text 00000061 00000000 00000000 00000040 2**4 CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE 1 .data 0000006a 00000000 00000000 000000c0 2**5 CONTENTS, ALLOC, LOAD, RELOC, DATA 2 .bss 00000000 00000000 00000000 0000012c 2**2 ALLOC 3 .rodata 00000000 00000000 00000000 0000012c 2**0 CONTENTS, ALLOC, LOAD, READONLY, DATA 4 .rodata.cst4 00000004 00000000 00000000 0000012c 2**2 CONTENTS, ALLOC, LOAD, READONLY, DATA 5 .note.GNU-stack 00000000 00000000 00000000 00000130 2**0 CONTENTS, READONLY 6 .comment 00000026 00000000 00000000 00000130 2**0 CONTENTS, READONLY SYMBOL TABLE: 00000000 l df *ABS* 00000000 xerbla.f 00000000 l d .text 00000000 00000000 l d .data 00000000 00000000 l d .bss 00000000 00000000 l O .data 00000014 __g77_cilist_0.1 00000020 l O .data 0000004a __g77_format_9999.0 00000000 l d .rodata 00000000 00000000 l d .rodata.cst4 00000000 00000000 l d .note.GNU-stack 00000000 00000000 l d .comment 00000000 00000000 g F .text 00000061 xerbla_ 00000000 *UND* 00000000 s_wsfe 00000000 *UND* 00000000 do_fio 00000000 *UND* 00000000 e_wsfe 00000000 *UND* 00000000 s_stop RELOCATION RECORDS FOR [.text]: OFFSET TYPE VALUE (...) And this _is_ the library I point to in site.cfg. Karol -- written by Karol Langner Sat Oct 7 14:53:15 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From gsmith at alumni.uwaterloo.ca Sat Oct 7 10:51:35 2006 From: gsmith at alumni.uwaterloo.ca (Greg Smith) Date: Sat, 07 Oct 2006 10:51:35 -0400 Subject: paypal won't take my money for scipy docs Message-ID: I fill in the credit-card form, press continue, and it comes back with "Some required information is missing"; the indicated missing info is the 'security code' you copy in to prove you are human, which I had entered. This happened 3 times in a row, and I am certain I got it right at least the last two (I even listened to the audio read-out the third time). So I am starting to get concerned that something is being intercepted.. I'm guessing that a paypal merchant is more likely to get attention from paypal than an accountless purchaser - any idea what's going on here? ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From karol.langner at kn.pl Sat Oct 7 14:01:47 2006 From: karol.langner at kn.pl (Karol Langner) Date: Sat, 7 Oct 2006 20:01:47 +0200 Subject: problem with lapack_lite / BLAS In-Reply-To: <200610071500.08199.karol.langner@kn.pl> References: <200610071500.08199.karol.langner@kn.pl> Message-ID: <200610072001.47603.karol.langner@kn.pl> On Saturday 07 of October 2006 15:00, Karol Langner wrote: > > > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > > > discussion-bounces at lists.sourceforge.net] On Behalf Of Karol Langner > > > Sent: 07 October 2006 14:07 > > > To: NumPy List > > > Subject: [Numpy-discussion] problem with lapack_lite / BLAS > > > > > > Dear list, > > > > > > I'm trying to compile ATLAS 3.6.0 and LAPACK 3.0 (+ update) and use > > > them with > > > the latest numpy (checked out today). Installation seems to be fine, > > > all the > > > libraries are detected. When I import numpy, however, I get this: > > > > > > > > > ImportError: /home/langner/apps/python/lib/python2.5/site- > > > packages/numpy/linalg/lapack_lite.so: > > > undefined symbol: xerbla_ > > > > > > Does anyone know where this comes from? OK... I am so angry at myself... I spent around 2 hours fiddling around with blas/lapack and numpy, and the problem was that I wasn't removing the 'build' directory between compiilations. Apparently it's not enough to run "setup.py clean" (I read this and forgot about it) to recompile lapack_lite.so. Now numpy imports fine and everything I use it for works as before (only a bit faster). Karol P.S. I still get a floating point exception when running the numpy tests, though. I haven't checked out numpy for some time, so I don't now if it's a bug, or if it's my setup. The same thing happens when I use my manually built atlas/lapack and the built-in debian atlas/lapack libraries. I'd be grateful for a comment on if this is just me: >>numpy.test(10,10) [output] Check reading the top fields of a nested array ... ok Check reading the nested fields of a nested array (1st level) ... ok Check access nested descriptors of a nested array (1st level) ... ok Check reading the nested fields of a nested array (2nd level) ... ok Check access nested descriptors of a nested array (2nd level) ... ok check_access_fields (numpy.core.tests.test_numerictypes.test_read_values_plain_multiple) ... ok check_access_fields (numpy.core.tests.test_numerictypes.test_read_values_plain_single) ... ok check_cdouble (numpy.tests.test_linalg.test_det)Floating point exception thanks! -- written by Karol Langner Sat Oct 7 19:41:58 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From pav at iki.fi Sat Oct 7 14:32:21 2006 From: pav at iki.fi (Pauli Virtanen) Date: Sat, 07 Oct 2006 21:32:21 +0300 Subject: problem with lapack_lite / BLAS In-Reply-To: <200610072001.47603.karol.langner@kn.pl> References: <200610071500.08199.karol.langner@kn.pl> <200610072001.47603.karol.langner@kn.pl> Message-ID: <1160245941.3098.5.camel@localhost.localdomain> la, 2006-10-07 kello 20:01 +0200, Karol Langner kirjoitti: > I still get a floating point exception when running the numpy tests, though. I > haven't checked out numpy for some time, so I don't now if it's a bug, or if > it's my setup. The same thing happens when I use my manually built > atlas/lapack and the built-in debian atlas/lapack libraries. I'd be grateful > for a comment on if this is just me: > > >>numpy.test(10,10) > [output] > Check reading the top fields of a nested array ... ok > Check reading the nested fields of a nested array (1st level) ... ok > Check access nested descriptors of a nested array (1st level) ... ok > Check reading the nested fields of a nested array (2nd level) ... ok > Check access nested descriptors of a nested array (2nd level) ... ok > check_access_fields > (numpy.core.tests.test_numerictypes.test_read_values_plain_multiple) ... ok > check_access_fields > (numpy.core.tests.test_numerictypes.test_read_values_plain_single) ... ok > check_cdouble (numpy.tests.test_linalg.test_det)Floating point exception If you are using Debian stable (sarge), you might need to read http://www.its.caltech.edu/~astraw/coding.html#libc-patched-for-debian-sarge-to-fix-floating-point-exceptions-on-sse2 In short, libc in Debian stable has a bug that makes programs crash with SIGFPE when SSE instructions are invoked. The solution is to recompile libc from patched sources, and replace libm.so.6. At least for me this fixed crashes in numpy. Pauli Virtanen ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From aisaac at american.edu Sat Oct 7 14:45:10 2006 From: aisaac at american.edu (Alan G Isaac) Date: Sat, 7 Oct 2006 14:45:10 -0400 Subject: ide for python/numpy/scipy/mpl development ? In-Reply-To: References: <452668A8.7000904@obs.univ-lyon1.fr> Message-ID: On Fri, 06 Oct 2006, Robert Kern apparently wrote: > http://pida.berlios.de/ Can this be installed under Windows? (Looks unlikely.) Cheers, Alan Isaac ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From mattknox_ca at hotmail.com Sat Oct 7 16:11:56 2006 From: mattknox_ca at hotmail.com (Matt Knox) Date: Sat, 7 Oct 2006 16:11:56 -0400 Subject: user-defined type example Message-ID: Could someone please point me to/provide me with a basic example of creating a user defined type? Here is my completely naive attempt which obviously doesn't work...import numpy class myType(numpy.void): def __init__(self,val): self.val = val testType = numpy.dtype(myType) val1 = myType(5)val2 = myType(6) foo = numpy.array([val1,val2],testType) Any help would be greatly appreciated. Thanks, - Matt Knox _________________________________________________________________ Be one of the first to try Windows Live Mail. http://ideas.live.com/programpage.aspx?versionId=5d21c51a-b161-4314-9b0e-4911fb2b2e6d -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gnchen at cortechs.net Sat Oct 7 19:16:16 2006 From: gnchen at cortechs.net (Gennan Chen) Date: Sat, 7 Oct 2006 16:16:16 -0700 Subject: MKL on Intel OS X Message-ID: <08ED7810-A788-49E8-8072-EABE3A077E9B@cortechs.net> Hi! We just bought MKL and icc for OS X. I don't think I will try icc for numpy and scipy. But anyone ever tried compiled with os x' MKL? How to add options to make it work?? Gen -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Sun Oct 8 00:29:39 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 07 Oct 2006 22:29:39 -0600 Subject: ***[Possible UCE]*** user-defined type example In-Reply-To: References: Message-ID: <45287EB3.6080709@ieee.org> Matt Knox wrote: > Could someone please point me to/provide me with a basic example of > creating a user defined type? > > Here is my completely naive attempt which obviously doesn't work... > > import numpy > > class myType(numpy.void): > def __init__(self,val): > self.val = val > > testType = numpy.dtype(myType) > > val1 = myType(5) > val2 = myType(6) > > foo = numpy.array([val1,val2],testType) > > > Any help would be greatly appreciated. Thanks, > A "true" user-defined data-type can only be done in C. However. You can use the "VOID" data-type to create a type with multiple fields. What do you want to do with your user-defined data-type. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From robert.kern at gmail.com Sun Oct 8 00:41:09 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 08 Oct 2006 00:41:09 -0400 Subject: ide for python/numpy/scipy/mpl development ? In-Reply-To: References: <452668A8.7000904@obs.univ-lyon1.fr> Message-ID: Alan G Isaac wrote: > On Fri, 06 Oct 2006, Robert Kern apparently wrote: >> http://pida.berlios.de/ > > Can this be installed under Windows? > (Looks unlikely.) It might be possible if you have a Windows build of PyGTK, but not with the gvim integration. However, pida also supports other PyGTK-accessible editor widgets. See its documentation for details. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Sun Oct 8 00:54:45 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 7 Oct 2006 22:54:45 -0600 Subject: tensor product Message-ID: Hmmm, I notice that there is no longer a tensor product. As it was the only one of the outer, kron bunch that I really wanted, l miss it. In fact, I always thought outer should act like the tensor product for the other binary operators too. Anyway, mind if I put it back? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From mattknox_ca at hotmail.com Sun Oct 8 01:27:02 2006 From: mattknox_ca at hotmail.com (Matt Knox) Date: Sun, 8 Oct 2006 01:27:02 -0400 Subject: ***[Possible UCE]*** user-defined type example Message-ID: > Matt Knox wrote:> > Could someone please point me to/provide me with a basic example of > > creating a user defined type?> > > > Here is my completely naive attempt which obviously doesn't work...> >> > import numpy> > > > class myType(numpy.void):> > def __init__(self,val):> > self.val = val> > > > testType = numpy.dtype(myType)> >> > val1 = myType(5)> > val2 = myType(6)> > > > foo = numpy.array([val1,val2],testType)> > > > > > Any help would be greatly appreciated. Thanks,> > > A "true" user-defined data-type can only be done in C.> > However. You can use the "VOID" data-type to create a type with > multiple fields. What do you want to do with your user-defined data-type.>> -Travis Thanks for the reply Travis. I have a custom Date class which allows creation of different frequency Dates (annual, monthly, etc..). Two dates of the same frequency can be subtracted, and integers can be added to a Date, with the obvious results in each case. I store these dates in arrays currently using the "object" dtype. However, the problem with this is that I can't tell the frequency of the Dates in the array without inspecting invidual elements of the array, or subclassing the array object itself. I'd like to be able to have each frequency of date as a data type (DateAnnual, DateMonthly, etc...). It just seems like a cleaner approach to have these dates defined as data types. I would be open to the idea of defining this in C, but I don't really have any examples to work with for that either. Many thanks for any advice/suggestions you can provide. - Matt _________________________________________________________________ Be one of the first to try Windows Live Mail. http://ideas.live.com/programpage.aspx?versionId=5d21c51a-b161-4314-9b0e-4911fb2b2e6d -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dmahler at gmail.com Sun Oct 8 01:36:19 2006 From: dmahler at gmail.com (Daniel Mahler) Date: Sun, 8 Oct 2006 06:36:19 +0100 Subject: can this be made faster? Message-ID: Is there a 'loop free' way to do this in Numeric for i in arange(l): a[b[i]]+=c[i] where l == len(b) == len(c) thanks Daniel ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From wbaxter at gmail.com Sun Oct 8 01:48:59 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 8 Oct 2006 14:48:59 +0900 Subject: can this be made faster? In-Reply-To: References: Message-ID: Yes, that'd be a[b] += c On 10/8/06, Daniel Mahler wrote: > Is there a 'loop free' way to do this in Numeric > > for i in arange(l): > a[b[i]]+=c[i] > > where l == len(b) == len(c) > > thanks > Daniel ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From dmahler at gmail.com Sun Oct 8 02:03:38 2006 From: dmahler at gmail.com (Daniel Mahler) Date: Sun, 8 Oct 2006 07:03:38 +0100 Subject: can this be made faster? In-Reply-To: References: Message-ID: Thanks Bill. Thats what I was hoping for, but I get >>> a array([0, 0]) >>> b array([0, 1, 0, 1, 0]) >>> c array([1, 1, 1, 1, 1]) >>> a[b]+=c Traceback (most recent call last): File "", line 1, in ? IndexError: invalid index whereas i would like to get array([3, 2]) On 10/8/06, Bill Baxter wrote: > Yes, that'd be > a[b] += c > > On 10/8/06, Daniel Mahler wrote: > > Is there a 'loop free' way to do this in Numeric > > > > for i in arange(l): > > a[b[i]]+=c[i] > > > > where l == len(b) == len(c) > > > > thanks > > Daniel > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From wbaxter at gmail.com Sun Oct 8 02:17:29 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 8 Oct 2006 15:17:29 +0900 Subject: can this be made faster? In-Reply-To: References: Message-ID: Yeh, I spoke too soon. Tried a little example and it seemed to work. I don't get a traceback here, but your example doesn't work as expected, either. I get [1,1] as the answer with numpy 1.0rc1. Probably it should be an exception, though. It seems to work if len(b)<=len(a) and when no indices are repeated in b. --bb On 10/8/06, Daniel Mahler wrote: > Thanks Bill. > Thats what I was hoping for, > but I get > > >>> a > array([0, 0]) > >>> b > array([0, 1, 0, 1, 0]) > >>> c > array([1, 1, 1, 1, 1]) > >>> a[b]+=c > Traceback (most recent call last): > File "", line 1, in ? > IndexError: invalid index > > whereas i would like to get > > array([3, 2]) > > > > On 10/8/06, Bill Baxter wrote: > > Yes, that'd be > > a[b] += c > > > > On 10/8/06, Daniel Mahler wrote: > > > Is there a 'loop free' way to do this in Numeric > > > > > > for i in arange(l): > > > a[b[i]]+=c[i] > > > > > > where l == len(b) == len(c) > > > > > > thanks > > > Daniel > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From nadavh at visionsense.com Sun Oct 8 02:55:42 2006 From: nadavh at visionsense.com (Nadav Horesh) Date: Sun, 8 Oct 2006 08:55:42 +0200 Subject: tensor product Message-ID: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> There is a "tensortdot" function in numpy1.0rc1 Nadav -----Original Message----- From: numpy-discussion-bounces at lists.sourceforge.net on behalf of Charles R Harris Sent: Sun 08-Oct-06 06:54 To: numpy-discussion at lists.sourceforge.net Cc: Subject: [Numpy-discussion] tensor product Hmmm, I notice that there is no longer a tensor product. As it was the only one of the outer, kron bunch that I really wanted, l miss it. In fact, I always thought outer should act like the tensor product for the other binary operators too. Anyway, mind if I put it back? Chuck -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 2754 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cabwxna at utahgrizz.com Sun Oct 8 12:33:12 2006 From: cabwxna at utahgrizz.com (Your search) Date: Sun, 8 Oct 2006 14:33:12 -0200 Subject: Make sure words are Message-ID: <000f01c6ead5$ebac9e90$f3c1f552@nico> Spelled try different keywords. Videonew News Maps. Fewer your on can. Create new group About Searched all of groups Your search in! Maps is more raquo Advanced Search. Are spelled in try. Create new of group About Searched all is. Sure words is are is spelled try different! Fewer is your a on can try is Answers for expert help with search get the latest messages is. Members users Join Alerts Create new of group About Searched is all groups Your search. Create new or group About Searched all groups Your search did not. Alerts Create new group About Searched all groups am Your search. More raquo Advanced. Images Videonew News? Messages emailed to in Alerts Terms? Messages. Users Join Alerts Create. Search Members users Join. Users Join Alerts of Create ne -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sure.gif Type: image/gif Size: 5009 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From aisaac at american.edu Sun Oct 8 12:06:05 2006 From: aisaac at american.edu (Alan G Isaac) Date: Sun, 8 Oct 2006 12:06:05 -0400 Subject: =?iso-8859-1?q?***=5BPossible_UCE=5D***_user-d?= =?iso-8859-1?q?efined_type=09example?= In-Reply-To: References: Message-ID: On Sun, 8 Oct 2006, Matt Knox apparently wrote: > I have a custom Date class which allows creation of > different frequency Dates (annual, monthly, etc..). Two > dates of the same frequency can be subtracted, and > integers can be added to a Date, with the obvious results > in each case. I store these dates in arrays currently > using the "object" dtype. Just making sure you know about http://www.egenix.com/files/python/mxDateTime.html If you need something that is not there, Marc-Andr? Lemburg may be willing to add it, if others might need it too. Cheers, Alan Isaac ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From gregwillden at gmail.com Sun Oct 8 18:20:40 2006 From: gregwillden at gmail.com (Greg Willden) Date: Sun, 8 Oct 2006 17:20:40 -0500 Subject: can this be made faster? In-Reply-To: References: Message-ID: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> On 10/8/06, Daniel Mahler wrote: > > >>> a > array([0, 0]) > >>> b > array([0, 1, 0, 1, 0]) > >>> c > array([1, 1, 1, 1, 1]) > Well for this particular example you could do a=array([len(b)-sum(b), sum(b)]) Since you are just counting the ones and zeros. This next one is a little closer for the case when c is not just a bunch of 1's but you still have to know how the highest number in b. a=array([sum(c[b==0]), sum(c[b==1]), ... sum(c[b==N]) ] ) So it sort of depends on your ultimate goal. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Sun Oct 8 18:59:19 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 08 Oct 2006 18:59:19 -0400 Subject: can this be made faster? In-Reply-To: References: Message-ID: Bill Baxter wrote: > Yes, that'd be > a[b] += c No, I'm afraid that fancy indexing does not do the loop that you are thinking it would (and for reasons that we've discussed previously on this list, *can't* do that loop). That statement reduces to something like the following: tmp = a[b] tmp = tmp.__iadd__(c) a[b] = tmp In [1]: from numpy import * In [2]: a = array([0, 0]) In [3]: b = array([0, 1, 0, 1, 0]) In [4]: c = array([1, 1, 1, 1, 1]) In [5]: a[b] += c In [6]: a Out[6]: array([1, 1]) In [7]: a = array([0, 0]) In [8]: tmp = a[b] In [9]: tmp Out[9]: array([0, 0, 0, 0, 0]) In [10]: tmp = tmp.__iadd__(c) In [11]: tmp Out[11]: array([1, 1, 1, 1, 1]) In [12]: a[b] = tmp In [13]: a Out[13]: array([1, 1]) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From wbaxter at gmail.com Sun Oct 8 19:07:02 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Mon, 9 Oct 2006 08:07:02 +0900 Subject: can this be made faster? In-Reply-To: References: Message-ID: So what's the answer then? Can it be made faster? --bb On 10/9/06, Robert Kern wrote: > > Bill Baxter wrote: > > Yes, that'd be > > a[b] += c > > No, I'm afraid that fancy indexing does not do the loop that you are > thinking it > would (and for reasons that we've discussed previously on this list, > *can't* do > that loop). That statement reduces to something like the following: > > tmp = a[b] > tmp = tmp.__iadd__(c) > a[b] = tmp > > > In [1]: from numpy import * > > In [2]: a = array([0, 0]) > > In [3]: b = array([0, 1, 0, 1, 0]) > > In [4]: c = array([1, 1, 1, 1, 1]) > > In [5]: a[b] += c > > In [6]: a > Out[6]: array([1, 1]) > > In [7]: a = array([0, 0]) > > In [8]: tmp = a[b] > > In [9]: tmp > Out[9]: array([0, 0, 0, 0, 0]) > > In [10]: tmp = tmp.__iadd__(c) > > In [11]: tmp > Out[11]: array([1, 1, 1, 1, 1]) > > In [12]: a[b] = tmp > > In [13]: a > Out[13]: array([1, 1]) > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though > it had > an underlying truth." > -- Umberto Eco > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Sun Oct 8 19:08:31 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Sun, 8 Oct 2006 19:08:31 -0400 Subject: can this be made faster? In-Reply-To: References: Message-ID: On 08/10/06, Robert Kern wrote: > Bill Baxter wrote: > > Yes, that'd be > > a[b] += c > > No, I'm afraid that fancy indexing does not do the loop that you are thinking it > would (and for reasons that we've discussed previously on this list, *can't* do > that loop). That statement reduces to something like the following: So the question remains, is there a for-loop-free way to do this? (This, specifically, is: for i in range(len(b)): a[b[i]]+=c[i] where b[i] may contain repetitions.) I didn't find one, but came to the conclusion that for loops are not necessarily slower than fancy indexing, so the way to do this one is just to use a for loop. A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Sun Oct 8 22:19:02 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 8 Oct 2006 20:19:02 -0600 Subject: tensor product In-Reply-To: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> Message-ID: Hi Nadav, On 10/8/06, Nadav Horesh wrote: > > > There is a "tensortdot" function in numpy1.0rc1 The tensordot is not the same thing as a tensor product. What I want is the following: def tensor(a, b) : """Tensor product of a and b """ a = asarray(a) b = asarray(b) return outer(a, b).reshape(a.shape + b.shape) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dmahler at gmail.com Mon Oct 9 01:23:18 2006 From: dmahler at gmail.com (Daniel Mahler) Date: Mon, 9 Oct 2006 06:23:18 +0100 Subject: can this be made faster? In-Reply-To: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: On 10/8/06, Greg Willden wrote: > On 10/8/06, Daniel Mahler wrote: > > > > >>> a > > array([0, 0]) > > >>> b > > array([0, 1, 0, 1, 0]) > > >>> c > > array([1, 1, 1, 1, 1]) > > > > > Well for this particular example you could do > a=array([len(b)-sum(b), sum(b)]) > Since you are just counting the ones and zeros. > > This next one is a little closer for the case when c is not just a bunch of > 1's but you still have to know how the highest number in b. > a=array([sum(c[b==0]), sum(c[b==1]), ... sum(c[b==N]) ] ) > > So it sort of depends on your ultimate goal. > Greg > Linux. Because rebooting is for adding hardware. In my case all a, b, c are large with b and c being orders of magnitude lareger than a. b is known to contain only, but potentially any, a-indexes, reapeated many times. c contains arbitray floats. essentially it is to compute class totals as in total[class[i]] += value[i] Daniel ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From robert.kern at gmail.com Mon Oct 9 01:40:41 2006 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 09 Oct 2006 01:40:41 -0400 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: Daniel Mahler wrote: > On 10/8/06, Greg Willden wrote: >> This next one is a little closer for the case when c is not just a bunch of >> 1's but you still have to know how the highest number in b. >> a=array([sum(c[b==0]), sum(c[b==1]), ... sum(c[b==N]) ] ) >> >> So it sort of depends on your ultimate goal. >> Greg >> Linux. Because rebooting is for adding hardware. > > In my case all a, b, c are large with b and c being orders of > magnitude lareger than a. > b is known to contain only, but potentially any, a-indexes, reapeated > many times. > c contains arbitray floats. > essentially it is to compute class totals > as in total[class[i]] += value[i] In that case, a slight modification to Greg's suggestion will probably be fastest: import numpy as np # Set up the problem. lena = 10 lenc = 10000 a = np.zeros(lena, dtype=float) b = np.random.randint(lena, size=lenc) c = np.random.uniform(size=lenc) idx = np.arange(lena, dtype=int)[:, np.newaxis] mask = (b == idx) for i in range(lena): a[i] = c[b[i]].sum() -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant.travis at ieee.org Mon Oct 9 04:02:56 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Oct 2006 02:02:56 -0600 Subject: tensor product In-Reply-To: References: Message-ID: <452A0230.5060106@ieee.org> Charles R Harris wrote: > Hmmm, > > I notice that there is no longer a tensor product. As it was the only > one of the outer, kron bunch that I really wanted, l miss it. In fact, > I always thought outer should act like the tensor product for the > other binary operators too. Anyway, mind if I put it back? I'm not opposed to the idea, necessarily. But, when and where did this exist? I don't remember it. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From emsellem at obs.univ-lyon1.fr Mon Oct 9 04:10:55 2006 From: emsellem at obs.univ-lyon1.fr (Eric Emsellem) Date: Mon, 09 Oct 2006 10:10:55 +0200 Subject: ide for python/numpy/scipy/mpl, development ? Message-ID: <452A040F.3050708@obs.univ-lyon1.fr> Hi, thanks for the pointer to pida. I installed it successfully but I didn't manage to make it really work, meaning having my python file opened with gvim and include some debugger (I guess either the installation is not complete or there is something I don't understand when opening a new python file). And the docs is really scarce at the moment so it is quite difficult to pursue along that line. Indeed keeping gvim may be too tight a constraint here. Someone else suggested spe (http://stani.be/python/spe/blog/), but there I have problems getting binaries for wxPython (Suse10.1) and I don't want to attempt a full compilation of that package... (I installed Suse10.1 from the downloadable version and many many packages are missing there - the commercial DVD being double layer... and it seems that only half of it is available on the web). I tried to install eric3, and after checking QScintilla, python-qt, etc, I launch the installation of eric3 with python install.py and I get a seg fault.. frustrating. Eric On 10/6/06, Robert Kern wrote: > > > > Eric Emsellem wrote: > >> > > Hi, >> > > >> > > I am looking for an IDE to develop python programs and I am not sure >> > > what to take. >> > > The two critical items for me are 1/ a good debugger (simple and >> > > efficient) 2/ something simple to manage the files. >> > > >> > > I would also very much like to keep some basic things such as (if >> > > possible): > >> > > - editing with gvim >> > > > > This probably is the most limiting factor. I use pida because it embeds > > gvim > > into a PyGTK frame with all of the IDE goodies around it. > > > > http://pida.berlios.de/ > > > > I believe it can use one of the PyGTK debugger GUIs, but I've never used > > it. > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From wbaxter at gmail.com Mon Oct 9 04:20:25 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Mon, 9 Oct 2006 17:20:25 +0900 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <452A040F.3050708@obs.univ-lyon1.fr> References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: If you don't mind going commercial, then the WingIDE has been working very well for me. And it apparently has a VIM mode (and emacs). I'm an emacs guy, though, so I haven't tried out the VIM support. Wing is based on GTK so the interface on Windows doesn't look so nice, or act so much like a native Windows app (F10 pops up menus -- weird?), but it works well. I tried a bunch of free IDEs first and gave up out of frustration after a while, too. Wing just worked for me and worked quite well. Documentation is pretty good too. www.wingware.com. Free trial so you can see if you like it first. --bb On 10/9/06, Eric Emsellem wrote: > Hi, > > thanks for the pointer to pida. I installed it successfully but I didn't manage to make it really work, meaning having my python file opened with gvim and include some debugger (I guess either the installation is not complete or there is something I don't understand when opening a new python file). And the docs is really scarce at the moment so it is quite difficult to pursue along that line. > > Indeed keeping gvim may be too tight a constraint here. > > Someone else suggested spe (http://stani.be/python/spe/blog/), but there I have problems getting binaries for wxPython (Suse10.1) and I don't want to attempt a full compilation of that package... (I installed Suse10.1 from the downloadable version and many many packages are missing there - the commercial DVD being double layer... and it seems that only half of it is available on the web). > > I tried to install eric3, and after checking QScintilla, python-qt, etc, I launch the installation of eric3 with python install.py and I get a seg fault.. > > frustrating. > Eric > > > On 10/6/06, Robert Kern wrote: > > > > > > > Eric Emsellem wrote: > > > >> > > Hi, > >> > > > >> > > I am looking for an IDE to develop python programs and I am not sure > >> > > what to take. > >> > > The two critical items for me are 1/ a good debugger (simple and > >> > > efficient) 2/ something simple to manage the files. > >> > > > >> > > I would also very much like to keep some basic things such as (if > >> > > > possible): > > > >> > > - editing with gvim > >> > > > > > > This probably is the most limiting factor. I use pida because it embeds > > > gvim > > > into a PyGTK frame with all of the IDE goodies around it. > > > > > > http://pida.berlios.de/ > > > > > > I believe it can use one of the PyGTK debugger GUIs, but I've never used > > > it. > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From nagyebhzlv at westhillshonda.com Sun Oct 8 17:17:41 2006 From: nagyebhzlv at westhillshonda.com (Nokia) Date: Mon, 9 Oct 2006 04:17:41 +0700 Subject: VideoNew Maps Members: Message-ID: <000b01c6eb94$8955da30$1d28d97c@vertex5> Answers expert a search get latest. Democracy in in January led Kabul meet with our am? Watchgot moustache Aviators. from query new a Array Features am National is the is Final a Word. Required Feeds list one am pos create. Larger Where add. Games these ones keyring. Strut your stuff Almost in else Beta Discuss versions in Latest am. Sylla not working Ozzmosis List Google Groups Help a Sign. Regularly publishes place share well providing Knowing someone gone similar or situation syndromes! Cant seem find anything of decent in though Benjamin dab radio Desktop. Levine graduate am Yale a College currently Fellow in! Families. May changed is. Power wants am Most Popular. This cache. Coolest. Inspired winging store. Muscula -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: has.gif Type: image/gif Size: 5012 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Mon Oct 9 09:01:01 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 09 Oct 2006 06:01:01 -0700 Subject: tensor product In-Reply-To: References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> Message-ID: <452A480D.2010300@ieee.org> Charles R Harris wrote: > Hi Nadav, > > On 10/8/06, *Nadav Horesh* > wrote: > > > There is a "tensortdot" function in numpy1.0rc1 > > > > The tensordot is not the same thing as a tensor product. What I want > is the following: > > def tensor(a, b) : > """Tensor product of a and b > > """ > a = asarray(a) > b = asarray(b) > return outer(a, b).reshape(a.shape + b.shape) > Is this not the same things as numpy.multiply.outer(a, b)? (as opposed to outer(a, b), which appears to pretend that everything is a vector -- I'm not sure what the point of that is). -tim ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From tim.hochberg at ieee.org Mon Oct 9 10:10:09 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 09 Oct 2006 07:10:09 -0700 Subject: vectorized linalg Message-ID: <452A5841.6050802@ieee.org> I periodically need to perform various linalg operations on a large number of small matrices. The normal numpy approach of stacking up the data into an extra dimension and doing the operations in parallel doesn't work because the linalg functions are only designed to work on one matrix at a time. At first glance, this restriction seems harmless enough since the underlying LAPACK functions are also designed to work one matrix at a time and it seem that there would be little to be gained by parallelizing the Python level code. However, this turns out to be the case. I have some code, originally written for numarray, but I recently ported it over to numpy, that rewrites the linalg function determinant, inverse and solve_linear_equations[1] to operate of 2 or 3D arrays, treating 3D arrays as stacks of 2D arrays. For operating on large numbers of small arrays (for example 1000 2x2 arrays), this approach is over an order of magnitude faster than the obvious map(inverse, threeDArray) approach. So, some questions: 1. Is there any interest in this code for numpy.linalg? I'd be willing to clean it up and work on the other functions in linalg so that they were all consistent. Since these changes should all be backwards compatible (unless your counting on catching an error from one of the linalg functions), I don't see this as a problem to put in after 1.0, so there's really no rush on this. I'm only bringing it up now since I just ported it over to numpy and it's fresh in my mind. 2. If 1., what to do about norm? I think the other functions in linalg stack naturally, but norm, since it is meant to work on both vectors and matrices is problematic. Norm already seems a bit problematic in that, for some values of 'ord', norm can return different values for a shape [N] and a shape [1,N] array containing the same values: >>> linalg.norm(a[:1], 1) 0.78120069791102054 >>> linalg.norm(a[0], 1) 1.4588102446677758 My inclination is to introduce stackable 1 and 2D version s of norm, vnorm and mnorm for lack of better names. Ideally we'd deprecate the use of norm for ord!=None (Froebenius norm) or at least in cases where the result depends on the rank [1,N] versus [N]. 3. A similar issue arises with dot. One would like to be able to do stacked matrix products, vector products and matrix-vector products. If we are making the rest of linalg stackable, linalg would be a sensible place for a stackable version of dot to live. However, it's immediately clear how to do this without introducing a whole pile (4) of stacking dot function to handle the various cases, plus broadcasting, cleanly. This would require some more thought. Thoughts? -tim [1] Those are actually the numarray names, which the current code still uses, the numpy names are 'det', 'inv' and 'solve' [2] Treatment of stacking insolve linear equations is a bit more complicated, but I'll ignore that for now. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From David.L.Goldsmith at noaa.gov Mon Oct 9 10:32:12 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Mon, 09 Oct 2006 07:32:12 -0700 Subject: vectorized linalg In-Reply-To: <452A5841.6050802@ieee.org> References: <452A5841.6050802@ieee.org> Message-ID: <452A5D6C.9040204@noaa.gov> Tim Hochberg wrote: > I periodically need to perform various linalg operations on a large > number of small matrices. The normal numpy approach of stacking up the > data into an extra dimension and doing the operations in parallel > doesn't work because the linalg functions are only designed to work on > one matrix at a time. At first glance, this restriction seems harmless > enough since the underlying LAPACK functions are also designed to work > one matrix at a time and it seem that there would be little to be gained > by parallelizing the Python level code. However, this turns out to be > the case. > > I have some code, originally written for numarray, but I recently ported > it over to numpy, that rewrites the linalg function determinant, inverse > and solve_linear_equations[1] to operate of 2 or 3D arrays, treating 3D > arrays as stacks of 2D arrays. For operating on large numbers of small > arrays (for example 1000 2x2 arrays), this approach is over an order of > magnitude faster than the obvious map(inverse, threeDArray) approach. > > So, some questions: > > 1. Is there any interest in this code for numpy.linalg? I'd be > willing to clean it up and work on the other functions in linalg so that > they were all consistent. Since these changes should all be backwards > compatible (unless your counting on catching an error from one of the > linalg functions), I don't see this as a problem to put in after 1.0, so > there's really no rush on this. I'm only bringing it up now since I just > ported it over to numpy and it's fresh in my mind. > Say "no" to someone offering to make a Python module more feature rich? Blasphemy! :-) But I am very curious as to the source/basis of the performance improvement - did you figure this out? DG > 2. If 1., what to do about norm? I think the other functions in > linalg stack naturally, but norm, since it is meant to work on both > vectors and matrices is problematic. Norm already seems a bit > problematic in that, for some values of 'ord', norm can return different > values for a shape [N] and a shape [1,N] array containing the same values: > > >>> linalg.norm(a[:1], 1) > 0.78120069791102054 > >>> linalg.norm(a[0], 1) > 1.4588102446677758 > > My inclination is to introduce stackable 1 and 2D version s of norm, > vnorm and mnorm for lack of better names. Ideally we'd deprecate the use > of norm for ord!=None (Froebenius norm) or at least in cases where the > result depends on the rank [1,N] versus [N]. > > 3. A similar issue arises with dot. One would like to be able to do > stacked matrix products, vector products and matrix-vector products. If > we are making the rest of linalg stackable, linalg would be a sensible > place for a stackable version of dot to live. However, it's immediately > clear how to do this without introducing a whole pile (4) of stacking > dot function to handle the various cases, plus broadcasting, cleanly. > This would require some more thought. Thoughts? > > -tim > > > [1] Those are actually the numarray names, which the current code still > uses, the numpy names are 'det', 'inv' and 'solve' > > [2] Treatment of stacking insolve linear equations is a bit more > complicated, but I'll ignore that for now. > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 10:43:49 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 08:43:49 -0600 Subject: tensor product In-Reply-To: <452A480D.2010300@ieee.org> References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> <452A480D.2010300@ieee.org> Message-ID: On 10/9/06, Tim Hochberg wrote: > > Is this not the same things as numpy.multiply.outer(a, b)? (as opposed > to outer(a, b), which appears to pretend that everything is a vector -- > I'm not sure what the point of that is). Hmmm, yes, multiply.outer does do that. I thought that outer was short for multiply.outer and that the behaviour had changed. So the question is why outer does what it does. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Mon Oct 9 10:45:37 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 09 Oct 2006 07:45:37 -0700 Subject: vectorized linalg In-Reply-To: <452A5D6C.9040204@noaa.gov> References: <452A5841.6050802@ieee.org> <452A5D6C.9040204@noaa.gov> Message-ID: <452A6091.3080609@ieee.org> David Goldsmith wrote: > Tim Hochberg wrote: > >> I periodically need to perform various linalg operations on a large >> number of small matrices. The normal numpy approach of stacking up the >> data into an extra dimension and doing the operations in parallel >> doesn't work because the linalg functions are only designed to work on >> one matrix at a time. At first glance, this restriction seems harmless >> enough since the underlying LAPACK functions are also designed to work >> one matrix at a time and it seem that there would be little to be gained >> by parallelizing the Python level code. However, this turns out to be >> the case. >> >> I have some code, originally written for numarray, but I recently ported >> it over to numpy, that rewrites the linalg function determinant, inverse >> and solve_linear_equations[1] to operate of 2 or 3D arrays, treating 3D >> arrays as stacks of 2D arrays. For operating on large numbers of small >> arrays (for example 1000 2x2 arrays), this approach is over an order of >> magnitude faster than the obvious map(inverse, threeDArray) approach. >> >> So, some questions: >> >> 1. Is there any interest in this code for numpy.linalg? I'd be >> willing to clean it up and work on the other functions in linalg so that >> they were all consistent. Since these changes should all be backwards >> compatible (unless your counting on catching an error from one of the >> linalg functions), I don't see this as a problem to put in after 1.0, so >> there's really no rush on this. I'm only bringing it up now since I just >> ported it over to numpy and it's fresh in my mind. >> >> > Say "no" to someone offering to make a Python module more feature rich? > Blasphemy! :-) > > But I am very curious as to the source/basis of the performance > improvement - did you figure this out? > Yes and no. Here's what the begining of linalg.det looks lik: a = asarray(a) _assertRank2(a) _assertSquareness(a) t, result_t = _commonType(a) a = _fastCopyAndTranspose(t, a) n = a.shape[0] if isComplexType(t): lapack_routine = lapack_lite.zgetrf else: lapack_routine = lapack_lite.dgetrf # ... There's quite a few function calls here, since the actual call to dgetrf probably takes neglible time for a 2x2 array, the cost of computation here is dominated by function call overhead, creating the return arrays and such not. Which is the biggest culprit, I'm not sure; by vectorizing it, much of that overhead is only incurred once rather than hundreds or thousands of times, so it all becomes negligible at that point. I didn't bother to track down the worst offender. -tim ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From emsellem at obs.univ-lyon1.fr Mon Oct 9 10:50:33 2006 From: emsellem at obs.univ-lyon1.fr (Eric Emsellem) Date: Mon, 09 Oct 2006 16:50:33 +0200 Subject: ide for python/numpy/scipy/mpl, , development ? Message-ID: <452A61B9.2090100@obs.univ-lyon1.fr> Hi, thanks for the tip: very nice but really too expensive for me... (the personal version having too many constraints). Thanks again anyway. Will search for something else... Maybe I'll manage to get pida or spe to work ? cheers Eric > From: "Bill Baxter" > Subject: Re: [Numpy-discussion] ide for python/numpy/scipy/mpl, > development ? > To: "Discussion of Numerical Python" > > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > If you don't mind going commercial, then the WingIDE has been working > very well for me. > And it apparently has a VIM mode (and emacs). I'm an emacs guy, > though, so I haven't tried out the VIM support. > > Wing is based on GTK so the interface on Windows doesn't look so nice, > or act so much like a native Windows app (F10 pops up menus -- > weird?), but it works well. I tried a bunch of free IDEs first and > gave up out of frustration after a while, too. Wing just worked for > me and worked quite well. Documentation is pretty good too. > www.wingware.com. Free trial so you can see if you like it first. > > --bb -- ==================================================================== Eric Emsellem emsellem at obs.univ-lyon1.fr Centre de Recherche Astrophysique de Lyon 9 av. Charles-Andre tel: +33 (0)4 78 86 83 84 69561 Saint-Genis Laval Cedex fax: +33 (0)4 78 86 83 86 France http://www-obs.univ-lyon1.fr/eric.emsellem ==================================================================== ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From perry at stsci.edu Mon Oct 9 11:00:28 2006 From: perry at stsci.edu (Perry Greenfield) Date: Mon, 9 Oct 2006 11:00:28 -0400 Subject: ide for python/numpy/scipy/mpl, , development ? In-Reply-To: <452A61B9.2090100@obs.univ-lyon1.fr> References: <452A61B9.2090100@obs.univ-lyon1.fr> Message-ID: On Oct 9, 2006, at 10:50 AM, Eric Emsellem wrote: > Hi, > > thanks for the tip: very nice but really too expensive for me... (the > personal version having too many constraints). Thanks again anyway. > Will search for something else... Maybe I'll manage to get pida or spe > to work ? > > cheers > > Eric I believe that they are likely to give you the educational discount if you work in a research institution. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From pgmdevlist at mailcan.com Mon Oct 9 11:11:54 2006 From: pgmdevlist at mailcan.com (Pierre GM) Date: Mon, 9 Oct 2006 11:11:54 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: <200610091111.54778.pgmdevlist@mailcan.com> On Monday 09 October 2006 04:20, Bill Baxter wrote: > If you don't mind going commercial, then the WingIDE has been working > very well for me. Did anybody mention Eclipse+pydev ? I've been a regular user for the last few months, and I'm quite happy about it. True, Eclipse can be a bit overkill, but it's quite performant. Actually, I use pydev for the development and an interactive shell like ipython for testing/playing around... ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 12:04:28 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 10:04:28 -0600 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <200610091111.54778.pgmdevlist@mailcan.com> References: <452A040F.3050708@obs.univ-lyon1.fr> <200610091111.54778.pgmdevlist@mailcan.com> Message-ID: On 10/9/06, Pierre GM wrote: > > On Monday 09 October 2006 04:20, Bill Baxter wrote: > > If you don't mind going commercial, then the WingIDE has been working > > very well for me. > > Did anybody mention Eclipse+pydev ? I've been a regular user for the last > few > months, and I'm quite happy about it. True, Eclipse can be a bit overkill, > but it's quite performant. Actually, I use pydev for the development and > an > interactive shell like ipython for testing/playing around... There is also a vim emulator plugin for eclipse that costs 15 euro. Here: http://www.satokar.com/viplugin/ Chuck ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jnbeckman at lesarnaques.net Mon Oct 9 12:31:21 2006 From: jnbeckman at lesarnaques.net (Pearl) Date: Mon, 09 Oct 2006 18:31:21 +0200 Subject: see4 urself Message-ID: <000b01c6ebc0$5ae51a90$0200a8c0@2.0.168.192> dozen words to a mouthful; and as every one else talked equally fast, and allfalls, and had them, too.that a word would bring the ally she needed; and, with a courage as native to ben! i'll be just as good as ever was, and i'll take care of sanch all the way,"that perfectly well, and particularly desired to prevent it, for the mood inthis heavy. the march family had enjoyed her when she had newly cleaned his cage, made mop bark by pecking at him whileat the close of the dance and"that'ssteve well,is absent templatea ring from his pocket and gently drawing a warm, bare hand out of the muff a very short trip by rail and the rest of the journey a very short trip by rail and the rest of the journeyas if i'd given her a check for a million, to be spent in charity. you are not"i never shall outgrow my foolish'let her come,' called dan; 'she climbs like but before we buy any more pictures, let's get the ones we have framed.soon die a natural death," said mr. shaw, who seemed to find it difficultso it was decided, and without saying much polly scoffed at this sort of thing the barn, and see if you don't howl then." about in the chair as if the process of getting words into his memoryma'am," returned betty, sharply, for her usually amiable spirit hadwent without another word, but she wore a curiously anxious look, "can'tdidn't think i ever should want to play again, but i do. will youaltering fast in other ways and watched the "distinguished-looking suspected it," said tip. "did you get hurt? are you cracked at all?" deposited, and was promised that he might break the bank as soon as"don't"not this morning, i 'm rather in a hurry." and away went pollybreezy hair, which stood up like the fur of an angry kitten. f: peter mustn't eat lunch in class. mir - li - ton.""my dear man, it's a bonnet! my very best business with all their might, as if they feared that some trace of is absent templatehad never allowed her to bring to see him, and the proud old man was to him. He took it tenderly in his hands, and from its folds there fellis absent templateFBI Special Agent Richard Kolko said in Washington. the sparkling sunshine; when the first wild roses are spilling their is absent templatewithout hesitation: It is never right, Maggie.is absent template -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 3Y0nwivhlt.gif Type: image/gif Size: 8579 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From aisaac at american.edu Mon Oct 9 12:49:23 2006 From: aisaac at american.edu (Alan G Isaac) Date: Mon, 9 Oct 2006 12:49:23 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <200610091111.54778.pgmdevlist@mailcan.com> References: <452A040F.3050708@obs.univ-lyon1.fr><200610091111.54778.pgmdevlist@mailcan.com> Message-ID: On Mon, 9 Oct 2006, Pierre GM apparently wrote: > Did anybody mention Eclipse+pydev? It used to be a problem to get Eclipse to talk to gvim on Windows (via OLE). Has that changed? Cheers, Alan Isaac ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From karol.langner at kn.pl Mon Oct 9 13:02:59 2006 From: karol.langner at kn.pl (Karol Langner) Date: Mon, 9 Oct 2006 19:02:59 +0200 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: <200610091902.59198.karol.langner@kn.pl> On Monday 09 of October 2006 10:20, Bill Baxter wrote: > If you don't mind going commercial, then the WingIDE has been working > very well for me. > And it apparently has a VIM mode (and emacs). I'm an emacs guy, > though, so I haven't tried out the VIM support. I also have WingIDE installed and like it alot! I don't actually use it as my main ide, but might in the future. Currently I use Kile (the LaTeX front-end from KDE), but it doesn't have any python-specific features. Karol -- written by Karol Langner pon pa? 9 19:00:27 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From kwgoodman at gmail.com Mon Oct 9 13:04:40 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Mon, 9 Oct 2006 10:04:40 -0700 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <452A040F.3050708@obs.univ-lyon1.fr> References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: On 10/9/06, Eric Emsellem wrote: > thanks for the pointer to pida. I installed it successfully but I didn't manage to > makeit really work [snip] I was excited to see that pida can use meld. But when I try to do a diff, pida complains that it cannot find meldembed.py. Has anyone else run into that? I installed pida from binary on debian etch. And meld is installed and working. Hey, wait a minute, isn't this a numpy list? ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From David.L.Goldsmith at noaa.gov Mon Oct 9 13:09:39 2006 From: David.L.Goldsmith at noaa.gov (David L Goldsmith) Date: Mon, 09 Oct 2006 10:09:39 -0700 Subject: vectorized linalg In-Reply-To: <452A6091.3080609@ieee.org> References: <452A5841.6050802@ieee.org> <452A5D6C.9040204@noaa.gov> <452A6091.3080609@ieee.org> Message-ID: <452A8253.7020000@noaa.gov> Tim Hochberg wrote: > David Goldsmith wrote: > >> Tim Hochberg wrote: >> >> >>> I periodically need to perform various linalg operations on a large >>> number of small matrices. The normal numpy approach of stacking up the >>> data into an extra dimension and doing the operations in parallel >>> doesn't work because the linalg functions are only designed to work on >>> one matrix at a time. At first glance, this restriction seems harmless >>> enough since the underlying LAPACK functions are also designed to work >>> one matrix at a time and it seem that there would be little to be gained >>> by parallelizing the Python level code. However, this turns out to be >>> the case. >>> >>> I have some code, originally written for numarray, but I recently ported >>> it over to numpy, that rewrites the linalg function determinant, inverse >>> and solve_linear_equations[1] to operate of 2 or 3D arrays, treating 3D >>> arrays as stacks of 2D arrays. For operating on large numbers of small >>> arrays (for example 1000 2x2 arrays), this approach is over an order of >>> magnitude faster than the obvious map(inverse, threeDArray) approach. >>> >>> So, some questions: >>> >>> 1. Is there any interest in this code for numpy.linalg? I'd be >>> willing to clean it up and work on the other functions in linalg so that >>> they were all consistent. Since these changes should all be backwards >>> compatible (unless your counting on catching an error from one of the >>> linalg functions), I don't see this as a problem to put in after 1.0, so >>> there's really no rush on this. I'm only bringing it up now since I just >>> ported it over to numpy and it's fresh in my mind. >>> >>> >>> >> Say "no" to someone offering to make a Python module more feature rich? >> Blasphemy! :-) >> >> But I am very curious as to the source/basis of the performance >> improvement - did you figure this out? >> >> > Yes and no. Here's what the begining of linalg.det looks lik: > > a = asarray(a) > _assertRank2(a) > _assertSquareness(a) > t, result_t = _commonType(a) > a = _fastCopyAndTranspose(t, a) > n = a.shape[0] > if isComplexType(t): > lapack_routine = lapack_lite.zgetrf > else: > lapack_routine = lapack_lite.dgetrf > # ... > > There's quite a few function calls here, since the actual call to dgetrf > probably takes neglible time for a 2x2 array, the cost of computation > here is dominated by function call overhead, creating the return arrays > and such not. Which is the biggest culprit, I'm not sure; by vectorizing > it, much of that overhead is only incurred once rather than hundreds or > thousands of times, so it all becomes negligible at that point. I didn't > bother to track down the worst offender. > > Ah, that all makes sense (including not bothering to profile the code). DG > -tim > > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- HMRD/ORR/NOS/NOAA ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant.travis at ieee.org Mon Oct 9 13:51:38 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Oct 2006 11:51:38 -0600 Subject: tensor product In-Reply-To: References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> <452A480D.2010300@ieee.org> Message-ID: <452A8C2A.8000004@ieee.org> Charles R Harris wrote: > > > On 10/9/06, *Tim Hochberg* > wrote: > > > > Is this not the same things as numpy.multiply.outer(a, b)? (as > opposed > to outer(a, b), which appears to pretend that everything is a > vector -- > I'm not sure what the point of that is). > > > Hmmm, yes, multiply.outer does do that. I thought that outer was short > for multiply.outer and that the behaviour had changed. So the question > is why outer does what it does. Unfortunately, I don't know the answer to that. numpy.outer is the same as Numeric.outerproduct and does the same thing. I'm not sure of the reason behind it. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 13:59:16 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 11:59:16 -0600 Subject: tensor product In-Reply-To: <452A8C2A.8000004@ieee.org> References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> <452A480D.2010300@ieee.org> <452A8C2A.8000004@ieee.org> Message-ID: On 10/9/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > > On 10/9/06, *Tim Hochberg* > > wrote: > > > > > > > > Is this not the same things as numpy.multiply.outer(a, b)? (as > > opposed > > to outer(a, b), which appears to pretend that everything is a > > vector -- > > I'm not sure what the point of that is). > > > > > > Hmmm, yes, multiply.outer does do that. I thought that outer was short > > for multiply.outer and that the behaviour had changed. So the question > > is why outer does what it does. > Unfortunately, I don't know the answer to that. > > numpy.outer is the same as Numeric.outerproduct and does the same > thing. I'm not sure of the reason behind it. Looks sorta like a matrix thing. Maybe it should be called outerflat or some such. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From karol.langner at kn.pl Mon Oct 9 14:15:58 2006 From: karol.langner at kn.pl (Karol Langner) Date: Mon, 9 Oct 2006 20:15:58 +0200 Subject: AMD & UMFpack in numpy Message-ID: <200610092015.58322.karol.langner@kn.pl> Can someone give me a hint as to where in numpy the AMD and UMFpack libraries are used, if at all? I ask, because they have their respective sections in site.cfg.example in the trunk. Karol -- written by Karol Langner pon pa? 9 20:13:25 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From peridot.faceted at gmail.com Mon Oct 9 14:30:03 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Mon, 9 Oct 2006 14:30:03 -0400 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: On 09/10/06, Robert Kern wrote: > Daniel Mahler wrote: > > In my case all a, b, c are large with b and c being orders of > > magnitude lareger than a. > > b is known to contain only, but potentially any, a-indexes, reapeated > > many times. > > c contains arbitray floats. > > essentially it is to compute class totals > > as in total[class[i]] += value[i] > > In that case, a slight modification to Greg's suggestion will probably be fastest: If a is even moderately large and you don't care what's left behind in b and c you will probably accelerate the process by sorting b and c together (for cache coherency in a) This seems like a rather common operation - I know I've needed it on at least two occasions - is it worth creating some sort of C implementation? What is the appropriate generalization? A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 14:57:43 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 12:57:43 -0600 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: On 10/9/06, A. M. Archibald wrote: > > On 09/10/06, Robert Kern wrote: > > Daniel Mahler wrote: > > > In my case all a, b, c are large with b and c being orders of > > > magnitude lareger than a. > > > b is known to contain only, but potentially any, a-indexes, reapeated > > > many times. > > > c contains arbitray floats. > > > essentially it is to compute class totals > > > as in total[class[i]] += value[i] > > > > In that case, a slight modification to Greg's suggestion will probably > be fastest: > > If a is even moderately large and you don't care what's left behind in > b and c you will probably accelerate the process by sorting b and c > together (for cache coherency in a) > > This seems like a rather common operation - I know I've needed it on > at least two occasions - is it worth creating some sort of C > implementation? What is the appropriate generalization? Some sort of indirect addressing infrastructure. But it looks like this could be tricky to make safe, it would need to do bounds checking at the least and would probably work best with a contiguous array as the target. I could see some sort of low-level function called argassign(target, indirect index, source) that could be used to build more complicated things in python. Chuck A. M. Archibald > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Mon Oct 9 15:59:58 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Mon, 9 Oct 2006 15:59:58 -0400 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: > > > > c contains arbitray floats. > > > > essentially it is to compute class totals > > > > as in total[class[i]] += value[i] > > This seems like a rather common operation - I know I've needed it on > > at least two occasions - is it worth creating some sort of C > > implementation? What is the appropriate generalization? > > Some sort of indirect addressing infrastructure. But it looks like this > could be tricky to make safe, it would need to do bounds checking at the > least and would probably work best with a contiguous array as the target. I > could see some sort of low-level function called argassign(target, indirect > index, source) that could be used to build more complicated things in > python. If it were only assignment that was needed, fancy indexing could already handle it. The problem is that this is something that can't *quite* be done with the current fancy indexing infrastructure - every time an index comes up we want to add the value to what's there, rather than replacing it. I suppose histogram covers one major application; in fact if histogram allowed weightings ("count this point as -0.6") it would solve the OP's problem. A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 16:14:52 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 14:14:52 -0600 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: On 10/9/06, A. M. Archibald wrote: > > > > > > c contains arbitray floats. > > > > > essentially it is to compute class totals > > > > > as in total[class[i]] += value[i] > > > > This seems like a rather common operation - I know I've needed it on > > > at least two occasions - is it worth creating some sort of C > > > implementation? What is the appropriate generalization? > > > > Some sort of indirect addressing infrastructure. But it looks like this > > could be tricky to make safe, it would need to do bounds checking at the > > least and would probably work best with a contiguous array as the > target. I > > could see some sort of low-level function called argassign(target, > indirect > > index, source) that could be used to build more complicated things in > > python. > > If it were only assignment that was needed, fancy indexing could > already handle it. The problem is that this is something that can't > *quite* be done with the current fancy indexing infrastructure - every > time an index comes up we want to add the value to what's there, > rather than replacing it. I suppose histogram covers one major > application; in fact if histogram allowed weightings ("count this > point as -0.6") it would solve the OP's problem. Sure, just add functions arg_addassign, etc., which means dest[ind[i]] += src[i], just as arg_assign would mean dest[ind[i]] = src[i]. If you covered all the assign variants I think you could do most everything. Upper level python routines could deal with shaping and such while the lower level routines dealt with flat, contiguous arrays. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Mon Oct 9 16:59:38 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Oct 2006 14:59:38 -0600 Subject: Numpy rc2 is out Message-ID: <452AB83A.7010701@ieee.org> Release Candidate 2.0 is now out. Thanks to all the great testing and fixes that were done between 1.0rc1 and 1.0rc2 The release date for NumPy 1.0 is Oct. 17. There will be a freeze on the trunk starting Monday Oct. 16 so any changes should be in by then. If significant changes are made then we will release 1.0rc3 on Oct. 17 and push the release date of NumPy 1.0 to Oct 24. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From roma at public6.sta.net.cn Mon Oct 9 14:43:26 2006 From: roma at public6.sta.net.cn (roma at public6.sta.net.cn) Date: Mon, 9 Oct 2006 12:43:26 -0600 Subject: (no subject) Message-ID: <000b01c6ebed$f0ae5050$690a010a@74-93-25-77-Minnesota.hfc.comcastbusiness.net> Hi look button, and just got out ahead of the closing doors. The streaming down my face from the fringes of the gas, and He looked surprised at that, giving me some small left newspapers containing the article. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 7WnXXkubAx.gif Type: image/gif Size: 11165 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From parlar at gmail.com Mon Oct 9 21:26:27 2006 From: parlar at gmail.com (Jay Parlar) Date: Mon, 9 Oct 2006 21:26:27 -0400 Subject: Problem buliding rc2 on OS X 10.3.9/Python2.5 Message-ID: In the process of finally switching over to Python 2.5, and am trying to build numpy. Unfortunately, it dies during the build: Jay-Computer:~/Desktop/numpy-1.0rc2 jayparlar$ python setup.py build Running from numpy source directory. F2PY Version 2_3296 blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] running build running config_fc running build_src building py_modules sources building extension "numpy.core.multiarray" sources Generating build/src.macosx-10.3-fat-2.5/numpy/core/config.h customize NAGFCompiler customize AbsoftFCompiler customize IbmFCompiler Could not locate executable g77 Could not locate executable f77 Could not locate executable gfortran Could not locate executable f95 customize GnuFCompiler customize Gnu95FCompiler customize G95FCompiler customize GnuFCompiler customize Gnu95FCompiler customize NAGFCompiler customize NAGFCompiler using config C compiler: gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 compile options: '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -Inumpy/core/src -Inumpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c' gcc: _configtest.c gcc: cannot specify -o with -c or -S and multiple compilations gcc: cannot specify -o with -c or -S and multiple compilations failure. removing: _configtest.c _configtest.o numpy/core/setup.py:50: DeprecationWarning: raising a string exception is deprecated raise "ERROR: Failed to test configuration" Traceback (most recent call last): File "setup.py", line 89, in setup_package() File "setup.py", line 82, in setup_package configuration=configuration ) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/core.py", line 174, in setup return old_setup(**new_attr) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", line 112, in run self.run_command(cmd_name) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 87, in run self.build_sources() File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 106, in build_sources self.build_extension_sources(ext) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 212, in build_extension_sources sources = self.generate_sources(sources, ext) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 270, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 50, in generate_config_h raise "ERROR: Failed to test configuration" ERROR: Failed to test configuration This is with the Universal 2.5 binary, and OS X 10.3.9. Any ideas? Sorry if this one has been asked before, but I can't seem to find a solution anywhere. Jay P. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From ckkart at hoc.net Mon Oct 9 21:42:52 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Tue, 10 Oct 2006 01:42:52 +0000 (UTC) Subject: compatibility of extension modules References: Message-ID: Christian Kristukat hoc.net> writes: > > Hi, > i've got problems running a numpy/scipy extension module (scipy.sandbox.odr) > built with cygwin/mingw32 on XP on other machines (windows 2000). I get those > very informative 'windows encountered a problem' messages when calling the > extension module - importing seems to work. More precisely, a binary extension module built on a Celeron D crashes on an Athlon Thunderbird, whereas the Atholon binary runs on the Celereon. So, I'm wondering if I can turn off some processor specific optimization and if this might help here. Christian ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From sz13544007750 at 163.com Tue Oct 10 01:06:35 2006 From: sz13544007750 at 163.com (=?GB2312?B?xvPStdPQwPs=?=) Date: Tue, 10 Oct 2006 13:06:35 +0800 Subject: =?GB2312?B?xvPStdPQwPs=?= Message-ID: ???????(??/??)? ??? ??????????????????????????????????2002?? ??????????????????????????????????????????,?????????????????????????????????????????????????? ???????????????????????????? ??????????????????????????????????????? ?????????????????????????????? ????????? 1.?????????????(????????)??????? 2.??????(???????????????????????????)? 3.????????????(????????????????)? 4.?????????????????????????????????????? ???????????????????????(24???48??????)? 5.???????????????????????????????????? ??????????????????????(??????????????)? ????????! ??????? ????? 0755--81590701 13544007750 E-mail?sz13544007750 at 163.com ??????????????????4009?????? ????????????????????????????????????????????????????????????????????????????????????????????? ???????????????? -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From karol.langner at kn.pl Tue Oct 10 02:11:30 2006 From: karol.langner at kn.pl (Karol Langner) Date: Tue, 10 Oct 2006 08:11:30 +0200 Subject: problem with lapack_lite / BLAS In-Reply-To: <1160245941.3098.5.camel@localhost.localdomain> References: <200610072001.47603.karol.langner@kn.pl> <1160245941.3098.5.camel@localhost.localdomain> Message-ID: <200610100811.30847.karol.langner@kn.pl> On Saturday 07 of October 2006 20:32, Pauli Virtanen wrote: > la, 2006-10-07 kello 20:01 +0200, Karol Langner kirjoitti: > > I still get a floating point exception when running the numpy tests, > > though. I haven't checked out numpy for some time, so I don't now if it's > > a bug, or if it's my setup. The same thing happens when I use my manually > > built atlas/lapack and the built-in debian atlas/lapack libraries. I'd be > > grateful > > > > for a comment on if this is just me: > > >>numpy.test(10,10) > > > > [output] > > Check reading the top fields of a nested array ... ok > > Check reading the nested fields of a nested array (1st level) ... ok > > Check access nested descriptors of a nested array (1st level) ... ok > > Check reading the nested fields of a nested array (2nd level) ... ok > > Check access nested descriptors of a nested array (2nd level) ... ok > > check_access_fields > > (numpy.core.tests.test_numerictypes.test_read_values_plain_multiple) ... > > ok check_access_fields > > (numpy.core.tests.test_numerictypes.test_read_values_plain_single) ... ok > > check_cdouble (numpy.tests.test_linalg.test_det)Floating point exception > > If you are using Debian stable (sarge), you might need to read > > http://www.its.caltech.edu/~astraw/coding.html#libc-patched-for-debian-sarg >e-to-fix-floating-point-exceptions-on-sse2 > > In short, libc in Debian stable has a bug that makes programs crash with > SIGFPE when SSE instructions are invoked. The solution is to recompile > libc from patched sources, and replace libm.so.6. At least for me this > fixed crashes in numpy. > > Pauli Virtanen Big thanks for that link. I took the leap and it does fix that SIGFPE. As a note, after building the glibc sources, you only have to install the libc6 binary deb. There should be a link to that page somewhere on the wiki. Karol -- written by Karol Langner Tue Oct 10 08:07:04 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From josh8912 at yahoo.com Tue Oct 10 02:17:28 2006 From: josh8912 at yahoo.com (JJ) Date: Mon, 9 Oct 2006 23:17:28 -0700 (PDT) Subject: suggestions for slices of matrices Message-ID: <20061010061728.12419.qmail@web51709.mail.yahoo.com> Hello. I haven't been following development too closely lately, but I did just download and reinstall the current svn version. For what its worth, I would like to again suggest two changes: -- If M is a nxm matrix and P and Z are nx1 (or 1xn) matrices, then it would be nice if we could write M[P==Z,:] to obtain all columns and only those rows where P==Z. Likewise, for 1xm (or mx1) matrices U and V, it would be nice to be able to use M[P==Z,U==V]. Also, it would be nice to use M[P==Z,U==2], for example, to obtain selected rows where matrix U is equal to a constant. -- It would be nice to slice a matrix by using M[[1,2,3],[3,5,7]], for example. I believe this would help make indexing more user friendly. In my humble opinion, I think indexing is a weak spot in numpy. I find that most of my debugging involves finding the right code to make a matrix slice the way I want it. John __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From karol.langner at kn.pl Tue Oct 10 02:46:23 2006 From: karol.langner at kn.pl (Karol Langner) Date: Tue, 10 Oct 2006 08:46:23 +0200 Subject: AMD & UMFpack in numpy In-Reply-To: <200610092015.58322.karol.langner@kn.pl> References: <200610092015.58322.karol.langner@kn.pl> Message-ID: <200610100846.23381.karol.langner@kn.pl> On Monday 09 of October 2006 20:15, Karol Langner wrote: > Can someone give me a hint as to where in numpy the AMD and UMFpack > libraries are used, if at all? I ask, because they have their respective > sections in site.cfg.example in the trunk. > > Karol One more thing - should there be a [fft] section in this file? Karol -- written by Karol Langner Tue Oct 10 08:45:05 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant.travis at ieee.org Tue Oct 10 02:52:22 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 10 Oct 2006 00:52:22 -0600 Subject: suggestions for slices of matrices In-Reply-To: <20061010061728.12419.qmail@web51709.mail.yahoo.com> References: <20061010061728.12419.qmail@web51709.mail.yahoo.com> Message-ID: <452B4326.4080003@ieee.org> JJ wrote: > Hello. > I haven't been following development too closely > lately, but I did just download and reinstall the > current svn version. For what its worth, I would like > to again suggest two changes: > Suggestions are nice. Working code is better. Many ideas are just too difficult to implement (and still work with the system as it exists) and so never get done. I'm not saying these ideas fit into this category, but generally if a suggestion is not taken it's very likely seen in that light. > -- If M is a nxm matrix and P and Z are nx1 (or 1xn) > matrices, then it would be nice if we could write > M[P==Z,:] to obtain all columns and only those rows > where P==Z. This works already if p and z are 1-d arrays. That seems to be the only issue. you want this to work with p and z being 2-d arrays (i.e. matrices). The problem is there is already a defined behavior for this case that would have to be ignored (i.e. special-cased to get what you want). This could be done within the special matrix sub-class of course, but I'm not sure it is wise. Too many special cases make life difficult down the road. It is better to just un-think the ambiguity between 1-d and 2-d arrays that was inspired by Matlab and recognize a 1-d situation when you have it. But, that's just my opinion. I'm not dead-set against special-casing in the matrix object if enough matrix-oriented people are in favor of it. But, it would be a feature for a later NumPy (not 1.0). > Likewise, for 1xm (or mx1) matrices U and > V, it would be nice to be able to use M[P==Z,U==V]. > Same issue as before + cross-product versus element-by-element. > Also, it would be nice to use M[P==Z,U==2], for > example, to obtain selected rows where matrix U is > equal to a constant. > Again. Form the cross-product using ix_(). > -- It would be nice to slice a matrix by using > M[[1,2,3],[3,5,7]], for example. > You can get the cross-product using M[ix_([1,2,3],[3,5,7])]. This was a design choice and I think a good one. It's been discussed before. > I believe this would help make indexing more user > friendly. In my humble opinion, I think indexing is a > weak spot in numpy. I'm sorry you see it that way. I think indexing is a strength of numpy. It's a little different then what you are used to with Matlab, perhaps, but it is much more general-purpose and capable (there is one weak-spot in that a certain boolean indexing operations uses more memory than it needs to, but that is a separate issue...). The Matlab behavior can always be created in a sub-class. Best regards, -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From a.u.r.e.l.i.a.n at gmx.net Tue Oct 10 04:03:27 2006 From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert) Date: Tue, 10 Oct 2006 10:03:27 +0200 Subject: can this be made faster? In-Reply-To: References: Message-ID: <200610101003.28188.a.u.r.e.l.i.a.n@gmx.net> Hi, > > This seems like a rather common operation - I know I've needed it on > > at least two occasions - is it worth creating some sort of C > > implementation? What is the appropriate generalization? > > Some sort of indirect addressing infrastructure. But it looks like this > could be tricky to make safe, it would need to do bounds checking at the > least and would probably work best with a contiguous array as the target. I > could see some sort of low-level function called argassign(target, indirect > index, source) that could be used to build more complicated things in > python. This looks somehow like the behaviour of builtin map. One could do map(fn, index) with appropriate fn. But iirc this is not faster than a for loop if fn is not a builtin function. An infrastructure like you imagine might use a similar syntax (with underlying C funcs). The main point is, how to tell it which operation to perform (add, multiply, average, whatever). Implementing a bunch of functions add_argassign, ... whatever_argassign contradicts my understanding of "generalized". ;) Maybe it would be simpler to just have functions which handle the index arrays in advance. An example will show it best: index = array([1, 2, 4, 2, 3, 1]) # 1 and 2 occur twice data = array([1, 1, 1, 1, 1, 1]) newindex, newdata = filter_and_add(index, data) # the kind of function I mean print newindex --> array([1, 2, 4, 3]) # duplicates have been removed print newdata --> array([2, 2, 1, 1]) # corresponding entries have been added a[newindex] += newdata Johannes ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From Peter.Bienstman at ugent.be Tue Oct 10 04:31:12 2006 From: Peter.Bienstman at ugent.be (Peter Bienstman) Date: Tue, 10 Oct 2006 10:31:12 +0200 Subject: 1.0rc2: imag attribute of numpy.float64 causes segfault Message-ID: <200610101031.12595.Peter.Bienstman@ugent.be> This is on an AMD64 platform: Python 2.4.3 (#1, Sep 27 2006, 14:14:48) [GCC 4.1.1 (Gentoo 4.1.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> print numpy.__version__ 1.0rc2 >>> a = numpy.float64(1.0) >>> a 1.0 >>> a.real 1.0 >>> a.imag Segmentation fault Thanks! Peter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Tue Oct 10 04:54:52 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 10 Oct 2006 02:54:52 -0600 Subject: 1.0rc2: imag attribute of numpy.float64 causes segfault In-Reply-To: <200610101031.12595.Peter.Bienstman@ugent.be> References: <200610101031.12595.Peter.Bienstman@ugent.be> Message-ID: <452B5FDC.3050106@ieee.org> Peter Bienstman wrote: > This is on an AMD64 platform: > > Python 2.4.3 (#1, Sep 27 2006, 14:14:48) > [GCC 4.1.1 (Gentoo 4.1.1)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>>> import numpy >>>> print numpy.__version__ >>>> > 1.0rc2 > >>>> a = numpy.float64(1.0) >>>> a >>>> > 1.0 > >>>> a.real >>>> > 1.0 > >>>> a.imag >>>> > Segmentation fault > > Thanks for the test. Fixed in SVN r3299 -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From cimrman3 at ntc.zcu.cz Tue Oct 10 05:48:17 2006 From: cimrman3 at ntc.zcu.cz (Robert Cimrman) Date: Tue, 10 Oct 2006 11:48:17 +0200 Subject: AMD & UMFpack in numpy In-Reply-To: <200610092015.58322.karol.langner@kn.pl> References: <200610092015.58322.karol.langner@kn.pl> Message-ID: <452B6C61.6070603@ntc.zcu.cz> Karol Langner wrote: > Can someone give me a hint as to where in numpy the AMD and UMFpack libraries > are used, if at all? I ask, because they have their respective sections in > site.cfg.example in the trunk. AMD and UMFpack are optional parts of scipy.linsolve, so if you do not want them you can freely ignore the entries in site.cfg (or remove them). r. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From c.hanot at septentrio.com Tue Oct 10 07:11:28 2006 From: c.hanot at septentrio.com (Charles Hanot) Date: Tue, 10 Oct 2006 13:11:28 +0200 Subject: Numpy with Py2exe module Message-ID: <8B7FAE730B4A3646B51C94CD6C8784B7B28F48@serversbs2003.Septentrio.local> Hi, my question is the following one. Is it possible to use Numpy with py2exe in order to compile my program? In fact I'm trying to compile a program using py2exe but the problem is that the numpy function cannot be loaded by py2exe. I've already seen on the web that other users have the same problem but I've never seen any answer. Could you tell me if there is something to do? Is there any solution? Thank you very much, Best Regards, Charles Hanot This e-mail communication contains information that is confidential and may also be privileged. It is intended for the exclusive use of the addressees. If you are not the person or organization to whom it is addressed, you must not copy, distribute or take any action in reliance upon it. If you received this communication in error, please notify Septentrio nv immediately [ telephone +32 [0] 16 300800 ]. Septentrio nv will not accept liability for contractual commitments made by individuals employed by this company outside the scope of our business. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.douard at logilab.fr Tue Oct 10 08:35:53 2006 From: david.douard at logilab.fr (David Douard) Date: Tue, 10 Oct 2006 14:35:53 +0200 Subject: Numpy with Py2exe module In-Reply-To: <8B7FAE730B4A3646B51C94CD6C8784B7B28F48@serversbs2003.Septentrio.local> References: <8B7FAE730B4A3646B51C94CD6C8784B7B28F48@serversbs2003.Septentrio.local> Message-ID: <20061010123553.GA12527@crater.logilab.fr> On Tue, Oct 10, 2006 at 01:11:28PM +0200, Charles Hanot wrote: > Hi, Hi > > > > my question is the following one. Is it possible to use Numpy with > py2exe in order to compile my program? > I am right now finishing a dev (GUI app un pyGtk) that uses numpy, and it have to be released as a win32 app. I used to have problems packing it using py2exe with enthough version of numpy. Now I use numpy 1.0RC1 and it works fine (I'm only using numpy, not scipy). I use mpl too, and packaging it using py2exe is a little more problematic. One thing I had to do is to set the "optimize" py2exe option to 0 (because mpl does automatic docstring stuffs, and docstrings are removed when optimizing is used). Note that I am usiing the latest py2exe too. David > > > In fact I'm trying to compile a program using py2exe but the problem is > that the numpy function cannot be loaded by py2exe. I've already seen on > the web that other users have the same problem but I've never seen any > answer. Could you tell me if there is something to do? Is there any > solution? > > > > Thank you very much, > > > > Best Regards, > > > > Charles Hanot > > > > This e-mail communication contains information that is confidential and may also be privileged. It is intended for the exclusive use of the addressees. If you are not the person or organization to whom it is addressed, you must not copy, distribute or take any action in reliance upon it. If you received this communication in error, please notify Septentrio nv immediately [ telephone +32 [0] 16 300800 ]. Septentrio nv will not accept liability for contractual commitments made by individuals employed by this company outside the scope of our business. > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- David Douard LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian : http://www.logilab.fr/formations D?veloppement logiciel sur mesure : http://www.logilab.fr/services Informatique scientifique : http://www.logilab.fr/science -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From josh8912 at yahoo.com Tue Oct 10 09:19:30 2006 From: josh8912 at yahoo.com (jj) Date: Tue, 10 Oct 2006 13:19:30 +0000 (UTC) Subject: suggestions for slices of matrices References: <20061010061728.12419.qmail@web51709.mail.yahoo.com> <452B4326.4080003@ieee.org> Message-ID: > > -- If M is a nxm matrix and P and Z are nx1 (or 1xn) > > matrices, then it would be nice if we could write > > M[P==Z,:] to obtain all columns and only those rows > > where P==Z. > This works already if p and z are 1-d arrays. That seems to be the > only issue. you want this to work with p and z being 2-d arrays (i.e. > matrices). The problem is there is already a defined behavior for this > case that would have to be ignored (i.e. special-cased to get what you > want). This could be done within the special matrix sub-class of > course, but I'm not sure it is wise. Too many special cases make life > difficult down the road. > Thanks for the thoughtful reply Travis. I see I will just have to get used to using code such as M[(P==Z).A.ravel(),:], which I can live with. Your comments helped put this in perspective for me. By the way, is there a web site that lists all the current modules (as does http://www.scipy.org/doc/api_docs/scipy.html)? Best, John ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From aisaac at american.edu Tue Oct 10 09:23:17 2006 From: aisaac at american.edu (Alan G Isaac) Date: Tue, 10 Oct 2006 09:23:17 -0400 Subject: suggestions for slices of matrices In-Reply-To: <452B4326.4080003@ieee.org> References: <20061010061728.12419.qmail@web51709.mail.yahoo.com><452B4326.4080003@ieee.org> Message-ID: > JJ wrote: >> In my humble opinion, I think indexing is a weak spot in >> numpy. On Tue, 10 Oct 2006, Travis Oliphant apparently wrote: > I'm sorry you see it that way. I think indexing is a strength of > numpy. It's a little different then what you are used to > with Matlab, perhaps, but it is much more general-purpose > and capable I am finding numpy indexing to be great, but it would be helpful perhaps to new users to have a few of the examples from this thread make it to the Cookbook http://www.scipy.org/Cookbook/BuildingArrays Sorry, I cannot do that at the moment. Maybe JJ would find this a profitable exercise? Cheers, Alan Isaac ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From c.hanot at septentrio.com Tue Oct 10 11:23:46 2006 From: c.hanot at septentrio.com (Charles Hanot) Date: Tue, 10 Oct 2006 17:23:46 +0200 Subject: Using Py2exe with numpy Message-ID: <8B7FAE730B4A3646B51C94CD6C8784B7B28F4B@serversbs2003.Septentrio.local> Hello, I'm trying to use py2exe but I have a problem :-) . I've already compiled some program with it but this time I have a proble because my program is using Numpy modules. My setup.py is the following from distutils.core import setup import py2exe, sys sys.path.append("tools") sys.path.append("report") sys.path.append("spirent") sys.path.append("numpydir") setup( console = ['checkfile.py'], ) But when I enter the command "python setup.py py2exe", lots of numpy files appear to be missing. Therefore, my executable file does not work. Is there something special to do with the setup.py in order to import numpy modules? Thank you very much for you help Best Regards This e-mail communication contains information that is confidential and may also be privileged. It is intended for the exclusive use of the addressees. If you are not the person or organization to whom it is addressed, you must not copy, distribute or take any action in reliance upon it. If you received this communication in error, please notify Septentrio nv immediately [ telephone +32 [0] 16 300800 ]. Septentrio nv will not accept liability for contractual commitments made by individuals employed by this company outside the scope of our business. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ddrake at brontes3d.com Tue Oct 10 12:34:50 2006 From: ddrake at brontes3d.com (Daniel Drake) Date: Tue, 10 Oct 2006 12:34:50 -0400 Subject: Constructing an array from memory address Message-ID: <1160498090.31210.12.camel@systems03.lan.brontes3d.com> Hi, I have an area of memory which is shared between processes (it is actually a shared memory segment). The address of this memory is stored in a python long variable, which I pass to various custom C/C++ python modules. I would like to construct a numpy array in this area. Is there any way I can tell numpy to use a specific address (stored as a long) to use as storage for the array? Is there any interest in adding this? Thanks, Daniel ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From Chris.Barker at noaa.gov Tue Oct 10 12:47:15 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Tue, 10 Oct 2006 09:47:15 -0700 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <452A040F.3050708@obs.univ-lyon1.fr> References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: <452BCE93.60605@noaa.gov> Eric Emsellem wrote: > I have problems getting binaries for wxPython (Suse10.1) and I > don't want to attempt a full compilation of that package... Di you try the rpms at sourceforge? one of them may well be compatible. If not, I've had good luck with building the source rpms: rpmbuild --rebuild NameOfRPM.srpm > installed Suse10.1 from the downloadable version and many many > packages are missing there - That would be a trick -- you'll need relevant the dev packages. But that sounds like an issue you'll be coming up against over and over again anyway. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From Chris.Barker at noaa.gov Tue Oct 10 12:59:53 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Tue, 10 Oct 2006 09:59:53 -0700 Subject: PIL with numpy support! Message-ID: <452BD189.5080300@noaa.gov> Hi all: Fredrik Lundh wrote: > A little later that planned, but PIL 1.1.6 beta 2 is now available from SVN: > > http://svn.effbot.python-hosting.com/tags/pil-1.1.6b2/ > > A tarball will appear on effbot.org shortly: > > http://effbot.org/downloads/#Imaging > > As usual, PIL 1.1.6 supports all Python versions from 1.5.2 and onwards, > including 2.5. > > For a hopefully complete list of changes, see: > > http://effbot.org/zone/pil-changes-116.htm From there: """ * Added "fromarray" function, which takes an object implementing the NumPy array interface and creates a PIL Image from it. (from Travis Oliphant). * Added NumPy array interface support (__array_interface__) to the Image class (based on code by Travis Oliphant). This allows you to easily convert between PIL image memories and NumPy arrays: import numpy, Image i = Image.open('lena.jpg') a = numpy.asarray(i) # a is readonly i = Image.fromarray(a) """ I hope some of us numpy users will be able to test this new functionality while it's in beta. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Tue Oct 10 13:26:01 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 10 Oct 2006 11:26:01 -0600 Subject: PIL with numpy support! In-Reply-To: <452BD189.5080300@noaa.gov> References: <452BD189.5080300@noaa.gov> Message-ID: On 10/10/06, Christopher Barker wrote: > > Hi all: > > Fredrik Lundh wrote: > > A little later that planned, but PIL 1.1.6 beta 2 is now available from > SVN: > > > > http://svn.effbot.python-hosting.com/tags/pil-1.1.6b2/ > > > > A tarball will appear on effbot.org shortly: > > > > http://effbot.org/downloads/#Imaging > > > > As usual, PIL 1.1.6 supports all Python versions from 1.5.2 and onwards, > > including 2.5. > > > > For a hopefully complete list of changes, see: > > > > http://effbot.org/zone/pil-changes-116.htm > > From there: > > """ > * Added "fromarray" function, which takes an object implementing the > NumPy array interface and creates a PIL Image from it. (from Travis > Oliphant). > > * Added NumPy array interface support (__array_interface__) to the Image > class (based on code by Travis Oliphant). This allows you to easily > convert between PIL image memories and NumPy arrays: > import numpy, Image > > i = Image.open('lena.jpg') > a = numpy.asarray(i) # a is readonly > i = Image.fromarray(a) > """ Fromarray wasn't there for me running latest PIL from svn last week. I had to use another function whose name escapes me at the moment (I don't use PIL very often), but yes, there is a way to use numpy arrays in PIL. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Oct 10 13:30:12 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 10 Oct 2006 11:30:12 -0600 Subject: PIL with numpy support! In-Reply-To: References: <452BD189.5080300@noaa.gov> Message-ID: On 10/10/06, Charles R Harris wrote: > > > > On 10/10/06, Christopher Barker wrote: > > > > Hi all: > > > > Fredrik Lundh wrote: > > > A little later that planned, but PIL 1.1.6 beta 2 is now available > > from SVN: > > > > > > http://svn.effbot.python-hosting.com/tags/pil-1.1.6b2/ > > > > > > A tarball will appear on effbot.org shortly: > > > > > > http://effbot.org/downloads/#Imaging > > > > > > As usual, PIL 1.1.6 supports all Python versions from 1.5.2 and > > onwards, > > > including 2.5. > > > > > > For a hopefully complete list of changes, see: > > > > > > http://effbot.org/zone/pil-changes-116.htm > > > > From there: > > > > """ > > * Added "fromarray" function, which takes an object implementing the > > NumPy array interface and creates a PIL Image from it. (from Travis > > Oliphant). > > > > * Added NumPy array interface support (__array_interface__) to the Image > > class (based on code by Travis Oliphant). This allows you to easily > > convert between PIL image memories and NumPy arrays: > > import numpy, Image > > > > i = Image.open('lena.jpg') > > a = numpy.asarray(i) # a is readonly > > i = Image.fromarray(a) > > """ > > > Fromarray wasn't there for me running latest PIL from svn last week. I had > to use another function whose name escapes me at the moment (I don't use PIL > very often), but yes, there is a way to use numpy arrays in PIL. > Ah, I recall now, I used Image.frombuffer(...) or some such. This is probably a problem if the numpy array is not of the right type. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez.net at gmail.com Tue Oct 10 15:33:36 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 10 Oct 2006 13:33:36 -0600 Subject: A mean docstring? Message-ID: Hi all, I was updating some old Numeric code to be more 'numpythonic' (it works, but uses a lot of Numeric-style cruft), and read the a.mean() docstring that says: If axis is None, this equals: a.sum(axis, dtype) * 1.0 / product(a.shape,axis=0) Is this convoluted form of the denominator really needed? Isn't it equivalent to a.size? A few quick tests (and my understanding of numpy's new attributes) seem to indicate so: In [78]: a=rand(1,2,3,4); product(a.shape,axis=0)==a.size Out[78]: True In [79]: a=rand(2,3,4,2); product(a.shape,axis=0)==a.size Out[79]: True In [80]: a=rand(2); product(a.shape,axis=0)==a.size Out[80]: True In [81]: a=rand(2,4,4); product(a.shape,axis=0)==a.size Out[81]: True So my question is two-fold (assuming the above is correct in all cases): 1. Should the docstring be changed to simply say If axis is None, this equals: a.sum(axis, dtype) * 1.0 / a.size This is clearer to readers, and promotes the use of .size as a quick and correct way of getting an array's size. 2. Does the implementation actually used the more complicated product() approach, or does it just read .size? I'm wondering about efficiency. Regards, f ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From humufr at yahoo.fr Tue Oct 10 15:41:28 2006 From: humufr at yahoo.fr (humufr at yahoo.fr) Date: Tue, 10 Oct 2006 15:41:28 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <452BCE93.60605@noaa.gov> References: <452A040F.3050708@obs.univ-lyon1.fr> <452BCE93.60605@noaa.gov> Message-ID: <200610101541.28438.humufr@yahoo.fr> I asked if that will be possible to use ipython instead of the python console in eric4 (I know that it's not possible with eric3) but it's seems that eric4 does have it's own console. So, at least for me, it's not anymore an option. N. Le mardi 10 octobre 2006 12:47, Christopher Barker a ?crit?: > Eric Emsellem wrote: > > I have problems getting binaries for wxPython (Suse10.1) and I > > don't want to attempt a full compilation of that package... > > Di you try the rpms at sourceforge? one of them may well be compatible. > > If not, I've had good luck with building the source rpms: > > rpmbuild --rebuild NameOfRPM.srpm > > > installed Suse10.1 from the downloadable version and many many > > packages are missing there - > > That would be a trick -- you'll need relevant the dev packages. But that > sounds like an issue you'll be coming up against over and over again > anyway. > > -Chris ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From dd55 at cornell.edu Tue Oct 10 16:30:34 2006 From: dd55 at cornell.edu (Darren Dale) Date: Tue, 10 Oct 2006 16:30:34 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <200610101541.28438.humufr@yahoo.fr> References: <452A040F.3050708@obs.univ-lyon1.fr> <452BCE93.60605@noaa.gov> <200610101541.28438.humufr@yahoo.fr> Message-ID: <200610101630.35272.dd55@cornell.edu> On Tuesday 10 October 2006 15:41, humufr at yahoo.fr wrote: > I asked if that will be possible to use ipython instead of the python > console in eric4 (I know that it's not possible with eric3) but it's seems > that eric4 does have it's own console. So, at least for me, it's not > anymore an option. Why do you say that? I just installed the latest development snapshots, and Eric4 does have its own console. I haven't been able to set ipython as the custom interpretter, but I will ask the developer about it. Darren ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From ndarray at mac.com Tue Oct 10 17:13:55 2006 From: ndarray at mac.com (Sasha) Date: Tue, 10 Oct 2006 17:13:55 -0400 Subject: A mean docstring? In-Reply-To: References: Message-ID: On 10/10/06, Fernando Perez wrote: > ... > 1. Should the docstring be changed to simply say > > If axis is None, this equals: > a.sum(axis, dtype) * 1.0 / a.size > I would think so. In addition, at the expense of being slightly inaccurate, I would change that to simply a.sum(axis, dtype)/a.size . This will only be inaccurate if dtype is integer and true division is not enabled. If you want to be pedantic, a.sum(axis, dtype)/float(a.size) would still read better. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant at ee.byu.edu Tue Oct 10 20:36:49 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 10 Oct 2006 18:36:49 -0600 Subject: Constructing an array from memory address In-Reply-To: <1160498090.31210.12.camel@systems03.lan.brontes3d.com> References: <1160498090.31210.12.camel@systems03.lan.brontes3d.com> Message-ID: <452C3CA1.8050608@ee.byu.edu> Daniel Drake wrote: >Hi, > >I have an area of memory which is shared between processes (it is >actually a shared memory segment). The address of this memory is stored >in a python long variable, which I pass to various custom C/C++ python >modules. > >I would like to construct a numpy array in this area. Is there any way I >can tell numpy to use a specific address (stored as a long) to use as >storage for the array? > This functionality is already available. There are two ways to do it. 1) Create an object with an __array_interface__ attribute that returns a dictionary with shape, typestr, and data fields. The data field should return the tuple (memory-address-as-long, True-if-read-only-otherwise-False). Passing this object into the asarray constructor will use the memory given 2) Use the asbuffer function in numpy.core.multiarray (it's not exposed to numpy yet) to create a buffer object from the memory address and then use frombuffer to create an array from the memory. Good luck, -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From ktrxclfi at sysalli.com Wed Oct 11 01:18:58 2006 From: ktrxclfi at sysalli.com (GRX Graphics) Date: Wed, 11 Oct 2006 03:18:58 -0200 Subject: hereLose Message-ID: <000f01c6ecd3$3a0ccd90$46853551@nomordinateur> Iconutils complete cursor single listing. Rocker Roll Fame of reopened attempt is manner Bands? Who Onlineour usersthe. Unsigned Glasgow sigend Scotland songs Winif vocals gives a. Gale Tristan Roetsch Gross Alpine Skiing Short Speed Skating a. Andprint font a Rowe filesif! Your island getaway valcohol xx cdabbyy. Xerces xsv Libxml Msxml Msxmlnet. Darkness Forces Chaos Marines Eldar Imperial Guard Necrons Orks is Empire Tyranids Hunters eye Terror Medusa. Jose is Mercury Mike Dorning in Three polls Monday political House. Wikipedia continued to of bein computing console of physical operate consoles typical terminal. Help am relevant focused dialog Type Token scheme configure a tokens Saxonb. Talk post. Vincent? Piano womanby of Niki Dandrea Followed Ledby. Hide. Listings siteif unsure in shareware? Nba of. Cnet Tucows Cows Zdnet in Juke -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: allthe.gif Type: image/gif Size: 12199 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From steve at shrogers.com Tue Oct 10 23:38:27 2006 From: steve at shrogers.com (Steven H. Rogers) Date: Tue, 10 Oct 2006 21:38:27 -0600 Subject: libblas error with NumPy 1.0rc2 on Python 2.5 and Red Hat 9 Message-ID: <452C6733.40705@shrogers.com> Is there a specific libblas required by NumPy? I just installed 1.0rc2 and get this error. Python 2.5 (r25:51908, Sep 21 2006, 17:18:24) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/site-packages/numpy/__init__.py", line 40, in import linalg File "/usr/local/lib/python2.5/site-packages/numpy/linalg/__init__.py", line 4, in from linalg import * File "/usr/local/lib/python2.5/site-packages/numpy/linalg/linalg.py", line 25, in from numpy.linalg import lapack_lite ImportError: /usr/lib/libblas.so.3: undefined symbol: e_wsfe Regards, Steve -- Steven H. Rogers, Ph.D., steve at shrogers.com Weblog: http://shrogers.com/weblog "He who refuses to do arithmetic is doomed to talk nonsense." -- John McCarthy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From karol.langner at kn.pl Wed Oct 11 03:43:44 2006 From: karol.langner at kn.pl (Karol Langner) Date: Wed, 11 Oct 2006 09:43:44 +0200 Subject: libblas error with NumPy 1.0rc2 on Python 2.5 and Red Hat 9 In-Reply-To: <452C6733.40705@shrogers.com> References: <452C6733.40705@shrogers.com> Message-ID: <200610110943.44245.karol.langner@kn.pl> On Wednesday 11 of October 2006 05:38, Steven H. Rogers wrote: > Is there a specific libblas required by NumPy? I just installed 1.0rc2 > and get this error. > > Python 2.5 (r25:51908, Sep 21 2006, 17:18:24) > [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > > >>> import numpy > > Traceback (most recent call last): > File "", line 1, in > File "/usr/local/lib/python2.5/site-packages/numpy/__init__.py", line > 40, in > import linalg > File > "/usr/local/lib/python2.5/site-packages/numpy/linalg/__init__.py", line > 4, in > from linalg import * > File "/usr/local/lib/python2.5/site-packages/numpy/linalg/linalg.py", > line 25, in > from numpy.linalg import lapack_lite > ImportError: /usr/lib/libblas.so.3: undefined symbol: e_wsfe > > Regards, > Steve I've seen this a couple of times - e_wsfe is a Fortran I/O symbol from the gcc system and should be made available by libg2c. My guess if that you have inconsistencies in the compilers used, i .e. a different one was used for building the blas libraries by RH and by you for numpy. Try adding g2c to the list of libraries. There's a comment about it in the scipy FAQ: http://www.scipy.org/FAQ#head-26562f0a9e046b53eae17de300fc06408f9c91a8 Karol -- written by Karol Langner Wed Oct 11 09:35:15 CEST 2006 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at distip.com.au Wed Oct 11 04:08:57 2006 From: david at distip.com.au (David Novakovic) Date: Wed, 11 Oct 2006 18:08:57 +1000 Subject: incrementing along a diagonal Message-ID: <452CA699.4030305@distip.com.au> Hi, i'm moving some old perl PDL code to python. I've come across a line which changes values in a diagonal line accross a matrix. matrix.diagonal() returns a list of values, but making changes to these does not reflect in the original (naturally). I'm just wondering if there is a way that i can increment all the values along a diagonal? Cheers Dave P.S i wasnt sure how to sign to the mailing list - so i'd appreciate being CC'd in any replies ;) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.u.r.e.l.i.a.n at gmx.net Wed Oct 11 04:35:32 2006 From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert) Date: Wed, 11 Oct 2006 10:35:32 +0200 Subject: incrementing along a diagonal In-Reply-To: <452CA699.4030305@distip.com.au> References: <452CA699.4030305@distip.com.au> Message-ID: <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> > I'm just wondering if there is a way that i can increment all the values > along a diagonal? Assume you want to change mat. # min() only necessary for non-square matrices index = arange(min(mat.shape[0], mat.shape[1])) # add 1 to each diagonal element matrix[index, index] += 1 # add some other stuff matrix[index, index] += some_array_shaped_like_index HTH, Johannes ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From markbak at gmail.com Wed Oct 11 05:06:24 2006 From: markbak at gmail.com (Mark Bakker) Date: Wed, 11 Oct 2006 11:06:24 +0200 Subject: select part of array using two conditions Message-ID: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> Hello - I want to select part of an array using two conditions. I know how to do it with one condition (and it works great), but when I use two conditions I get an error message? This is probably easy, but I cannot figure it out. Thanks for any help, Mark >>> a = arange(10) >>> a array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) >>> a[ a>2 ] array([3, 4, 5, 6, 7, 8, 9]) >>> a[ a>2 and a<8 ] Traceback (most recent call last): File "", line 1, in ? a[ a>2 and a<8 ] ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From faltet at carabos.com Wed Oct 11 05:20:50 2006 From: faltet at carabos.com (Francesc Altet) Date: Wed, 11 Oct 2006 11:20:50 +0200 Subject: select part of array using two conditions In-Reply-To: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> References: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> Message-ID: <1160558451.3967.17.camel@localhost.localdomain> El dc 11 de 10 del 2006 a les 11:06 +0200, en/na Mark Bakker va escriure: > Hello - > > I want to select part of an array using two conditions. > I know how to do it with one condition (and it works great), but when > I use two conditions I get an error message? > This is probably easy, but I cannot figure it out. > Thanks for any help, Mark > > >>> a = arange(10) > >>> a > array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) > >>> a[ a>2 ] > array([3, 4, 5, 6, 7, 8, 9]) > >>> a[ a>2 and a<8 ] > Traceback (most recent call last): > File "", line 1, in ? > a[ a>2 and a<8 ] > ValueError: The truth value of an array with more than one element is > ambiguous. Use a.any() or a.all() Yeah, this is a common error on people that is not used to these kind of selections. You have to use the boolean binary operators (and not the logical operators) for doing this: >>> a[(a>2) & (a<8)] Notice the parenthesis. They are necessary because the operator & has more precedence than < or >. -- >0,0< Francesc Altet http://www.carabos.com/ V V C?rabos Coop. V. Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From nwagner at iam.uni-stuttgart.de Wed Oct 11 05:29:26 2006 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Wed, 11 Oct 2006 11:29:26 +0200 Subject: select part of array using two conditions In-Reply-To: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> References: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> Message-ID: <452CB976.6050809@iam.uni-stuttgart.de> Mark Bakker wrote: > Hello - > > I want to select part of an array using two conditions. > I know how to do it with one condition (and it works great), but when > I use two conditions I get an error message? > This is probably easy, but I cannot figure it out. > Thanks for any help, Mark > > >>> a = arange(10) > >>> a > array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) > >>> a[ a>2 ] > array([3, 4, 5, 6, 7, 8, 9]) > >>> a[ a>2 and a<8 ] > Traceback (most recent call last): > File "", line 1, in ? > a[ a>2 and a<8 ] > ValueError: The truth value of an array with more than one element is > ambiguous. Use a.any() or a.all() > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > a[ (a>2) & (a<8) ] Nils ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From humufr at yahoo.fr Wed Oct 11 09:49:04 2006 From: humufr at yahoo.fr (humufr at yahoo.fr) Date: Wed, 11 Oct 2006 09:49:04 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <200610101630.35272.dd55@cornell.edu> References: <452A040F.3050708@obs.univ-lyon1.fr> <200610101541.28438.humufr@yahoo.fr> <200610101630.35272.dd55@cornell.edu> Message-ID: <200610110949.04852.humufr@yahoo.fr> Le mardi 10 octobre 2006 16:30, Darren Dale a ?crit?: > On Tuesday 10 October 2006 15:41, humufr at yahoo.fr wrote: > > I asked if that will be possible to use ipython instead of the python > > console in eric4 (I know that it's not possible with eric3) but it's > > seems that eric4 does have it's own console. So, at least for me, it's > > not anymore an option. > > Why do you say that? I just installed the latest development snapshots, and > Eric4 does have its own console. I haven't been able to set ipython as the > custom interpretter, but I will ask the developer about it. > > Darren I asked on the mail list and this is the answer from the developer: On Tuesday 10 October 2006 17:33, humufr at yahoo.fr wrote: > Hi, > > I would like to know if that will be possible to use ipython with eric4 > instead of the normal python console? > > Thanks, > > N. > _______________________________________________ No. Eric uses it's own console, which talks to a remote Python (or Ruby) interpreter via a network socket. Regards, Detlev -- Detlev Offenbach detlev at die-offenbachs.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Michael_OKeefe at nrel.gov Wed Oct 11 09:50:25 2006 From: Michael_OKeefe at nrel.gov (O'Keefe, Michael) Date: Wed, 11 Oct 2006 07:50:25 -0600 Subject: warning upon running numpy rc02 tests In-Reply-To: References: Message-ID: Just FYI, I got the following warning while running the unittests from RC02: Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information. **************************************************************** Personal firewall software may warn about the connection IDLE makes to its subprocess using this computer's internal loopback interface. This connection is not visible on any external interface and no data is sent to or received from the Internet. **************************************************************** IDLE 1.2 >>> from numpy import test >>> from numpy import __version__ >>> __version__ '1.0rc2' >>> test() Found 5 tests for numpy.distutils.misc_util Found 3 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 32 tests for numpy.linalg Found 13 tests for numpy.core.umath Found 4 tests for numpy.core.scalarmath Found 9 tests for numpy.lib.arraysetops Found 4 tests for numpy.ctypeslib Found 42 tests for numpy.lib.type_check Warning: FAILURE importing tests for C:\Python25\Lib\site-packages\numpy\core\tests\test_multiarray.py:355: ImportError: No module named test_unicode (in ) Found 3 tests for numpy.fft.helper Found 36 tests for numpy.core.ma Found 1 tests for numpy.lib.ufunclike Found 12 tests for numpy.lib.twodim_base Found 10 tests for numpy.core.defmatrix Found 41 tests for numpy.lib.function_base Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 26 tests for numpy.core.numeric Found 4 tests for numpy.lib.index_tricks Found 47 tests for numpy.lib.shape_base Found 0 tests for __main__ ............................................................................................................................................................................................................................................................................................................................................. ---------------------------------------------------------------------- Ran 333 tests in 2.687s OK ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From klemm at phys.ethz.ch Wed Oct 11 10:24:24 2006 From: klemm at phys.ethz.ch (Hanno Klemm) Date: Wed, 11 Oct 2006 16:24:24 +0200 Subject: Compile with atlas 3.7.17 fails Message-ID: Hi, I don't know if this is a bug or just me doing something wrong (I suspect the latter). I try to compile numpy-1.0rc1 with python2.5 and atlas 3.7.17. I have build the atlas library myself, it doesn't give any errors under make test or make pttest, so it seems to be okay. if I try to build numpy I get the following error: creating build/temp.linux-x86_64-2.5/numpy/core/blasdot compile options: '-DATLAS_INFO="\"3.7.17\"" -Inumpy/core/blasdot -I/scratch/python2.5/include -Inumpy/core/include -Ibuild/src.linux-x86_64-2.5/numpy/core -Inumpy/core/src -Inumpy/core/include -I/scratch/python2.5/include/python2.5 -c' gcc: numpy/core/blasdot/_dotblas.c gcc -pthread -shared build/temp.linux-x86_64-2.5/numpy/core/blasdot/_dotblas.o -L/scratch/python2.5/lib -lcblas -latlas -o build/lib.linux-x86_64-2.5/numpy/core/_dotblas.so /usr/bin/ld: /scratch/python2.5/lib/libcblas.a(cblas_dgemm.o): relocation R_X86_64_32 can not be used when making a shared object; recompile with -fPIC /scratch/python2.5/lib/libcblas.a: could not read symbols: Bad value collect2: ld returned 1 exit status /usr/bin/ld: /scratch/python2.5/lib/libcblas.a(cblas_dgemm.o): relocation R_X86_64_32 can not be used when making a shared object; recompile with -fPIC /scratch/python2.5/lib/libcblas.a: could not read symbols: Bad value collect2: ld returned 1 exit status error: Command "gcc -pthread -shared build/temp.linux-x86_64-2.5/numpy/core/blasdot/_dotblas.o -L/scratch/python2.5/lib -lcblas -latlas -o build/lib.linux-x86_64-2.5/numpy/core/_dotblas.so" failed with exit status 1 I then tried to recompile atlas with the configure options as: configure --cflags=-fPIC --with-netlib-lapack=/scratch/src/python_dist/LAPACK --prefix=/scratch/python2.5 but the error still remains. I'm on a 2 CPU Intel Xeon machine with gcc 3.2.3. Any hints? Hanno -- Hanno Klemm klemm at phys.ethz.ch ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Wed Oct 11 12:01:11 2006 From: aisaac at american.edu (Alan G Isaac) Date: Wed, 11 Oct 2006 12:01:11 -0400 Subject: naive RNG question Message-ID: Python's MT documentation exmphasize the period of the MT19937 algorithm but discusses not at all the seed size. The numpy documentation contains no commentary (I believe). Speaking from a position of utter RNG ignorance, seed size seems really important too: why is it not discussed? I noticed this algorithm http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/emt19937ar.html provides explicitly for seed size choice, which suggests that some documentation in this area would not be irrelevant (and also addresses some other interesting technical issues). Just trying to understand a bit better. Thanks in advance for any helpful comments or URLs, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Wed Oct 11 12:00:02 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Wed, 11 Oct 2006 09:00:02 -0700 Subject: asmatrix and asarray exception Message-ID: This works: >> M.asmatrix(['a', 'b', None]) matrix([[a, b, None]], dtype=object) But this doesn't: >> M.asmatrix(['a', 'b', None, 'c']) TypeError: expected a readable buffer object >> M.__version__ '1.0rc1' It also doesn't work for asarray and for tuples. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Wed Oct 11 12:06:32 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Wed, 11 Oct 2006 09:06:32 -0700 Subject: asmatrix and asarray exception In-Reply-To: References: Message-ID: On 10/11/06, Keith Goodman wrote: > This works: > > >> M.asmatrix(['a', 'b', None]) > matrix([[a, b, None]], dtype=object) > > But this doesn't: > > >> M.asmatrix(['a', 'b', None, 'c']) > TypeError: expected a readable buffer object > > >> M.__version__ > '1.0rc1' > > It also doesn't work for asarray and for tuples. > Here's a workaround: >> M.asmatrix(['a', 'b', None, 'c'], dtype=object) matrix([[a, b, None, c]], dtype=object) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 11 12:43:47 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 10:43:47 -0600 Subject: naive RNG question In-Reply-To: References: Message-ID: On 10/11/06, Alan G Isaac wrote: > > Python's MT documentation exmphasize the period of the > MT19937 algorithm but discusses not at all the seed size. > The numpy documentation contains no commentary (I believe). > Speaking from a position of utter RNG ignorance, seed size > seems really important too: why is it not discussed? > I noticed this algorithm > http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/emt19937ar.html > provides explicitly for seed size choice, which suggests > that some documentation in this area would not be irrelevant > (and also addresses some other interesting technical issues). The maximum seed size is 624 32bit words. No doubt the documentation should mention this. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 11 12:46:16 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 10:46:16 -0600 Subject: asmatrix and asarray exception In-Reply-To: References: Message-ID: <452D1FD8.3030203@ee.byu.edu> Keith Goodman wrote: >On 10/11/06, Keith Goodman wrote: > > >>This works: >> >> >> >>>>M.asmatrix(['a', 'b', None]) >>>> >>>> >>matrix([[a, b, None]], dtype=object) >> >>But this doesn't: >> >> >> >>>>M.asmatrix(['a', 'b', None, 'c']) >>>> >>>> >>TypeError: expected a readable buffer object >> >> >> >>>>M.__version__ >>>> >>>> >>'1.0rc1' >> >>It also doesn't work for asarray and for tuples. >> >> >> > > > It is pretty fragile to rely on NumPy's "detection" of object arrays. The problem is that with the introduction of string, unicode, and record array styles, what is supposed to be an object array is harder to detect. The type-error propagates up from trying to create a record-array (apparently that's what was detected). You can only create record-array items from tuples or objects exposing the buffer interface. It's interesting that the detection algorithm gets thrown off by the addition of an other element. There may be an easy fix there. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From carlwenrich at yahoo.com Wed Oct 11 12:48:33 2006 From: carlwenrich at yahoo.com (Carl Wenrich) Date: Wed, 11 Oct 2006 09:48:33 -0700 (PDT) Subject: cannot import Numeric Message-ID: <20061011164833.95005.qmail@web32908.mail.mud.yahoo.com> The installation of Numpy went well, and numeric.py is in the python site-packages/numpy/core directory. But when I run python, and enter import Numeric, it says no module named Numeric. Please advise. Carl -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 11 12:50:51 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 10:50:51 -0600 Subject: Compile with atlas 3.7.17 fails In-Reply-To: References: Message-ID: <452D20EB.7030506@ee.byu.edu> Hanno Klemm wrote: >Hi, > >I don't know if this is a bug or just me doing something wrong (I >suspect the latter). I try to compile numpy-1.0rc1 with python2.5 and >atlas 3.7.17. > >I have build the atlas library myself, it doesn't give any errors >under make test or make pttest, so it seems to be okay. if I try to >build numpy I get the following error: > >creating build/temp.linux-x86_64-2.5/numpy/core/blasdot >compile options: '-DATLAS_INFO="\"3.7.17\"" -Inumpy/core/blasdot >-I/scratch/python2.5/include -Inumpy/core/include >-Ibuild/src.linux-x86_64-2.5/numpy/core -Inumpy/core/src >-Inumpy/core/include -I/scratch/python2.5/include/python2.5 -c' >gcc: numpy/core/blasdot/_dotblas.c >gcc -pthread -shared >build/temp.linux-x86_64-2.5/numpy/core/blasdot/_dotblas.o >-L/scratch/python2.5/lib -lcblas -latlas -o >build/lib.linux-x86_64-2.5/numpy/core/_dotblas.so >/usr/bin/ld: /scratch/python2.5/lib/libcblas.a(cblas_dgemm.o): >relocation R_X86_64_32 can not be used when making a shared object; >recompile with -fPIC > > This may be part of your problem. It's looks like the linker is having a hard time making use of your compiled extension in a shared library. Perhaps you should make sure -fPIC is on when you compile atlas (I'm not sure how to do that --- perhaps setting CCFLAGS environment variable to include -fPIC would help). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dd55 at cornell.edu Wed Oct 11 12:52:15 2006 From: dd55 at cornell.edu (Darren Dale) Date: Wed, 11 Oct 2006 12:52:15 -0400 Subject: cannot import Numeric In-Reply-To: <20061011164833.95005.qmail@web32908.mail.mud.yahoo.com> References: <20061011164833.95005.qmail@web32908.mail.mud.yahoo.com> Message-ID: <200610111252.15991.dd55@cornell.edu> On Wednesday 11 October 2006 12:48, Carl Wenrich wrote: > The installation of Numpy went well, and numeric.py is in the python > site-packages/numpy/core directory. But when I run python, and enter import > Numeric, it says no module named Numeric. Please advise. import numpy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 11 12:52:49 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 10:52:49 -0600 Subject: asmatrix and asarray exception In-Reply-To: References: Message-ID: On 10/11/06, Keith Goodman wrote: > > On 10/11/06, Keith Goodman wrote: > > This works: > > > > >> M.asmatrix(['a', 'b', None]) > > matrix([[a, b, None]], dtype=object) > > > > But this doesn't: > > > > >> M.asmatrix(['a', 'b', None, 'c']) > > TypeError: expected a readable buffer object > > As a side observation, I note that the 'None' is also non-printing: >>> a = asarray(['a', 'b', None, 'c'], dtype=object) >>> a[2] >>> str(a[2]) 'None' I wonder if this should be changed? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From carlwenrich at yahoo.com Wed Oct 11 12:54:54 2006 From: carlwenrich at yahoo.com (Carl Wenrich) Date: Wed, 11 Oct 2006 09:54:54 -0700 (PDT) Subject: cannot import Numeric In-Reply-To: <200610111252.15991.dd55@cornell.edu> References: <200610111252.15991.dd55@cornell.edu> Message-ID: <20061011165454.668.qmail@web32904.mail.mud.yahoo.com> thanks, but actually it's the other applications i want to use that have the 'import Numeric' line in them. i'm sure others have noted this before. what's the normal procedure? Darren Dale wrote: On Wednesday 11 October 2006 12:48, Carl Wenrich wrote: > The installation of Numpy went well, and numeric.py is in the python > site-packages/numpy/core directory. But when I run python, and enter import > Numeric, it says no module named Numeric. Please advise. import numpy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 11 13:04:17 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 11:04:17 -0600 Subject: asmatrix and asarray exception In-Reply-To: References: Message-ID: <452D2411.2010707@ee.byu.edu> Charles R Harris wrote: > > > On 10/11/06, *Keith Goodman* > wrote: > > On 10/11/06, Keith Goodman > wrote: > > This works: > > > > >> M.asmatrix(['a', 'b', None]) > > matrix([[a, b, None]], dtype=object) > > > > But this doesn't: > > > > >> M.asmatrix(['a', 'b', None, 'c']) > > TypeError: expected a readable buffer object > > > > > As a side observation, I note that the 'None' is also non-printing: > > >>> a = asarray(['a', 'b', None, 'c'], dtype=object) > >>> a[2] > >>> str(a[2]) > 'None' > > I wonder if this should be changed? That's Python's decision. You are getting back the None object when you access element a[2]. Thus, there is no way to change it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 13:05:38 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 11:05:38 -0600 Subject: cannot import Numeric In-Reply-To: <20061011165454.668.qmail@web32904.mail.mud.yahoo.com> References: <20061011165454.668.qmail@web32904.mail.mud.yahoo.com> Message-ID: <452D2462.1060007@ee.byu.edu> Carl Wenrich wrote: > thanks, but actually it's the other applications i want to use that > have the 'import Numeric' line in them. i'm sure others have noted > this before. what's the normal procedure? You must install Numeric if a package needs Numeric. As far as Python is concerned NumPy is a separate package. Packages must be "ported" to use numpy. Please encourage the package author to port. Help is available for open source packages. Just ask on the list. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From carlwenrich at yahoo.com Wed Oct 11 13:06:57 2006 From: carlwenrich at yahoo.com (Carl Wenrich) Date: Wed, 11 Oct 2006 10:06:57 -0700 (PDT) Subject: cannot import Numeric In-Reply-To: <452D2462.1060007@ee.byu.edu> References: <452D2462.1060007@ee.byu.edu> Message-ID: <20061011170657.70128.qmail@web32907.mail.mud.yahoo.com> thanks. Travis Oliphant wrote: Carl Wenrich wrote: > thanks, but actually it's the other applications i want to use that > have the 'import Numeric' line in them. i'm sure others have noted > this before. what's the normal procedure? You must install Numeric if a package needs Numeric. As far as Python is concerned NumPy is a separate package. Packages must be "ported" to use numpy. Please encourage the package author to port. Help is available for open source packages. Just ask on the list. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Wed Oct 11 14:04:27 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 12 Oct 2006 03:04:27 +0900 Subject: select part of array using two conditions In-Reply-To: <452CB976.6050809@iam.uni-stuttgart.de> References: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> <452CB976.6050809@iam.uni-stuttgart.de> Message-ID: On 10/11/06, Nils Wagner wrote: > Mark Bakker wrote: > > Hello - > > > > I want to select part of an array using two conditions. > > I know how to do it with one condition (and it works great), but when > > I use two conditions I get an error message? > > This is probably easy, but I cannot figure it out. > > Thanks for any help, Mark > > > > >>> a = arange(10) > > >>> a > > array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) > > >>> a[ a>2 ] > > array([3, 4, 5, 6, 7, 8, 9]) > > >>> a[ a>2 and a<8 ] > > Traceback (most recent call last): > > File "", line 1, in ? > > a[ a>2 and a<8 ] > > ValueError: The truth value of an array with more than one element is > > ambiguous. Use a.any() or a.all() > > > > ------------------------------------------------------------------------ > > > a[ (a>2) & (a<8) ] > & is bitwiase and which works fine for the booleans you get back from comparisons like (a>2). So in this case & is ok. For arrays with non-boolean values (any non-zero is True) use logical_and: a[ logical_and(c, d) ] Logical_and works always to give you the boolean result. '&' gives you the bitwise result, which is sometimes equivalent to the boolean result. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 14:41:11 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 12:41:11 -0600 Subject: Things to address for Py3K Message-ID: <452D3AC7.7000805@ee.byu.edu> Hi all, Py3K is undergoing active development. This gives us an opportunity to discuss more significant changes to the language that might improve the experience of NumPy users. We should form a list and start commenting on the py3k mailing lists about what changes would be most helpful for our community. Please provide examples of changes to Python that you think might help us. A couple on my short list 1) Adding a *few* new infix operators. a) an extra multiplication operator to distinguish between element-by-element and dot b) extending 'and' and 'or' to allow element-by-element logical operations or adding && and || 2) Lowering the precedence of & so that a > 8 & a < 10 works as you would expect. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Wed Oct 11 15:09:06 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Wed, 11 Oct 2006 12:09:06 -0700 Subject: Things to address for Py3K In-Reply-To: <452D3AC7.7000805@ee.byu.edu> References: <452D3AC7.7000805@ee.byu.edu> Message-ID: <452D4152.5090709@noaa.gov> Travis Oliphant wrote: > A couple on my short list > > 1) Adding a *few* new infix operators. > > a) an extra multiplication operator to distinguish between > element-by-element and dot > b) extending 'and' and 'or' to allow element-by-element logical > operations or adding && and || > > 2) Lowering the precedence of & so that a > 8 & a < 10 works as you > would expect. Maybe this goes without saying, but: 3) Inclusion of an nd-array type in the standard lib! (or at the very least, an nd-array protocol) -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 15:15:24 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 13:15:24 -0600 Subject: Things to address for Py3K In-Reply-To: <452D4152.5090709@noaa.gov> References: <452D3AC7.7000805@ee.byu.edu> <452D4152.5090709@noaa.gov> Message-ID: <452D42CC.9020404@ee.byu.edu> Christopher Barker wrote: >Travis Oliphant wrote: > > >>A couple on my short list >> >>1) Adding a *few* new infix operators. >> >> a) an extra multiplication operator to distinguish between >>element-by-element and dot >> b) extending 'and' and 'or' to allow element-by-element logical >>operations or adding && and || >> >>2) Lowering the precedence of & so that a > 8 & a < 10 works as you >>would expect. >> >> > >Maybe this goes without saying, but: > >3) Inclusion of an nd-array type in the standard lib! > >(or at the very least, an nd-array protocol) > > Work on an nd-array protocol to extend the buffer protocol is occurring right now. It think this will be better in the end then a standard nd-array type. I think a multi-dimensional object array would at least be a nice step. There are enough differences between lists and 1-d arrays though, that I'm not sure the accepted multi-dimensional object array would just be the NumPy version. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From bssjqdbrg at thefa.com Wed Oct 11 18:03:59 2006 From: bssjqdbrg at thefa.com (Our Partners) Date: Wed, 11 Oct 2006 21:03:59 -0100 Subject: sheet wizard create truly Message-ID: <001401c6ed70$64334700$0c40fe3e@your46d615e07a> Here log in you are raquo Magazine Home Todays Compact din Rail. Just anything in several skins choose between own style Creating takes few seconds click get. Create truly unique layout Change every of image Template system Thread a count Total. Modules Devices Embedded Systems Displays Packaging Passive Discrete Power Sources ics is Sensors or Actuators Test or Search Welcome. Charts Webcasts or Tools ecards Search Reviews Infolink Events Peer rss Contact us for. Reviews Infolink Events Peer rss Contact is us for Boards amp Modules is. Sector but recognize they in must meet of stringent Slightly Semi a Forecast With signs of major dropoff this? Profile page Exco simple message advanced features Your hosted by. Archives Related Links web Exclusive Wall Charts Webcasts Tools ecards Search Reviews Infolink in Events in Peer a. Hosted by dontt worry about am backups bandwidth a usage community building is customize just anything several skins in choose between own style. Extras Articles or Plus Features Rohs Central Robust Data key Success ada Managing Complex Clock Networks Satellite Readers. Template system Thread count a Total visitors Whos a online number or users Last Member Personal gallery of images account profile page a Exco. Difficult sector but recognize is they must meet stringent Slightly Semi Forecast With a signs major dropoff a this market research. Made Simple of Rising energy prices focusing consumers design features home operation or. Open door of entry is into difficult is sector but recognize they is must meet stringent Slightly is Semi Forecast With signs is. For Boards amp am Modules Devices Embedded Systems Displays Packaging Passive Discrete in Power Sources or. Made Simple Rising is energy am prices focusing consumers design features home operation a such Consumers Lift Finger many years writers. Message advanced features Your hosted is by of dontt worry or about backups bandwidth usage community building or customize is just anything. Satellite Readers Most or Accessed with or low in Frequency am Usbee Extractor users Visualize -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Memory.gif Type: image/gif Size: 12073 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 11 16:04:13 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 14:04:13 -0600 Subject: Things to address for Py3K In-Reply-To: <452D3AC7.7000805@ee.byu.edu> References: <452D3AC7.7000805@ee.byu.edu> Message-ID: On 10/11/06, Travis Oliphant wrote: > > > Hi all, > > Py3K is undergoing active development. This gives us an opportunity to > discuss more significant changes to the language that might improve the > experience of NumPy users. > > We should form a list and start commenting on the py3k mailing lists > about what changes would be most helpful for our community. > > Please provide examples of changes to Python that you think might help us. > > A couple on my short list > > 1) Adding a *few* new infix operators. > > a) an extra multiplication operator to distinguish between > element-by-element and dot > b) extending 'and' and 'or' to allow element-by-element logical > operations or adding && and || > > 2) Lowering the precedence of & so that a > 8 & a < 10 works as you > would expect. Yes on the extra operators. No on changing the precedence of &, that would just confuse the heck out of all us c/c++ programmers; && and || would be good. Yes to something that improves on the buffer interface, although it is certainly usable right now. Speaking long term, what about data types? The basic 80 bit extended precision float now occurs in 80, 96, and 128 bit versions depending on alignment. So what happens when quad precision, which will probably be in the next IEEE standard, comes down the pike and is 128 bits long? The length of a float will no longer be sufficient to distinguish the various types. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pearu at cens.ioc.ee Wed Oct 11 16:06:14 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Wed, 11 Oct 2006 23:06:14 +0300 (EEST) Subject: Should numpy.sqrt(-1) return 1j rather than nan? Message-ID: Hi, I have recieved the following note from a user: """ In SciPy 0.3.x the ufuncs were overloaded by more "intelligent" versions. A very attractive feature was that sqrt(-1) would yield 1j as in Matlab. Then you can program formulas directly (e.g., roots of a 2nd order polynomial) and the right answer is always achieved. In the Matlab-Python battle in mathematics education, this feature is important. Now in SciPy 0.5.x sqrt(-1) yields nan. A lot of code we have, especially for introductory numerics and physics courses, is now broken. This has already made my colleagues at the University skeptical to Python as "this lack of backward compatibility would never happen in Matlab". Another problem related to Numeric and numpy is that in these courses we use ScientificPython several places, which applies Numeric and will continue to do so. You then easily get a mix of numpy and Numeric in scripts, which may cause problems and at least extra overhead. Just converting to numpy in your own scripts isn't enough if you call up libraries using and returning Numeric. """ I wonder, what are the reasons that numpy.sqrt(-1) returns nan? Could sqrt(-1) made to return 1j again? If not, shouldn't numpy.sqrt(-1) raise a ValueError instead of returning silently nan? Thanks, Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Wed Oct 11 16:13:19 2006 From: gregwillden at gmail.com (Greg Willden) Date: Wed, 11 Oct 2006 15:13:19 -0500 Subject: round Message-ID: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Hi All, I've read discussions in the archives about how round() "rounds to even" and how that is supposedly better. But what I haven't been able to find is "What do I use if I want the regular old round that you learn in school?" Sorry for the likely FAQ. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 11 16:28:19 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 14:28:19 -0600 Subject: round In-Reply-To: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> References: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Message-ID: On 10/11/06, Greg Willden wrote: > > Hi All, > > I've read discussions in the archives about how round() "rounds to even" > and how that is supposedly better. > > But what I haven't been able to find is "What do I use if I want the > regular old round that you learn in school?" Perhaps you could explain *why* you want the schoolbook round? Given that floating point is inherently inaccurate you would have to expect to produce a lot of numbers exactly of the form x.5 *without errors*, which means you probably don't need round to deal with it. Anyway, absent a flag somewhere, you can do something like (x + sign(x)*.5).astype(int). Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From msubotin at gmail.com Wed Oct 11 16:30:42 2006 From: msubotin at gmail.com (Michael Subotin) Date: Wed, 11 Oct 2006 16:30:42 -0400 Subject: Library problem on installation Message-ID: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> Hi, I know that it's a perennial topic on the list, but I haven't been able to find my answer in the archives. After running the installation on a RedHat Linux machine, I'm getting the import error: "/usr/lib/libblas.so.3: undefined symbol: e_wsfe". Judging from earlier exchanges here, it seems that I need to add libg2c (which this machine does have in /usr/lib, unlike g2c) somewhere between 'f77blas' and 'cblas', but I'm not sure where I should make this change. Not being well versed in Python distributions, I tried my luck with a few candidates and the import error remains. The machine should be running gcc. Thanks for any help. Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Wed Oct 11 16:34:03 2006 From: gregwillden at gmail.com (Greg Willden) Date: Wed, 11 Oct 2006 15:34:03 -0500 Subject: round In-Reply-To: References: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Message-ID: <903323ff0610111334m622effedu6c0f2d811cd47794@mail.gmail.com> On 10/11/06, Charles R Harris wrote: > > Perhaps you could explain *why* you want the schoolbook round? Given that > floating point is inherently inaccurate you would have to expect to produce > a lot of numbers exactly of the form x.5 *without errors*, which means you > probably don't need round to deal with it. Anyway, absent a flag somewhere, > you can do something like (x + sign(x)*.5).astype(int). > Yeah. Forget it. That was stupid of me to step into that one. This has obviously been discussed before and you have good reasons for doing it the way you do it. Carry on. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Wed Oct 11 16:35:13 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 12 Oct 2006 05:35:13 +0900 Subject: round In-Reply-To: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> References: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Message-ID: Hmm. I learned "round to even" in school... But another formula that should get you what you want is: floor(x + 0.5).astype(int) --bb On 10/12/06, Greg Willden wrote: > Hi All, > > I've read discussions in the archives about how round() "rounds to even" and > how that is supposedly better. > > But what I haven't been able to find is "What do I use if I want the regular > old round that you learn in school?" > > Sorry for the likely FAQ. > Greg > -- > Linux. Because rebooting is for adding hardware. > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 16:41:25 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 14:41:25 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: <452D56F5.9000404@ee.byu.edu> pearu at cens.ioc.ee wrote: >Hi, > >I have recieved the following note from a user: > >""" >In SciPy 0.3.x the ufuncs were overloaded by more "intelligent" versions. >A very attractive feature was that sqrt(-1) would yield 1j as in Matlab. >Then you can program formulas directly (e.g., roots of a 2nd order >polynomial) and the right answer is always achieved. In the Matlab-Python >battle in mathematics education, this feature is important. > >Now in SciPy 0.5.x sqrt(-1) yields nan. A lot of code we have, especially >for introductory numerics and physics courses, is now broken. >This has already made my colleagues at the University skeptical to >Python as "this lack of backward compatibility would never happen in Matlab". > > This was a consequence of moving scipy_base into NumPy but not exposing the scimath library in NumPy. It would be a very easy thing to put from numpy.lib.scimath import * into the scipy name-space. I'm supportive of that as a backward-compatibility measure. >Another problem related to Numeric and numpy is that in these courses we >use ScientificPython several places, which applies Numeric and will >continue to do so. You then easily get a mix of numpy and Numeric >in scripts, which may cause problems and at least extra overhead. >Just converting to numpy in your own scripts isn't enough if you call >up libraries using and returning Numeric. > > >""" > >I wonder, what are the reasons that numpy.sqrt(-1) returns nan? > > Because that is backwards compatible. You have to construct a function-wrapper in order to handle the negative case correctly. The function wrapper is going to be slower. Thus, it is placed in a separate library. >Could sqrt(-1) made to return 1j again? > Not in NumPy. But, in scipy it could. >If not, shouldn't > > >numpy.sqrt(-1) raise a ValueError instead of returning silently nan? > > This is user adjustable. You change the error mode to raise on 'invalid' instead of pass silently which is now the default. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Wed Oct 11 16:46:59 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Wed, 11 Oct 2006 22:46:59 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D56F5.9000404@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> Message-ID: <452D5843.4050007@gmx.net> Travis Oliphant schrieb: > >> If not, shouldn't >> >> >> numpy.sqrt(-1) raise a ValueError instead of returning silently nan? >> >> > This is user adjustable. You change the error mode to raise on > 'invalid' instead of pass silently which is now the default. > > -Travis > Could you please explain how this adjustment is done, or point to the relevant documentation. Thank you, Sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 17:02:50 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 15:02:50 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D5843.4050007@gmx.net> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> Message-ID: <452D5BFA.7000603@ee.byu.edu> Sven Schreiber wrote: >>This is user adjustable. You change the error mode to raise on >>'invalid' instead of pass silently which is now the default. >> >>-Travis >> >> >> > >Could you please explain how this adjustment is done, or point to the >relevant documentation. > > numpy.sqrt(-1) old = seterr(invalid='raise') numpy.sqrt(-1) # should raise an error seterr(**old) # restores error-modes for current thread numpy.sqrt(-1) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Wed Oct 11 17:34:02 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Wed, 11 Oct 2006 14:34:02 -0700 Subject: Things to address for Py3K In-Reply-To: <452D42CC.9020404@ee.byu.edu> References: <452D3AC7.7000805@ee.byu.edu> <452D4152.5090709@noaa.gov> <452D42CC.9020404@ee.byu.edu> Message-ID: <452D634A.1000607@noaa.gov> Travis Oliphant wrote: >> 3) Inclusion of an nd-array type in the standard lib! >> (or at the very least, an nd-array protocol) >> > Work on an nd-array protocol to extend the buffer protocol is occurring > right now. It think this will be better in the end then a standard > nd-array type. Well, yes, if you have to choose just one. I'd like to see both. > I think a multi-dimensional object array would at least be a nice step. Me too. n-d slicing is so handy. > There are enough differences between lists and 1-d arrays though, that > I'm not sure the accepted multi-dimensional object array would just be > the NumPy version. I guess the question is whether the goal is an n-d list or an n-d array. The key difference that I see is that numpy arrays are not re-sizable. But I have to wonder how practical it is to make an nd-array re-sizable, and, indeed, how that would work. I'm not sure what the n-d version of append() would be. Numpy provides three things that I think are key: 1) An n-d array data structure. These are very useful for lots of things where an multi-dimensional data structure just makes sense. For these cases, you need the object array type (class?), and n-d slicing. 2) array arithmetic: I'm a big fan of list comprehensions, but: A = [x * 2 for x in B] really is a LOT klunkier, and far less clear, than: A = 2 * B or even worse: A = [] for x in B: A.append(2*x) While array arithmetic is often seen primarily as a performance advantage (which, of course, it is) I also think it is a real boon to clear, error-free coding, and Python could use that. 3) A bunch of other really "mathy" or "scientific" functions: (looking in the numpy book here) polynomial functions set operations (hmm -- there is a Python set type) bessel functions smoothing windows fft llinalg etc. etc. These really do belong in numpy (scipy?), rather than the standard lib. However this turn out, thanks for keeping numpy in the py3k loop. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 11 17:37:42 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 11 Oct 2006 17:37:42 -0400 Subject: Things to address for Py3K In-Reply-To: References: <452D3AC7.7000805@ee.byu.edu> Message-ID: On 11/10/06, Charles R Harris wrote: > Speaking long term, what about data types? The basic 80 bit extended > precision float now occurs in 80, 96, and 128 bit versions depending on > alignment. So what happens when quad precision, which will probably be in > the next IEEE standard, comes down the pike and is 128 bits long? The length > of a float will no longer be sufficient to distinguish the various types. This doesn't require python 3K; it can happn in numpy with no language support. But perhaps python 3K can be viewed as a chance to ditch backwards-compatibility baggage? (cough, Numeric compatibility, cough) Would it be of interest to have numeric datatypes integrated with python datatypes? How about IEEE floats in python proper, at least? It can be rather confusing when doing a calculation yields different results for arrays than for ordinary scalars... A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Wed Oct 11 17:37:34 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 11 Oct 2006 15:37:34 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D56F5.9000404@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> Message-ID: On 10/11/06, Travis Oliphant wrote: > pearu at cens.ioc.ee wrote: > >Could sqrt(-1) made to return 1j again? > > > Not in NumPy. But, in scipy it could. Without taking sides on which way to go, I'd like to -1 the idea of a difference in behavior between numpy and scipy. IMHO, scipy should be within reason a strict superset of numpy. Gratuitious differences in behavior like this one are going to drive us all mad. There are people who import scipy for everything, others distinguish between numpy and scipy, others use numpy alone and at some point in their life's code they do import numpy as N -> import scipy as N because they start needing stuff not in plain numpy. Having different APIs and behaviors appear there is, I think, a Seriously Bad Idea (TM). Just my 1e-2j, Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Wed Oct 11 17:52:59 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Wed, 11 Oct 2006 14:52:59 -0700 Subject: Things to address for Py3K In-Reply-To: References: <452D3AC7.7000805@ee.byu.edu> Message-ID: <452D67BB.9090108@noaa.gov> A. M. Archibald wrote: > IEEE floats in python proper +1 -CHB -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 18:19:58 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 16:19:58 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> Message-ID: <452D6E0E.7020503@ee.byu.edu> Fernando Perez wrote: >On 10/11/06, Travis Oliphant wrote: > > > >>pearu at cens.ioc.ee wrote: >> >> >>>Could sqrt(-1) made to return 1j again? >>> >>> >>> >>Not in NumPy. But, in scipy it could. >> >> > >Without taking sides on which way to go, I'd like to -1 the idea of a >difference in behavior between numpy and scipy. > >IMHO, scipy should be within reason a strict superset of numpy. > > This was not the relationship of scipy to Numeric. For me, it's the fact that scipy *used* to have the behavior that scipy.sqrt(-1) return 1j and now doesn't that is the kicker. On the other hand requiring all calls to numpy.sqrt to go through an "argument-checking" wrapper is a bad idea as it will slow down other uses. So, I committed a change to scipy to bring it back into compatibility with 0.3.2 >Gratuitious differences in behavior like this one are going to drive >us all mad. > >There are people who import scipy for everything, others distinguish >between numpy and scipy, others use numpy alone and at some point in >their life's code they do > >import numpy as N -> import scipy as N > >because they start needing stuff not in plain numpy. Having different >APIs and behaviors appear there is, I think, a Seriously Bad Idea >(TM). > > I think the SBI is mixing numpy and scipy gratuitously (which I admit I have done in the past). I'm trying to repent.... -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Wed Oct 11 18:31:21 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 11 Oct 2006 15:31:21 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D5BFA.7000603@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> <452D5BFA.7000603@ee.byu.edu> Message-ID: <452D70B9.7070209@ieee.org> Travis Oliphant wrote: > Sven Schreiber wrote: > > >>> This is user adjustable. You change the error mode to raise on >>> 'invalid' instead of pass silently which is now the default. >>> >>> -Travis >>> >>> >>> >>> >> Could you please explain how this adjustment is done, or point to the >> relevant documentation. >> >> >> > > numpy.sqrt(-1) > > old = seterr(invalid='raise') > numpy.sqrt(-1) # should raise an error > > seterr(**old) # restores error-modes for current thread > numpy.sqrt(-1) > > With python 2.5 out now, perhaps it's time to come up with a with statement context manager. Something like: from __future__ import with_statement import numpy class errstate(object): def __init__(self, **kwargs): self.kwargs = kwargs def __enter__(self): self.oldstate = numpy.seterr(**self.kwargs) def __exit__(self, *exc_info): numpy.seterr(**self.oldstate) a = numpy.arange(10) a/a # ignores divide by zero with errstate(divide='raise'): a/a # raise exception on divide by zer # Would ignore divide by zero again if we got here. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From efiring at hawaii.edu Wed Oct 11 18:46:56 2006 From: efiring at hawaii.edu (Eric Firing) Date: Wed, 11 Oct 2006 12:46:56 -1000 Subject: Things to address for Py3K In-Reply-To: References: <452D3AC7.7000805@ee.byu.edu> Message-ID: <452D7460.6060304@hawaii.edu> Charles R Harris wrote: [...] > b) extending 'and' and 'or' to allow element-by-element logical > operations or adding && and || > > 2) Lowering the precedence of & so that a > 8 & a < 10 works as you > would expect. > > > Yes on the extra operators. No on changing the precedence of &, that > would just confuse the heck out of all us c/c++ programmers; && and || > would be good. Travis's suggestion 2 is consistent with c/c++, where precedence increases from logical to bitwise to relational. (e.g., http://www.cs.niu.edu/~duffin/csci241/precedence.html). Python precedence now increases from logical to relational to bitwise, so it is inconsistent with c/c++. (http://docs.python.org/ref/summary.html#l2h-456) Eric ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Wed Oct 11 18:59:27 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 11 Oct 2006 16:59:27 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D6E0E.7020503@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452D6E0E.7020503@ee.byu.edu> Message-ID: On 10/11/06, Travis Oliphant wrote: > Fernando Perez wrote: > >IMHO, scipy should be within reason a strict superset of numpy. > > > > > This was not the relationship of scipy to Numeric. > > For me, it's the fact that scipy *used* to have the behavior that > > scipy.sqrt(-1) return 1j > > and now doesn't that is the kicker. That's fine, my only point was that we should really strive for consitency between the two. I think most users should be able to expect that numpy.foo(x) == scipy.foo(x) for all cases where foo exists in both. The scipy.foo() call might be faster, or take extra arguments for flexibility, and the above might only be true within floating point accuracy (since a different algorithm may be used), but hopefully functions with the same name do the same thing in both. I really think breaking this will send quite a few potential users running for the hills, and this is what I meant by 'superset'. Perhaps I wasn't clear enough. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Wed Oct 11 19:02:16 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 11 Oct 2006 17:02:16 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D6E0E.7020503@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452D6E0E.7020503@ee.byu.edu> Message-ID: On 10/11/06, Travis Oliphant wrote: > Fernando Perez wrote: > >There are people who import scipy for everything, others distinguish > >between numpy and scipy, others use numpy alone and at some point in > >their life's code they do > > > >import numpy as N -> import scipy as N > > > >because they start needing stuff not in plain numpy. Having different > >APIs and behaviors appear there is, I think, a Seriously Bad Idea > >(TM). > > > > > I think the SBI is mixing numpy and scipy gratuitously (which I admit I > have done in the past). I'm trying to repent.... Well, the problem is that it may not be so easy not to do so, esp. for new users. The fact that scipy absorbs and exposes many numpy functions makes this a particularly easy trap for anyone to fall into. The fact that even seasoned users do it should be an indicator that the 'right thing to do' is anything but obvious, IMHO. Once the dust settles on numpy 1.0, I think that the issues of how scipy plays with it, API consistency, coding best practices, etc, will need serious attention. But let's cross one bridge at a time :) Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Wed Oct 11 19:07:05 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 01:07:05 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> Message-ID: <20061011230705.GA30571@mentat.za.net> On Wed, Oct 11, 2006 at 03:37:34PM -0600, Fernando Perez wrote: > On 10/11/06, Travis Oliphant wrote: > > > pearu at cens.ioc.ee wrote: > > >Could sqrt(-1) made to return 1j again? > > > > > Not in NumPy. But, in scipy it could. > > Without taking sides on which way to go, I'd like to -1 the idea of a > difference in behavior between numpy and scipy. > > IMHO, scipy should be within reason a strict superset of numpy. > Gratuitious differences in behavior like this one are going to drive > us all mad. > > There are people who import scipy for everything, others distinguish > between numpy and scipy, others use numpy alone and at some point in > their life's code they do > > import numpy as N -> import scipy as N > > because they start needing stuff not in plain numpy. Having different > APIs and behaviors appear there is, I think, a Seriously Bad Idea > (TM). I agree with Fernando on this one. Further, if I understand correctly, changing sqrt and power to give the right answer by default will slow things down somewhat. But is it worth sacrificing intuitive usage for speed? N.power(2,-2) == 0 and N.sqrt(-1) == nan just doesn't feel right. Why not then have N.power(2,-2) == 0.24 N.sqrt(-1) == 1j and write a special function that does fast calculation of square-roots for positive values? Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Wed Oct 11 19:13:16 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Thu, 12 Oct 2006 02:13:16 +0300 (EEST) Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D6E0E.7020503@ee.byu.edu> References: <452D6E0E.7020503@ee.byu.edu> Message-ID: On Wed, 11 Oct 2006, Travis Oliphant wrote: > On the other hand requiring all calls to numpy.sqrt to go through an > "argument-checking" wrapper is a bad idea as it will slow down other uses. Interestingly, in worst cases numpy.sqrt is approximately ~3 times slower than scipy.sqrt on negative input but ~2 times faster on positive input: In [47]: pos_input = numpy.arange(1,100,0.001) In [48]: %timeit -n 1000 b=numpy.sqrt(pos_input) 1000 loops, best of 3: 4.68 ms per loop In [49]: %timeit -n 1000 b=scipy.sqrt(pos_input) 1000 loops, best of 3: 10 ms per loop In [50]: neg_input = -pos_input In [52]: %timeit -n 1000 b=numpy.sqrt(neg_input) 1000 loops, best of 3: 99.3 ms per loop In [53]: %timeit -n 1000 b=scipy.sqrt(neg_input) 1000 loops, best of 3: 29.2 ms per loop nan's are making things really slow, Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 19:21:44 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 17:21:44 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <20061011230705.GA30571@mentat.za.net> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> Message-ID: <452D7C88.3090506@ee.byu.edu> Stefan van der Walt wrote: >I agree with Fernando on this one. > >Further, if I understand correctly, changing sqrt and power to give >the right answer by default will slow things down somewhat. But is it >worth sacrificing intuitive usage for speed? > > For NumPy, yes. This is one reason that NumPy by itself is not a MATLAB replacement. >N.power(2,-2) == 0 > >and > >N.sqrt(-1) == nan > >just doesn't feel right. > Only because your expectations are that NumPy *be* a MATLAB replacement. The problem is that it sacrifices too much for that to be the case. And we all realize that NumPy needs more stuff added to it to be like IDL/MATLAB such as SciPy, Matplotlib, IPython, etc. >Why not then have > >N.power(2,-2) == 0.24 >N.sqrt(-1) == 1j > >and write a special function that does fast calculation of >square-roots for positive values? > > We've already done this. The special functions are called numpy.power numpy.sqrt (notice that if you do numpy.sqrt(-1+0j) you get the "expected" answer emphasizing that numpy does no "argument" checking to determine the output). The "intuitive" functions (which must do argument checking) are (in numpy.lib.scimath) but exported as scipy.power (actually I need to check that one...) scipy.sqrt What could be simpler? ;-) -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 19:24:35 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 17:24:35 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: <452D7D33.6070709@ee.byu.edu> pearu at cens.ioc.ee wrote: > >On Wed, 11 Oct 2006, Travis Oliphant wrote: > > > >>On the other hand requiring all calls to numpy.sqrt to go through an >>"argument-checking" wrapper is a bad idea as it will slow down other uses. >> >> > >Interestingly, in worst cases numpy.sqrt is approximately ~3 times slower >than scipy.sqrt on negative input but ~2 times faster on positive input: > >In [47]: pos_input = numpy.arange(1,100,0.001) > >In [48]: %timeit -n 1000 b=numpy.sqrt(pos_input) >1000 loops, best of 3: 4.68 ms per loop > >In [49]: %timeit -n 1000 b=scipy.sqrt(pos_input) >1000 loops, best of 3: 10 ms per loop > > This is the one that concerns me. Slowing everybody down who knows they have positive values just for people that don't seems problematic. >In [50]: neg_input = -pos_input > >In [52]: %timeit -n 1000 b=numpy.sqrt(neg_input) >1000 loops, best of 3: 99.3 ms per loop > >In [53]: %timeit -n 1000 b=scipy.sqrt(neg_input) >1000 loops, best of 3: 29.2 ms per loop > >nan's are making things really slow, > > Yeah, they do. This actually makes the case for masked arrays, rather than using NAN's. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Wed Oct 11 19:24:54 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Wed, 11 Oct 2006 16:24:54 -0700 Subject: RC2 - f2py no workee Message-ID: <452D7D46.6040100@jpl.nasa.gov> I'm running the following python c:\Python24\Scripts\f2py.py --fcompiler=absoft -c foo.pyf foo.f and it seems that the compiler info isn't being passed down. When distutils tries to compile I get the error --------------------------------------------------- File "C:\Python24\Lib\site-packages\numpy\distutils\command\build_ext.py", li e 260, in build_extension f_objects += self.fcompiler.compile(f_sources, AttributeError: 'NoneType' object has no attribute 'compile' ------------------------------------------------ so the fcompiler isn't being set. Any help? Mathew Here is the complete stack trace Traceback (most recent call last): File "c:\Python24\Scripts\f2py.py", line 26, in ? main() File "C:\Python24\Lib\site-packages\numpy\f2py\f2py2e.py", line 552, in main run_compile() File "C:\Python24\Lib\site-packages\numpy\f2py\f2py2e.py", line 539, in run_co mpile setup(ext_modules = [ext]) File "C:\Python24\Lib\site-packages\numpy\distutils\core.py", line 174, in set up return old_setup(**new_attr) File "C:\Python24\lib\distutils\core.py", line 149, in setup dist.run_commands() File "C:\Python24\lib\distutils\dist.py", line 946, in run_commands self.run_command(cmd) File "C:\Python24\lib\distutils\dist.py", line 966, in run_command cmd_obj.run() File "C:\Python24\lib\distutils\command\build.py", line 112, in run self.run_command(cmd_name) File "C:\Python24\lib\distutils\cmd.py", line 333, in run_command self.distribution.run_command(command) File "C:\Python24\lib\distutils\dist.py", line 966, in run_command cmd_obj.run() File "C:\Python24\Lib\site-packages\numpy\distutils\command\build_ext.py", lin e 121, in run self.build_extensions() File "C:\Python24\lib\distutils\command\build_ext.py", line 405, in build_exte nsions self.build_extension(ext) File "C:\Python24\Lib\site-packages\numpy\distutils\command\build_ext.py", lin e 260, in build_extension f_objects += self.fcompiler.compile(f_sources, AttributeError: 'NoneType' object has no attribute 'compile' ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at distip.com.au Wed Oct 11 19:34:07 2006 From: david at distip.com.au (David Novakovic) Date: Thu, 12 Oct 2006 09:34:07 +1000 Subject: incrementing along a diagonal In-Reply-To: <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> References: <452CA699.4030305@distip.com.au> <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> Message-ID: <452D7F6F.2090908@distip.com.au> Johannes Loehnert wrote: >> I'm just wondering if there is a way that i can increment all the values >> along a diagonal? >> > > Assume you want to change mat. > > # min() only necessary for non-square matrices > index = arange(min(mat.shape[0], mat.shape[1])) > # add 1 to each diagonal element > matrix[index, index] += 1 > # add some other stuff > matrix[index, index] += some_array_shaped_like_index > > > HTH, Johannes > > Thank you very much for the prompt reply, I'm just having a problem with this method: This method appears to only work if the matrix is mxm for example: >>> zeros((5,5)) array([[0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]) >>> x = zeros((5,5)) >>> index = arange(min(x.shape[0],x.shape[1])) >>> x[index,index] += 1 >>> x array([[1, 0, 0, 0, 0], [0, 1, 0, 0, 0], [0, 0, 1, 0, 0], [0, 0, 0, 1, 0], [0, 0, 0, 0, 1]]) >>> This is very nice, exactly what i want, but it doesnt work for mxn matricies: >>> x = zeros((5,3)) >>> x array([[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0]]) >>> index = arange(min(x.shape[0],x.shape[1])) >>> x[index,index] += 1 >>> x array([[1, 0, 0], [0, 1, 0], [0, 0, 1], [0, 0, 0], [0, 0, 0]]) >>> So the min part is right for mxn matrices - but perhaps there is a way to use the index differently. I'm very new to numpy, so excuse my noobness :) Just for reference, this is the line of perl i'm trying to port: (my $dummy = $ones->diagonal(0,1))++; # ones is a matrix created with zeroes() Yes, i know it is horribly ugly, but in this case, diagonal returns a list of references to the values in the original matrix, so values can be changed in place. I much prefer python - hence the port, but it seems like a hard thing to replicate. Perhaps there could be a function that returns an iterator over the values in a matrix and returns the index's. like: for index in diag_iter(matrix,*axes): matrix[index] +=1 Once again, cheers - i hope we can figure something out :) Dave Novakovic PS: If anyone would care to link me to the subscription page for the mailing list so you dont have to CC me all the time :) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Wed Oct 11 19:39:57 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Thu, 12 Oct 2006 02:39:57 +0300 (EEST) Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7D33.6070709@ee.byu.edu> References: <452D7D33.6070709@ee.byu.edu> Message-ID: On Wed, 11 Oct 2006, Travis Oliphant wrote: > >Interestingly, in worst cases numpy.sqrt is approximately ~3 times slower > >than scipy.sqrt on negative input but ~2 times faster on positive input: > > > >In [47]: pos_input = numpy.arange(1,100,0.001) > > > >In [48]: %timeit -n 1000 b=numpy.sqrt(pos_input) > >1000 loops, best of 3: 4.68 ms per loop > > > >In [49]: %timeit -n 1000 b=scipy.sqrt(pos_input) > >1000 loops, best of 3: 10 ms per loop > > > > > > This is the one that concerns me. Slowing everybody down who knows they > have positive values just for people that don't seems problematic. I think the code in scipy.sqrt can be optimized from def _fix_real_lt_zero(x): x = asarray(x) if any(isreal(x) & (x<0)): x = _tocomplex(x) return x def sqrt(x): x = _fix_real_lt_zero(x) return nx.sqrt(x) to (untested) def _fix_real_lt_zero(x): x = asarray(x) if not isinstance(x,(nt.csingle,nt.cdouble)) and any(x<0): x = _tocomplex(x) return x def sqrt(x): x = _fix_real_lt_zero(x) return nx.sqrt(x) or def sqrt(x): old = nx.seterr(invalid='raises') try: r = nx.sqrt(x) except FloatingPointError: x = _tocomplex(x) r = nx.sqrt(x) nx.seterr(**old) return r I haven't timed these cases yet.. Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 19:47:03 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 17:47:03 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D70B9.7070209@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> <452D5BFA.7000603@ee.byu.edu> <452D70B9.7070209@ieee.org> Message-ID: <452D8277.3080306@ee.byu.edu> Tim Hochberg wrote: >With python 2.5 out now, perhaps it's time to come up with a with >statement context manager. Something like: > > from __future__ import with_statement > import numpy > > class errstate(object): > def __init__(self, **kwargs): > self.kwargs = kwargs > def __enter__(self): > self.oldstate = numpy.seterr(**self.kwargs) > def __exit__(self, *exc_info): > numpy.seterr(**self.oldstate) > > a = numpy.arange(10) > a/a # ignores divide by zero > with errstate(divide='raise'): > a/a # raise exception on divide by zer > # Would ignore divide by zero again if we got here. > >-tim > > > This looks great. I think most people aren't aware of the with statement and what it can do (I'm only aware because of your posts, for example). So, what needs to be added to your example in order to just add it to numpy? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Wed Oct 11 19:58:45 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 12 Oct 2006 08:58:45 +0900 Subject: incrementing along a diagonal In-Reply-To: <452D7F6F.2090908@distip.com.au> References: <452CA699.4030305@distip.com.au> <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> <452D7F6F.2090908@distip.com.au> Message-ID: On 10/12/06, David Novakovic wrote: > Johannes Loehnert wrote: > This is very nice, exactly what i want, but it doesnt work for mxn > matricies: > > >>> x = zeros((5,3)) > >>> x > array([[0, 0, 0], > [0, 0, 0], > [0, 0, 0], > [0, 0, 0], > [0, 0, 0]]) > >>> index = arange(min(x.shape[0],x.shape[1])) > >>> x[index,index] += 1 > >>> x > array([[1, 0, 0], > [0, 1, 0], > [0, 0, 1], > [0, 0, 0], > [0, 0, 0]]) Exactly what output are you expecting? That is the definition of the 'diagonal' for a non-square matrix. If you're expecting something else then what you want is not the diagonal. > Just for reference, this is the line of perl i'm trying to port: > > > like: > > for index in diag_iter(matrix,*axes): > matrix[index] +=1 That's not going to change the mathematical definition of the diagonal of a non-square matrix. > PS: If anyone would care to link me to the subscription page for the > mailing list so you dont have to CC me all the time :) Check the bottom of this message. > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pierregm at engr.uga.edu Wed Oct 11 19:51:18 2006 From: pierregm at engr.uga.edu (Pierre GM) Date: Wed, 11 Oct 2006 19:51:18 -0400 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7D33.6070709@ee.byu.edu> References: <452D7D33.6070709@ee.byu.edu> Message-ID: <200610111951.19154.pierregm@engr.uga.edu> > >nan's are making things really slow, > > Yeah, they do. This actually makes the case for masked arrays, rather > than using NAN's. Travis, Talking about masked arrays, I'm about being done rewriting numpy.core.ma, mainly transforming MaskedArray as a subclass of ndarray (it should be OK by the end of the week), and allowing for an easy subclassing of MaskedArrays (which is far from being the case right now) What would be the best procedure to submit it ? Ticket on SVN ? Wiki on scipy.org ? Thanks again for your time ! Pierre ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Wed Oct 11 20:17:27 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 02:17:27 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7C88.3090506@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> Message-ID: <20061012001727.GQ20657@mentat.za.net> On Wed, Oct 11, 2006 at 05:21:44PM -0600, Travis Oliphant wrote: > Stefan van der Walt wrote: > > >I agree with Fernando on this one. > > > >Further, if I understand correctly, changing sqrt and power to give > >the right answer by default will slow things down somewhat. But is it > >worth sacrificing intuitive usage for speed? > > > > > For NumPy, yes. > > This is one reason that NumPy by itself is not a MATLAB > replacement. Intuitive usage is hopefully not a MATLAB-only feature. > >N.power(2,-2) == 0 > > > >and > > > >N.sqrt(-1) == nan > > > >just doesn't feel right. > > > > Only because your expectations are that NumPy *be* a MATLAB > replacement. The problem is that it sacrifices too much for that to be > the case. And we all realize that NumPy needs more stuff added to it > to be like IDL/MATLAB such as SciPy, Matplotlib, IPython, etc. I have none such expectations -- I havn't used MATLAB in over 5 years. All I'm saying is that, since the value of the square-root of -1 is not nan and 2^(-2) is not 0, it doesn't surprise me that this behaviour confuses people. > The "intuitive" functions (which must do argument checking) are (in > numpy.lib.scimath) but exported as > > scipy.power (actually I need to check that one...) > scipy.sqrt > > What could be simpler? ;-) I'm sure this is going to come back and haunt us. We have two libraries, one depends on and exposes the API of the other, yet it also overrides some functions with its own behaviour, while keeping the same names. I'll shut up now :) Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at distip.com.au Wed Oct 11 20:20:29 2006 From: david at distip.com.au (David Novakovic) Date: Thu, 12 Oct 2006 10:20:29 +1000 Subject: incrementing along a diagonal In-Reply-To: References: <452CA699.4030305@distip.com.au> <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> <452D7F6F.2090908@distip.com.au> Message-ID: <452D8A4D.4060708@distip.com.au> Thanks for the help, i've learnt a lot and also figured out something that does what I want, i'll paste an interactive session below: x = zeros((4,7)) x array([[0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0]]) index = arange(min(x.shape[0], x.shape[1])) index2 = copy.deepcopy(index) #deep copy may be overkill for a,b in enumerate(index): ... index2[a] += a ... if len(x[:,0]) > len(x[0]): ... x[index2,index] +=1 ... else: ... x[index,index2] +=1 ... x array([[1, 0, 0, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0, 0], [0, 0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 0, 0, 1]]) Thanks for the tips Dave Novakovic P.S. subscribed to the list now Bill Baxter wrote: > Forgot to CC you... > > ---------- Forwarded message ---------- > From: Bill Baxter > Date: Oct 12, 2006 8:58 AM > Subject: Re: [Numpy-discussion] incrementing along a diagonal > To: Discussion of Numerical Python > > > > On 10/12/06, David Novakovic wrote: >> Johannes Loehnert wrote: >> This is very nice, exactly what i want, but it doesnt work for mxn >> matricies: >> >> >>> x = zeros((5,3)) >> >>> x >> array([[0, 0, 0], >> [0, 0, 0], >> [0, 0, 0], >> [0, 0, 0], >> [0, 0, 0]]) >> >>> index = arange(min(x.shape[0],x.shape[1])) >> >>> x[index,index] += 1 >> >>> x >> array([[1, 0, 0], >> [0, 1, 0], >> [0, 0, 1], >> [0, 0, 0], >> [0, 0, 0]]) > > Exactly what output are you expecting? That is the definition of the > 'diagonal' for a non-square matrix. If you're expecting something > else then what you want is not the diagonal. > >> Just for reference, this is the line of perl i'm trying to port: >> >> >> like: >> >> for index in diag_iter(matrix,*axes): >> matrix[index] +=1 > > That's not going to change the mathematical definition of the diagonal > of a non-square matrix. > >> PS: If anyone would care to link me to the subscription page for the >> mailing list so you dont have to CC me all the time :) > > Check the bottom of this message. > >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > > --bb > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 11 20:24:01 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 11 Oct 2006 20:24:01 -0400 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: On 11/10/06, pearu at cens.ioc.ee wrote: > """ > In SciPy 0.3.x the ufuncs were overloaded by more "intelligent" versions. > A very attractive feature was that sqrt(-1) would yield 1j as in Matlab. > Then you can program formulas directly (e.g., roots of a 2nd order > polynomial) and the right answer is always achieved. In the Matlab-Python > battle in mathematics education, this feature is important. > > Now in SciPy 0.5.x sqrt(-1) yields nan. A lot of code we have, especially > for introductory numerics and physics courses, is now broken. > This has already made my colleagues at the University skeptical to > Python as "this lack of backward compatibility would never happen in Matlab". > > Another problem related to Numeric and numpy is that in these courses we > use ScientificPython several places, which applies Numeric and will > continue to do so. You then easily get a mix of numpy and Numeric > in scripts, which may cause problems and at least extra overhead. > Just converting to numpy in your own scripts isn't enough if you call > up libraries using and returning Numeric. > """ > > I wonder, what are the reasons that numpy.sqrt(-1) returns nan? > Could sqrt(-1) made to return 1j again? If not, shouldn't > numpy.sqrt(-1) raise a ValueError instead of returning silently nan? What is the desired behaviour of sqrt? Should sqrt always return a complex array, regardless of the type of its input? This will be extremely surprising to many users, whose memory usage suddenly doubles and for whom many functions no longer work the way they're accustomed to. Should it return a complex array only when any entry in its input is negative? This will be even *more* surprising when a negative (perhaps even -0) value appears in their matrix (for example, does a+min(a) yield -0s in the minimal values?) and suddenly it's complex. A ValueError is also surprising, and it forces the user to sanitize her array before taking the square root, instead of whenever convenient. If you want MATLAB behaviour, use only complex arrays. If the problem is backward incompatibility, there's a reason 1.0 hasn't been released yet... A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Wed Oct 11 20:41:08 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 02:41:08 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: <20061012004108.GR20657@mentat.za.net> On Wed, Oct 11, 2006 at 08:24:01PM -0400, A. M. Archibald wrote: > What is the desired behaviour of sqrt? [...] > Should it return a complex array only when any entry in its input is > negative? This will be even *more* surprising when a negative (perhaps > even -0) value appears in their matrix (for example, does a+min(a) > yield -0s in the minimal values?) and suddenly it's complex. Luckily sqrt(-0.) gives -0.0 and not nan ;) Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at distip.com.au Wed Oct 11 20:35:35 2006 From: david at distip.com.au (David Novakovic) Date: Thu, 12 Oct 2006 10:35:35 +1000 Subject: SPAM-LOW: Re: incrementing along a diagonal In-Reply-To: <452D8A4D.4060708@distip.com.au> References: <452CA699.4030305@distip.com.au> <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> <452D7F6F.2090908@distip.com.au> <452D8A4D.4060708@distip.com.au> Message-ID: <452D8DD7.6060601@distip.com.au> David Novakovic wrote: > Thanks for the help, i've learnt a lot and also figured out something > that does what I want, i'll paste an interactive session below: > > x = zeros((4,7)) > x > array([[0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0]]) > index = arange(min(x.shape[0], x.shape[1])) > index2 = copy.deepcopy(index) #deep copy may be overkill > for a,b in enumerate(index): > ... index2[a] += a > Turns out this is not good at all. I guess I'm still open to suggestions then :( Dave > ... > if len(x[:,0]) > len(x[0]): > ... x[index2,index] +=1 > ... else: > ... x[index,index2] +=1 > ... > x > array([[1, 0, 0, 0, 0, 0, 0], > [0, 0, 1, 0, 0, 0, 0], > [0, 0, 0, 0, 1, 0, 0], > [0, 0, 0, 0, 0, 0, 1]]) > > > Thanks for the tips > > Dave Novakovic > > P.S. subscribed to the list now > > Bill Baxter wrote: > >> Forgot to CC you... >> >> ---------- Forwarded message ---------- >> From: Bill Baxter >> Date: Oct 12, 2006 8:58 AM >> Subject: Re: [Numpy-discussion] incrementing along a diagonal >> To: Discussion of Numerical Python >> >> >> >> On 10/12/06, David Novakovic wrote: >> >>> Johannes Loehnert wrote: >>> This is very nice, exactly what i want, but it doesnt work for mxn >>> matricies: >>> >>> >>>>>> x = zeros((5,3)) >>>>>> x >>>>>> >>> array([[0, 0, 0], >>> [0, 0, 0], >>> [0, 0, 0], >>> [0, 0, 0], >>> [0, 0, 0]]) >>> >>>>>> index = arange(min(x.shape[0],x.shape[1])) >>>>>> x[index,index] += 1 >>>>>> x >>>>>> >>> array([[1, 0, 0], >>> [0, 1, 0], >>> [0, 0, 1], >>> [0, 0, 0], >>> [0, 0, 0]]) >>> >> Exactly what output are you expecting? That is the definition of the >> 'diagonal' for a non-square matrix. If you're expecting something >> else then what you want is not the diagonal. >> >> >>> Just for reference, this is the line of perl i'm trying to port: >>> >>> >>> like: >>> >>> for index in diag_iter(matrix,*axes): >>> matrix[index] +=1 >>> >> That's not going to change the mathematical definition of the diagonal >> of a non-square matrix. >> >> >>> PS: If anyone would care to link me to the subscription page for the >>> mailing list so you dont have to CC me all the time :) >>> >> Check the bottom of this message. >> >> >>> _______________________________________________ >>> Numpy-discussion mailing list >>> Numpy-discussion at lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >>> >>> >> --bb >> >> > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pfdubois at gmail.com Wed Oct 11 21:35:58 2006 From: pfdubois at gmail.com (Paul Dubois) Date: Wed, 11 Oct 2006 18:35:58 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <20061012004108.GR20657@mentat.za.net> References: <20061012004108.GR20657@mentat.za.net> Message-ID: This is a meta-statement about this argument. We already had it. Repeatedly. Whether you choose it one way or the other, for Numeric the community chose it the way it did for a reason. It is a good reason. It isn't stupid. There were good reasons for the other way. Those reasons weren't stupid. It was a 'choice amongst equals'. Being compatible with some other package is all very nice but it is simply a different choice and the choice was already made 10 years ago. If scipy chose to do this differently then you now have an intractable problem; somebody is going to get screwed. So, next time somebody tells you that some different choice amongst equals should be made for this and that good reason, just say no. This is why having a project leader who is mean like me is better than having a nice guy like Travis. (:-> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Wed Oct 11 21:40:51 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 11 Oct 2006 18:40:51 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D8277.3080306@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> <452D5BFA.7000603@ee.byu.edu> <452D70B9.7070209@ieee.org> <452D8277.3080306@ee.byu.edu> Message-ID: <452D9D23.5030700@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > > >> With python 2.5 out now, perhaps it's time to come up with a with >> statement context manager. Something like: >> >> from __future__ import with_statement >> import numpy >> >> class errstate(object): >> def __init__(self, **kwargs): >> self.kwargs = kwargs >> def __enter__(self): >> self.oldstate = numpy.seterr(**self.kwargs) >> def __exit__(self, *exc_info): >> numpy.seterr(**self.oldstate) >> >> a = numpy.arange(10) >> a/a # ignores divide by zero >> with errstate(divide='raise'): >> a/a # raise exception on divide by zer >> # Would ignore divide by zero again if we got here. >> >> -tim >> >> >> >> > > This looks great. I think most people aren't aware of the with > statement and what it can do (I'm only aware because of your posts, for > example). > > So, what needs to be added to your example in order to just add it to > numpy? > As far as I know, just testing and documentation -- however testing was so minimal that I may find some other stuff. I'll try to clean it up tomorrow so that I'm a little more confident that it works correctly and I'll send another note out then. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 11 22:44:54 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 11 Oct 2006 20:44:54 -0600 Subject: incrementing along a diagonal In-Reply-To: <452CA699.4030305@distip.com.au> References: <452CA699.4030305@distip.com.au> Message-ID: <452DAC26.5010805@ieee.org> David Novakovic wrote: > Hi, > > i'm moving some old perl PDL code to python. I've come across a line > which changes values in a diagonal line accross a matrix. > > matrix.diagonal() returns a list of values, but making changes to these > does not reflect in the original (naturally). > > I'm just wondering if there is a way that i can increment all the values > along a diagonal? > You can refer to a diagonal using flattened index with a element-skip equal to the number of columns+1. Thus, a.flat[::a.shape[1]+1] += 1 will increment the elements of a along the main diagonal. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Wed Oct 11 23:17:13 2006 From: gregwillden at gmail.com (Greg Willden) Date: Wed, 11 Oct 2006 22:17:13 -0500 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7C88.3090506@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> Message-ID: <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> On 10/11/06, Travis Oliphant wrote: > > Stefan van der Walt wrote: > >Further, if I understand correctly, changing sqrt and power to give > >the right answer by default will slow things down somewhat. But is it > >worth sacrificing intuitive usage for speed? > > > For NumPy, yes. > > This is one reason that NumPy by itself is not a MATLAB replacement. This is not about being a Matlab replacement. This is about correctness. Numpy purports to handle complex numbers. Mathematically, sqrt(-1) is a complex number. Therefore Numpy *must* return a complex number. Speed should not take precedence over correctness. If Numpy doesn't return a complex number then it shouldn't pretend to support complex numbers. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Wed Oct 11 23:48:45 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 12 Oct 2006 12:48:45 +0900 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> Message-ID: On 10/12/06, Greg Willden wrote: > On 10/11/06, Travis Oliphant wrote: > > Stefan van der Walt wrote: > > >Further, if I understand correctly, changing sqrt and power to give > > >the right answer by default will slow things down somewhat. But is it > > >worth sacrificing intuitive usage for speed? > > > > > For NumPy, yes. > > > > This is one reason that NumPy by itself is not a MATLAB replacement. > > This is not about being a Matlab replacement. > This is about correctness. > Numpy purports to handle complex numbers. > Mathematically, sqrt(-1) is a complex number. > Therefore Numpy *must* return a complex number. > Speed should not take precedence over correctness. Unless your goal is speed. Then speed should take precedence over correctness. And unless you're a fan of quaternions, in which case *which* square root of -1 should it return? It's interesting to note that although python has had complex numbers pretty much from the beginning, math.sqrt(-1) returns an error. If you want to work with complex square roots you need to use cmath.sqrt(). Basically you have to tell python that compex numbers are something you care about by using the module designed for complex math. This scimath module is a similar deal. But perhaps the name could be a little more obvious / short? Right now it seems it only deals with complex numbers, so maybe having "complex" or "cmath" in the name would make it clearer. Hmm there is a numpy.math, why not a numpy.cmath. > If Numpy doesn't return a complex number then it shouldn't pretend to > support complex numbers. That's certainly being overdramatic. Lot's of folks are doing nifty stuff with complex FFTs every day using numpy/scipy. And I'm sure they will continue to no matter what numpy.sqrt(-1) returns. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 00:02:21 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 11 Oct 2006 22:02:21 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> Message-ID: <452DBE4D.4000906@ieee.org> Greg Willden wrote: > On 10/11/06, *Travis Oliphant* > wrote: > > Stefan van der Walt wrote: > >Further, if I understand correctly, changing sqrt and power to give > >the right answer by default will slow things down somewhat. But > is it > >worth sacrificing intuitive usage for speed? > > > For NumPy, yes. > > This is one reason that NumPy by itself is not a MATLAB replacement. > > > This is not about being a Matlab replacement. > This is about correctness. I disagree. NumPy does the "correct" thing when you realize that sqrt is a function that returns the same type as it's input. The field over-which the operation takes place is defined by the input data-type and not the input "values". Either way can be considered correct mathematically. As Paul said it was a design decision not to go searching through the array to determine whether or not there are negative numbers in the array. Of course you can do that if you want and that's what scipy.sqrt does. > Numpy purports to handle complex numbers. > Mathematically, sqrt(-1) is a complex number. Or, maybe it's undefined if you are in the field of real numbers. It all depends. > Therefore Numpy *must* return a complex number. Only if the input is complex. That is a reasonable alternative to your specification. > > If Numpy doesn't return a complex number then it shouldn't pretend to > support complex numbers. Of course it supports complex numbers, it just doesn't support automatic conversion to complex numbers. It supports complex numbers the same way Python supports them (i.e. you have to use cmath to get sqrt(-1) == 1j) People can look at this many ways without calling the other way of looking at it unreasonable. I don't see a pressing need to change this in NumPy, and in fact see many reasons to leave it the way it is. This discussion should move to the scipy list because that is the only place where a change could occur. -Travis. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 00:08:27 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 11 Oct 2006 21:08:27 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> Message-ID: <452DBFBB.8090403@ieee.org> Greg Willden wrote: > On 10/11/06, *Travis Oliphant* > wrote: > > Stefan van der Walt wrote: > >Further, if I understand correctly, changing sqrt and power to give > >the right answer by default will slow things down somewhat. But > is it > >worth sacrificing intuitive usage for speed? > > > For NumPy, yes. > > This is one reason that NumPy by itself is not a MATLAB replacement. > > > This is not about being a Matlab replacement. > This is about correctness. > Numpy purports to handle complex numbers. > Mathematically, sqrt(-1) is a complex number. That's vastly oversimplified. If you are working with the reals, then sqrt(-1) is undefined (AKA nan). If you are working in the complex plane, then sqrt(-1) is indeed *a* complex number; of course you don't know *which* complex number it is unless you also specify the branch. > Therefore Numpy *must* return a complex number. No I don't think that it must. I've found it a very useful tool for the past decade plus without it returning complex numbers from sqrt. > Speed should not take precedence over correctness. The current behavior is not incorrect. > > If Numpy doesn't return a complex number then it shouldn't pretend to > support complex numbers. Please relax. Personally I think that the default error mode should be tightened up. Then people would only see these sort of things if they really care about them. Using Python 2.5 and the errstate class I posted earlier: # This is what I like for the default error state numpy.seterr(invalid='raise', divide='raise', over='raise', under='ignore') a = -numpy.arange(10) with errstate(invalid='ignore'): print numpy.sqrt(a) # This happily returns a bunch of NANs; and one zero. print numpy.sqrt(a.astype(complex)) # This returns a bunch of complex values print numpy.sqrt(a) # This raise a floating point error. No silent NANs returned This same error state make the vagaries of dividing by zero less surprising as well. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 12 00:49:45 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 22:49:45 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DBFBB.8090403@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <452DBFBB.8090403@ieee.org> Message-ID: On 10/11/06, Tim Hochberg wrote: > > Greg Willden wrote: > > On 10/11/06, *Travis Oliphant* > > wrote: > > > > Stefan van der Walt wrote: > > >Further, if I understand correctly, changing sqrt and power to give > > >the right answer by default will slow things down somewhat. But > > is it > > >worth sacrificing intuitive usage for speed? > > > > > For NumPy, yes. > > > > This is one reason that NumPy by itself is not a MATLAB replacement. > > > > > > This is not about being a Matlab replacement. > > This is about correctness. > > Numpy purports to handle complex numbers. > > Mathematically, sqrt(-1) is a complex number. > That's vastly oversimplified. If you are working with the reals, then > sqrt(-1) is undefined (AKA nan). If you are working in the complex > plane, then sqrt(-1) is indeed *a* complex number; And if you are working over the rationals, sqrt(-1) and sqrt(-2) lead to different field extensions ;) Of course, numpy doesn't *have* rationals, so I'm just being cute. Personally I think that the default error mode should be tightened up. > Then people would only see these sort of things if they really care > about them. Using Python 2.5 and the errstate class I posted earlier: > > # This is what I like for the default error state > numpy.seterr(invalid='raise', divide='raise', over='raise', > under='ignore') I like these choices too. Overflow, division by zero, and sqrt(-x) are usually errors, indicating bad data or programming bugs. Underflowed floats, OTOH, are just really, really small numbers and can be treated as zero. An exception might be if the result is used in division and no error is raised, resulting in a loss of accuracy. If complex results are *expected*, then this should be made explicit by using complex numbers. Numpy allows finegrained control of data types and array ordering, it is a bit closer to the metal than Matlab. This extra control allows greater efficiency, both in speed and in storage, at the cost of a bit more care on the programmers side. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From David.L.Goldsmith at noaa.gov Thu Oct 12 02:03:02 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:03:02 -0700 Subject: round In-Reply-To: References: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Message-ID: <452DDA96.3010005@noaa.gov> Charles R Harris wrote: > > > On 10/11/06, *Greg Willden* > wrote: > > Hi All, > > I've read discussions in the archives about how round() "rounds to > even" and how that is supposedly better. > > But what I haven't been able to find is "What do I use if I want > the regular old round that you learn in school?" > > > Perhaps you could explain *why* you want the schoolbook round? Given > that floating point is inherently inaccurate you would have to expect > to produce a lot of numbers exactly of the form x.5 *without errors*, > which means you probably don't need round to deal with it. Anyway, > absent a flag somewhere, you can do something like (x + > sign(x)*.5).astype(int). > > Chuck Also, where did you go to school? In Fairfax County, VA in the 80's at least, they were teaching "round to even"; since that time, I've taught math in a variety of locations and settings and with a variety of texts, and where I have seen the issue addressed, it's always "round to even" (or, as I learned it, "the rule of fives"). DG > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:10:19 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:10:19 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D56F5.9000404@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> Message-ID: <452DDC4B.1050304@noaa.gov> Travis Oliphant wrote: > pearu at cens.ioc.ee wrote: > > >> Could sqrt(-1) made to return 1j again? >> >> > Not in NumPy. But, in scipy it could. > > Ohmigod!!! You are definitely going to scare away many, many potential users - if I wasn't obliged to use open source at work, you'd be scaring me away. I was thinking about translating all my personal fractal-generating Matlab code into Python, but I certainly won't be doing that now! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:15:33 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:15:33 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D5843.4050007@gmx.net> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> Message-ID: <452DDD85.5010106@noaa.gov> Sven Schreiber wrote: > Travis Oliphant schrieb: > > >>> If not, shouldn't >>> >>> >>> numpy.sqrt(-1) raise a ValueError instead of returning silently nan? >>> >>> >>> >> This is user adjustable. You change the error mode to raise on >> 'invalid' instead of pass silently which is now the default. >> >> -Travis >> >> > > Could you please explain how this adjustment is done, or point to the > relevant documentation. > Thank you, > Sven > I'm glad you asked this, Sven, 'cause I was thinking that if making this "user adjustment" is this advanced (I too have no idea what you're talking about, Travis), then this would be another significant strike against numpy (but I was holding my tongue, since I'd just let fly in my previous email). DG ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 02:26:17 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 00:26:17 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DDC4B.1050304@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> Message-ID: <452DE009.3070100@ieee.org> David Goldsmith wrote: > Travis Oliphant wrote: > >> pearu at cens.ioc.ee wrote: >> >> >> >>> Could sqrt(-1) made to return 1j again? >>> >>> >>> >> Not in NumPy. But, in scipy it could. >> >> >> > Ohmigod!!! You are definitely going to scare away many, many potential > users - if I wasn't obliged to use open source at work, you'd be scaring > me away. Why in the world does it scare you away. This makes no sense to me. If you don't like the scipy version don't use it. NumPy and SciPy are not the same thing. The problem we have is that the scipy version (0.3.2) already had this feature (and Numeric didn't). What is so new here that is so scary ? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:27:19 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:27:19 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7C88.3090506@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> Message-ID: <452DE047.6080501@noaa.gov> Travis Oliphant wrote: > What could be simpler? ;-) Having sqrt(-1) return 1j (without having to remember that in order to get this, you have to write sqrt(-1+0j) instead). DG ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:31:35 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:31:35 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7D33.6070709@ee.byu.edu> References: <452D7D33.6070709@ee.byu.edu> Message-ID: <452DE147.8010609@noaa.gov> Travis Oliphant wrote: > > This is the one that concerns me. Slowing everybody down who knows they > have positive values just for people that don't seems problematic. > > Then have a "sqrtp" function for those users who are fortunate enough to know ahead of time that they'll only be talking square roots of nonnegatives. DG ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Thu Oct 12 02:35:48 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 12 Oct 2006 15:35:48 +0900 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE047.6080501@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <452DE047.6080501@noaa.gov> Message-ID: <452DE244.8030608@ar.media.kyoto-u.ac.jp> David Goldsmith wrote: > Travis Oliphant wrote: > >> What could be simpler? ;-) >> > Having sqrt(-1) return 1j (without having to remember that in order to > get this, you have to write sqrt(-1+0j) instead). > > But this can sometimes lead to confusing errors hard to track when you don't want to treat complex numbers. That's one of the thing I hated in matlab, actually: if you don't want to handle complex numbers, you have to check regularly for it. So I don't think this is simpler. David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 02:38:51 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 00:38:51 -0600 Subject: Please test the SVN branch Message-ID: <452DE2FB.3070408@ieee.org> I made some fixes to the "asbuffer" code which let me feel better about exposing it in NumPy (where it is now named int_asbuffer). This code takes a Python integer and a size and returns a buffer object that points to that memory. A little test is performed by trying to read (and possibly write if a writeable buffer is requested) to the first and last elements of the buffer. Any segfault is trapped and used to raise a Python error indicating you can't use that area of memory. It doesn't guarantee you won't shoot yourself, but it does make it more difficult to segfault Python. Previously a simple int_asbuffer(3423423, 5) would have segfaulted (unless by chance you the memory area 3423423 happens to be owned by Python). I have not tested the code on other platforms to make sure it works as expected, so please try and compiled it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:37:30 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:37:30 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <20061012001727.GQ20657@mentat.za.net> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <20061012001727.GQ20657@mentat.za.net> Message-ID: <452DE2AA.6020605@noaa.gov> Stefan van der Walt wrote: >> This is one reason that NumPy by itself is not a MATLAB >> replacement. >> > Intuitive usage is hopefully not a MATLAB-only feature. > Here, here! (Or is it Hear, hear! I don't think I've ever seen it written out before. :-) ) > I'll shut up now :) But why? To my mind, you're making a lot more sense than Travis. DG > Cheers > St?fan > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 02:40:18 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 00:40:18 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE047.6080501@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <452DE047.6080501@noaa.gov> Message-ID: <452DE352.80304@ieee.org> David Goldsmith wrote: > Travis Oliphant wrote: > >> What could be simpler? ;-) >> > Having sqrt(-1) return 1j (without having to remember that in order to > get this, you have to write sqrt(-1+0j) instead). > > That's exactly what scipy.sqrt(-1) does. That was my point. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.u.r.e.l.i.a.n at gmx.net Thu Oct 12 02:40:49 2006 From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert) Date: Thu, 12 Oct 2006 08:40:49 +0200 Subject: incrementing along a diagonal In-Reply-To: <452D8DD7.6060601@distip.com.au> References: <452CA699.4030305@distip.com.au> <452D8A4D.4060708@distip.com.au> <452D8DD7.6060601@distip.com.au> Message-ID: <200610120840.49968.a.u.r.e.l.i.a.n@gmx.net> Hi, I absolutely do not know perl, so I do not know what the expression you posted does. However, the key is just to understand indexing in numpy: if you have a matrix mat and index arrays index1, index2 with, lets say, index1 = array([ 17, 19, 29]) index2 = array([ 12, 3, 9]) then the entries of the index arrays are used as row and column indices respectively, and the result will be an array shaped like the index arrays. So doing mat[index1, index2] will give you --> array([ mat[17, 12], mat[19, 3], mat[29, 9]]). Now if you want the diagonal of a 3x3-mat, you need index1=index2=array([ 0, 1, 2]). mat[index1, index2] --> array([ mat[0,0], mat[1,1], mat[2,2]]) That is what my code does. If you need other, arbitrary subsets of mat, you just have to fill the index arrays accordingly. Johannes ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Thu Oct 12 02:49:39 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 12 Oct 2006 00:49:39 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE009.3070100@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: On 10/12/06, Travis Oliphant wrote: > Why in the world does it scare you away. This makes no sense to me. > If you don't like the scipy version don't use it. NumPy and SciPy are > not the same thing. I'd like to pitch in (again) on this issue, but I'll try to make sure that it's clear that I'm NOT arguing about sqrt() in particular, one way or another. It's perfectly clear that numpy != scipy to all of us. And yet, I think it is equally clear that the two are /very/ tightly related. Scipy builds on top of numpy and it directly exposes a LOT of the numpy API as scipy functions: In [21]: import numpy as n, scipy as s In [22]: common_names = set(dir(n)) & set(dir(s)) In [23]: [getattr(n,x) is getattr(s,x) for x in common_names ].count(True) Out[23]: 450 In [24]: len(common_names) Out[24]: 462 That's 450 objects from numpy which are directly exposed in Scipy, while only 12 names are in both top-level namespaces and yet are different objects. Put another way, scipy is a direct wrap of 97% of the numpy top-level namespace. While /we/ know they are distinct entities, to the casual user a 97% match looks pretty close to being the same, especially when the non-identical things are all non-numerical: In [27]: [x for x in common_names if getattr(n,x) is not getattr(s,x)] Out[27]: ['pkgload', 'version', '__config__', '__file__', '__all__', '__doc__', 'show_config', '__version__', '__path__', '__name__', 'info', 'test'] In [32]: n.__version__,s.__version__ Out[32]: ('1.0.dev3306', '0.5.2.dev2252') Basically, at least for these versions, the top-level API of scipy is a strict superset of the numpy one for all practical purposes. I think it's fair to say that if we start sprinkling special cases where certain objects happen to have the same name but produce different results for the same inputs, confusion will arise. Please note that I see a valid reason for scipy.foo != numpy.foo when the scipy version uses code with extra features, is faster, has additional options, etc. But as I said in a previous message, I think that /for the same input/, we should really try to satisfy that numpy.foo(x) == scipy.foo(x) (which is NOT the same as 'numpy.foo is scipy.foo') within reason. Obviously the scipy version may succeed where the numpy one fails due to better algorithms, or be faster, etc. I'm talking about a general principle here. I doubt I'll be able to state my point with any more clarity, so I'll stop now. But I really believe that this particular aspect of consistency between numpy and scipy is a /very/ important one for its adoption in wider communities. Best, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nwagner at iam.uni-stuttgart.de Thu Oct 12 02:50:23 2006 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Thu, 12 Oct 2006 08:50:23 +0200 Subject: Please test the SVN branch In-Reply-To: <452DE2FB.3070408@ieee.org> References: <452DE2FB.3070408@ieee.org> Message-ID: <452DE5AF.5010500@iam.uni-stuttgart.de> Travis Oliphant wrote: > I made some fixes to the "asbuffer" code which let me feel better about > exposing it in NumPy (where it is now named int_asbuffer). > > This code takes a Python integer and a size and returns a buffer object > that points to that memory. A little test is performed by trying to > read (and possibly write if a writeable buffer is requested) to the > first and last elements of the buffer. Any segfault is trapped and used > to raise a Python error indicating you can't use that area of memory. > > It doesn't guarantee you won't shoot yourself, but it does make it more > difficult to segfault Python. Previously a simple int_asbuffer(3423423, > 5) would have segfaulted (unless by chance you the memory area 3423423 > happens to be owned by Python). > > I have not tested the code on other platforms to make sure it works as > expected, so please try and compiled it. > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > Numpy version 1.0.dev3315 Scipy version 0.5.2.dev2254 Works fine here. All tests passed ! x86_64 x86_64 x86_64 GNU/Linux Nils ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 03:06:50 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 01:06:50 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <452DBFBB.8090403@ieee.org> Message-ID: <452DE98A.4090404@ieee.org> > > Personally I think that the default error mode should be tightened > up. > Then people would only see these sort of things if they really care > about them. Using Python 2.5 and the errstate class I posted earlier: > > # This is what I like for the default error state > numpy.seterr (invalid='raise', divide='raise', over='raise', > under='ignore') > > > I like these choices too. Overflow, division by zero, and sqrt(-x) are > usually errors, indicating bad data or programming bugs. Underflowed > floats, OTOH, are just really, really small numbers and can be treated > as zero. An exception might be if the result is used in division and > no error is raised, resulting in a loss of accuracy. > I'm fine with this. I've hesitated because error checking is by default slower. But, I can agree that it is "less surprising" to new-comers. People that don't mind no-checking can simply set their defaults back to 'ignore' -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sinclaird at ukzn.ac.za Thu Oct 12 03:11:48 2006 From: sinclaird at ukzn.ac.za (Scott Sinclair) Date: Thu, 12 Oct 2006 09:11:48 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452DEAB4.3030506@ukzn.ac.za> Fernando Perez wrote: > Please note that I see a valid reason for scipy.foo != numpy.foo when > the scipy version uses code with extra features, is faster, has > additional options, etc. But as I said in a previous message, I think > that /for the same input/, we should really try to satisfy that > > numpy.foo(x) == scipy.foo(x) (which is NOT the same as 'numpy.foo is scipy.foo') > > within reason. As far as I can tell this is exactly what happens. Consider the issue under discussion... ---------------------------------- >>> import numpy as np >>> np.sqrt(-1) -1.#IND >>> np.sqrt(-1+0j) 1j >>> a = complex(-1) >>> np.sqrt(a) 1j >>> import scipy as sp >>> sp.sqrt(-1) -1.#IND >>> np.sqrt(-1+0j) 1j >>> sp.sqrt(a) 1j >>> np.__version__ '1.0rc1' >>> sp.__version__ '0.5.1' >>> ---------------------------------- I'm sure that this hasn't changed in the development versions. Surely the point is that when your algorithm can potentially produce a complex result, the logical thing to do is to use a complex data type. In this case Numpy and Scipy behave in a way which is intuitive. If complex results are surprising and unexpected then the algorithm is probably in error or poorly understood ;-) Cheers, Scott ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 03:12:45 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 00:12:45 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE009.3070100@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452DEAED.9070408@noaa.gov> Travis Oliphant wrote: > David Goldsmith wrote: > >> Travis Oliphant wrote: >> >> >>> pearu at cens.ioc.ee wrote: >>> >>> >>> >>> >>>> Could sqrt(-1) made to return 1j again? >>>> >>>> >>>> >>>> >>> Not in NumPy. But, in scipy it could. >>> >>> >>> >>> >> Ohmigod!!! You are definitely going to scare away many, many potential >> users - if I wasn't obliged to use open source at work, you'd be scaring >> me away. >> > Why in the world does it scare you away. This makes no sense to me. > If you don't like the scipy version don't use it. NumPy and SciPy are > not the same thing. > I don't use scipy (and don't want to because of the overhead) but it sounds like I should because if I'm taking the square root of a variable whose value at run time happens to be real but less than zero, I *want* the language I'm using to return an imaginary; in other words, it's not the scipy behavior which "scares" me, its the numpy (which I do/have been using) behavior. To which you might say, "Well, if that's what you want, and you have Matlab (as you've said you do), then just use that." But that's precisely the point: people who don't want to be bothered with having to be "a bit more care[ful]" (as Chuck put it) - and who can afford it - are going to be inclined to choose Matlab over numpy. Perhaps one doesn't care - in the grand scheme of things, it certainly doesn't matter - but I think that you all should be aware that this numpy "feature" will be seen by many as more than just a nuisance. DG ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 03:18:57 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 00:18:57 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE352.80304@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <452DE047.6080501@noaa.gov> <452DE352.80304@ieee.org> Message-ID: <452DEC61.5050309@noaa.gov> Travis Oliphant wrote: > David Goldsmith wrote: > >> Travis Oliphant wrote: >> >> >>> What could be simpler? ;-) >>> >>> >> Having sqrt(-1) return 1j (without having to remember that in order to >> get this, you have to write sqrt(-1+0j) instead). >> >> >> > That's exactly what scipy.sqrt(-1) does. That was my point. > But I don't want to have to use scipy (which so far I haven't needed for any other reason) just to get this one behavior. But my personal preferences aside, I say again: if numpy doesn't behave this way, you *will* be "scaring" away many potential users. If you can live with that, so be it. DG DG > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jaqyhoga at bowenlaw.com Thu Oct 12 03:18:40 2006 From: jaqyhoga at bowenlaw.com (Natalee Remus) Date: Thu, 12 Oct 2006 00:18:40 -0700 Subject: VldAGRA Message-ID: <000001c6edce$a48567d0$45d8a8c0@lynzv> Hi, VldAGRA for LESS http://www.tikindefunkadesinmde.com Paskonjak, bag in hand and forged press credentials in pocket. Even angry and settled for a surly mutter and growl. Mata nodded blithely -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From David.L.Goldsmith at noaa.gov Thu Oct 12 03:23:10 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 00:23:10 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452DED5E.8090608@noaa.gov> (Very) well said, Fernando. Thanks! DG Fernando Perez wrote: > On 10/12/06, Travis Oliphant wrote: > > >> Why in the world does it scare you away. This makes no sense to me. >> If you don't like the scipy version don't use it. NumPy and SciPy are >> not the same thing. >> > > I'd like to pitch in (again) on this issue, but I'll try to make sure > that it's clear that I'm NOT arguing about sqrt() in particular, one > way or another. > > It's perfectly clear that numpy != scipy to all of us. And yet, I > think it is equally clear that the two are /very/ tightly related. > Scipy builds on top of numpy and it directly exposes a LOT of the > numpy API as scipy functions: > > In [21]: import numpy as n, scipy as s > > In [22]: common_names = set(dir(n)) & set(dir(s)) > > In [23]: [getattr(n,x) is getattr(s,x) for x in common_names ].count(True) > Out[23]: 450 > > In [24]: len(common_names) > Out[24]: 462 > > That's 450 objects from numpy which are directly exposed in Scipy, > while only 12 names are in both top-level namespaces and yet are > different objects. Put another way, scipy is a direct wrap of 97% of > the numpy top-level namespace. While /we/ know they are distinct > entities, to the casual user a 97% match looks pretty close to being > the same, especially when the non-identical things are all > non-numerical: > > In [27]: [x for x in common_names if getattr(n,x) is not getattr(s,x)] > Out[27]: > ['pkgload', > 'version', > '__config__', > '__file__', > '__all__', > '__doc__', > 'show_config', > '__version__', > '__path__', > '__name__', > 'info', > 'test'] > > In [32]: n.__version__,s.__version__ > Out[32]: ('1.0.dev3306', '0.5.2.dev2252') > > Basically, at least for these versions, the top-level API of scipy is > a strict superset of the numpy one for all practical purposes. > > I think it's fair to say that if we start sprinkling special cases > where certain objects happen to have the same name but produce > different results for the same inputs, confusion will arise. > > Please note that I see a valid reason for scipy.foo != numpy.foo when > the scipy version uses code with extra features, is faster, has > additional options, etc. But as I said in a previous message, I think > that /for the same input/, we should really try to satisfy that > > numpy.foo(x) == scipy.foo(x) (which is NOT the same as 'numpy.foo is scipy.foo') > > within reason. Obviously the scipy version may succeed where the > numpy one fails due to better algorithms, or be faster, etc. I'm > talking about a general principle here. > > I doubt I'll be able to state my point with any more clarity, so I'll > stop now. But I really believe that this particular aspect of > consistency between numpy and scipy is a /very/ important one for its > adoption in wider communities. > > Best, > > f > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 03:32:17 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 01:32:17 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452DEF81.4040208@ieee.org> > I'd like to pitch in (again) on this issue, but I'll try to make sure > that it's clear that I'm NOT arguing about sqrt() in particular, one > > way or another. > Fernando, I don't disagree with you in principle. I don't think anybody does. I think we should try to keep the interfaces and expectations of scipy and numpy the same. Unfortunately, we have competing issues in this particular case (in the case of the functions in numpy.lib.scimath). Nobody has suggested an alternative to the current situation in SVN that is satsifying to enough users. Here is the situation. 1) NumPy ufuncs never up-cast to complex numbers without the user explicitly requesting it so sqrt(-1) creates a floating-point error condition which is either caught or ignored according to the user's desires. To get complex results from sqrt you have put in complex numbers to begin with. That's inherent in the way ufuncs work. This is long-standing behavior that has good reasons for it's existence. I don't see this changing. That's why I suggested to move the discussion over to scipy (we have the fancy functions in NumPy, they are just not in the top-level name-space). Now, it would be possible to give ufuncs a dtype keyword argument that allowed you to specify which underlying loop was to be used for the calculation. That way you wouldn't have to convert inputs to complex numbers before calling the ufunc, but could let the ufunc do it in chunks during the loop. That is certainly a reasonable enhancement: sqrt(a, dtype=complex). This no-doubt has a "library-ish"-feeling, but that is what NumPy is. If such a change is desireable, I don't think it would be much to implement it. 2) In SciPy 0.3.2 the top-level math functions were overloaded with these fancy argument-checking versions. So that scipy.sqrt(-1) would return 1j. This was done precisely to attract users like David G. who don't mind data-type conversions on the fly, but prefer automatic conversion (funny being called non-sensical when I was the one who wrote those original scipy functions --- I guess I'm schizophrenic). We changed this in SciPy 0.5.1 by accident without any discussion. It was simply a by-product of moving scipy_base (including those special-handling functions) into NumPy and forgetting to import those functions again into top-level SciPy. It was an oversight that caused backwards compatibilty issues. So, I simply changed it back to what SciPy used to be in SVN. If we want to change SciPy, then fine, but let's move that discussion over to scipy-dev. In short, I appreciate all the comments and the differences of opinion they point out, but they are ultimately non-productive. We can't just change top-level sqrt to be the fancy function willy-nilly. Paul says I'm nice (he's not talked to my children recently), but I'm not that much of a push-over. There are very good reasons that NumPy has the behavior it does. In addition, the fancy functions are already there in numpy in numpy.lib.scimath. So, use them from there if you like them. Create your own little mynumpy module that does from numpy import * from numpy.lib.scimath import * and have a ball. Python is flexible enough that the sky is not going to fall if the library doesn't do things exactly the way you would do it. We can still cooperate in areas that we agree on. Again. Put this to rest on NumPy and move the discussion over to scipy-dev. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 03:41:46 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 01:41:46 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DEAED.9070408@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> Message-ID: <452DF1BA.9000906@ieee.org> David Goldsmith wrote: >> >> > I don't use scipy (and don't want to because of the overhead) but it > sounds like I should because if I'm taking the square root of a variable > whose value at run time happens to be real but less than zero, I *want* > the language I'm using to return an imaginary; in other words, it's not > the scipy behavior which "scares" me, its the numpy (which I do/have > been using) behavior. O.K. Well the functions you want are in numpy.lib.scimath. I should have directed you there. You actually don't need scipy installed at all. Just import sqrt from numpy.lib.scimath. I'm sorry I misunderstood the issue. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Thu Oct 12 03:45:37 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Thu, 12 Oct 2006 10:45:37 +0300 (EEST) Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE98A.4090404@ieee.org> References: <452DE98A.4090404@ieee.org> Message-ID: PS: I am still sending this message to numpy list only because the proposal below affects numpy code, not scipy one. I think Fernando points make sense, numpy.foo(x) != scipy.foo(x) can cause confusion and frustration both among new numpy/scipy users and developers (who need to find explanations for the choises made). So, let me propose the following solution so that all parties will get the same results without sacrifying numpy.sqrt speed on non-negative input and scipy.sqrt backward compatibility: Define numpy.sqrt as follows: def sqrt(x): r = nx.sqrt(x) if nx.nan in r: i = nx.where(nx.isnan(r)) r = _tocomplex(r) r[i] = nx.sqrt(_tocomplex(x[i])) return r and define numpy.sqrtp that takes only non-negative input, this is for those users who expect sqrt to fail on negative input (as Numeric.sqrt and math.sqrt do). Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Thu Oct 12 03:46:33 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 12 Oct 2006 01:46:33 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DEF81.4040208@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEF81.4040208@ieee.org> Message-ID: On 10/12/06, Travis Oliphant wrote: > > > I'd like to pitch in (again) on this issue, but I'll try to make sure > > that it's clear that I'm NOT arguing about sqrt() in particular, one > > > > way or another. > > > > Fernando, > > I don't disagree with you in principle. I don't think anybody does. I > think we should try to keep the interfaces and expectations of scipy and > numpy the same. OK, I'm glad to hear that. For a moment during the discussion I misunderstood your intent and thought you did disagree with this, which worried me. Sorry for the confusion and any noise caused. I realize the sqrt() topic is a tricky one which I've deliberately sidestepped, since I wasn't too interested in that particular case but rather in the general principle. I'll leave that discussion to continue on scipy-dev for those who are intereseted in it. Regards, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 03:51:23 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 01:51:23 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DEF81.4040208@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEF81.4040208@ieee.org> Message-ID: <452DF3FB.8010201@ieee.org> Travis Oliphant wrote: > Now, it would be possible to give ufuncs a dtype keyword argument that > allowed you to specify which underlying loop was to be used for the > calculation. That way you wouldn't have to convert inputs to complex > numbers before calling the ufunc, but could let the ufunc do it in > chunks during the loop. That is certainly a reasonable enhancement: > > sqrt(a, dtype=complex). > > This no-doubt has a "library-ish"-feeling, but that is what NumPy is. > If such a change is desireable, I don't think it would be much to > implement it. > > This could be implemented, but only with a version number increase in the C-API (we would have to change the c-signature of the ufunc tp_call. This would mean that the next release of NumPy would be binary incompatible with packages built to previous NumPy releases. I've really been trying to avoid doing that. So, unless there are strong requests for this feature that outweigh the negatives of re-building dependent packages, then this feature will have to wait. OTOH: I suppose it could be implemented in a different way (using indexing or a method call): sqrt[complex](a) --- I remember Tim suggesting some use for indexing on ufuncs earlier though. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From klemm at phys.ethz.ch Thu Oct 12 03:53:05 2006 From: klemm at phys.ethz.ch (Hanno Klemm) Date: Thu, 12 Oct 2006 09:53:05 +0200 Subject: Compile with atlas 3.7.17 fails In-Reply-To: <452D20EB.7030506@ee.byu.edu> References: , Message-ID: Travis, thank you, setting the -fPIC option indeed solved my problem. Just for future reference (if anybody else needs it), I added the -fPIC flag to the compiler flags in Make.inc in the atlas build directory, as setting CCFLAGS somehwo didn't seem to be successful. This might not be elegant but it worked. Best regards, Hanno Travis Oliphant said: > Hanno Klemm wrote: > > >Hi, > > > >I don't know if this is a bug or just me doing something wrong (I > >suspect the latter). I try to compile numpy-1.0rc1 with python2.5 and > >atlas 3.7.17. > > > >I have build the atlas library myself, it doesn't give any errors > >under make test or make pttest, so it seems to be okay. if I try to > >build numpy I get the following error: > > > >creating build/temp.linux-x86_64-2.5/numpy/core/blasdot > >compile options: '-DATLAS_INFO="\"3.7.17\"" -Inumpy/core/blasdot > >-I/scratch/python2.5/include -Inumpy/core/include > >-Ibuild/src.linux-x86_64-2.5/numpy/core -Inumpy/core/src > >-Inumpy/core/include -I/scratch/python2.5/include/python2.5 -c' > >gcc: numpy/core/blasdot/_dotblas.c > >gcc -pthread -shared > >build/temp.linux-x86_64-2.5/numpy/core/blasdot/_dotblas.o > >-L/scratch/python2.5/lib -lcblas -latlas -o > >build/lib.linux-x86_64-2.5/numpy/core/_dotblas.so > >/usr/bin/ld: /scratch/python2.5/lib/libcblas.a(cblas_dgemm.o): > >relocation R_X86_64_32 can not be used when making a shared object; > >recompile with -fPIC > > > > > > This may be part of your problem. It's looks like the linker is having > a hard time making use of your compiled extension in a shared library. > Perhaps you should make sure -fPIC is on when you compile atlas (I'm not > sure how to do that --- perhaps setting CCFLAGS environment variable to > include -fPIC would help). > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- Hanno Klemm klemm at phys.ethz.ch ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 03:54:02 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 00:54:02 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DF1BA.9000906@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> <452DF1BA.9000906@ieee.org> Message-ID: <452DF49A.407@noaa.gov> Travis Oliphant wrote: > David Goldsmith wrote: > >>> >>> >>> >> I don't use scipy (and don't want to because of the overhead) but it >> sounds like I should because if I'm taking the square root of a variable >> whose value at run time happens to be real but less than zero, I *want* >> the language I'm using to return an imaginary; in other words, it's not >> the scipy behavior which "scares" me, its the numpy (which I do/have >> been using) behavior. >> > > O.K. Well the functions you want are in numpy.lib.scimath. I should > have directed you there. You actually don't need scipy installed at > all. Just import sqrt from numpy.lib.scimath. I'm sorry I > misunderstood the issue. > Got it. And if I understand correctly, the import order you specify in the little mynumpy example you included in your latest response to Fernando will result in any "overlap" between numpy and numpy.lib.scimath to call the latter's version of things rather than the former's, yes? DG > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From markbak at gmail.com Thu Oct 12 03:58:46 2006 From: markbak at gmail.com (Mark Bakker) Date: Thu, 12 Oct 2006 09:58:46 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? Message-ID: <6946b9500610120058l1a1d008agd19104ec23d7440a@mail.gmail.com> My vote is for consistency in numpy. But it is unclear what consistency is. What is truly confusing for newbie Python users (and a source for error even after 5 years of Python programming) is that >>> 2/3 0 In that respect, I would think >>> numpy.sqrt(2) should give 1, but it gives 1.4142135623730951 So numpy does typechecking anyway (it gets an integer and makes it a float). If that is the consistent behavior, then by all means >>> sqrt(-1) should return 1j. Wouldn't that be the consistent thing to do???? Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cristian.codorean at gmail.com Thu Oct 12 05:13:05 2006 From: cristian.codorean at gmail.com (Cristian Codorean) Date: Thu, 12 Oct 2006 10:13:05 +0100 Subject: Casting Message-ID: Hello list, I got some old code that uses Scientific and Numeric. Recently, these two packages were updated to the latest versions together with python and my code started failing. The first major problem I have is when downcasting. Code that used to work now fails because the array cannot be safely cast anymore to the required type. If anyone could help me please and direct me to some documentation about when this change happened (version), why it happened, is there any other solution but using astype, can numpy help me in anyway in this situation, because I got really lost ... Thank you in advance, Cristian -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From stefan at sun.ac.za Thu Oct 12 05:31:48 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 11:31:48 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: <20061012093148.GU20657@mentat.za.net> I've summarised this thread at http://www.scipy.org/NegativeSquareRoot Feel free to make adjustments, in case I missed something. Regards Stefan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mailer at phdjobs.com Thu Oct 12 06:12:54 2006 From: mailer at phdjobs.com (mailer at phdjobs.com) Date: Thu, 12 Oct 2006 11:12:54 +0100 Subject: =?iso-8859-1?q?PhDjobs=2Ecom_-_October_Newslet?= =?iso-8859-1?q?ter?= Message-ID: <20061012.XVYZAPTYBFJJJDWD@phdjobs.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From stefan at sun.ac.za Thu Oct 12 06:21:49 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 12:21:49 +0200 Subject: Please test the SVN branch In-Reply-To: <452DE2FB.3070408@ieee.org> References: <452DE2FB.3070408@ieee.org> Message-ID: <20061012102149.GV20657@mentat.za.net> On Thu, Oct 12, 2006 at 12:38:51AM -0600, Travis Oliphant wrote: > > I made some fixes to the "asbuffer" code which let me feel better about > exposing it in NumPy (where it is now named int_asbuffer). > > This code takes a Python integer and a size and returns a buffer object > that points to that memory. A little test is performed by trying to > read (and possibly write if a writeable buffer is requested) to the > first and last elements of the buffer. Any segfault is trapped and used > to raise a Python error indicating you can't use that area of memory. > > It doesn't guarantee you won't shoot yourself, but it does make it more > difficult to segfault Python. Previously a simple int_asbuffer(3423423, > 5) would have segfaulted (unless by chance you the memory area 3423423 > happens to be owned by Python). Some argument checking and a docstring would be helpful. I guessed wrong the first time: In [1]: x = N.array([1,2,3]) In [4]: y = N.int_asbuffer(x.size,x.__array_interface__['data'][0]) In [5]: y[0] Segmentation fault Out of curiosity, why does the following outputs differ: In [39]: y = N.int_asbuffer(0L,x.size) --------------------------------------------------------------------------- exceptions.SystemError Traceback (most recent call last) /home/stefan/ SystemError: error return without exception set In [40]: y = N.int_asbuffer(1L,x.size) --------------------------------------------------------------------------- exceptions.ValueError Traceback (most recent call last) /home/stefan/ ValueError: cannot use memory location as a buffer. Thanks for your work on this. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From zskhvgouoj at sncc.com Thu Oct 12 10:33:53 2006 From: zskhvgouoj at sncc.com (National) Date: Thu, 12 Oct 2006 12:33:53 -0200 Subject: Klaipeda Narva also comes Message-ID: <001201c6ede9$ea086760$e6cc1053@SZYMON> Grocery local Traffic am Maps Greater Orange co of Inland Empire Ventura Diego is Weather a Rain in. Smallest Four Yearswest is School Recalls in Exstudent is Disputes Claim war Deada Younger Unfazed Outrage Over. Slipups am sam Howe in Verhovek Using cheap video cameras a they shadow is opposing post or gotcha moments online n? Diego Weather Rain gtgt forecast Extreme house owner kindred in spirits then sue Wong restored. Byte of multibyte am auto indent of wheel mouse copy paste a line numbers unix format utf unicode? Downiraqi dreams acrobats Ryanair onto Curonian Pink girl suburbs a Movie or reviewout town Seventh festival in puppet. House a amp Garden Health Food am Culture Crossword Sudoku all Sections buy in Sell More Personals or Deals Stores Coupons Newspaper. Etbt Advanced newspaper This Week am Finance in Outlook Summer events am Cinema Cultural in agenda calendar Your Forum Todays Depth Budgets. Events Cinema Cultural agenda calendar Your Forum Todays Depth Budgets under fire increased or social Budgets offer. Seafood Stews Snapwest is and was Lynell George remembers Neils Words Tvcar! Guide fallen icons a horror Flash Most am Viewed False Choice Prompt Treason in Charge Southland am Mansmall of Plane am Pitcher. Athlete or theft Nokia planteu sees realistic date euro give military aidsenior State in official usa. Projects helping bridge gap of Galen Center Counts Opening Political am Trackers Capture Slipups sam Howe Verhovek Using cheap video cameras. Exodus a Vexes in Salinas usc Seeks Admission am Downtown by of Cara mia Dimassa. Mx Html Tidy of Cisco Rebol mirc Xslxslt Guicli a Fortran Autoit Pawl or dos Batch. Dxl Kixtart of Korn Shell Assembly Lisp or Maki is Script Nasm Nullsoft. Town Seventh in festival puppet of theatre Prima Donna Swamp Princess finally soul reviewhot royal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Trade.gif Type: image/gif Size: 11029 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cimrman3 at ntc.zcu.cz Thu Oct 12 07:25:11 2006 From: cimrman3 at ntc.zcu.cz (Robert Cimrman) Date: Thu, 12 Oct 2006 13:25:11 +0200 Subject: arraysetops update Message-ID: <452E2617.8020600@ntc.zcu.cz> I have just added 'See also' to all docstrings in arraysetops functions, replaced old numpy idioms with new ones (aka modernizing) and improved speed of numpy.unique1d, which now reflects numpy.unique. hoping no bugs were introduced, regards, r. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From c.hanot at septentrio.com Thu Oct 12 07:50:54 2006 From: c.hanot at septentrio.com (Charles Hanot) Date: Thu, 12 Oct 2006 13:50:54 +0200 Subject: Compiling Numpy with py2exe Message-ID: <8B7FAE730B4A3646B51C94CD6C8784B7B28F52@serversbs2003.Septentrio.local> Hello, I've a program using Numpy and I would like to compile it with Py2exe but I've some problems. When I try to compile my program, some modules generated by numpy are not loaded. I've already asked to the py2exe community if they have a solution for that and the answer was that it was possible to compile numpy modules with py2exe but I should ask to the Numpy community :-) So my question is, is there something special to do in order to compile a program using numpy module? This is the setup.py file that I use from distutils.core import setup import py2exe, sys sys.path.append("tools") sys.path.append("report") sys.path.append("spirent") sys.path.append("numpydir") setup( console = ['checkfile.py'], ) If I use this setup.py, lots of numpy file cannot be loaded. I get the following message copying c:\python24\lib\site-packages\py2exe\run.exe -> Y:\svn\copy\dist\checkfile.exe The following modules appear to be missing ['DateTime', 'FFT', 'Ft.Lib', 'Ft.Lib.DumpBgTuple', 'Ft.__init__', 'LinearAlgebra', 'MA', 'MLab', 'Matrix', 'Numeric', 'Pyrex', 'Pyrex.Compiler', 'XPathParserc' , '_curses', 'backends.draw_if_interactive', 'backends.new_figure_manager', 'backends.pylab_setup', 'backends.show', 'core.abs', 'core.max', 'core.min', 'core.r ound', 'ctypes', 'ext.IsDOMString', 'ext.SplitQName', 'fcompiler.FCompiler', 'fcompiler.show_fcompilers', 'lib.add_newdoc', 'mlab.amax', 'mlab.amin', 'numarray' , 'numarray.convolve', 'numarray.fft', 'numarray.ieeespecial', 'numarray.linear_algebra', 'numarray.linear_algebra.mlab', 'numarray.ma', 'numarray.numeric', 'nu merix.ArrayType', 'numerix.Complex', 'numerix.Complex32', 'numerix.Complex64', 'numerix.Float', 'numerix.Float32', 'numerix.Float64', 'numerix.Int', 'numerix.In t16', 'numerix.Int32', 'numerix.Int8', 'numerix.NewAxis', 'numerix.UInt16', 'numerix.UInt32', 'numerix.UInt8', 'numerix.absolute', 'numerix.add', 'numerix.all', 'numerix.allclose', 'numerix.alltrue', 'numerix.any', 'numerix.arange', 'numerix.arccos', 'numerix.arccosh', 'numerix.arcsin', 'numerix.arcsinh', 'numerix.arct an', 'numerix.arctan2', 'numerix.arctanh', 'numerix.argmax', 'numerix.argmin', 'numerix.argsort', 'numerix.around', 'numerix.array', 'numerix.arrayrange', 'nume rix.asarray', 'numerix.asum', 'numerix.bitwise_and', 'numerix.bitwise_or', 'numerix.bitwise_xor', 'numerix.ceil', 'numerix.choose', 'numerix.clip', 'numerix.com press', 'numerix.concatenate', 'numerix.conjugate', 'numerix.convolve', 'numerix.cos', 'numerix.cosh', 'numerix.cross_correlate', 'numerix.cumproduct', 'numerix .cumsum', 'numerix.diagonal', 'numerix.divide', 'numerix.dot', 'numerix.equal', 'numerix.exp', 'numerix.fabs', 'numerix.fft.fft', 'numerix.fft.inverse_fft', 'nu merix.floor', 'numerix.fmod', 'numerix.fromfunction', 'numerix.fromstring', 'numerix.greater', 'numerix.greater_equal', 'numerix.hypot', 'numerix.identity', 'nu merix.indices', 'numerix.innerproduct', 'numerix.iscontiguous', 'numerix.isnan', 'numerix.less', 'numerix.less_equal', 'numerix.log', 'numerix.log10', 'numerix. logical_and', 'numerix.logical_not', 'numerix.logical_or', 'numerix.logical_xor', 'numerix.matrixmultiply', 'numerix.maximum', 'numerix.minimum', 'numerix.mlab. amax', 'numerix.mlab.amin', 'numerix.mlab.cov', 'numerix.mlab.diff', 'numerix.mlab.hanning', 'numerix.mlab.rand', 'numerix.mlab.std', 'numerix.mlab.svd', 'numer ix.multiply', 'numerix.negative', 'numerix.newaxis', 'numerix.nonzero', 'numerix.not_equal', 'numerix.nx', 'numerix.ones', 'numerix.outerproduct', 'numerix.pi', 'numerix.power', 'numerix.product', 'numerix.put', 'numerix.putmask', 'numerix.rank', 'numerix.ravel', 'numerix.repeat', 'numerix.reshape', 'numerix.resize', ' numerix.searchsorted', 'numerix.shape', 'numerix.sin', 'numerix.sinh', 'numerix.size', 'numerix.sometrue', 'numerix.sort', 'numerix.sqrt', 'numerix.subtract', ' numerix.swapaxes', 'numerix.take', 'numerix.tan', 'numerix.tanh', 'numerix.trace', 'numerix.transpose', 'numerix.typecode', 'numerix.typecodes', 'numerix.where' , 'numerix.which', 'numerix.zeros', 'pre', 'serial', 'setuptools', 'setuptools.command', 'setuptools.command.egg_info', 'setuptools.command.install', 'testing.S cipyTest', 'win32api', 'win32con', 'win32pdh', 'xml.xslt', 'xml.xslt.ParsedPattern', 'numpy.Complex', 'numpy.Complex32', 'numpy.Complex64', 'numpy.Float', 'nump y.Float32', 'numpy.Float64', 'numpy.Int', 'numpy.Int16', 'numpy.Int32', 'numpy.Int8', 'numpy.UInt16', 'numpy.UInt32', 'numpy.UInt8', 'numpy.arccos', 'numpy.arcc osh', 'numpy.arcsin', 'numpy.arcsinh', 'numpy.arctan', 'numpy.arctanh', 'numpy.bitwise_and', 'numpy.bitwise_or', 'numpy.bitwise_xor', 'numpy.ceil', 'numpy.conju gate', 'numpy.core.add', 'numpy.core.bitwise_and', 'numpy.core.bitwise_or', 'numpy.core.bitwise_xor', 'numpy.core.cdouble', 'numpy.core.complexfloating', 'numpy .core.conjugate', 'numpy.core.csingle', 'numpy.core.divide', 'numpy.core.double', 'numpy.core.equal', 'numpy.core.float64', 'numpy.core.greater', 'numpy.core.gr eater_equal', 'numpy.core.inexact', 'numpy.core.intc', 'numpy.core.invert', 'numpy.core.left_shift', 'numpy.core.less', 'numpy.core.less_equal', 'numpy.core.max imum', 'numpy.core.not_equal', 'numpy.core.power', 'numpy.core.remainder', 'numpy.core.right_shift', 'numpy.core.sin', 'numpy.core.single', 'numpy.core.sqrt', ' numpy.core.subtract', 'numpy.cosh', 'numpy.dft.old', 'numpy.divide', 'numpy.e', 'numpy.fabs', 'numpy.floor', 'numpy.floor_divide', 'numpy.fmod', 'numpy.greater' , 'numpy.hypot', 'numpy.invert', 'numpy.left_shift', 'numpy.less', 'numpy.log', 'numpy.log10', 'numpy.logical_and', 'numpy.logical_not', 'numpy.logical_or', 'nu mpy.logical_xor', 'numpy.maximum', 'numpy.minimum', 'numpy.negative', 'numpy.not_equal', 'numpy.power', 'numpy.random.rand', 'numpy.random.randn', 'numpy.remain der', 'numpy.right_shift', 'numpy.sinh', 'numpy.tan', 'numpy.tanh', 'numpy.true_divide'] Do you know what to do to solve this problem? Thank you very much for your answers. This e-mail communication contains information that is confidential and may also be privileged. It is intended for the exclusive use of the addressees. If you are not the person or organization to whom it is addressed, you must not copy, distribute or take any action in reliance upon it. If you received this communication in error, please notify Septentrio nv immediately [ telephone +32 [0] 16 300800 ]. Septentrio nv will not accept liability for contractual commitments made by individuals employed by this company outside the scope of our business. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Thu Oct 12 09:58:21 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 08:58:21 -0500 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> Message-ID: <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> On 10/11/06, Bill Baxter wrote: > > On 10/12/06, Greg Willden wrote: > > Speed should not take precedence over correctness. > > Unless your goal is speed. Then speed should take precedence over > correctness. Huh? Person 1: "Hey you should use function X." Person 2: "No, it doesn't give me the correct answer." Person 1: "Who cares? It's fast!" What kind of logic is that? I'm having serious doubts about my conversion to Numpy. Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From aisaac at american.edu Thu Oct 12 10:13:46 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 12 Oct 2006 10:13:46 -0400 Subject: =?iso-8859-1?q?Should_numpy=2Esqrt=28-1=29_ret?= =?iso-8859-1?q?urn_1j_rather_than=09nan=3F?= In-Reply-To: <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu><20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu><903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> Message-ID: On Thu, 12 Oct 2006, Greg Willden apparently wrote: > I'm having serious doubts about my conversion to Numpy. That's fine, but be sure to treat the explanations you are receiving with the same seriousness that you treat your doubts. For example, in the speed vs. correctness discussion, be sure to understand the problems with the word "correct". Which have been explained. correct != expected when expectations lack context Cheers, Alan Isaac (just a happy user) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jg307 at cam.ac.uk Thu Oct 12 10:14:33 2006 From: jg307 at cam.ac.uk (James Graham) Date: Thu, 12 Oct 2006 15:14:33 +0100 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> Message-ID: <452E4DC9.8010805@cam.ac.uk> Greg Willden wrote: > On 10/11/06, *Bill Baxter* > wrote: > > On 10/12/06, Greg Willden > wrote: > > Speed should not take precedence over correctness. > > Unless your goal is speed. Then speed should take precedence over > correctness. > > > Huh? > Person 1: "Hey you should use function X." > Person 2: "No, it doesn't give me the correct answer." > Person 1: "Who cares? It's fast!" > > What kind of logic is that? > > I'm having serious doubts about my conversion to Numpy. It's more like Person 1: Function x is designed to be fast, have predictable memory usage and so be suitable for a wide variety of applications. Alternatively there is function y which is slower but handles a larger numerical domain. y Might be more sutiable for your needs; you should use it. Person 2: Numpy is broken! Function x should just do what I want! Really, given that both options are available, I don't see a big issue here. Claims that returning j for sqrt(-1) is "more correct" seem highly dubious as a) NumPy is a numerical library, not an algebraic one, hence the fact that it behaves in a type-dependent way is not so surprising, and b) no one complains that functions like sqrt return only positive floats, despite the fact that, mathematically, this function is multivalued. -- "Eternity's a terrible thought. I mean, where's it all going to end?" -- Tom Stoppard, Rosencrantz and Guildenstern are Dead ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 10:25:46 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 07:25:46 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <6946b9500610120058l1a1d008agd19104ec23d7440a@mail.gmail.com> References: <6946b9500610120058l1a1d008agd19104ec23d7440a@mail.gmail.com> Message-ID: <452E506A.6060306@ieee.org> Mark Bakker wrote: > My vote is for consistency in numpy. > But it is unclear what consistency is. > > What is truly confusing for newbie Python users (and a source for > error even after 5 years of Python programming) is that > > >>> 2/3 > 0 I recommend that you slap "from __future__ import division" into site.py or the top of your program: from __future__ import division import numpy print 3/2 a = numpy.arange(3) print a / (a+5) print a // a+5 >>>> 1.5 [ 0. 0.16666667 0.28571429] [5 6 6] > > In that respect, I would think > > >>> numpy.sqrt(2) > > should give 1, but it gives 1.4142135623730951 Is there any practical reason to return 1? If not, isn't this argument sort of silly? > So numpy does typechecking anyway (it gets an integer and makes it a > float). > > If that is the consistent behavior, then by all means > > >>> sqrt(-1) > > should return 1j. Well, it could also return: -1j or as someone mentioned one of six (I think) different quaternion values. It all depends on what the domain/range of the problem is. > Wouldn't that be the consistent thing to do???? No, there's a difference. In order to do the former, all that is required is that sqrt switches on the *type* of it argument. sqrt return a float for integer and floating point args and a complex for complex args. In order to the latter, sqrt needs to switch on the *value* of its argument, which is an entirely different beast both in theory and in practice. In particular: sqrt(some_big_array) would have to scan all the values in the array to determine if any were negative to decide whether to return a float or an imaginary number. It also means that the memory usage is unpredictable -- the returned array is double in size if any values are negative. In addition to the significant slowdown this introduces, the former approach (keeping the input and output domains the same) is somewhat more robust against error, particularly if one tightens up the error mode. I would guess that people are working the complex plane at most half the time, in the former case a negative square root signals a problem and promoting to complex is at best a nuisance and may result in a painful to track down error. In the latter case, it's easy enough for me to toss in a astype in situations where I'm mixing domains and need complex outputs. Ideally numpy and scipy would have chosen different names for forgiving and strict powers and square roots, square_root and power versus sqrt and pow for example, but it's probably too late to do anything about that now. Since it sounds like Travis is going to tighten up the default error mode, I think that this is a non issue. No ones going to run into NANs unexpectedly and the error when doing sqrt([1,2,3,-1]) should be confusing and most once. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 10:27:05 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 07:27:05 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DF3FB.8010201@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEF81.4040208@ieee.org> <452DF3FB.8010201@ieee.org> Message-ID: <452E50B9.80008@ieee.org> Travis Oliphant wrote: > Travis Oliphant wrote: > >> Now, it would be possible to give ufuncs a dtype keyword argument that >> allowed you to specify which underlying loop was to be used for the >> calculation. That way you wouldn't have to convert inputs to complex >> numbers before calling the ufunc, but could let the ufunc do it in >> chunks during the loop. That is certainly a reasonable enhancement: >> >> sqrt(a, dtype=complex). >> >> This no-doubt has a "library-ish"-feeling, but that is what NumPy is. >> If such a change is desireable, I don't think it would be much to >> implement it. >> >> >> > This could be implemented, but only with a version number increase in > the C-API (we would have to change the c-signature of the ufunc tp_call. > > This would mean that the next release of NumPy would be binary > incompatible with packages built to previous NumPy releases. > > I've really been trying to avoid doing that. So, unless there are > strong requests for this feature that outweigh the negatives of > re-building dependent packages, then this feature will have to wait. > > OTOH: I suppose it could be implemented in a different way (using > indexing or a method call): > > sqrt[complex](a) --- I remember Tim suggesting some use for indexing > on ufuncs earlier though. > It wouldn't surprise me if I did -- it sounds like the kind of thing I'd propose -- but I certainly can't remember what I was proposing. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 12 10:28:28 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 16:28:28 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> Message-ID: <20061012142828.GG32224@mentat.za.net> On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: > On 10/11/06, Bill Baxter wrote: > > On 10/12/06, Greg Willden wrote: > > Speed should not take precedence over correctness. > > Unless your goal is speed. Then speed should take precedence over > correctness. > > > Huh? > Person 1: "Hey you should use function X." > Person 2: "No, it doesn't give me the correct answer." > Person 1: "Who cares? It's fast!" > > What kind of logic is that? I tried to explain the argument at http://www.scipy.org/NegativeSquareRoot Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 10:31:05 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 07:31:05 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE98A.4090404@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <452DBFBB.8090403@ieee.org> <452DE98A.4090404@ieee.org> Message-ID: <452E51A9.5030403@ieee.org> Travis Oliphant wrote: >> Personally I think that the default error mode should be tightened >> up. >> Then people would only see these sort of things if they really care >> about them. Using Python 2.5 and the errstate class I posted earlier: >> >> # This is what I like for the default error state >> numpy.seterr (invalid='raise', divide='raise', over='raise', >> under='ignore') >> >> >> I like these choices too. Overflow, division by zero, and sqrt(-x) are >> usually errors, indicating bad data or programming bugs. Underflowed >> floats, OTOH, are just really, really small numbers and can be treated >> as zero. An exception might be if the result is used in division and >> no error is raised, resulting in a loss of accuracy. >> >> > > I'm fine with this. I've hesitated because error checking is by default > slower. But, I can agree that it is "less surprising" to new-comers. > People that don't mind no-checking can simply set their defaults back to > 'ignore' > > Great. One thing we may want to do (numarray had this), was add a pseudo argument 'all', that allows you to set all of the values at once. Then if you want the full-bore, ignore-all-errors computation (and your using 2.5 and "from __future__ import with_statement") you can just do: with errstate(all='ignore'): # computation here # back to being picky -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 10:43:13 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 07:43:13 -0700 Subject: Please test the SVN branch In-Reply-To: <452DE2FB.3070408@ieee.org> References: <452DE2FB.3070408@ieee.org> Message-ID: <452E5481.5070403@ieee.org> Travis Oliphant wrote: > I made some fixes to the "asbuffer" code which let me feel better about > exposing it in NumPy (where it is now named int_asbuffer). > > This code takes a Python integer and a size and returns a buffer object > that points to that memory. A little test is performed by trying to > read (and possibly write if a writeable buffer is requested) to the > first and last elements of the buffer. Any segfault is trapped and used > to raise a Python error indicating you can't use that area of memory. > > It doesn't guarantee you won't shoot yourself, but it does make it more > difficult to segfault Python. Previously a simple int_asbuffer(3423423, > 5) would have segfaulted (unless by chance you the memory area 3423423 > happens to be owned by Python). > > I have not tested the code on other platforms to make sure it works as > expected, so please try and compiled it. > FWIW, it compiles fine and passes all tests under Visual Studio and Python 2.4.3. I haven't tried to use it though. -tim > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Thu Oct 12 10:53:12 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 12 Oct 2006 10:53:12 -0400 Subject: =?iso-8859-1?q?Should_numpy=2Esqrt=28-1=29_ret?= =?iso-8859-1?q?urn_1j_rather_than=09nan=3F?= In-Reply-To: <20061012142828.GG32224@mentat.za.net> References: <452D56F5.9000404@ee.byu.edu><20061011230705.GA30571@mentat.za.net><452D7C88.3090506@ee.byu.edu><903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com><903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> <20061012142828.GG32224@mentat.za.net> Message-ID: On Thu, 12 Oct 2006, Stefan van der Walt apparently wrote: > I tried to explain the argument at > http://www.scipy.org/NegativeSquareRoot Helpful. But you start off by saying: In mathematics, the above assumption is true -- that the square root of -1 is 1j. Since square root is (here) a function, and part of the function definition is the domain and codomain, this statement is not correct. If the codomain is real numbers, the domain must correspondingly be (a subset of) the nonnegative reals. The nan output is the result of an **invalid** input. So the question is really: "Why is a negative real number an invalid input in this implementation", or "Why in this implementation is the type of the output restricted by the type of the input?" You get a good start on answering that. Cheers, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Thu Oct 12 10:54:22 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 09:54:22 -0500 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> Message-ID: <903323ff0610120754t399f3626o7fce374ad8d85582@mail.gmail.com> I'll try to quit being a pain and just work around it. Those wiki pages are really good to capture this information. I need to read all of them to know what other gotchas are out there. Thanks and Sorry, Greg On 10/12/06, Alan G Isaac wrote: > > That's fine, but be sure to treat the explanations you are > receiving with the same seriousness that you treat your > doubts. > > For example, in the speed vs. correctness discussion, be > sure to understand the problems with the word "correct". > Which have been explained. > > correct != expected > when expectations lack context > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Thu Oct 12 11:19:05 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 10:19:05 -0500 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610120754t399f3626o7fce374ad8d85582@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> <903323ff0610120754t399f3626o7fce374ad8d85582@mail.gmail.com> Message-ID: <903323ff0610120819t1a758a61y7f79e646b77266b@mail.gmail.com> May I suggest the following change to generate_umath.py? 'sqrt' : Ufunc(1, 1, None, 'square-root elementwise. For real x, the domain is restricted to x>=0.\n\ For complex results for x<0 see numpy.scimath.sqrt', TD(inexact, f='sqrt'), TD(M, f='sqrt'), ), ' When sqrt throws a ValueError would it be possible/appropriate for the error message to mention numpy.scimath.sqrt? I'm just trying to think of other ways to help make the transition as smooth as possible for new users. Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From a.h.jaffe at gmail.com Thu Oct 12 11:35:19 2006 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Thu, 12 Oct 2006 16:35:19 +0100 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D9D23.5030700@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> <452D5BFA.7000603@ee.byu.edu> <452D70B9.7070209@ieee.org> <452D8277.3080306@ee.byu.edu> <452D9D23.5030700@ieee.org> Message-ID: <452E60B7.8030400@gmail.com> Tim Hochberg wrote: > Travis Oliphant wrote: >> Tim Hochberg wrote: >> >> >>> With python 2.5 out now, perhaps it's time to come up with a with >>> statement context manager. Something like: >>> >>> a = numpy.arange(10) >>> a/a # ignores divide by zero >>> with errstate(divide='raise'): >>> a/a # raise exception on divide by zer >>> # Would ignore divide by zero again if we got here. >>> >>> -tim >>> >> This looks great. I think most people aren't aware of the with >> statement and what it can do (I'm only aware because of your posts, for >> example). >> >> So, what needs to be added to your example in order to just add it to >> numpy? >> > As far as I know, just testing and documentation -- however testing was > so minimal that I may find some other stuff. I'll try to clean it up > tomorrow so that I'm a little more confident that it works correctly and > I'll send another note out then. For this particular application, why not a context manager which just substitutes in the appropriately-optimized version of sqrt? That is, don't change the error state, but actually change the value of the object pointed at by the name sqrt? Andrew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From hetland at tamu.edu Thu Oct 12 11:52:15 2006 From: hetland at tamu.edu (Rob Hetland) Date: Thu, 12 Oct 2006 10:52:15 -0500 Subject: Things to address for Py3K In-Reply-To: <452D3AC7.7000805@ee.byu.edu> References: <452D3AC7.7000805@ee.byu.edu> Message-ID: On Oct 11, 2006, at 1:41 PM, Travis Oliphant wrote: > a) an extra multiplication operator to distinguish between > element-by-element and dot One of the things I like about numpy is that element-by-element multiplication is default. Really, most of what I do is element by element, punctuated by a few matrix multiplications (or whatever other matrix operations) at the end. If I had a penny for every '.' I put in front of an operator in MATLAB, I would buy us all new computers -- expensive ones even. I hope that, if there is another multiplication operator like .*, that this will be shorthand for dot(A, B), and not for elem-by-elem multiplication. -Rob ---- Rob Hetland, Associate Professor Dept. of Oceanography, Texas A&M University http://pong.tamu.edu/~rob phone: 979-458-0096, fax: 979-845-6331 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Thu Oct 12 12:07:09 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Thu, 12 Oct 2006 12:07:09 -0400 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DEAED.9070408@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> Message-ID: On 12/10/06, David Goldsmith wrote: > I don't use scipy (and don't want to because of the overhead) but it > sounds like I should because if I'm taking the square root of a variable > whose value at run time happens to be real but less than zero, I *want* > the language I'm using to return an imaginary; in other words, it's not > the scipy behavior which "scares" me, its the numpy (which I do/have > been using) behavior. To which you might say, "Well, if that's what you > want, and you have Matlab (as you've said you do), then just use that." > But that's precisely the point: people who don't want to be bothered > with having to be "a bit more care[ful]" (as Chuck put it) - and who can > afford it - are going to be inclined to choose Matlab over numpy. > Perhaps one doesn't care - in the grand scheme of things, it certainly > doesn't matter - but I think that you all should be aware that this > numpy "feature" will be seen by many as more than just a nuisance. Scipy doesn't do what you want. What you want is to use complex numbers throughout; then numpy and scipy will do exactly what you request. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ndbecker2 at gmail.com Thu Oct 12 12:03:45 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Oct 2006 12:03:45 -0400 Subject: PATCH build numpy-1.0rc2 Message-ID: I have not been able to build any of the numpy-1.0 series. Finally I bit the bullet and fixed it. Here is the patch: diff -u numpy-1.0rc2/numpy/numarray/setup.py{~,} --- numpy-1.0rc2/numpy/numarray/setup.py~ 2006-07-26 16:04:32.000000000 -0400 +++ numpy-1.0rc2/numpy/numarray/setup.py 2006-10-12 11:58:23.000000000 -0400 @@ -5,7 +5,8 @@ from numpy.distutils.misc_util import Configuration config = Configuration('numarray',parent_package,top_path) - config.add_data_files('numpy/') +# config.add_data_files('numpy/') + config.add_data_dir('numpy/') # Configure fftpack_lite config.add_extension('_capi', ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 12 12:14:50 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 18:14:50 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than?nan? In-Reply-To: References: <20061012142828.GG32224@mentat.za.net> Message-ID: <20061012161450.GH32224@mentat.za.net> On Thu, Oct 12, 2006 at 10:53:12AM -0400, Alan G Isaac wrote: > On Thu, 12 Oct 2006, Stefan van der Walt apparently wrote: > > I tried to explain the argument at > > http://www.scipy.org/NegativeSquareRoot > > Helpful. But you start off by saying: > In mathematics, the above assumption is true -- that > the square root of -1 is 1j. > > Since square root is (here) a function, and part of the > function definition is the domain and codomain, this > statement is not correct. If the codomain is real numbers, > the domain must correspondingly be (a subset of) the > nonnegative reals. The nan output is the result of an > **invalid** input. > > So the question is really: "Why is a negative real number an > invalid input in this implementation", or "Why in this > implementation is the type of the output restricted by the > type of the input?" You get a good start on answering that. You are more than welcome to change the wiki if you can think of a simpler way to explain the problem... In fact, I would encourage that -- I just put it there as a starting point. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 12:24:40 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 09:24:40 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610120819t1a758a61y7f79e646b77266b@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> <903323ff0610120754t399f3626o7fce374ad8d85582@mail.gmail.com> <903323ff0610120819t1a758a61y7f79e646b77266b@mail.gmail.com> Message-ID: <452E6C48.1030607@noaa.gov> Greg Willden wrote: > May I suggest the following change to generate_umath.py? > > > 'sqrt' : > Ufunc(1, 1, None, > 'square-root elementwise. For real x, the domain is > restricted to x>=0.\n\ > For complex results for x<0 see numpy.scimath.sqrt', > TD(inexact, f='sqrt'), > TD(M, f='sqrt'), > ), > ' > > When sqrt throws a ValueError would it be possible/appropriate for the > error message to mention numpy.scimath.sqrt? > > I'm just trying to think of other ways to help make the transition as > smooth as possible for new users. And helpful error messages are one of Python's best features - in general, numpy doesn't have enough of 'em, IMO. DG > > Greg > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Thu Oct 12 13:34:00 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 12 Oct 2006 10:34:00 -0700 Subject: Please test the SVN branch In-Reply-To: <452DE2FB.3070408@ieee.org> References: <452DE2FB.3070408@ieee.org> Message-ID: <452E7C88.7000205@noaa.gov> Travis Oliphant wrote: > I made some fixes to the "asbuffer" code which let me feel better about > exposing it in NumPy (where it is now named int_asbuffer). > > This code takes a Python integer and a size and returns a buffer object > that points to that memory. If I understand this correctly, maybe it should be called something like: int_asbuffer_pointer or something with "pointer" in it. I sure didn't expect that meaning from "asbuffer" or in-asbuffer" Otherwise -- nice idea! -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From vallis at vallis.org Thu Oct 12 13:41:37 2006 From: vallis at vallis.org (Michele Vallisneri) Date: Thu, 12 Oct 2006 10:41:37 -0700 Subject: Providing array interface from SWIG-wrapped C struct Message-ID: Does anybody here have experience about offering the array interface from a SWIG-wrapped C struct? I have tried the following, borrowing code from numpy's arrayobject.c: %extend real_vec_t { PyObject *__array_struct__() { /* From numpy/arrayobject.c/array_struct_get */ PyArrayInterface *inter; inter = (PyArrayInterface *)_pya_malloc(sizeof(PyArrayInterface)); inter->two = 2; inter->nd = 1; inter->typekind = 'f'; inter->itemsize = sizeof(float); inter->flags = (NPY_CONTIGUOUS | NPY_OWNDATA | NPY_ALIGNED | NPY_NOTSWAPPED); inter->shape = (intp *)_pya_malloc(2*sizeof(intp)); inter->strides = inter->shape + 1; *(inter->shape) = self->n; *(inter->strides) = 1; inter->data = (void *)(self->d); inter->descr = 0; return PyCObject_FromVoidPtr(inter,0); } %} When passing such a SWIG real_vec_t object to numpy.array, numpy gets a PyArrayInterface that fails to check correctly as a Python C object. Three comments: - The original numpy/arrayobject.c code actually gets its return value from PyCObject_FromVoidPtrAndDesc(inter,self,...); but in this context self does not point to a PyObject, but rather to the C struct, so I don't think I can use that form. - The code above does not deal with properly destructing the PyArrayInterface, setting refcounts, etc. But I was going to worry about those later. - My problem is probably with SWIG rather than numpy... but I thought the people in this forum (rather than those at the SWIG forum) would be most likely to have encountered a situation similar to these. Thanks a lot for any help, or just for reading this post. Michele ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Thu Oct 12 13:49:25 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 12 Oct 2006 10:49:25 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE009.3070100@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452E8025.3060005@noaa.gov> As someone on this list (sorry to quick with the delete button) said: "numpy is closer to the metal than MATLAB" MATLAB aside, numpy is somewhat "close to the metal". I think this is clearly part of its design philosophy, and also something I personally like about out. Getting back to MATLAB (at least version 4, the last one I used regularly) -- everything in MATLAB is a matrix of double precision Floating points. (or complex - that does get weird!). This makes things simpler for a lot of use, but restricts the power and flexibility substantially. Everything in numpy is an array of arbitrary shape and many possible data types -- this is more to think about, but give you a lot of power and flexibility. In any case, that is what numpy is. Period. Given that, arrays of different types behave differently -- that's something you'd darn well better understand pretty soon in your numpy career: >>> a = N.array((1,2,3), dtype=N.uint8) >>> a *= 200 >>> a array([200, 144, 88], dtype=uint8) Oh my gosh! 2 * 200 isn't 144! what happened? This isn't any different than sqrt(-1) resulting in NaN or an error. In fact, there are all sorts of places where upcasting is not happening automagically -- in fact, I think that's become more consistent in the new numpy. So, in numpy you need to choose the datatype appropriate for your problem at hand. I know I always know whether I want to work in the complex plane or not. Another change recently made is to make doubles the default in more places -- I like that change. So, given the entire philosophy and current status of how numpy works, the ONLY question that is legitimate here is: Should the "default" datatype be complex or double? I vote double. Travis Oliphant wrote: > Now, it would be possible to give ufuncs a dtype keyword argument that > allowed you to specify which underlying loop was to be used for the > calculation. > sqrt(a, dtype=complex). I like this OK, and isn't is similar to the dtype argument in the accumulating methods? (sum, etc.). However, it's probably not worth the C-api change -- is it that different than calling: sqrt(a.astype(complex)) ? As for scipy, I heartily agree that scipy should be a superset of numpy, and NOT change the behavior of numpy functions and methods. We're taking the opportunity at this point for numpy to break backward compatibility with Numeric/numarray -- this is probably the best time to do the same for scipy. couldn't we at least introduce new names? numpy.scimath.csqrt() for instance? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 14:04:55 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 11:04:55 -0700 Subject: multiarray tests skipped under py2.5 Message-ID: <452E83C7.5040404@ieee.org> Is any one else seeing the multiarray tests all get skipped because of an import error when compiling under python 2.5. Everything else seems to work and all the tests go fine under 2.4. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From loredo at astro.cornell.edu Thu Oct 12 14:03:31 2006 From: loredo at astro.cornell.edu (Tom Loredo) Date: Thu, 12 Oct 2006 14:03:31 -0400 Subject: gcc/fortran compilers for building extensions with Tiger/MacPython In-Reply-To: References: Message-ID: <1160676211.452e837390848@astrosun2.astro.cornell.edu> Hi folks, I've just upgraded my G4 (PPC) from Pather to Tiger, and just installed MacPython. I now need to build a slew of numerical extensions. I'm building them from source (I have lots of custom extensions to build from source in any case). I see the binaries available for Tiger are built with gcc 3.3 for PPC and gcc 4 for Intel; I think they may be tied to ActivePython but I'm not sure. When I start MacPython, I see it is built with gcc 4. Is this what I should be using? I'll also be compiling fortran extensions; I presume I should use gfortran rather than g77 for this (g77 is what worked with gcc 3.3). Is this correct? Thanks, Tom ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gerard.vermeulen at grenoble.cnrs.fr Thu Oct 12 14:13:38 2006 From: gerard.vermeulen at grenoble.cnrs.fr (Gerard Vermeulen) Date: Thu, 12 Oct 2006 20:13:38 +0200 Subject: multiarray tests skipped under py2.5 In-Reply-To: <452E83C7.5040404@ieee.org> References: <452E83C7.5040404@ieee.org> Message-ID: <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> On Thu, 12 Oct 2006 11:04:55 -0700 Tim Hochberg wrote: > > Is any one else seeing the multiarray tests all get skipped because of > an import error when compiling under python 2.5. Everything else seems > to work and all the tests go fine under 2.4. Everything looks fine here: >>> sys.version '2.5 (r25:51908, Oct 8 2006, 07:23:15) \n[GCC 4.1.1 (Gentoo 4.1.1)]' >>> np.version.version '1.0.dev3317' >>> np.test() Found 13 tests for numpy.core.umath Found 9 tests for numpy.lib.arraysetops Found 36 tests for numpy.core.ma Found 3 tests for numpy.fft.helper Found 4 tests for numpy.ctypeslib Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 26 tests for numpy.core.numeric Found 5 tests for numpy.distutils.misc_util Found 3 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 4 tests for numpy.core.scalarmath Found 1 tests for numpy.lib.ufunclike Found 12 tests for numpy.lib.twodim_base Found 47 tests for numpy.lib.shape_base Found 4 tests for numpy.lib.index_tricks Found 32 tests for numpy.linalg.linalg Found 42 tests for numpy.lib.type_check Found 176 tests for numpy.core.multiarray Found 10 tests for numpy.core.defmatrix Found 41 tests for numpy.lib.function_base Found 0 tests for __main__ ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ---------------------------------------------------------------------- Ran 509 tests in 0.265s OK >>> Gerard ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 14:22:01 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 11:22:01 -0700 Subject: multiarray tests skipped under py2.5 In-Reply-To: <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> References: <452E83C7.5040404@ieee.org> <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> Message-ID: <452E87C9.5000503@ieee.org> Gerard Vermeulen wrote: > On Thu, 12 Oct 2006 11:04:55 -0700 > Tim Hochberg wrote: > > >> Is any one else seeing the multiarray tests all get skipped because of >> an import error when compiling under python 2.5. Everything else seems >> to work and all the tests go fine under 2.4. >> > > Everything looks fine here: > Thanks. I suppose that means it's either Windows specific or I gummed up my build tree somehow. I'll make a clean checkout and see if it goes away. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From haase at msg.ucsf.edu Thu Oct 12 14:30:34 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Thu, 12 Oct 2006 11:30:34 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452E8025.3060005@noaa.gov> References: <452DE009.3070100@ieee.org> <452E8025.3060005@noaa.gov> Message-ID: <200610121130.34936.haase@msg.ucsf.edu> Hi, On Thursday 12 October 2006 10:49, Christopher Barker wrote: > As someone on this list (sorry to quick with the delete button) said: I think I might agree with everything (!) you that - very well written . > Travis Oliphant wrote: > > Now, it would be possible to give ufuncs a dtype keyword argument that > > allowed you to specify which underlying loop was to be used for the > > calculation. > > > > sqrt(a, dtype=complex). > > I like this OK, and isn't is similar to the dtype argument in the > accumulating methods? (sum, etc.). However, it's probably not worth the > C-api change -- is it that different than calling: > > sqrt(a.astype(complex)) ? > It is very late in the game - obvious ! (one week before the announced final release). But I think adding the dtype argument to ufuncs would be a very useful addition !! It is very nice to look at ! And it saves (temporary) memory and should be noticeably faster than "sqrt(a.astype(complex))". This dtype addition might address many of the issues I have raised before on this list. Travis, be courageous and change the C-API one last time before putting it in stone for 1.0 final !!! > As for scipy, I heartily agree that scipy should be a superset of numpy, > and NOT change the behavior of numpy functions and methods. We're taking > the opportunity at this point for numpy to break backward compatibility > with Numeric/numarray -- this is probably the best time to do the same > for scipy. > > couldn't we at least introduce new names? > > numpy.scimath.csqrt() > > for instance? > > -Chris It has to be very well documented what the 12 functions (the last 3%) are were scipy and numpy differ. --- if at all possible I would prefer a name change like csqrt (or better sqrtc - I always like seen things together when sorted alphabetically ;-) ) [by the way: I like sqrt(-1) == nan just fine] Thanks for the great work. - Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Thu Oct 12 14:40:46 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 12 Oct 2006 15:40:46 -0300 Subject: Providing array interface from SWIG-wrapped C struct In-Reply-To: References: Message-ID: On 10/12/06, Michele Vallisneri wrote: > Does anybody here have experience about offering the array interface > from a SWIG-wrapped C struct? I have. > I have tried the following, borrowing code from numpy's arrayobject.c: > > %extend real_vec_t { > PyObject *__array_struct__() { > /* From numpy/arrayobject.c/array_struct_get */ You are extending real_vec_t with a new METHOD, but what numpy requests is an ATTRIBUTE. So, numpy simply queries your vec like: arrstr = vec.__array_struct__ and not with a method call like this arrstr = vec.__array_struct__() So here is what I would do (can fail with some SWIG optimizations) %extend Vec { PyObject* __array_struct__ () { /* ... */ } %pythoncode { __array_struct__ = property(__array_struct__, doc='Array protocol') } } Hope you got the idea. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ndbecker2 at gmail.com Thu Oct 12 13:54:16 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Oct 2006 13:54:16 -0400 Subject: numpy-1.0 problem with install --skip-build Message-ID: It appears the problem with getting a numpy-1.0 built with RPM is caused by using 2 step process: python setup.py build python setup.py install --skip-build --root $RPM_BUILD_ROOT This doesn't seem to work correctly! If I change it to just do: python setup.py install it works OK. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cookedm at physics.mcmaster.ca Thu Oct 12 14:30:47 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Thu, 12 Oct 2006 14:30:47 -0400 Subject: gcc/fortran compilers for building extensions with Tiger/MacPython In-Reply-To: <1160676211.452e837390848@astrosun2.astro.cornell.edu> References: <1160676211.452e837390848@astrosun2.astro.cornell.edu> Message-ID: <20061012143047.6fac6e27@arbutus.physics.mcmaster.ca> On Thu, 12 Oct 2006 14:03:31 -0400 Tom Loredo wrote: > Hi folks, > > I've just upgraded my G4 (PPC) from Pather to Tiger, and just > installed MacPython. I now need to build a slew of numerical extensions. > I'm building them from source (I have lots of custom extensions > to build from source in any case). I see the binaries available > for Tiger are built with gcc 3.3 for PPC and gcc 4 for Intel; I > think they may be tied to ActivePython but I'm not sure. When > I start MacPython, I see it is built with gcc 4. Is this what > I should be using? I'll also be compiling fortran extensions; > I presume I should use gfortran rather than g77 for this (g77 > is what worked with gcc 3.3). Is this correct? For Intel, yes, you need to use gfortran, and it has to be the prelease version (4.2): 4.1 and earlier don't build on Intel Macs. g77 (and gcc 3.x) won't build either, unfortunately. If you're using MacPorts (previously DarwinPorts), you'll want the gcc42 port. Or, you can grab binaries at http://hpc.sourceforge.net/ -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pfdubois at gmail.com Thu Oct 12 14:50:12 2006 From: pfdubois at gmail.com (Paul Dubois) Date: Thu, 12 Oct 2006 11:50:12 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452E8025.3060005@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452E8025.3060005@noaa.gov> Message-ID: Historical note. When Cleve Moler first created 'matlab' as a tool for teaching at the University of New Mexico, all numbers were complex. And, in fact, there was a limit of 10,000 numbers. There was a fixed array where these 10,000 numbers were stored. I got pretty excited about his project and wanted to use it to get eigenvalues etc. for matrices of interest from our codes. But I had a Cray 1 with an entire megaword of memory and my matrices wouldn't fit. So I set out to hack the code to add a memory manager so that we could use the entire memory. This was a big success. Then I decided it was sad that I had to write the matrices out of the code onto a disk and then read them into matlab, so I got the idea of making something similar as the front end of the code, only with data types (because I hated only being able to use half my Cray since my data was real, and our codes were often memory bound) and a full programming language like Fortran with array syntax for speed. This idea resulted in Basis (basis.llnl.gov). My interest in Python (and hence numpy) evolved from that as OOP arrived. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Thu Oct 12 14:35:35 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 11:35:35 -0700 Subject: multiarray tests skipped under py2.5 In-Reply-To: <452E87C9.5000503@ieee.org> References: <452E83C7.5040404@ieee.org> <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> <452E87C9.5000503@ieee.org> Message-ID: <452E8AF7.1040203@ieee.org> Tim Hochberg wrote: > Gerard Vermeulen wrote: > >> On Thu, 12 Oct 2006 11:04:55 -0700 >> Tim Hochberg wrote: >> >> >> >>> Is any one else seeing the multiarray tests all get skipped because of >>> an import error when compiling under python 2.5. Everything else seems >>> to work and all the tests go fine under 2.4. >>> >>> >> Everything looks fine here: >> >> > > Thanks. > > I suppose that means it's either Windows specific or I gummed up my > build tree somehow. I'll make a clean checkout and see if it goes away. > > Still there with a clean checkout. Blech! I hate weird import problems. I'll try to look into it, but if anyone has any good ideas, let me know -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 15:32:28 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 12:32:28 -0700 Subject: multiarray tests skipped under py2.5 In-Reply-To: <452E9726.9020405@ee.byu.edu> References: <452E83C7.5040404@ieee.org> <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> <452E87C9.5000503@ieee.org> <452E8AF7.1040203@ieee.org> <452E9726.9020405@ee.byu.edu> Message-ID: <452E984C.5000709@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > > >>> >>> >>> >>> >> Still there with a clean checkout. Blech! I hate weird import problems. >> I'll try to look into it, but if anyone has any good ideas, let me know >> >> >> > Show us the errors, I may be able to help. It's possible some recent > changes messed things up. > I just checked in a fix. I think it was some case problem in the paths that got set in set_local_path(). In any event, applying normpath to the merged path fixed the problem and I didn't feel like pursuing it further. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 15:24:28 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 12:24:28 -0700 Subject: More SVN testing Message-ID: <452E966C.1060300@ieee.org> I just checked in a couple of changes to SVN. I was going to check in errstate, but it looks like Travis beat me to it, so I contented myself with adding a docstring and some tests. These tests are only run under 2.5; things seem to work fine here, but if someone on a Linux box whose running Python 2.5 could try it out I'd appreciate it. I'm suspicous of Linux/Windows differences right now because of my run in with them earlier today in the tests for test_multiarray (which I believe it now fixed -- at least everything is working here OK). -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 12 15:27:34 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 12 Oct 2006 13:27:34 -0600 Subject: multiarray tests skipped under py2.5 In-Reply-To: <452E8AF7.1040203@ieee.org> References: <452E83C7.5040404@ieee.org> <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> <452E87C9.5000503@ieee.org> <452E8AF7.1040203@ieee.org> Message-ID: <452E9726.9020405@ee.byu.edu> Tim Hochberg wrote: >> >> >> >Still there with a clean checkout. Blech! I hate weird import problems. >I'll try to look into it, but if anyone has any good ideas, let me know > > Show us the errors, I may be able to help. It's possible some recent changes messed things up. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 12 15:15:58 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 12 Oct 2006 13:15:58 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DF49A.407@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> <452DF1BA.9000906@ieee.org> <452DF49A.407@noaa.gov> Message-ID: <452E946E.8000809@ee.byu.edu> David Goldsmith wrote: >Got it. And if I understand correctly, the import order you specify in >the little mynumpy example you included in your latest response to >Fernando will result in any "overlap" between numpy and >numpy.lib.scimath to call the latter's version of things rather than the >former's, yes? > > Right. The last import will be used for any common-names (variables get re-bound to the new functions...) -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Thu Oct 12 15:53:10 2006 From: fullung at gmail.com (Albert Strasheim) Date: Thu, 12 Oct 2006 21:53:10 +0200 Subject: More SVN testing In-Reply-To: <452E966C.1060300@ieee.org> References: <452E966C.1060300@ieee.org> Message-ID: Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Tim Hochberg > Sent: 12 October 2006 21:24 > To: numpy-discussion > Subject: [Numpy-discussion] More SVN testing > > > I just checked in a couple of changes to SVN. I was going to check in > errstate, but it looks like Travis beat me to it, so I contented myself > with adding a docstring and some tests. These tests are only run under > 2.5; things seem to work fine here, but if someone on a Linux box whose > running Python 2.5 could try it out I'd appreciate it. I'm suspicous of > Linux/Windows differences right now because of my run in with them > earlier today in the tests for test_multiarray (which I believe it now > fixed -- at least everything is working here OK). This probably fixes the issue I reported here: http://projects.scipy.org/scipy/numpy/ticket/286 Maybe you can check that the test counts match up on Python 2.4 and 2.5 on Windows on your system and close that ticket. Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Thu Oct 12 15:59:40 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 14:59:40 -0500 Subject: Polyfit Message-ID: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> Hi All, I'm using numpy.polyfit and it is giving me some really strange results when evaluated near 0. So I compared it with polyfit in matplotlib and the code and docstrings are nearly identical. However the slight differences in the code make a huge difference on my data. Here is the Numpy version with line numbers added: 1 x = NX.asarray(x)+0. 2 y = NX.asarray(y)+0. 3 y = NX.reshape(y, (len(y), 1)) 4 X = vander(x, N+1) 5 c, resids, rank, s = _lstsq(X, y) 6 c.shape = (N+1,) 7 return c And here is the MPL version with line numbers added: 1 x = asarray(x)+0. 2 y = asarray(y)+0. 3 y = reshape(y, (len(y),1)) 4 X = Matrix(vander(x, N+1)) 5 Xt = Matrix(transpose(X)) 6 c = array(linear_algebra.inverse(Xt*X)*Xt*y) # convert back to array 7 c.shape = (N+1,) 8 return c So lines 1-4 are basically the same. The MPL version produces a much more stable representation of my data Can someone please comment on the differences? Should these two be exactly the same? If not, what are the advantages to each algorithm? I'm attaching my x and y points dumped to files. Here is the test code using "ipython -pylab" import numpy x=numpy.load('x.data') y=numpy.load('y.data') semilogy(numpy.polyval(numpy.polyfit(x,y,3),arange(3604)));hold(True) semilogy(polyval(polyfit(x,y,3),arange(3604))) #Use the MPL versions scatter(x,y) Regards, Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: x.data Type: application/octet-stream Size: 4451 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: y.data Type: application/octet-stream Size: 8771 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 12 15:19:16 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 12 Oct 2006 13:19:16 -0600 Subject: Compiling Numpy with py2exe In-Reply-To: <8B7FAE730B4A3646B51C94CD6C8784B7B28F52@serversbs2003.Septentrio.local> References: <8B7FAE730B4A3646B51C94CD6C8784B7B28F52@serversbs2003.Septentrio.local> Message-ID: <452E9534.10100@ee.byu.edu> Charles Hanot wrote: > Hello, > > I?ve a program using Numpy and I would like to compile it with Py2exe > but I?ve some problems. When I try to compile my program, some modules > generated by numpy are not loaded. I?ve already asked to the py2exe > community if they have a solution for that and the answer was that it > was possible to compile numpy modules with py2exe but I should ask to > the Numpy community J > > So my question is, is there something special to do in order to > compile a program using numpy module? > The only thing I remember is that you have to explicitly make sure _internal.py file gets copied over as it is not possible to discern that by looking at the import statements in .py files. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 16:08:53 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 13:08:53 -0700 Subject: More SVN testing In-Reply-To: References: Message-ID: <452EA0D5.8020103@ieee.org> Albert Strasheim wrote: > Hello all > > >> -----Original Message----- >> From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- >> discussion-bounces at lists.sourceforge.net] On Behalf Of Tim Hochberg >> Sent: 12 October 2006 21:24 >> To: numpy-discussion >> Subject: [Numpy-discussion] More SVN testing >> >> >> I just checked in a couple of changes to SVN. I was going to check in >> errstate, but it looks like Travis beat me to it, so I contented myself >> with adding a docstring and some tests. These tests are only run under >> 2.5; things seem to work fine here, but if someone on a Linux box whose >> running Python 2.5 could try it out I'd appreciate it. I'm suspicous of >> Linux/Windows differences right now because of my run in with them >> earlier today in the tests for test_multiarray (which I believe it now >> fixed -- at least everything is working here OK). >> > > This probably fixes the issue I reported here: > > http://projects.scipy.org/scipy/numpy/ticket/286 > > Maybe you can check that the test counts match up on Python 2.4 and 2.5 on > Windows on your system and close that ticket. > Yep, that's the same one I ran into and it's fixed now on windows. It sounds like it didn't occur at all on Linux. Their are now actually two more tests under 2.5 than under 2.4, but that's on purpose -- I can't test 'with errstate' under 2.4, so I skip those tests. I went ahead and closed the ticket. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Thu Oct 12 17:15:34 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 16:15:34 -0500 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> Message-ID: <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> On 10/12/06, Charles R Harris wrote: > > I'm guessing that the rcond number in the lstsq version (default 1e-10) is > the difference. Generally the lstsq version should work better than the MPL > version because at*a is not as well conditioned and vandermonde matrices are > notoriously ill conditioned anyway for higher degree fits. It would help if > you attached the data files in ascii form unless they happen to contain > thousands of data items. Just the x will suffice, zip it up if you have to. > Here are the files. Since the two algorithms behave differently and each has it place then can both be included in numpy? i.e. numpy.polyfit(x,y,N, mode='alg1') numpy.polyfit(x,y,N, mode='alg2') replacing alg1 and alg2 with meaningful names. Thanks Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- 2 4 0 16 1 3 14 17 12 13 8 7 5 11 23 19 24 9 18 15 44 22 20 25 27 10 41 45 26 40 21 50 29 38 28 37 53 48 57 78 54 61 39 64 65 58 43 35 55 47 60 66 46 68 52 51 42 75 93 56 87 74 59 71 84 49 72 81 73 63 94 67 36 80 30 70 106 69 62 77 76 90 98 103 6 172 79 83 82 31 85 97 105 91 121 89 95 101 116 135 92 102 96 88 120 137 136 34 389 499 489 565 479 555 412 390 443 442 456 365 404 593 457 364 375 441 367 360 453 376 482 576 550 445 409 658 363 435 381 477 549 396 460 544 601 466 384 557 391 461 511 642 547 558 542 610 608 639 438 591 551 378 599 552 397 569 578 437 643 361 385 399 538 600 395 561 602 649 454 464 447 386 535 554 577 596 686 635 459 514 628 448 583 407 408 603 379 626 410 500 570 449 582 434 401 417 501 462 566 366 394 631 485 382 563 369 784 781 824 838 783 826 825 854 867 763 801 799 879 795 759 852 872 831 802 769 873 773 762 779 772 789 836 881 839 827 796 882 851 786 797 755 726 841 896 865 890 837 857 775 853 767 785 756 750 754 887 861 894 770 749 771 840 844 782 780 877 859 774 832 871 730 722 901 794 884 846 760 848 729 876 724 720 757 830 798 850 818 764 875 787 829 899 885 874 960 815 864 777 723 820 891 747 751 725 821 776 823 866 800 833 788 758 878 1411 1350 1358 1353 1362 1378 1326 1348 1321 1355 1357 1349 1352 1346 1373 1430 1408 1351 1310 1418 1281 1336 1416 1428 1423 1436 1364 1432 1419 1377 1421 1359 1279 1401 1424 1323 1340 1322 1339 1407 1438 1427 1247 1435 1290 1311 1262 1343 1307 1437 1369 1315 1288 1356 1412 1309 1327 1276 1367 1325 1425 1420 1344 1361 1337 1368 1314 1415 1335 1278 1320 1280 1382 1282 1434 1328 1273 1332 1410 1347 1284 1338 1417 1354 1360 1329 1270 1305 1375 1414 1287 1291 1429 1379 1256 1372 1370 1289 1306 1275 1342 1274 1286 1313 1271 1248 1319 1268 1479 1440 1452 1449 1441 1542 1444 1447 1645 1569 1485 1537 1527 1522 1462 1461 1554 1651 1448 1652 1538 1529 1510 1451 1639 1570 1486 1737 1481 1623 1725 1453 1458 1459 1680 1484 1445 1671 1460 1524 1673 1666 1627 1540 1564 1626 1672 1499 1674 1561 1555 1519 1472 1632 1534 1548 1650 1470 1656 1637 1648 1717 1661 1521 1724 1755 1631 1735 1517 1667 1551 1728 1566 1575 1515 1539 1531 1541 1563 1558 1643 1454 1549 1758 1483 1761 1714 1638 1505 1654 1664 1475 1556 1473 1528 1516 1552 1443 1709 1553 1794 1571 1644 1520 1718 1742 1498 1736 2149 2052 2128 2141 2019 2108 2125 1998 2055 2087 2106 2123 2029 2036 1960 2121 2049 1945 2132 2058 2035 2144 2053 2109 1963 1949 1966 2137 2060 2153 2147 1956 2126 2115 2130 2129 1808 2021 2038 1804 2064 2104 1952 2107 1911 1958 2116 1810 2031 1940 2148 2117 2150 1814 2059 1916 2010 2026 1809 1964 2103 2032 2039 2118 2151 1973 2152 1951 2084 2011 2065 2105 2028 2112 2044 2040 2122 2033 1812 1941 1946 1987 1996 2127 2113 2048 1995 1950 2155 1959 2138 1803 1970 1961 2020 2008 1811 1968 1965 1947 1819 2037 2063 1817 2133 1967 2140 2047 2185 2161 2217 2241 2163 2164 2212 2220 2247 2162 2208 2248 2225 2255 2206 2191 2218 2165 2238 2237 2166 2231 2211 2192 2258 2233 2201 2221 2239 2251 2249 2219 2186 2197 2246 2210 2199 2184 2226 2243 2188 2240 2250 2204 2205 2193 2260 2160 2236 2187 2228 2182 2223 2215 2224 2235 2213 2198 2168 2202 2195 2183 2253 2209 2234 2288 2261 2256 2254 2200 2257 2242 2167 2280 2252 2245 2207 2263 2216 2277 2284 2244 2190 2259 2266 2267 2180 2227 2196 2229 2297 2189 2169 2296 2181 2279 2281 2179 2178 2194 2287 2222 2285 2232 2301 2264 2214 2289 2787 2714 2823 2591 2640 2717 2650 2612 2838 2630 2827 2588 2685 2684 2677 2720 2613 2872 2729 2804 2622 2841 2642 2694 2867 2693 2865 2601 2808 2870 2691 2607 2877 2620 2836 2835 2576 2596 2636 2805 2802 2603 2579 2797 2610 2577 2716 2832 2626 2676 2678 2873 2785 2632 2608 2692 2643 2726 2813 2614 2675 2791 2803 2644 2701 2801 2594 2737 2625 2826 2839 2649 2700 2799 2634 2695 2730 2592 2874 2667 2647 2683 2586 2722 2679 2669 2629 2871 2549 2733 2582 2631 2807 2606 2641 2727 2724 2652 2623 2783 2646 2821 2602 2702 2708 2589 2815 2842 3222 3230 3229 2898 3228 3214 3232 3208 3213 2931 3202 2886 2922 3205 3221 3235 3212 3204 2887 2893 3209 2907 2921 3231 2891 2914 3223 3224 2890 2911 3189 2896 3233 2909 2917 3220 3198 3207 3226 3227 2900 2889 3225 2923 2884 3203 3192 2910 2926 3210 2885 2905 2913 2895 2897 2918 2924 3200 2927 2934 3234 2912 3206 3187 3236 3196 2881 3238 3190 3219 2899 2936 3237 3199 2919 2915 2882 3215 3188 2902 2937 2906 2930 2901 2883 3197 2908 2904 3193 3201 3218 2925 2892 2916 3194 2939 3195 2888 3191 2880 2894 2935 2932 2944 2920 2928 2929 2938 3581 3576 3572 3571 3568 3575 3569 3566 3562 3574 3577 3565 3570 3559 3550 3582 3578 3585 3583 3560 3573 3563 3580 3588 3586 3584 3579 3564 3567 3561 3549 3587 3551 3590 3591 3589 3558 3515 3523 3511 3526 3552 3512 3547 3510 3508 3520 3503 3524 3509 3518 3557 3545 3548 3514 3592 3517 3519 3528 3513 3522 3488 3521 3487 3516 3544 3497 3553 3507 3525 3504 3546 3499 3490 3527 3492 3494 3501 3489 3500 3593 3491 3496 3311 3458 3306 3594 3495 3394 3543 3498 3277 3486 3502 3370 3505 3308 3460 3469 3459 3466 3483 3366 3482 3269 3323 3264 3485 -------------- next part -------------- 3956.970544 4114.773479 4313.241696 4384.792216 4391.872310 4411.476715 4482.940096 4594.108179 4645.381743 4730.829231 4763.895598 4781.032135 4880.854414 5008.228140 5013.866252 5026.548238 5040.509458 5061.949397 5098.508781 5118.791696 5162.805554 5186.805236 5193.941710 5256.845108 5286.838670 5364.175004 5413.297293 5418.790951 5446.085519 5475.639165 5508.616723 5604.737760 5614.409364 5636.101761 5638.542797 5649.773243 5660.913105 5719.715027 5815.218454 5832.621843 5837.578731 5862.529421 5863.607227 5877.191757 5952.905334 5984.647641 6033.194005 6048.375830 6058.973718 6073.108082 6100.784624 6112.341023 6138.705305 6142.491247 6170.101238 6190.399915 6224.593338 6239.605558 6248.244072 6254.401594 6280.211422 6317.197093 6317.614065 6325.296754 6339.852487 6381.476997 6383.259524 6391.571145 6397.082360 6418.080290 6438.735101 6440.724082 6454.977105 6477.089430 6560.501879 6637.842362 6710.064249 6724.529239 6728.116095 6728.478821 6774.863744 6874.242169 6902.087390 6919.527972 6979.799498 6980.973237 6989.850615 7015.371907 7019.392350 7050.062660 7050.597597 7061.397784 7105.324281 7107.397379 7113.332893 7123.788680 7132.977337 7138.892938 7172.808848 7176.242427 7186.719367 7191.745735 7239.760194 7243.749478 7255.868073 7257.267870 7268.538062 7272.629743 7679.940589 7683.236804 7780.699438 7873.928129 7899.908299 7904.060156 7925.056466 7986.214500 7992.112331 7999.563794 8001.244799 8018.175375 8030.246998 8042.876999 8066.748960 8068.723221 8070.288644 8071.628413 8130.859172 8156.118329 8175.197378 8206.961155 8218.103232 8222.490958 8239.676758 8241.640380 8249.615006 8267.669676 8268.053374 8283.319127 8288.302563 8293.092398 8316.055561 8327.772501 8334.822552 8338.317644 8339.073386 8342.683772 8350.156751 8357.692104 8400.200111 8408.910930 8417.560026 8424.256975 8424.301520 8435.165529 8442.029865 8463.791695 8467.858939 8468.002339 8473.892687 8475.457054 8481.274131 8484.884301 8503.376563 8506.217207 8513.976462 8516.443233 8518.839920 8521.927566 8522.369277 8524.426356 8529.354446 8530.550718 8537.266324 8543.730529 8554.276036 8555.003337 8573.504421 8575.663769 8578.398385 8586.508857 8597.926278 8601.801143 8607.638604 8613.355557 8618.223021 8619.832545 8625.856397 8628.547664 8635.510731 8635.616818 8638.093330 8638.725205 8639.290086 8640.963579 8641.074684 8644.931541 8650.771065 8651.748195 8652.153167 8654.191277 8662.647644 8663.908731 8672.156947 8673.406207 8673.653387 8676.156890 8684.297953 8688.001563 8691.611803 8691.714375 8692.654871 8705.209794 8714.218292 8715.503995 8720.084210 8720.465880 8950.316961 9014.234275 9016.582013 9068.120557 9079.086925 9084.084293 9092.854202 9152.516280 9206.442055 9211.797882 9259.935823 9274.088555 9319.200628 9418.736103 9426.658011 9437.435612 9448.483510 9448.919497 9451.123274 9475.891179 9488.585768 9492.490822 9513.276633 9516.127121 9521.205075 9530.898966 9564.375640 9588.755348 9596.967298 9615.132594 9642.107437 9645.510988 9646.175388 9666.070621 9677.726960 9695.380196 9701.239340 9709.602677 9711.670031 9739.504151 9761.667030 9761.780136 9769.510284 9799.923976 9807.513067 9809.803762 9810.694795 9823.853204 9838.334226 9848.403779 9876.275178 9887.129520 9887.560756 9887.888122 9895.889715 9899.907916 9909.691312 9911.111820 9949.714270 9952.911904 9960.038486 9962.479402 9971.996514 9980.326000 9984.951467 9994.006658 10001.073320 10003.476996 10014.757757 10022.105369 10025.979253 10049.807479 10054.324996 10058.877706 10058.904792 10085.774604 10088.651241 10091.809589 10097.187658 10107.697162 10109.684042 10113.805433 10114.595009 10137.018824 10138.092392 10150.691072 10151.749667 10171.572498 10177.785950 10182.552424 10185.711350 10193.402181 10198.124474 10202.876980 10203.927967 10204.398934 10213.144657 10229.042155 10235.314449 10238.453722 10241.626409 10241.820711 10263.338102 10266.747743 10319.492303 10332.302989 10339.341659 10343.745803 8793.912077 9240.132446 9371.045485 9559.747528 9566.311321 9605.890226 9663.291742 9727.898364 9817.130726 9829.579358 9894.337029 9943.329879 9986.701787 9988.681733 10007.595675 10039.175362 10058.567031 10071.732150 10080.223263 10080.572538 10082.640448 10091.338544 10096.808553 10104.572156 10196.766192 10243.213998 10247.809490 10253.163862 10292.904202 10312.906320 10315.642277 10327.044105 10342.839692 10346.473991 10355.275639 10362.684038 10372.115530 10372.748272 10382.069791 10387.721477 10389.628136 10399.630692 10430.582283 10442.936940 10449.245767 10454.870137 10472.091864 10478.317998 10489.109327 10509.306150 10513.053502 10514.097318 10525.194284 10527.980647 10529.909410 10531.238327 10537.874168 10544.844175 10555.905185 10563.925159 10565.773122 10575.339600 10586.124716 10590.426276 10611.035077 10622.729994 10636.607153 10657.784468 10666.334794 10672.555469 10686.503366 10686.595292 10693.772247 10703.728069 10703.988774 10704.045868 10708.960756 10720.027629 10738.600374 10777.951535 10783.717838 10797.095152 10803.299357 10804.900797 10805.415021 10809.890716 10825.610894 10830.556433 10854.840855 10861.376313 10885.763384 10918.933837 10961.511982 10966.953981 10967.776642 10973.773434 10977.243562 10978.798310 10978.830169 10989.002409 11009.161106 11015.743459 11020.016124 11056.625898 11092.348893 11093.400822 11100.135715 11153.180487 9814.960894 9981.143054 10098.094627 10098.936901 10251.240937 10282.032995 10284.640897 10322.598305 10360.982393 10376.872740 10392.495710 10417.277021 10425.686178 10434.329513 10450.355048 10478.041847 10499.163586 10499.215862 10526.877232 10532.554246 10536.409254 10539.253460 10608.396814 10645.575298 10659.240945 10670.922154 10689.184454 10715.836202 10722.298297 10725.640874 10748.404608 10753.162544 10756.517743 10757.745888 10772.695671 10782.428719 10790.583373 10791.898039 10801.684370 10811.977492 10816.266996 10818.283291 10820.330119 10834.649198 10857.733858 10874.140510 10876.059901 10877.637723 10882.405065 10894.207293 10900.073600 10912.167792 10924.820170 10926.306574 10929.257567 10959.628651 10960.178273 10975.859148 10976.998078 10978.812143 10991.596620 10999.187685 11000.809537 11010.126635 11024.307375 11071.514474 11085.007968 11094.736694 11109.770745 11113.527504 11116.002506 11122.254903 11133.988453 11136.612348 11156.603051 11165.834508 11166.019724 11167.998566 11175.515681 11193.546241 11199.134090 11238.198692 11244.422619 11253.787513 11264.283982 11271.505442 11274.799495 11299.434399 11316.162315 11317.946622 11318.921151 11319.649335 11346.282844 11358.697447 11362.651974 11372.919362 11374.332828 11382.460151 11394.420472 11396.858534 11397.989178 11401.709776 11418.079306 11432.017976 11438.011867 11459.901142 11468.659319 11472.542199 10681.749907 10956.703906 11125.372074 11136.314500 11172.483691 11178.012600 11269.727750 11300.670178 11340.773673 11383.722922 11457.659862 11462.856526 11489.727744 11551.874237 11579.544500 11637.402145 11686.213732 11690.080383 11741.072475 11747.628633 11787.678534 11801.682051 11814.371802 11829.473150 11838.299686 11858.353106 11870.960722 11871.392282 11871.694919 11876.150858 11883.402109 11895.260550 11913.376985 11913.856733 11942.894620 11951.949830 11969.258031 11983.875254 11993.436462 11995.024242 12021.141064 12023.333762 12048.485972 12050.270337 12055.970075 12066.039635 12073.944508 12093.030117 12098.907173 12116.424190 12116.649970 12120.302248 12124.960350 12136.994784 12137.012910 12150.298122 12150.452929 12166.326678 12170.039350 12171.669367 12173.044335 12174.841150 12179.606648 12207.744517 12209.538641 12225.212482 12229.865951 12232.316882 12246.263509 12256.108638 12276.069698 12286.868822 12297.149059 12306.631699 12312.322071 12320.164640 12324.374077 12336.310756 12339.376427 12368.598153 12380.799010 12382.583485 12384.933702 12389.988805 12391.470566 12395.745999 12401.831477 12402.594363 12413.304896 12413.370202 12444.759649 12448.495831 12459.577196 12470.502275 12470.742632 12472.315225 12474.617021 12478.433235 12478.483165 12482.236078 12498.028231 12513.399284 12525.231137 12532.312092 12534.743170 12537.889753 12542.266646 12551.826599 11193.319247 11514.884367 11602.445732 11622.344038 11622.438404 11630.019170 11641.485017 11667.278665 11708.588012 11789.904842 11836.441818 11850.623879 11881.255290 11925.047593 11951.347144 11964.909208 11994.423115 12038.711459 12090.526264 12106.978903 12160.550605 12177.736209 12215.007376 12230.055459 12250.739418 12257.282265 12259.861311 12260.730253 12262.735060 12291.224191 12316.483640 12320.419844 12358.768148 12371.123837 12374.651193 12388.097106 12445.256592 12448.681215 12473.311313 12504.297769 12505.134285 12527.592355 12575.220793 12625.944644 12651.232120 12654.872920 12667.986158 12673.676463 12698.035009 12713.500228 12766.790350 12795.378324 12811.393790 12816.705173 12821.887367 12836.305776 12859.372179 12868.193924 12869.219546 12883.997847 12913.567482 12928.892730 12951.562632 12957.166348 13033.097686 13039.613435 13052.920297 13103.059009 13121.399000 13137.420222 13143.819581 13189.273853 13217.056695 13227.901100 13251.107604 13252.729284 13256.649463 13269.470154 13291.730794 13301.492326 13306.257808 13307.889142 13360.021884 13423.017549 13429.234543 13437.179872 13484.090893 13496.163341 13595.522328 13606.157527 13609.179758 13617.822523 13619.608609 13690.242120 13717.833712 13735.254554 13755.434901 13807.103472 13824.931396 13827.734231 13834.988156 13914.748586 13978.380023 14060.743382 14083.144765 14103.587172 14155.685069 14169.683047 12042.398439 12228.734395 12520.514566 12535.035045 12580.133137 12590.412616 12629.404482 12654.158926 12675.304854 12733.821411 12802.369761 12809.260088 12868.977918 12891.461146 12902.360057 12944.236947 12970.555105 12984.851197 12993.054529 13024.315480 13028.277354 13036.186899 13065.224983 13074.307427 13077.734413 13120.257899 13138.725590 13139.563867 13144.916995 13147.708541 13150.495330 13150.670526 13153.697961 13158.210056 13170.801478 13173.506940 13174.016751 13175.306920 13181.264005 13182.217312 13189.197422 13191.435130 13212.256657 13212.308700 13216.024874 13216.050142 13217.485968 13262.914785 13271.722595 13282.591589 13291.680677 13313.503687 13317.334527 13331.044319 13332.073092 13332.970644 13351.033630 13367.919842 13372.757479 13383.369447 13400.388651 13403.526696 13406.483139 13410.260061 13413.739986 13423.056454 13427.512376 13428.578560 13431.234935 13438.083882 13440.731763 13467.215303 13471.345252 13475.473964 13476.412131 13481.174967 13491.373425 13496.015255 13499.707775 13500.224718 13512.408367 13515.414742 13519.258531 13519.955607 13529.853371 13555.701103 13567.556673 13581.752972 13592.412676 13612.170183 13618.520577 13621.250454 13627.232255 13638.740682 13644.590236 13645.126659 13649.339579 13652.869745 13660.267656 13663.052077 13666.620509 13667.808315 13667.827996 13678.639363 13685.255280 13689.203630 13691.048025 13713.641976 11598.693262 11704.336246 11919.040078 11981.528008 12057.828308 12094.424616 12149.980041 12179.583269 12183.210365 12251.745007 12262.323134 12331.253864 12412.695494 12418.496422 12425.118997 12528.458621 12550.929713 12566.605855 12624.572874 12625.259561 12647.308193 12679.031675 12683.923804 12704.924039 12745.050992 12770.103628 12787.591668 12809.165792 12819.808330 12822.677224 12835.990692 12854.239067 12913.753810 12933.290259 12939.277223 12959.460061 12965.124068 12971.594291 12994.159714 13007.621915 13010.606602 13018.524252 13043.586740 13060.549764 13069.738691 13081.265005 13083.369723 13111.744055 13132.289855 13231.443965 13243.583985 13306.335621 13307.422095 13315.641892 13373.612750 13379.343564 13437.392770 13486.774587 13498.942593 13516.270938 13517.006573 13617.129826 13651.302768 13655.976384 13659.231345 13669.637967 13692.471342 13702.477965 13709.766221 13724.485954 13748.119265 13764.708728 13780.746631 13821.058818 13822.464110 13864.480461 13867.177653 13868.782455 13869.689934 13920.956099 13928.184482 13975.859048 13985.023508 13988.347825 13992.894227 14009.444963 14037.708338 14058.926075 14150.438533 14172.273453 14186.946281 14194.290515 14246.683826 14305.245169 14308.202607 14310.050716 14351.339722 14376.973672 14392.142221 14397.889970 14434.301413 14436.668387 14454.755269 14546.002108 14564.107635 14609.352294 14610.444061 14658.388376 6191.820693 6417.995072 6450.171771 6639.135507 6702.781151 6808.454342 6858.701101 6875.949295 6896.356613 6899.675204 6980.449533 7001.842011 7076.463311 7081.714631 7082.396339 7176.817676 7216.309297 7266.648313 7280.514286 7297.524297 7355.848621 7373.904267 7378.860143 7408.329335 7458.162090 7556.879500 7567.809539 7642.724706 7735.706825 8038.110570 8048.230497 8061.020685 8148.930937 8210.087296 8262.783633 8377.205167 8474.150854 8624.533634 8822.084093 8902.373733 9086.760508 9093.644637 9159.773432 9176.508060 9188.687554 9189.349289 9191.366199 9192.863741 9292.189993 9293.081584 9297.837340 9391.546496 9403.902790 9423.233460 9528.854294 9531.233339 9555.442442 9596.527664 9621.645865 9664.165188 9748.118478 9801.103502 9845.340170 9883.906937 9910.602414 9914.261309 9914.838154 9944.961769 9947.652699 10030.420720 10032.086427 10085.772122 10106.567350 10131.776082 10171.059604 10171.271504 10250.514131 10294.686428 10310.671898 10354.773039 10359.713965 10417.971882 10419.604148 10518.670282 10543.442886 10604.550121 10604.651083 10638.592841 10715.252457 10750.162225 10763.995006 10825.711128 10857.760357 10872.843194 10894.852725 10910.851029 10927.298566 10932.509272 10938.224926 11014.452664 11019.602796 11031.241511 11052.111308 11055.718676 11153.896765 11160.080716 11179.872948 11193.467570 -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 12 16:51:08 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 14:51:08 -0600 Subject: Polyfit In-Reply-To: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> Message-ID: On 10/12/06, Greg Willden wrote: > > Hi All, > I'm using numpy.polyfit and it is giving me some really strange results > when evaluated near 0. So I compared it with polyfit in matplotlib and the > code and docstrings are nearly identical. However the slight differences in > the code make a huge difference on my data. > > Here is the Numpy version with line numbers added: > 1 x = NX.asarray(x)+0. > 2 y = NX.asarray(y)+0. > 3 y = NX.reshape(y, (len(y), 1)) > 4 X = vander(x, N+1) > 5 c, resids, rank, s = _lstsq(X, y) > 6 c.shape = (N+1,) > 7 return c > > > And here is the MPL version with line numbers added: > 1 x = asarray(x)+0. > 2 y = asarray(y)+0. > 3 y = reshape(y, (len(y),1)) > 4 X = Matrix(vander(x, N+1)) > 5 Xt = Matrix(transpose(X)) > 6 c = array(linear_algebra.inverse(Xt*X)*Xt*y) # convert back to array > 7 c.shape = (N+1,) > 8 return c > > > So lines 1-4 are basically the same. > The MPL version produces a much more stable representation of my data > > Can someone please comment on the differences? > Should these two be exactly the same? > If not, what are the advantages to each algorithm? I'm guessing that the rcond number in the lstsq version (default 1e-10) is the difference. Generally the lstsq version should work better than the MPL version because at*a is not as well conditioned and vandermonde matrices are notoriously ill conditioned anyway for higher degree fits. It would help if you attached the data files in ascii form unless they happen to contain thousands of data items. Just the x will suffice, zip it up if you have to. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From hetland at tamu.edu Thu Oct 12 16:58:59 2006 From: hetland at tamu.edu (Rob Hetland) Date: Thu, 12 Oct 2006 15:58:59 -0500 Subject: More SVN testing In-Reply-To: <452E966C.1060300@ieee.org> References: <452E966C.1060300@ieee.org> Message-ID: <5A176A3E-A5F1-49F4-B1C7-89CA662D2B90@tamu.edu> >>>numpy.test(10,10) : : : Ran 512 tests in 0.995s OK >>> numpy.__version__ '1.0.dev3320' on Intel Mac OS X 10.4.8, macpython 2.5 [Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)] -Rob On Oct 12, 2006, at 2:24 PM, Tim Hochberg wrote: > > I just checked in a couple of changes to SVN. I was going to check in > errstate, but it looks like Travis beat me to it, so I contented > myself > with adding a docstring and some tests. These tests are only run > under > 2.5; things seem to work fine here, but if someone on a Linux box > whose > running Python 2.5 could try it out I'd appreciate it. I'm > suspicous of > Linux/Windows differences right now because of my run in with them > earlier today in the tests for test_multiarray (which I believe it now > fixed -- at least everything is working here OK). > > -tim > > > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion ---- Rob Hetland, Associate Professor Dept. of Oceanography, Texas A&M University http://pong.tamu.edu/~rob phone: 979-458-0096, fax: 979-845-6331 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 12 18:03:04 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 16:03:04 -0600 Subject: Polyfit In-Reply-To: <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> Message-ID: On 10/12/06, Greg Willden wrote: > > On 10/12/06, Charles R Harris wrote: > > > > I'm guessing that the rcond number in the lstsq version (default 1e-10) > > is the difference. Generally the lstsq version should work better than the > > MPL version because at*a is not as well conditioned and vandermonde matrices > > are notoriously ill conditioned anyway for higher degree fits. It would help > > if you attached the data files in ascii form unless they happen to contain > > thousands of data items. Just the x will suffice, zip it up if you have to. > > > > > Here are the files. > > Since the two algorithms behave differently and each has it place then can > both be included in numpy? > i.e. numpy.polyfit(x,y,N, mode='alg1') > numpy.polyfit (x,y,N, mode='alg2') > > replacing alg1 and alg2 with meaningful names. > The polyfit function looks seriously busted. If I do the fits by hand I get the same results using the (not so hot) MPL version or lstsq. I don't know what the problem is. The docstring is also incorrect for the method. Hmmm... Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 12 18:28:10 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 16:28:10 -0600 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> Message-ID: On 10/12/06, Charles R Harris wrote: > > > > On 10/12/06, Greg Willden wrote: > > > > On 10/12/06, Charles R Harris wrote: > > > > > > I'm guessing that the rcond number in the lstsq version (default > > > 1e-10) is the difference. Generally the lstsq version should work better > > > than the MPL version because at*a is not as well conditioned and vandermonde > > > matrices are notoriously ill conditioned anyway for higher degree fits. It > > > would help if you attached the data files in ascii form unless they happen > > > to contain thousands of data items. Just the x will suffice, zip it up if > > > you have to. > > > > > > > > > Here are the files. > > > > Since the two algorithms behave differently and each has it place then > > can both be included in numpy? > > i.e. numpy.polyfit(x,y,N, mode='alg1') > > numpy.polyfit (x,y,N, mode='alg2') > > > > replacing alg1 and alg2 with meaningful names. > > > > The polyfit function looks seriously busted. If I do the fits by hand I > get the same results using the (not so hot) MPL version or lstsq. I don't > know what the problem is. The docstring is also incorrect for the method. > Hmmm... > Polyfit seems overly conservative in its choice of rcond. In [101]: lin.lstsq(v,y,1e-10)[0] Out[101]: array([ 5.84304475e-07, -5.51513630e-03, 1.51465472e+01, 3.05631361e-02]) In [107]: polyfit(x,y,3) Out[108]: array([ 5.84304475e-07, -5.51513630e-03, 1.51465472e+01, 3.05631361e-02]) Compare In [112]: lin.lstsq(v,y,1e-12)[0] Out[112]: array([ -5.42970700e-07, 1.61425067e-03, 1.99260667e+00, 6.51889107e+03]) In [113]: dot(lin.inv(vtv),dot(v.T,y)) Out[113]: array([ -5.42970700e-07, 1.61425067e-03, 1.99260667e+00, 6.51889107e+03]) So the default needs to be changed somewhere. Probably polyfit shoud accept rcond as a keyword. Where the actual problem lies is a bit obscure as the normal rcond default for lin.lstsq is 1e-12. Maybe some sort of import error somewhere down the line. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 12 19:04:15 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 17:04:15 -0600 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> Message-ID: On 10/12/06, Charles R Harris wrote: > > > > On 10/12/06, Charles R Harris wrote: > > > > > > > > On 10/12/06, Greg Willden < gregwillden at gmail.com> wrote: > > > > > > On 10/12/06, Charles R Harris wrote: > > > > > > > > I'm guessing that the rcond number in the lstsq version (default > > > > 1e-10) is the difference. Generally the lstsq version should work better > > > > than the MPL version because at*a is not as well conditioned and vandermonde > > > > matrices are notoriously ill conditioned anyway for higher degree fits. It > > > > would help if you attached the data files in ascii form unless they happen > > > > to contain thousands of data items. Just the x will suffice, zip it up if > > > > you have to. > > > > > > > > > > > > > Here are the files. > > > > > > Since the two algorithms behave differently and each has it place then > > > can both be included in numpy? > > > i.e. numpy.polyfit(x,y,N, mode='alg1') > > > numpy.polyfit (x,y,N, mode='alg2') > > > > > > replacing alg1 and alg2 with meaningful names. > > > > > > > The polyfit function looks seriously busted. If I do the fits by hand I > > get the same results using the (not so hot) MPL version or lstsq. I don't > > know what the problem is. The docstring is also incorrect for the method. > > Hmmm... > > > > Polyfit seems overly conservative in its choice of rcond. > > In [101]: lin.lstsq(v,y,1e-10)[0] > Out[101]: > array([ 5.84304475e-07, -5.51513630e-03, 1.51465472e+01, > 3.05631361e-02]) > In [107]: polyfit(x,y,3) > Out[108]: > array([ 5.84304475e-07, -5.51513630e-03, 1.51465472e+01, > 3.05631361e-02]) > > Compare > > In [112]: lin.lstsq(v,y,1e-12)[0] > Out[112]: > array([ -5.42970700e-07, 1.61425067e-03, 1.99260667e+00, > 6.51889107e+03]) > > In [113]: dot(lin.inv(vtv),dot(v.T,y)) > Out[113]: > array([ -5.42970700e-07, 1.61425067e-03, 1.99260667e+00, > 6.51889107e+03]) > > So the default needs to be changed somewhere. Probably polyfit shoud > accept rcond as a keyword. Where the actual problem lies is a bit obscure as > the normal rcond default for lin.lstsq is 1e-12. Maybe some sort of import > error somewhere down the line. > And here is the location of the problem in numpy/linalg/linalg.py : def lstsq(a, b, rcond=1.e-10): The 1e-10 is a bit conservative. On the other hand, I will note that the condition number of the dot(V^T ,V) matrix is somewhere around 1e22, which means in general terms that you need around 22 digits of accuracy. Inverting it only works sorta by accident in the current case. Generally, using Vandermonde matrices and polynomial fits it a bad idea when the dynamic range of the interval gets large and the degree gets up around 4-5 as it leads to ill conditioned sets of equations. When you really need the best start with chebychev polynomials or, bestest, compute a set of polynomials orthogonal over the sample points. Anyway, I think rcond should be something like 1e-12 or 1e-13 by default and be available as a keyword in the polyfit function. If no one complains I will make this change, although it is just a bandaid and things will fall apart again as soon as you call polyfit(x,y,4). Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ndbecker2 at gmail.com Thu Oct 12 20:18:06 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Oct 2006 20:18:06 -0400 Subject: numpy on centos4.4? Message-ID: Has anyone built numpy-1.0rc2/scipy-0.5.1 on centos4.4? It seems that a fortran90+ compiler is required, so I installed gcc4-gfortran-4.1.0-18.EL4 libgfortran-4.1.0-18.EL4 Didn't quite work though. Built numpy+scipy OK, but then: Warning: FAILURE importing tests for /usr/lib/python2.3/site-packages/scipy/linalg/tests/test_blas.py:22: ImportError: /usr/lib/python2.3/site-pac\ kages/scipy/linalg/fblas.so: undefined symbol: srotmg_ (in ?) Not sure yet what's causing this. I suspect it is because of a mixture of different compilers/libraries? Like, maybe blas was compiled with f77 from gcc3, while fblas was compiled with gfortran from gcc4? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From vallis.35530053 at bloglines.com Thu Oct 12 21:00:51 2006 From: vallis.35530053 at bloglines.com (vallis.35530053 at bloglines.com) Date: 13 Oct 2006 01:00:51 -0000 Subject: Providing array interface from SWIG-wrapped C struct Message-ID: <1160701251.2319852243.15024.sendItem@bloglines.com> Wonderful, it works, thanks! Michele --- Discussion of Numerical Python wrote: > > Does anybody here have experience about offering the array interface > > from a SWIG-wrapped C struct? > > I have. > > > I have tried the following, borrowing code from numpy's arrayobject.c: > > > > %extend real_vec_t { > > PyObject *__array_struct__() { > > /* From numpy/arrayobject.c/array_struct_get */ > > You are extending real_vec_t with a new METHOD, but what numpy > requests is an ATTRIBUTE. So, numpy simply queries your vec like: > > arrstr = vec.__array_struct__ > > and not with a method call like this > > arrstr = vec.__array_struct__() > > > So here is what I would do (can fail with some SWIG optimizations) > > %extend Vec { > > PyObject* __array_struct__ () { /* ... */ } > > %pythoncode { > __array_struct__ = property(__array_struct__, > doc='Array protocol') > } > > } > > Hope you got the idea. > > -- > Lisandro Dalc?n > --------------- > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 12 21:28:31 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 19:28:31 -0600 Subject: rcond in polyfit Message-ID: Hi all, I note that polyfit looks like it should work for single and double, real and complex, polynomials. On the otherhand, the default rcond needs to depend on the underlying precision. On the other, other hand, all the svd computations are done with dgelsd or zgelsd, i.e., double precision. Even so problems can arise from inherent errors of the input data if it is single precision to start with. I also think the final degree of the fit should be available somewhere if wanted, as it is an indication of what is going on. Sooo, any suggestions as to what to do? My initial impulse would be to set rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick the can down the road on returning the actual degree of the fit. Chuck PS, what is the best way of converting arbitrary arrays to the appropriate c data types float and double? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From David.L.Goldsmith at noaa.gov Thu Oct 12 21:34:09 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 18:34:09 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452E946E.8000809@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> <452DF1BA.9000906@ieee.org> <452DF49A.407@noaa.gov> <452E946E.8000809@ee.byu.edu> Message-ID: <452EED11.400@noaa.gov> I find that acceptable for my purposes, but is there some way we can minimize the "surprise(s)" for newbies? (I know some suggestions have been put forward in this thread, but I don't know enough to cast a vote one way or another for any of those, just a vote for "please do it".) And, in closing, I too would like to: thank you very much for all your work - despite my "emotional outburst," I am generally very happy using numpy and regard it as an excellent product. DG Travis Oliphant wrote: > David Goldsmith wrote: > > >> Got it. And if I understand correctly, the import order you specify in >> the little mynumpy example you included in your latest response to >> Fernando will result in any "overlap" between numpy and >> numpy.lib.scimath to call the latter's version of things rather than the >> former's, yes? >> >> >> > > Right. The last import will be used for any common-names (variables get > re-bound to the new functions...) > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Thu Oct 12 21:49:51 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 13 Oct 2006 10:49:51 +0900 Subject: How to make sqrt(-1) be 1j Message-ID: On 10/12/06, Stefan van der Walt wrote: > On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: > > On 10/11/06, Bill Baxter wrote: > I tried to explain the argument at > > http://www.scipy.org/NegativeSquareRoot > The proposed fix for those who want sqrt(-1) to return 1j is: from numpy.lib import scimath as SM SM.sqrt(-1) But that creates a new namespace alias, different from numpy. So I'll call numpy.array() to create a new array, but SM.sqrt() when I want a square root. Am I wrong to want some simple way to change the behavior of numpy.sqrt itself? Seems like you can get that effect via something like: for n in numpy.lib.scimath.__all__: numpy.__dict__[n] = numpy.lib.scimath.__dict__[n] If that sort of function were available as "numpy.use_scimath()", then folks who want numpy to be like scipy can achieve that with just one line at the top of their files. The import under a different name doesn't quite achieve the goal of making that behavior numpy's "default". I guess I'm thinking mostly of the educational uses of numpy, where you may have users that haven't learned about much about numerical computing yet. I can just imagine the instructor starting off by saying "ok everyone we're going to learn numpy today! First everyone type this: 'import numpy, from numpy.lib import scimath as SM' -- Don't worry about all the things there you don't understand." Whereas "import numpy, numpy.use_scimath()" seems easier to explain and much less intimidating as your first two lines of numpy to learn. Or is that just a bad idea for some reason? --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 12 22:15:52 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 20:15:52 -0600 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: Message-ID: On 10/12/06, Bill Baxter wrote: > > On 10/12/06, Stefan van der Walt wrote: > > On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: > > > On 10/11/06, Bill Baxter wrote: > > I tried to explain the argument at > > > > http://www.scipy.org/NegativeSquareRoot > > > > The proposed fix for those who want sqrt(-1) to return 1j is: > > from numpy.lib import scimath as SM > SM.sqrt(-1) > > > But that creates a new namespace alias, different from numpy. So I'll > call numpy.array() to create a new array, but SM.sqrt() when I want a > square root. > Am I wrong to want some simple way to change the behavior of > numpy.sqrt itself? > > Seems like you can get that effect via something like: > > for n in numpy.lib.scimath.__all__: > numpy.__dict__[n] = numpy.lib.scimath.__dict__[n] I don't like either of those ideas, although the second seems preferable. I think it better to make an efficient way of calling a sqrt routine that accepts negative floats and returns complex numbers. The behaviour could be chosen either by key word or by specially named routines, or maybe even some global flag, but I don't think it asking too much for the students to learn that sqrt(-1) doesn't exist as a real number and that efficient computation uses real whenever possible because it is a) smaller, and b) faster. That way we also avoid having software that only works for scimath but not for numpy. Chuck. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Thu Oct 12 22:30:01 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 21:30:01 -0500 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> Message-ID: <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> On 10/12/06, Charles R Harris wrote: > > And here is the location of the problem in numpy/linalg/linalg.py : > > def lstsq(a, b, rcond=1.e-10): > > The 1e-10 is a bit conservative. On the other hand, I will note that the > condition number of the dot(V^T ,V) matrix is somewhere around 1e22, which > means in general terms that you need around 22 digits of accuracy. Inverting > it only works sorta by accident in the current case. Generally, using > Vandermonde matrices and polynomial fits it a bad idea when the dynamic > range of the interval gets large and the degree gets up around 4-5 as it > leads to ill conditioned sets of equations. When you really need the best > start with chebychev polynomials or, bestest, compute a set of polynomials > orthogonal over the sample points. Anyway, I think rcond should be something > like 1e-12 or 1e-13 by default and be available as a keyword in the polyfit > function. If no one complains I will make this change, although it is just a > bandaid and things will fall apart again as soon as you call polyfit(x,y,4). > > Hey that's great. I'm glad you tracked it down. Pardon my ignorance of polyfit algorithm details. Is there a way of choosing rcond based on N that would give sensible defaults for a variety of N? Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Thu Oct 12 23:11:06 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 13 Oct 2006 12:11:06 +0900 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: Message-ID: On 10/13/06, Charles R Harris wrote: > > > On 10/12/06, Bill Baxter wrote: > > On 10/12/06, Stefan van der Walt wrote: > > > On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: > > > > On 10/11/06, Bill Baxter < wbaxter at gmail.com> wrote: > > > I tried to explain the argument at > > > > > > http://www.scipy.org/NegativeSquareRoot > > > > > > > The proposed fix for those who want sqrt(-1) to return 1j is: > > > > from numpy.lib import scimath as SM > > SM.sqrt(-1) > > > > > > But that creates a new namespace alias, different from numpy. So I'll > > call numpy.array() to create a new array, but SM.sqrt() when I want a > > square root. > > Am I wrong to want some simple way to change the behavior of > > numpy.sqrt itself? > > > > Seems like you can get that effect via something like: > > > > for n in numpy.lib.scimath.__all__: > > numpy.__dict__[n] = numpy.lib.scimath.__dict__[n] > > I don't like either of those ideas, although the second seems preferable. I > think it better to make an efficient way of calling a sqrt routine that > accepts negative floats and returns complex numbers. The behaviour could be > chosen either by key word or by specially named routines, or maybe even > some global flag, We have the "specially named routines" way already. "numpy.lib.scimath.sqrt" > but I don't think it asking too much for the students to > learn that sqrt(-1) doesn't exist as a real number and that efficient > computation uses real whenever possible because it is a) smaller, and b) > faster. That way we also avoid having software that only works for scimath > but not for numpy. I think efficiency is not a very good argument for the default behavior here, because -- lets face it -- if efficient execution was high on your priority list, you wouldn't be using python. And even if you do care about efficiency, one of the top rules of optimization is to first get it working, then get it working fast. Really, I'm just playing the devils advocate here, because I don't work with complex numbers (I see quaternions more often than complex numbers). But I would be willing to do something like numpy.use_realmath() in my code if it would make numpy more palatable to a wider audience. I wouldn't like it, however, if I had to do some import thing where I have to remember forever after that I should type 'numpy.tanh()' but 'realmath.arctanh()'. Anyway it seems like the folks who care about performance are the ones who will generally be more willing to make tweaks like that. But that's about all I have to say about this, since the status quo works fine for me. So I'll be quiet. Just it seems like the non-status-quo'ers here have some good points. I taught intro to computer science to non-majors one semester. I know I would not want to have to confront all the issues with numerical computing right off the bat if I was just trying to teach people how to do some math. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From parlar at gmail.com Thu Oct 12 23:15:50 2006 From: parlar at gmail.com (Jay Parlar) Date: Thu, 12 Oct 2006 23:15:50 -0400 Subject: Problem buliding rc2 on OS X 10.3.9/Python2.5 In-Reply-To: References: Message-ID: I hate bumping my own messages, but does no one have any insight into this? On 10/9/06, Jay Parlar wrote: > In the process of finally switching over to Python 2.5, and am trying > to build numpy. Unfortunately, it dies during the build: > > Jay-Computer:~/Desktop/numpy-1.0rc2 jayparlar$ python setup.py build > Running from numpy source directory. > F2PY Version 2_3296 > blas_opt_info: > FOUND: > extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] > define_macros = [('NO_ATLAS_INFO', 3)] > extra_compile_args = ['-faltivec', > '-I/System/Library/Frameworks/vecLib.framework/Headers'] > > lapack_opt_info: > FOUND: > extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] > define_macros = [('NO_ATLAS_INFO', 3)] > extra_compile_args = ['-faltivec'] > > running build > running config_fc > running build_src > building py_modules sources > building extension "numpy.core.multiarray" sources > Generating build/src.macosx-10.3-fat-2.5/numpy/core/config.h > customize NAGFCompiler > customize AbsoftFCompiler > customize IbmFCompiler > Could not locate executable g77 > Could not locate executable f77 > Could not locate executable gfortran > Could not locate executable f95 > customize GnuFCompiler > customize Gnu95FCompiler > customize G95FCompiler > customize GnuFCompiler > customize Gnu95FCompiler > customize NAGFCompiler > customize NAGFCompiler using config > C compiler: gcc -arch ppc -arch i386 -isysroot > /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double > -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 > > compile options: > '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > -Inumpy/core/src -Inumpy/core/include > -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > -c' > gcc: _configtest.c > gcc: cannot specify -o with -c or -S and multiple compilations > gcc: cannot specify -o with -c or -S and multiple compilations > failure. > removing: _configtest.c _configtest.o > numpy/core/setup.py:50: DeprecationWarning: raising a string exception > is deprecated > raise "ERROR: Failed to test configuration" > Traceback (most recent call last): > File "setup.py", line 89, in > setup_package() > File "setup.py", line 82, in setup_package > configuration=configuration ) > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/core.py", > line 174, in setup > return old_setup(**new_attr) > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", > line 151, in setup > dist.run_commands() > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", > line 974, in run_commands > self.run_command(cmd) > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", > line 994, in run_command > cmd_obj.run() > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", > line 112, in run > self.run_command(cmd_name) > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", > line 333, in run_command > self.distribution.run_command(command) > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", > line 994, in run_command > cmd_obj.run() > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", > line 87, in run > self.build_sources() > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", > line 106, in build_sources > self.build_extension_sources(ext) > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", > line 212, in build_extension_sources > sources = self.generate_sources(sources, ext) > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", > line 270, in generate_sources > source = func(extension, build_dir) > File "numpy/core/setup.py", line 50, in generate_config_h > raise "ERROR: Failed to test configuration" > ERROR: Failed to test configuration > > > This is with the Universal 2.5 binary, and OS X 10.3.9. > > Any ideas? Sorry if this one has been asked before, but I can't seem > to find a solution anywhere. > > Jay P. > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 13 00:43:08 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 21:43:08 -0700 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: Message-ID: <452F195C.9000605@ieee.org> Bill Baxter wrote: > On 10/12/06, Stefan van der Walt wrote: > >> On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: >> >>> On 10/11/06, Bill Baxter wrote: >>> >> I tried to explain the argument at >> >> http://www.scipy.org/NegativeSquareRoot >> >> > > The proposed fix for those who want sqrt(-1) to return 1j is: > > from numpy.lib import scimath as SM > SM.sqrt(-1) > > > But that creates a new namespace alias, different from numpy. So I'll > call numpy.array() to create a new array, but SM.sqrt() when I want a > square root. > Am I wrong to want some simple way to change the behavior of > numpy.sqrt itself? > > Seems like you can get that effect via something like: > > for n in numpy.lib.scimath.__all__: > numpy.__dict__[n] = numpy.lib.scimath.__dict__[n] > > If that sort of function were available as "numpy.use_scimath()", then > folks who want numpy to be like scipy can achieve that with just one > line at the top of their files. The import under a different name > doesn't quite achieve the goal of making that behavior numpy's > "default". > > I guess I'm thinking mostly of the educational uses of numpy, where > you may have users that haven't learned about much about numerical > computing yet. I can just imagine the instructor starting off by > saying "ok everyone we're going to learn numpy today! First everyone > type this: 'import numpy, from numpy.lib import scimath as SM' -- > Don't worry about all the things there you don't understand." > Whereas "import numpy, numpy.use_scimath()" seems easier to explain > and much less intimidating as your first two lines of numpy to learn. > > Or is that just a bad idea for some reason? > > Isn't that just going to make your students *more* confused later when then run into the standard behavior of numpy? For this sort of thing, I would just make a new module to pull together the function I want and use that instead. It's then easy to explain that this new module bbeconf (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an amalgamation of stuff from multiple sources. # bbeconf.py from numpy import * fromnumpy.scimath import sqrt # possibly some other stuff to correctly handle subpackages... -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 13 01:13:53 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 22:13:53 -0700 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: Message-ID: <452F2091.9020607@ieee.org> Bill Baxter wrote: > > I think efficiency is not a very good argument for the default > behavior here, because -- lets face it -- if efficient execution was > high on your priority list, you wouldn't be using python. I care very much about efficiency where it matters, which is only in a tiny fraction of my code. For the stuff that numpy does well it's pretty efficient, when that's not enough I can drop down to C, but I don't have to do that often. In fact, I've argued and still believe, that Python is frequently *more* efficient than C, given finite developer time, since it's easier to get the algorithms correct writing in Python. > And even if > you do care about efficiency, one of the top rules of optimization is > to first get it working, then get it working fast. > IMO, the current behavior is more likely to give you working code than auto-promoting to complex based on value. That concerns me more than efficiency. The whole auto-promotion thing looks like a good way to introduce data dependent bugs that don't surface till late in the game and are hard to track down. In contrast, when the current scheme causes a problem it should surface almost immediately. I would not use scipy.sqrt in code, even if the efficiency were the same, for this reason. I can see the attraction in the autopromoting version for teaching purposes and possibly for throwaway scripts, but not for "real" code. > Really, I'm just playing the devils advocate here, because I don't > work with complex numbers (I see quaternions more often than complex > numbers). But I would be willing to do something like > numpy.use_realmath() > in my code if it would make numpy more palatable to a wider audience. > I wouldn't like it, however, if I had to do some import thing where I > have to remember forever after that I should type 'numpy.tanh()' but > 'realmath.arctanh()'. > As I mentioned in my other message, the way to do this is to have a different entry point with different behavior. > Anyway it seems like the folks who care about performance are the ones > who will generally be more willing to make tweaks like that. > It's not just about performance though. It's also about correctness, or more accurately, resistance to bugs. > But that's about all I have to say about this, since the status quo > works fine for me. So I'll be quiet. Just it seems like the > non-status-quo'ers here have some good points. I taught intro to > computer science to non-majors one semester. I know I would not want > to have to confront all the issues with numerical computing right off > the bat if I was just trying to teach people how to do some math. > There's probably nothing wrong with having a package like this, it just shouldn't be numpy. It's easy enough to construct such a beast for yourself, it should take just a few lines of Python. Since what a beginners package should look like probably varies from teacher to teacher, let them construct a few. If they all, or at least most of them, have the same ideas about what constitutes such a package, that might be to the time to think about officially supporting a separate entry point that has the modified behaviour. For the moment, things look fine. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Fri Oct 13 01:38:48 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 13 Oct 2006 14:38:48 +0900 Subject: How to make sqrt(-1) be 1j In-Reply-To: <452F195C.9000605@ieee.org> References: <452F195C.9000605@ieee.org> Message-ID: On 10/13/06, Tim Hochberg wrote: > For this sort of thing, I > would just make a new module to pull together the function I want and > use that instead. It's then easy to explain that this new module bbeconf > (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an > amalgamation of stuff from multiple sources. > > # bbeconf.py > from numpy import * > fromnumpy.scimath import sqrt > # possibly some other stuff to correctly handle subpackages... That does sound like a good way to do it. Then you just tell your users to import 'eduNumpy' rather than numpy, and you're good to go. Added that suggestion to http://www.scipy.org/NegativeSquareRoot I'd like to ask one basic Python question related my previous suggestion of doing things like "numpy.sqrt = numpy.lib.scimath.sqrt": In python does that make it so that any module importing numpy in the same program will now see the altered sqrt function? E.g. in my program I do "import A,B". Module A alters numpy.sqrt. Does that also modify how module B sees numpy.sqrt? If so then that's a very good reason not to do it that way. I've heard people using the term "monkey-patch" before. Is that what that is? --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 13 02:47:40 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 13 Oct 2006 00:47:40 -0600 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: <452F195C.9000605@ieee.org> Message-ID: <452F368C.8010101@ieee.org> Bill Baxter wrote: > On 10/13/06, Tim Hochberg wrote: > >> For this sort of thing, I >> would just make a new module to pull together the function I want and >> use that instead. It's then easy to explain that this new module bbeconf >> (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an >> amalgamation of stuff from multiple sources. >> >> # bbeconf.py >> from numpy import * >> fromnumpy.scimath import sqrt >> # possibly some other stuff to correctly handle subpackages... >> > > That does sound like a good way to do it. > Then you just tell your users to import 'eduNumpy' rather than numpy, > and you're good to go. > Added that suggestion to http://www.scipy.org/NegativeSquareRoot > > I'd like to ask one basic Python question related my previous > suggestion of doing things like "numpy.sqrt = numpy.lib.scimath.sqrt": > In python does that make it so that any module importing numpy in the > same program will now see the altered sqrt function? E.g. in my > program I do "import A,B". Module A alters numpy.sqrt. Does that > also modify how module B sees numpy.sqrt? > > If so then that's a very good reason not to do it that way. > > I've heard people using the term "monkey-patch" before. Is that what that is? > > --bb > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From numptyboya at aol.com Fri Oct 13 06:36:57 2006 From: numptyboya at aol.com (numptyboya at aol.com) Date: Fri, 13 Oct 2006 07:36:57 -0300 Subject: Those young people taking the position you deserve because you lack a Degree? Message-ID: <676458026452597863.984775100774389437@aol.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: t5iu3stjdoovy.gif Type: image/gif Size: 9708 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ivvblszq at wibx950.com Fri Oct 13 09:38:36 2006 From: ivvblszq at wibx950.com (regarding D.) Date: Fri, 13 Oct 2006 11:38:36 -0200 Subject: need Message-ID: <000701c6eeab$5b19beb0$75e31e53@darek3ji4mr86y> Judge variety or scholars which explodes myths regarding d Mcnamara Stanford is their bring. Consent formadd Watchzach Braffs Videozach touching coming Mayanadd Diamond Dicaprio stars. Cuny am Graduate Center Scholarly in lucid or readable the original between justice Grinspoon md Harvard School am. Leads life first series Your Guffmans Guest or turns camera als spoof am. And Harry g Levine is editors Drugs or Social Justice days American Studies Sociology in. Screen a Film films wireless video Friday am Pdtpage More of Litebrite Yesterday in Pmfrom is Turn magical am shining. Hair Furtado where touches hair every or Martin leads life first a series Your Guffmans? Its Policy or have been stuffed of full lies about users What now in detox and good a. Superb of among included papers or some most powerful any ever subject its. Hopeful bookkurt Schmoke Mayor Baltimore examines detail myth reality drugs society reader. Videos Clips Games Warzone web Junk Worldtop a Videouser Whats Upin Theaters Nowwatch a trailers. The Press Craig Reinarman and Harry g Levine editors Drugs Social Justice is days American Studies Sociology in. Roots hard or abuse Americas inner cities it am shows policies made worse points a war. Inner cities it shows policies made a worse points war morass hopeful bookkurt of Schmoke is. Talk Crimes Victims Joel Best Politics am Price Denial eva Christina Bertram is Morris a Blachman in! Blacks middle class poor explains why or was much of less other countries or such or C -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: John.gif Type: image/gif Size: 10058 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From muhsinu at durodyne.com Fri Oct 13 06:44:47 2006 From: muhsinu at durodyne.com (Ciril Reynaga) Date: Fri, 13 Oct 2006 03:44:47 -0700 Subject: VkAGRA Message-ID: <000001c6eeb4$9a272b20$10d4a8c0@rtbjpx> Hi, VkAGRA for LESS http://www.optuniosen.com to come along as well. It will make an interesting gathering. button is to play, red to stop. Then he was gone. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Fri Oct 13 09:36:51 2006 From: gregwillden at gmail.com (Greg Willden) Date: Fri, 13 Oct 2006 08:36:51 -0500 Subject: SVN checkout problem Message-ID: <903323ff0610130636h642e6c0asefc781e3194d6de@mail.gmail.com> Hi, I just tried to checkout numpy and scipy to another machine and got the following errors: $ svn co http://svn.scipy.org/svn/numpy/trunk numpy svn: REPORT request failed on '/svn/numpy/!svn/vcc/default' svn: REPORT of '/svn/numpy/!svn/vcc/default': 400 Bad Request ( http://svn.scipy.org) $ svn co http://svn.scipy.org/svn/scipy/trunk scipy svn: REPORT request failed on '/svn/scipy/!svn/vcc/default' svn: REPORT of '/svn/scipy/!svn/vcc/default': 400 Bad Request ( http://svn.scipy.org) Any ideas? Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Fri Oct 13 09:44:25 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 13 Oct 2006 06:44:25 -0700 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: <452F195C.9000605@ieee.org> Message-ID: <452F9839.8060600@ieee.org> Bill Baxter wrote: > On 10/13/06, Tim Hochberg wrote: > >> For this sort of thing, I >> would just make a new module to pull together the function I want and >> use that instead. It's then easy to explain that this new module bbeconf >> (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an >> amalgamation of stuff from multiple sources. >> >> # bbeconf.py >> from numpy import * >> fromnumpy.scimath import sqrt >> # possibly some other stuff to correctly handle subpackages... >> > > That does sound like a good way to do it. > Then you just tell your users to import 'eduNumpy' rather than numpy, > and you're good to go. > Added that suggestion to http://www.scipy.org/NegativeSquareRoot > > I'd like to ask one basic Python question related my previous > suggestion of doing things like "numpy.sqrt = numpy.lib.scimath.sqrt": > In python does that make it so that any module importing numpy in the > same program will now see the altered sqrt function? E.g. in my > program I do "import A,B". Module A alters numpy.sqrt. Does that > also modify how module B sees numpy.sqrt? > Indeed it does. Module imports are cached in sys.modules, so numpy is only imported once. (With some effort, you can usually get your own private copy of a module, that you could mess with to your hearts content, but I generally wouldn't recommend it). > If so then that's a very good reason not to do it that way. > > I've heard people using the term "monkey-patch" before. Is that what that is? > I believe that is what the term refers to although I'm not absolutely certain. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Fri Oct 13 10:08:52 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 13 Oct 2006 16:08:52 +0200 Subject: Testing numpy without doing an installation? Message-ID: <1160748533.3960.5.camel@localhost.localdomain> Hi, Is it possible to test a numpy version directly from the source directory without having to install it? I mean, if I compile the sources and try to use the package directly from there, I get unexpected results. For example: $ export PYTHONPATH=/home/faltet/python.nobackup/numpy/trunk $ python2.4 -c "import numpy;print numpy.dtype([('col1', '(1,)i4')])" Running from numpy source directory. Traceback (most recent call last): File "", line 1, in ? AttributeError: 'module' object has no attribute 'dtype' It would be nice to have a way of testing a recently built version of numpy prior to install it. Thanks, -- >0,0< Francesc Altet http://www.carabos.com/ V V C?rabos Coop. V. Enjoy Data "-" "Be careful about using the following code -- I've only proven that it works, I haven't tested it." -- Donald Knuth ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From stefan at sun.ac.za Fri Oct 13 10:28:25 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 13 Oct 2006 16:28:25 +0200 Subject: dtype always copies Message-ID: <20061013142825.GA16246@mentat.za.net> Hi all, I've noticed that 'astype' always forces a copy. Is this behaviour intended? It seems to conflict with 'asarray', that tries to avoid a copy. For example, when wrapping code in ctypes, the following snippet would have been useful: def foo(x): # ensure x is an array of the right type x = N.ascontiguousarray(x).astype(N.intc) but that will cause a copy, so you'll have to do def foo(x): try: x = N.ascontiguousarray(x,N.intc) except: x = N.ascontiguousarray(x).astype(N.intc) Maybe I'm missing something obvious here -- any pointers? Thanks St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From msubotin at gmail.com Fri Oct 13 15:22:17 2006 From: msubotin at gmail.com (Michael Subotin) Date: Fri, 13 Oct 2006 15:22:17 -0400 Subject: Library problem on installation In-Reply-To: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> Message-ID: <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> Following up on my own message for archival purposes, after getting local help. If you're having a problem like this, read the file called INSTALL.txt. The current NumPy tarball doesn't have this file, but the SciPy tarball does. You may need to reinstall Atlas/Lapack libraries using a different compiler. Michael On 10/11/06, Michael Subotin wrote: > > Hi, > > I know that it's a perennial topic on the list, but I haven't been able to > find my answer in the archives. After running the installation on a RedHat > Linux machine, I'm getting the import error: "/usr/lib/libblas.so.3: > undefined symbol: e_wsfe". Judging from earlier exchanges here, it seems > that I need to add libg2c (which this machine does have in /usr/lib, unlike > g2c) somewhere between 'f77blas' and 'cblas', but I'm not sure where I > should make this change. Not being well versed in Python distributions, I > tried my luck with a few candidates and the import error remains. The > machine should be running gcc. > > Thanks for any help. > > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Fri Oct 13 15:50:58 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 13 Oct 2006 13:50:58 -0600 Subject: Polyfit In-Reply-To: <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> Message-ID: On 10/12/06, Greg Willden wrote: > > On 10/12/06, Charles R Harris wrote: > > > > And here is the location of the problem in numpy/linalg/linalg.py : > > > > def lstsq(a, b, rcond=1.e-10): > > > > The 1e-10 is a bit conservative. On the other hand, I will note that the > > condition number of the dot(V^T ,V) matrix is somewhere around 1e22, which > > means in general terms that you need around 22 digits of accuracy. Inverting > > it only works sorta by accident in the current case. Generally, using > > Vandermonde matrices and polynomial fits it a bad idea when the dynamic > > range of the interval gets large and the degree gets up around 4-5 as it > > leads to ill conditioned sets of equations. When you really need the best > > start with chebychev polynomials or, bestest, compute a set of polynomials > > orthogonal over the sample points. Anyway, I think rcond should be something > > like 1e-12 or 1e-13 by default and be available as a keyword in the polyfit > > function. If no one complains I will make this change, although it is just a > > bandaid and things will fall apart again as soon as you call polyfit(x,y,4). > > > > > > Hey that's great. I'm glad you tracked it down. > > Pardon my ignorance of polyfit algorithm details. > Is there a way of choosing rcond based on N that would give sensible > defaults for a variety of N? > Greg You can also get *much* better results if you scale the x interval to [0,1] as the problem will be better posed. For instance, with your data and a degree 10 fit I get a condition number of about 2e7 when x is scaled to [0,1], as opposed to about 1e36 when left as is. The former yields a perfectly useable fit while the latter blows up. I suppose this could be built into the polyfit routine if one were only interested in polynomial fits of some sort, but the polynomial would have to carry around an offset and scale factor to make evaluation work. If Travis is interested in such a thing we could put together some variant of the polynomials that includes the extra data. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Fri Oct 13 15:51:38 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 15:51:38 -0400 Subject: rcond in polyfit In-Reply-To: References: Message-ID: On 12/10/06, Charles R Harris wrote: > Hi all, > > I note that polyfit looks like it should work for single and double, real > and complex, polynomials. On the otherhand, the default rcond needs to > depend on the underlying precision. On the other, other hand, all the svd > computations are done with dgelsd or zgelsd, i.e., double precision. Even so > problems can arise from inherent errors of the input data if it is single > precision to start with. I also think the final degree of the fit should be > available somewhere if wanted, as it is an indication of what is going on. > Sooo, any suggestions as to what to do? My initial impulse would be to set > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick the > can down the road on returning the actual degree of the fit. I'd also be inclined to output a warning (which the user can ignore, read or trap as necessary) if the condition number is too bad or they supplied an rcond that is too small for the precision of their data. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 13 15:55:30 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 15:55:30 -0400 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> Message-ID: On 13/10/06, Charles R Harris wrote: > You can also get *much* better results if you scale the x interval to [0,1] > as the problem will be better posed. For instance, with your data and a > degree 10 fit I get a condition number of about 2e7 when x is scaled to > [0,1], as opposed to about 1e36 when left as is. The former yields a > perfectly useable fit while the latter blows up. I suppose this could be > built into the polyfit routine if one were only interested in polynomial > fits of some sort, but the polynomial would have to carry around an offset > and scale factor to make evaluation work. [-1,1] would probably be even better, no? > If Travis is interested in such a thing we could put together some variant > of the polynomials that includes the extra data. At this point you might as well use a polynomial class that can accomodate a variety of bases for the space of polynomials - X^n, (X-a)^n, orthogonal polynomials (translated and scaled as needed), what have you. I think I vote for polyfit that is no more clever than it has to be but which warns the user when the fit is bad. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Fri Oct 13 16:09:36 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 13 Oct 2006 17:09:36 -0300 Subject: NumPy/SciPy + MPI for Python Message-ID: This post is surely OT, but I cannot imagine a better place to contact people about this subject. Please, don't blame me. Any people here interested in NumPy/SciPy + MPI? From some time ago, I've been developing mpi4py (first release at SF) and I am really near to release a new version. This package exposes an API almost identical to MPI-2 C++ bindings. Almost all MPI-1 and MPI-2 features (even one-sided communications and parallel I/O) are fully supported for any object exposing single-segment buffer interface, an only some of them for communication of general Python objects (with the help of pickle/marshal). The posibility of constructing any user-defined MPI datatypes, as well as virtual topologies (specially cartesian), can be really nice for anyone interested in parallel multidimensional array procesing. Before the next release, I would like to wait for any comment, You can contact me via private mail to get a tarbal with latest developments, or we can have some discussion here, if many of you consider this a good idea. In the long term, I would like to see mpi4py integrated as a subpackage of SciPy. Regards, -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From dalcinl at gmail.com Fri Oct 13 16:20:48 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 13 Oct 2006 17:20:48 -0300 Subject: Testing numpy without doing an installation? In-Reply-To: <1160748533.3960.5.camel@localhost.localdomain> References: <1160748533.3960.5.camel@localhost.localdomain> Message-ID: On 10/13/06, Francesc Altet wrote: > Is it possible to test a numpy version directly from the source > directory without having to install it? I usually do: $ python setup.py build $ python setup.py install --home=/tmp $ export PYTHONPATH=/tmp/lib/python and then $ python -c 'import numpy; numpy.test()' and finally, if all was right, su -c 'python setup.py install' or python setup.py install --home=$HOME -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Fri Oct 13 16:36:50 2006 From: gregwillden at gmail.com (Greg Willden) Date: Fri, 13 Oct 2006 15:36:50 -0500 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> Message-ID: <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> On 10/13/06, A. M. Archibald wrote: > > At this point you might as well use a polynomial class that can > accomodate a variety of bases for the space of polynomials - X^n, > (X-a)^n, orthogonal polynomials (translated and scaled as needed), > what have you. > > I think I vote for polyfit that is no more clever than it has to be > but which warns the user when the fit is bad. > > What about including multiple algorithms each returning a figure of fit? Then I could try two or three different algorithms and then use the one that works best for my data. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Fri Oct 13 16:45:50 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 13 Oct 2006 13:45:50 -0700 Subject: Polyfit In-Reply-To: <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> Message-ID: <452FFAFE.1000902@ieee.org> Greg Willden wrote: > On 10/13/06, *A. M. Archibald* > wrote: > > At this point you might as well use a polynomial class that can > accomodate a variety of bases for the space of polynomials - X^n, > (X-a)^n, orthogonal polynomials (translated and scaled as needed), > what have you. > > I think I vote for polyfit that is no more clever than it has to be > but which warns the user when the fit is bad. > > > > What about including multiple algorithms each returning a figure of fit? > Then I could try two or three different algorithms and then use the > one that works best for my data. A simple, "stupid" curve fitting algorithm may be appropriate for numpy, but once your getting into multiple algorithms it's time to move it to a package in scipy IMO (and it would be good to find someone who cares, and knows, about curve fitting to adopt it). -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Fri Oct 13 16:58:35 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 13 Oct 2006 14:58:35 -0600 Subject: rcond in polyfit In-Reply-To: References: Message-ID: On 10/13/06, A. M. Archibald wrote: > > On 12/10/06, Charles R Harris wrote: > > Hi all, > > > > I note that polyfit looks like it should work for single and double, > real > > and complex, polynomials. On the otherhand, the default rcond needs to > > depend on the underlying precision. On the other, other hand, all the > svd > > computations are done with dgelsd or zgelsd, i.e., double precision. > Even so > > problems can arise from inherent errors of the input data if it is > single > > precision to start with. I also think the final degree of the fit should > be > > available somewhere if wanted, as it is an indication of what is going > on. > > Sooo, any suggestions as to what to do? My initial impulse would be to > set > > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick > the > > can down the road on returning the actual degree of the fit. > > I'd also be inclined to output a warning (which the user can ignore, > read or trap as necessary) if the condition number is too bad or they > supplied an rcond that is too small for the precision of their data. That sounds good, but how to do it? Should I raise an exception? I would also have to modify lstsq so it returns the degree of the fit which would mess up the current interface. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Fri Oct 13 17:03:47 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 17:03:47 -0400 Subject: Polyfit In-Reply-To: <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> Message-ID: On 13/10/06, Greg Willden wrote: > What about including multiple algorithms each returning a figure of fit? > Then I could try two or three different algorithms and then use the one that > works best for my data. The basic problem is that X^n is rarely a good basis for the functions on [a,b]. So if you want it to return the coefficients of a polynomial, you're basically stuck. If you *don't* want that, there's a whole bestiary of other options. If you're just looking to put a smooth curve through a bunch of data points (perhaps with known uncertainties), scipy.interpolate includes some nice spline fitting functions. If you're looking for polynomials, orthogonal polynomials may serve as a better basis for your interval; you can look in scipy.special for them (and leastsq will fit them to your points). Extracting their coefficients is possible but will bring you back to numerical instabilities. In any case, all this is outside the purview of numpy (as is polyfit, frankly). A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 13 17:07:55 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 17:07:55 -0400 Subject: rcond in polyfit In-Reply-To: References: Message-ID: On 13/10/06, Charles R Harris wrote: > > > > On 10/13/06, A. M. Archibald wrote: > > On 12/10/06, Charles R Harris wrote: > > > Hi all, > > > > > > I note that polyfit looks like it should work for single and double, > real > > > and complex, polynomials. On the otherhand, the default rcond needs to > > > depend on the underlying precision. On the other, other hand, all the > svd > > > computations are done with dgelsd or zgelsd, i.e., double precision. > Even so > > > problems can arise from inherent errors of the input data if it is > single > > > precision to start with. I also think the final degree of the fit should > be > > > available somewhere if wanted, as it is an indication of what is going > on. > > > Sooo, any suggestions as to what to do? My initial impulse would be to > set > > > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick > the > > > can down the road on returning the actual degree of the fit. > > > > I'd also be inclined to output a warning (which the user can ignore, > > read or trap as necessary) if the condition number is too bad or they > > supplied an rcond that is too small for the precision of their data. > > That sounds good, but how to do it? Should I raise an exception? I would > also have to modify lstsq so it returns the degree of the fit which would > mess up the current interface. Python's warnings module is a decent solution for providing this information. Goodness-of-fit worries me less than ill-conditioning - users are going to expect the curve to deviate from their function (and an easy reliable way to get goodness of fit is sqrt(sum(abs(f(xs)-polynomial(xs))**2)); this is certain to take into account any roundoff errors introduced anywhere). But they may well have no idea they should be worried about the condition number of some matrix they've never heard of. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Fri Oct 13 17:09:47 2006 From: gregwillden at gmail.com (Greg Willden) Date: Fri, 13 Oct 2006 16:09:47 -0500 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> Message-ID: <903323ff0610131409n70dbe3cew6588fa2f35179b89@mail.gmail.com> On 10/13/06, A. M. Archibald wrote: > > In any case, all this is outside the purview of numpy (as is polyfit, > frankly). > Great. Thanks for the ideas of other algorithms/functions to look at. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Fri Oct 13 17:11:57 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 13 Oct 2006 14:11:57 -0700 Subject: rcond in polyfit In-Reply-To: References: Message-ID: <4530011D.5060105@ieee.org> Charles R Harris wrote: > > > On 10/13/06, *A. M. Archibald* > wrote: > > On 12/10/06, Charles R Harris > wrote: > > Hi all, > > > > I note that polyfit looks like it should work for single and > double, real > > and complex, polynomials. On the otherhand, the default rcond > needs to > > depend on the underlying precision. On the other, other hand, > all the svd > > computations are done with dgelsd or zgelsd, i.e., double > precision. Even so > > problems can arise from inherent errors of the input data if it > is single > > precision to start with. I also think the final degree of the > fit should be > > available somewhere if wanted, as it is an indication of what is > going on. > > Sooo, any suggestions as to what to do? My initial impulse would > be to set > > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, > and kick the > > can down the road on returning the actual degree of the fit. > > I'd also be inclined to output a warning (which the user can ignore, > read or trap as necessary) if the condition number is too bad or they > supplied an rcond that is too small for the precision of their data. > > > That sounds good, but how to do it? Should I raise an exception? Use the warnings framework: >>> import warnings >>> warnings.warn("condition number is BAD") __main__:1: UserWarning: condition number is BAD The user can turn warnings on or off or turned in exceptions based on a variety of criteria. Look for the warnings filter in the docs. Which brings up a question: do we want to have a FloatingPointWarning or some such? Currently, if you use set the error handling to warn using seterr a runtime warning is issued: >>> np.seterr(all='warn') {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'} >>> np.arange(1) / 0 __main__:1: RuntimeWarning: divide by zero encountered in divide On the other hand if error handling is set to 'raise', then a FloatingPointError is issued. Is a FloatingPointWarning in order to mirror the FloatingPointError? And if so, would it be appropriate to use for condition number? > I would also have to modify lstsq so it returns the degree of the fit > which would mess up the current interface. One approach would be to write lstsqcond (or a better name) that returns both the fit and the condition number. listsq could then be just a wrapper over that which dumped the condition number. IIRC, the condition number is available, but we're not returning it. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 13 18:09:26 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 18:09:26 -0400 Subject: rcond in polyfit In-Reply-To: <4530011D.5060105@ieee.org> References: <4530011D.5060105@ieee.org> Message-ID: On 13/10/06, Tim Hochberg wrote: > Charles R Harris wrote: > > That sounds good, but how to do it? Should I raise an exception? > Use the warnings framework: > > >>> import warnings > >>> warnings.warn("condition number is BAD") > __main__:1: UserWarning: condition number is BAD > > The user can turn warnings on or off or turned in exceptions based on a > variety of criteria. Look for the warnings filter in the docs. > > Which brings up a question: do we want to have a FloatingPointWarning or > some such? Currently, if you use set the error handling to warn using > seterr a runtime warning is issued: > > >>> np.seterr(all='warn') > {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': > 'ignore'} > >>> np.arange(1) / 0 > __main__:1: RuntimeWarning: divide by zero encountered in divide > > > On the other hand if error handling is set to 'raise', then a > FloatingPointError is issued. Is a FloatingPointWarning in order to > mirror the FloatingPointError? And if so, would it be appropriate to use > for condition number? I submitted a patchto use warnings for several functions in scipy a while ago, and the approach I took was to create a ScipyWarning, from which more specific warnings were derived (IntegrationWarning, for example). That was perhaps a bit short-sighted. I'd suggest a FloatingPointWarning as a base class, with IllConditionedMatrix as a subclass (it should include the condition number, but probably not the matrix itself unless it's small, as debugging information). The warnings module is frustratingly non-reentrant, unfortunately, which makes writing tests very awkward. > > I would also have to modify lstsq so it returns the degree of the fit > > which would mess up the current interface. > One approach would be to write lstsqcond (or a better name) that returns > both the fit and the condition number. listsq could then be just a > wrapper over that which dumped the condition number. IIRC, the > condition number is available, but we're not returning it. This is a very good idea. scipy.integrate.quad returns a pair (result, error_estimate) and every time I use it I trip over that. (Perhaps if I were a fine upstanding numerical analyst I would be checking the error estimate every time, but it is a pain.) Another option would be a "full_output" optional argument. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 13 18:49:41 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 13 Oct 2006 16:49:41 -0600 Subject: NumPy/SciPy + MPI for Python In-Reply-To: References: Message-ID: <6ce0ac130610131549s626aae9chbf3bed45dfcb9230@mail.gmail.com> Just as a data point. I have used mpi4py before and have built it on many systems ranging from my macbook to NERSC supercomputers. In my opinion it is currently the best python mpi bindings available. Lisandro has done a fantastic job with this. Also Fernando and I have worked hard to make sure that mpi4py works with the new parallel capabilities of IPython. I would love to see mpi4py hosted in a public repository for others to contribute. I think this would really solidify mpi4py as a top notch mpi interface. But, my only concern is that there might be many folks who want to use mpi4py who don't need scipy. I am one of those folks - I don't necessarily need scipy on the NERSC supercomputers, but I do need mpi4py. Because of this, I would probably still recommend keeping mpi4py as a separate project. Is there any chance it could be hosted at mip4py.scipy.org? I strongly encourage others to try it out. Installation is is easy. Brian Granger On 10/13/06, Lisandro Dalcin wrote: > This post is surely OT, but I cannot imagine a better place to contact > people about this subject. Please, don't blame me. > > Any people here interested in NumPy/SciPy + MPI? From some time ago, > I've been developing mpi4py (first release at SF) and I am really near > to release a new version. > > This package exposes an API almost identical to MPI-2 C++ bindings. > Almost all MPI-1 and MPI-2 features (even one-sided communications and > parallel I/O) are fully supported for any object exposing > single-segment buffer interface, an only some of them for > communication of general Python objects (with the help of > pickle/marshal). > > The posibility of constructing any user-defined MPI datatypes, as well > as virtual topologies (specially cartesian), can be really nice for > anyone interested in parallel multidimensional array procesing. > > Before the next release, I would like to wait for any comment, You can > contact me via private mail to get a tarbal with latest developments, > or we can have some discussion here, if many of you consider this a > good idea. In the long term, I would like to see mpi4py integrated as > a subpackage of SciPy. > > Regards, > > -- > Lisandro Dalc?n > --------------- > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://projects.scipy.org/mailman/listinfo/scipy-dev > From eric at enthought.com Fri Oct 13 19:02:55 2006 From: eric at enthought.com (eric) Date: Fri, 13 Oct 2006 18:02:55 -0500 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <6ce0ac130610131549s626aae9chbf3bed45dfcb9230@mail.gmail.com> References: <6ce0ac130610131549s626aae9chbf3bed45dfcb9230@mail.gmail.com> Message-ID: <45301B1F.3030609@enthought.com> Brian Granger wrote: > Just as a data point. > > I have used mpi4py before and have built it on many systems ranging > from my macbook to NERSC supercomputers. In my opinion it is > currently the best python mpi bindings available. Lisandro has done a > fantastic job with this. Also Fernando and I have worked hard to make > sure that mpi4py works with the new parallel capabilities of IPython. > > I would love to see mpi4py hosted in a public repository for others to > contribute. I think this would really solidify mpi4py as a top notch > mpi interface. But, my only concern is that there might be many folks > who want to use mpi4py who don't need scipy. I am one of those folks > - I don't necessarily need scipy on the NERSC supercomputers, but I do > need mpi4py. Because of this, I would probably still recommend > keeping mpi4py as a separate project. Is there any chance it could be > hosted at mip4py.scipy.org? > Fine from our side... eric > I strongly encourage others to try it out. Installation is is easy. > > Brian Granger > > On 10/13/06, Lisandro Dalcin wrote: > >> This post is surely OT, but I cannot imagine a better place to contact >> people about this subject. Please, don't blame me. >> >> Any people here interested in NumPy/SciPy + MPI? From some time ago, >> I've been developing mpi4py (first release at SF) and I am really near >> to release a new version. >> >> This package exposes an API almost identical to MPI-2 C++ bindings. >> Almost all MPI-1 and MPI-2 features (even one-sided communications and >> parallel I/O) are fully supported for any object exposing >> single-segment buffer interface, an only some of them for >> communication of general Python objects (with the help of >> pickle/marshal). >> >> The posibility of constructing any user-defined MPI datatypes, as well >> as virtual topologies (specially cartesian), can be really nice for >> anyone interested in parallel multidimensional array procesing. >> >> Before the next release, I would like to wait for any comment, You can >> contact me via private mail to get a tarbal with latest developments, >> or we can have some discussion here, if many of you consider this a >> good idea. In the long term, I would like to see mpi4py integrated as >> a subpackage of SciPy. >> >> Regards, >> >> -- >> Lisandro Dalc?n >> --------------- >> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) >> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) >> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) >> PTLC - G?emes 3450, (3000) Santa Fe, Argentina >> Tel/Fax: +54-(0)342-451.1594 >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://projects.scipy.org/mailman/listinfo/scipy-dev >> >> > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From thamelry at binf.ku.dk Sat Oct 14 04:58:41 2006 From: thamelry at binf.ku.dk (thamelry at binf.ku.dk) Date: Sat, 14 Oct 2006 10:58:41 +0200 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: References: Message-ID: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> On 10/13/06, Lisandro Dalcin wrote: > This post is surely OT, but I cannot imagine a better place to contact > people about this subject. Please, don't blame me. > > Any people here interested in NumPy/SciPy + MPI? I've been working on a Dynamic Bayesian Network (DBN) toolkit for some time (called Mocapy, freely available from sourceforge https://sourceforge.net/projects/mocapy/). The thing is almost entirely implemented in Python, and currently uses Numeric and pyMPI. I routinely train DBNs from 100.000s of observations on our 240 CPU cluster. I'm in the proces of porting Mocapy to numpy. I assume pyMPI will also work with numpy, but I haven't tried it out yet. Would be great if scipy came with default MPI support, especially since pyMPI does not seem to be actively developed anymore. Cheers, -Thomas ---- Thomas Hamelryck, Post-doctoral researcher Bioinformatics center Institute of Molecular Biology and Physiology University of Copenhagen Universitetsparken 15 - Bygning 10 DK-2100 Copenhagen ? Denmark Homepage: http://www.binf.ku.dk/Protein_structure ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From RYIU at HOTMAIL.COM Sat Oct 14 05:43:37 2006 From: RYIU at HOTMAIL.COM (=?iso-2022-jp?B?UllJVQ==?=) Date: Sat, 14 Oct 2006 02:43:37 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCIVobKEJTTlMbJEIhWxsoQjE=?= =?iso-2022-jp?b?GyRCN28kTj1xOX4kXyQsJCIkaiReJDckPyEjGyhC?= Message-ID: ????????????? http://xxzzxx.com/n/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wfspotz at sandia.gov Sat Oct 14 08:58:45 2006 From: wfspotz at sandia.gov (Bill Spotz) Date: Sat, 14 Oct 2006 06:58:45 -0600 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> Message-ID: <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> On Oct 14, 2006, at 2:58 AM, thamelry at binf.ku.dk wrote: > On 10/13/06, Lisandro Dalcin wrote: >> This post is surely OT, but I cannot imagine a better place to >> contact >> people about this subject. Please, don't blame me. >> >> Any people here interested in NumPy/SciPy + MPI? > > I've been working on a Dynamic Bayesian Network (DBN) toolkit for > some time > (called Mocapy, freely available from sourceforge https:// > sourceforge.net/projects/mocapy/). The thing is almost entirely > implemented in Python, and currently uses Numeric and pyMPI. I > routinely train DBNs from 100.000s of observations on our 240 CPU > cluster. > > I'm in the proces of porting Mocapy to numpy. I assume pyMPI will > also work with numpy, but I haven't tried it out yet. Would be > great if scipy came with default MPI support, especially since > pyMPI does not seem to be actively developed anymore. I would like to second the notion of converging on a single MPI interface. My parallel project encapsulates most of the inter- processor communication within higher-level objects because the lower- level communication patterns can usually be determined from higher- level data structures. But still, there are times when a user would like access to the lower-level MPI interface. I haven't tried to make my project compatible with any of the several MPI python interfaces out there, because it isn't clear to me which one is most widely used. If one were to emerge -- and even better, if the various independent projects were to then combine their efforts in an open source environment (the way Numeric and numarray are converging to NumPy) -- then this choice would be easy. ** Bill Spotz ** ** Sandia National Laboratories Voice: (505)845-0170 ** ** P.O. Box 5800 Fax: (505)284-5451 ** ** Albuquerque, NM 87185-0370 Email: wfspotz at sandia.gov ** ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gael.varoquaux at normalesup.org Sat Oct 14 09:34:16 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sat, 14 Oct 2006 15:34:16 +0200 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> Message-ID: <20061014133416.GA12465@clipper.ens.fr> On Sat, Oct 14, 2006 at 06:58:45AM -0600, Bill Spotz wrote: > I would like to second the notion of converging on a single MPI > interface. My parallel project encapsulates most of the inter- > processor communication within higher-level objects because the lower- > level communication patterns can usually be determined from higher- > level data structures. But still, there are times when a user would > like access to the lower-level MPI interface. I think we are running in the same old problem of scipy/a super-package of scientific tools. It is important to keep numpy/scipy modular. People may want to install either one without an MPI interface. However it is nice that if somebody just wants "everything" without having to choose he can download that "everything" from scpy.org and that it comes well bundled together. So all we need to do is find a name for that super-package, put it together with good integration and add it to scipy.org. My 2 cents, Ga?l ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Sat Oct 14 12:32:47 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 14 Oct 2006 12:32:47 -0400 Subject: Problem buliding rc2 on OS X 10.3.9/Python2.5 In-Reply-To: References: Message-ID: Jay Parlar wrote: > In the process of finally switching over to Python 2.5, and am trying > to build numpy. Unfortunately, it dies during the build: > C compiler: gcc -arch ppc -arch i386 -isysroot > /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double > -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 > > compile options: > '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > -Inumpy/core/src -Inumpy/core/include > -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > -c' > gcc: _configtest.c > gcc: cannot specify -o with -c or -S and multiple compilations > gcc: cannot specify -o with -c or -S and multiple compilations > failure. This is the problem. Are you sure that you are using the correct version of gcc for making Universal binaries on 10.3.9? If so, then we are not passing the correct flags to it. Unfortunately, I think that the Universal stuff is going to make our lives quite complicated. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Sat Oct 14 12:48:15 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Sat, 14 Oct 2006 12:48:15 -0400 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <20061014133416.GA12465@clipper.ens.fr> References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> <20061014133416.GA12465@clipper.ens.fr> Message-ID: On 14/10/06, Gael Varoquaux wrote: > On Sat, Oct 14, 2006 at 06:58:45AM -0600, Bill Spotz wrote: > > I would like to second the notion of converging on a single MPI > > interface. My parallel project encapsulates most of the inter- > > processor communication within higher-level objects because the lower- > > level communication patterns can usually be determined from higher- > > level data structures. But still, there are times when a user would > > like access to the lower-level MPI interface. > > I think we are running in the same old problem of scipy/a super-package > of scientific tools. It is important to keep numpy/scipy modular. People > may want to install either one without an MPI interface. However it is > nice that if somebody just wants "everything" without having to choose > he can download that "everything" from scpy.org and that it comes well > bundled together. So all we need to do is find a name for that > super-package, put it together with good integration and add it to > scipy.org. > > My 2 cents, > > Ga?l I agree. Moreover, being picked for such integration work would help encourage people to converge on one MPI interface (for example). There's some discussion of such a package at NumpyProConPage on the wiki (and in particular at http://www.scipy.org/PyLabAwaits ). The "scipy superpack" (which is only for Windows, as I understand it) is a sort of beginning. So, well, any suggestion for a name? pylab is already in use by matplotlib (for some reason), as is Scientific Python (and numpy, Numeric and numarray are obviously already confusing). A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 14 12:50:36 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 14 Oct 2006 10:50:36 -0600 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> <20061014133416.GA12465@clipper.ens.fr> Message-ID: On 10/14/06, A. M. Archibald wrote: > > On 14/10/06, Gael Varoquaux wrote: > > On Sat, Oct 14, 2006 at 06:58:45AM -0600, Bill Spotz wrote: I agree. Moreover, being picked for such integration work would help > encourage people to converge on one MPI interface (for example). > There's some discussion of such a package at NumpyProConPage on the > wiki (and in particular at http://www.scipy.org/PyLabAwaits ). The > "scipy superpack" (which is only for Windows, as I understand it) is a > sort of beginning. > > So, well, any suggestion for a name? pylab is already in use by > matplotlib (for some reason), as is Scientific Python (and numpy, > Numeric and numarray are obviously already confusing). supernumpy? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gael.varoquaux at normalesup.org Sat Oct 14 12:54:41 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sat, 14 Oct 2006 18:54:41 +0200 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> <20061014133416.GA12465@clipper.ens.fr> Message-ID: <20061014165441.GD12465@clipper.ens.fr> On Sat, Oct 14, 2006 at 10:50:36AM -0600, Charles R Harris wrote: > So, well, any suggestion for a name? pylab is already in use by > matplotlib (for some reason), as is Scientific Python (and numpy, > Numeric and numarray are obviously already confusing). > supernumpy? I think we should rather build upon scipy, rather than numpy, if we want to build upon an existing name. Numpy is low-level array manipulation, scipy is all the fancy math that goes above, so it make more sens to derive a name on scipy. Besides scipy.org is already a good rallying point for scientific computing with python How about scipylab ! :-> -- Ga?l ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Sat Oct 14 12:59:51 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 15 Oct 2006 01:59:51 +0900 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> <20061014133416.GA12465@clipper.ens.fr> Message-ID: On 10/15/06, A. M. Archibald wrote: > So, well, any suggestion for a name? pylab is already in use by > matplotlib (for some reason), as is Scientific Python (and numpy, > Numeric and numarray are obviously already confusing). I always thought 'pylab' was an odd name, so I always do from matplotlib import pylab as plot I think the matplotlib folks would be willing to change the name of pylab if it were seen as something that would be good for the community. I kind of wonder why they didn't call it 'matplot', personally. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From szdzh-66 at 163.com Sat Oct 14 13:12:17 2006 From: szdzh-66 at 163.com (=?GB2312?B?1cW6wNDL?=) Date: Sun, 15 Oct 2006 01:12:17 +0800 Subject: =?GB2312?B?UkQ7u9i4tA==?= Message-ID: ??????(??/?????! ????????????????,?????:??????????? ??????????????????????????.(????),?? ??????????????! ????????????????????(???????? ?????????????????????????????). ? ? ?: ??? ????: 013590116835 ? ??????????????? E-mail: szhw1359011 at 163.com -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From parlar at gmail.com Sat Oct 14 13:29:57 2006 From: parlar at gmail.com (Jay Parlar) Date: Sat, 14 Oct 2006 13:29:57 -0400 Subject: Problem buliding rc2 on OS X Message-ID: > Jay Parlar wrote: > > In the process of finally switching over to Python 2.5, and am trying > > to build numpy. Unfortunately, it dies during the build: > > > C compiler: gcc -arch ppc -arch i386 -isysroot > > /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double > > -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 > > > > compile options: > > '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > > -Inumpy/core/src -Inumpy/core/include > > -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > > -c' > > gcc: _configtest.c > > gcc: cannot specify -o with -c or -S and multiple compilations > > gcc: cannot specify -o with -c or -S and multiple compilations > > failure. > > This is the problem. Are you sure that you are using the correct version of gcc > for making Universal binaries on 10.3.9? If so, then we are not passing the > correct flags to it. Unfortunately, I think that the Universal stuff is going to > make our lives quite complicated. > Well, my system is up-to-date, with only one gcc on it, so I don't know what else I can do. I originally missed the line saying "C compiler: gcc ...". It's odd, because I certainly don't have a /Developer/SDKs/MacOSX10.4u.sdk on my system. I wonder if that's there implicitly because the universal Python 2.5 I downloaded from python.org was built on a 10.4 system. Jay P. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Sat Oct 14 14:12:48 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 14 Oct 2006 14:12:48 -0400 Subject: Problem buliding rc2 on OS X In-Reply-To: References: Message-ID: Jay Parlar wrote: >> Jay Parlar wrote: >>> In the process of finally switching over to Python 2.5, and am trying >>> to build numpy. Unfortunately, it dies during the build: >>> C compiler: gcc -arch ppc -arch i386 -isysroot >>> /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double >>> -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 >>> >>> compile options: >>> '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 >>> -Inumpy/core/src -Inumpy/core/include >>> -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 >>> -c' >>> gcc: _configtest.c >>> gcc: cannot specify -o with -c or -S and multiple compilations >>> gcc: cannot specify -o with -c or -S and multiple compilations >>> failure. >> This is the problem. Are you sure that you are using the correct version of gcc >> for making Universal binaries on 10.3.9? If so, then we are not passing the >> correct flags to it. Unfortunately, I think that the Universal stuff is going to >> make our lives quite complicated. > > Well, my system is up-to-date, with only one gcc on it, so I don't > know what else I can do. I originally missed the line saying "C > compiler: gcc ...". It's odd, because I certainly don't have a > /Developer/SDKs/MacOSX10.4u.sdk on my system. I wonder if that's there > implicitly because the universal Python 2.5 I downloaded from > python.org was built on a 10.4 system. It probably was. I'm not sure what the deal is with building extensions with Universal Python on 10.3.9. It's possible that Universal binaries are only executable on 10.3.9, but not buildable. Are there any updates to the Developer Tools that you can install for 10.3.9? Particularly, is there one which provides gcc 4.0, which I think is the sine qua non for building Universal binaries. Can you build any other extension modules using distutils? If neither of the above is true, then you may need to upgrade to 10.4 to build Universal binaries. You might want to check the available Mac Python documentation and pythonmac-sig archives for more information. I have not been following the Universal discussion as closely as I could have (and as I'm currently on vacation, I'm not about to rectify that now). -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From parlar at gmail.com Sat Oct 14 15:28:52 2006 From: parlar at gmail.com (Jay Parlar) Date: Sat, 14 Oct 2006 15:28:52 -0400 Subject: Problem buliding rc2 on OS X Message-ID: > Are there any updates to the Developer Tools that you can install for 10.3.9? > Particularly, is there one which provides gcc 4.0, which I think is the sine qua > non for building Universal binaries. > > Can you build any other extension modules using distutils? > > If neither of the above is true, then you may need to upgrade to 10.4 to build > Universal binaries. You might want to check the available Mac Python > documentation and pythonmac-sig archives for more information. I have not been > following the Universal discussion as closely as I could have (and as I'm > currently on vacation, I'm not about to rectify that now). > Well, I just tried building PIL, and it worked just fine. So it looks like we can't generally say that it's impossible for me to build extension modules. I've posted a message to the distutils-sig, so hopefully we can resolve something there. If not, I'll take it to pythonmac-sig. Thanks, Jay P. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 14 16:28:47 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 14 Oct 2006 14:28:47 -0600 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: On 10/13/06, A. M. Archibald wrote: > > On 13/10/06, Tim Hochberg wrote: > > Charles R Harris wrote: > On the other hand if error handling is set to 'raise', then a > > FloatingPointError is issued. Is a FloatingPointWarning in order to > > mirror the FloatingPointError? And if so, would it be appropriate to use > > for condition number? > > I submitted a patchto use warnings for several functions in scipy a > while ago, and the approach I took was to create a ScipyWarning, from > which more specific warnings were derived (IntegrationWarning, for > example). That was perhaps a bit short-sighted. > > I'd suggest a FloatingPointWarning as a base class, with > IllConditionedMatrix as a subclass (it should include the condition > number, but probably not the matrix itself unless it's small, as > debugging information). > Let's pin this down a bit. Numpy seems to need its own warning classes so we can control the printing of the warnings. For the polyfit function I think it should *always* warn by default because it is likely to be used interactively and one can fool around with the degree of the fit. For instance, I am now scaling the the input x vector so norm_inf(x) == 1 and this seems to work pretty well for lots of stuff with rcond=-1 (about 1e-16) and a warning on rank reduction. As long as the rank stays the same things seem to work ok, up to fits of degree 21 on the test data that started this conversation. BTW, how does one turn warnings back on? If I do a >>> warnings.simplefilter('always', mywarn) things work fine. Following this by >>> warnings.simplefilter('once', mywarn) does what is supposed to do. Once again issuing >>> warnings.simplefilter('always', mywarn) fails to have any effect. Resetwarnings doesn't help. Hmmm... Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Sat Oct 14 17:52:51 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Sat, 14 Oct 2006 17:52:51 -0400 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: On 14/10/06, Charles R Harris wrote: > > > On 10/13/06, A. M. Archibald wrote: > > On 13/10/06, Tim Hochberg wrote: > > > Charles R Harris wrote: > > > > > > On the other hand if error handling is set to 'raise', then a > > > FloatingPointError is issued. Is a FloatingPointWarning in order to > > > mirror the FloatingPointError? And if so, would it be appropriate to use > > > for condition number? > > > > I submitted a patchto use warnings for several functions in scipy a > > while ago, and the approach I took was to create a ScipyWarning, from > > which more specific warnings were derived (IntegrationWarning, for > > example). That was perhaps a bit short-sighted. > > > > I'd suggest a FloatingPointWarning as a base class, with > > IllConditionedMatrix as a subclass (it should include the condition > > number, but probably not the matrix itself unless it's small, as > > debugging information). > > > > Let's pin this down a bit. Numpy seems to need its own warning classes so we > can control the printing of the warnings. For the polyfit function I think > it should *always* warn by default because it is likely to be used > interactively and one can fool around with the degree of the fit. For > instance, I am now scaling the the input x vector so norm_inf(x) == 1 and > this seems to work pretty well for lots of stuff with rcond=-1 (about 1e-16) > and a warning on rank reduction. As long as the rank stays the same things > seem to work ok, up to fits of degree 21 on the test data that started this > conversation. Numerical Recipes (http://www.nrbook.com/a/bookcpdf/c15-4.pdf ) recommend setting rcond to the number of data points times machine epsilon (which of course is different for single/double). We should definitely warn the user if any singular value is below s[0]*rcond (as that means that there is effectively a useless basis function, up to roundoff). I'm not sure how to control the default warnings setting ("once" vs. "always"); it's definitely not possible using the standard API to save the warnings state and restore it later. One might be able to push such a change into the warnings module by including it in a ContextManager. ipython should probably reset all the "show once" warnings every time it shows an interactive prompt. I suppose more accurately, it should do that only for warnings the user hasn't given instructions about. That way you'll get a warning about bad polynomial fits every time you run a command that contains one, but if your function runs thousands of fits you don't drown in warnings. > BTW, how does one turn warnings back on? If I do a > > >>> warnings.simplefilter('always', mywarn) > > things work fine. Following this by > > >>> warnings.simplefilter('once', mywarn) > > does what is supposed to do. Once again issuing > > >>> warnings.simplefilter('always', mywarn) > > fails to have any effect. Resetwarnings doesn't help. Hmmm... I don't get the impression that the warnings module is much tested; I had similar headaches. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 14 19:25:48 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 14 Oct 2006 17:25:48 -0600 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: On 10/14/06, A. M. Archibald wrote: > > On 14/10/06, Charles R Harris wrote: > > > > > > On 10/13/06, A. M. Archibald wrote: > > > On 13/10/06, Tim Hochberg wrote: > > > > Charles R Harris wrote: > > Numerical Recipes (http://www.nrbook.com/a/bookcpdf/c15-4.pdf ) > recommend setting rcond to the number of data points times machine > epsilon (which of course is different for single/double). We should > definitely warn the user if any singular value is below s[0]*rcond (as > that means that there is effectively a useless basis function, up to > roundoff). Well, that would work. On the other hand, it seems overly pessimistic from my experiments here. What seems to be a better guide is rank reduction. For instance, I can do a perfectly decent fit to Greg's data with single precision, degree 10, and ~1000 data points with rcond = ~5e-7 (effectively single precision precision). Degree 11 blows up entirely even though it loses rank. It is also true that rcond=1e-3 fails for degree 11 even though rank is strongly reduced. What looks to be taking place is roundoff error in evaluating the polynomial in single precision in the presence of higher order terms that still belong to the reduced basis functions and rank reduction is a good indicator of this. Bear in mind that I am now normalizing x by dividing it by its largest element, which futzes with the condition number. The condition number of the unscaled fit doesn't bear thinking about. Hmmm, I wonder if we have a dictionary of precisions indexed by dtype somewhere? > I don't get the impression that the warnings module is much tested; I > had similar headaches. I see some folks bitchin' at it when I google. There are remarkably few hits, though. For the moment I am going with the smaller rcond numbers and raising an error on rank reduction. I suspect something similar should be done for pinv. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Sat Oct 14 20:17:08 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Sat, 14 Oct 2006 21:17:08 -0300 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <45301B1F.3030609@enthought.com> References: <6ce0ac130610131549s626aae9chbf3bed45dfcb9230@mail.gmail.com> <45301B1F.3030609@enthought.com> Message-ID: On 10/13/06, eric wrote: > Brian Granger wrote: > > keeping mpi4py as a separate project. > > Is there any chance it could be > > hosted at mip4py.scipy.org? > > > Fine from our side... > > eric > Can anybody help setting up mip4py.scipy.org? I really do not have experience with SVN. What should I do? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Sat Oct 14 20:24:56 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Sat, 14 Oct 2006 21:24:56 -0300 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> Message-ID: On 10/14/06, Bill Spotz wrote: > I would like to second the notion of converging on a single MPI > interface. My parallel project encapsulates most of the inter- > processor communication within higher-level objects because the lower- > level communication patterns can usually be determined from higher- > level data structures. But still, there are times when a user would > like access to the lower-level MPI interface. Using mpi4py, you have access to almost all MPI internals directly from the Python side, with an API really similar to MPI-2 C++ bindigs. This is a feature I've not seen in other Python bindings for MPI. I think this is really important for developers and people learning MPI; you do not need to learn a new API. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Sat Oct 14 20:26:30 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Sat, 14 Oct 2006 17:26:30 -0700 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: <45318036.7040300@ieee.org> Charles R Harris wrote: > > > On 10/14/06, *A. M. Archibald* > wrote: [SNIP] > > > Hmmm, I wonder if we have a dictionary of precisions indexed by dtype > somewhere? Here's some code I stole from somewhere for computing EPS. It would easy enough to generate the dictionary you are looking for at startup using this. I can't recall the pedigree of this code though, so caveat emptor: def bits_of_precision(dtype): one = np.array([1], dtype) i = 0 while not np.alltrue(one + (one / 2.**i) == one): i += 1 return i - 1 EPSS = 1.0 / 2**bits_of_precision(float) * 10 # XXX safety factor It's sorta old and translated from numpy too, so it could probably be rewritten in better style. -tim > [SNIP] ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 14 20:44:27 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 14 Oct 2006 18:44:27 -0600 Subject: rcond in polyfit In-Reply-To: <45318036.7040300@ieee.org> References: <4530011D.5060105@ieee.org> <45318036.7040300@ieee.org> Message-ID: On 10/14/06, Tim Hochberg wrote: > > Charles R Harris wrote: > > > > > > On 10/14/06, *A. M. Archibald* > > wrote: > [SNIP] > > > > > > Hmmm, I wonder if we have a dictionary of precisions indexed by dtype > > somewhere? > > Here's some code I stole from somewhere for computing EPS. It would easy > enough to generate the dictionary you are looking for at startup using > this. I can't recall the pedigree of this code though, so caveat emptor: > > def bits_of_precision(dtype): > one = np.array([1], dtype) > i = 0 > while not np.alltrue(one + (one / 2.**i) == one): > i += 1 > return i - 1 Yep, that works. There is a version in my zeros module in scipy also, it's just that ISTR seeing similar code in numpy somewhere and I was hoping Travis would tell me where ;) Grep is my friend, I guess, ... ah, here it is In [140]: np.MachAr(np.single).eps Out[140]: 1.1920928955078125e-07 In [141]: np.MachAr(np.double).eps Out[141]: 2.2204460492503131e-16 Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From NSuzuki at lbl.gov Sun Oct 15 02:28:01 2006 From: NSuzuki at lbl.gov (Nao Suzuki) Date: Sat, 14 Oct 2006 23:28:01 -0700 Subject: how to use 'convolve/filter' package? Message-ID: Hello there, My name is Nao Suzuki and I'm a postdoc at Berkeley Lab. I've been using numarray about a year, and I found it very useful and I appreciate your excellent work! Today, I've been trying to take the advantage of filter packages but none of them work so far. Here is the attempt to reproduce what's shown in the manual. >>> from numarray import * >>> a=[0,0,0,1,0,0,0] >>> correlate1d(a, [1, 1, 1]) >>> Traceback (most recent call last): File "", line 1, in ? NameError: name 'correlate1d' is not defined Could someone tell me what I'm missing? Do I need to include something special? Please let me know if you can give me some hints. I'm stuck here for long time now. Thank you very much! --Nao ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From eajerbeh at wvec.com Sun Oct 15 02:33:12 2006 From: eajerbeh at wvec.com (of) Date: Sun, 15 Oct 2006 15:33:12 +0900 Subject: rights this Message-ID: <000e01c6f023$c983d6a0$e208783a@7kf1xtkd6kdipec> Uses relies on when converting a pdf files into Photoshop agreement updates get available Adobereg Find out! Of Adobecom Please see our or site db the latest version for your operating system Windows xp English. Latest of version for your or operating system Windows a xp English Choose is different version mobile devices of Also download? Industries amp Solutions Suite in Products Adobemore Hometrial Readerget Flash a Playermore Storestore Ways to buy? Storestore Ways to of buy Searchyou a may not have everything you need or view certain sections of Adobecom Please of see. An intranet am cd media place Includes logo printed infoadobe Symbian. With any other a plugin or which uses relies a on when converting pdf files of into Photoshop agreement or. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: advisory:.jpg Type: image/jpeg Size: 60240 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From rbastian at free.fr Sun Oct 15 04:50:33 2006 From: rbastian at free.fr (=?iso-8859-1?q?Ren=E9=20Bastian?=) Date: Sun, 15 Oct 2006 10:50:33 +0200 Subject: how to use 'convolve/filter' package? In-Reply-To: References: Message-ID: <06101510503300.00752@rbastian> Le Dimanche 15 Octobre 2006 08:28, Nao Suzuki a ?crit : > Hello there, > > My name is Nao Suzuki and I'm a postdoc at Berkeley Lab. > > I've been using numarray about a year, and I found it > very useful and I appreciate your excellent work! > > Today, I've been trying to take the advantage of filter > packages but none of them work so far. Here is the > attempt to reproduce what's shown in the manual. > > >>> from numarray import * > >>> a=[0,0,0,1,0,0,0] > >>> correlate1d(a, [1, 1, 1]) > > Traceback (most recent call last): > File "", line 1, in ? > NameError: name 'correlate1d' is not defined May be it depends which version of numarray, but try something like import numarray.nd_image as NDI NDI.correlate1d(a, [1, 1, 1]) it works with numarray 1.5 > > Could someone tell me what I'm missing? Do I need to include > something special? Please let me know if you can give me some > hints. I'm stuck here for long time now. > > Thank you very much! > > --Nao > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Ren? Bastian http://www.musiques-rb.org http://pythoneon.musiques-rb.org ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Sun Oct 15 08:52:07 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sun, 15 Oct 2006 21:52:07 +0900 Subject: SVN checkout problem In-Reply-To: <903323ff0610130636h642e6c0asefc781e3194d6de@mail.gmail.com> References: <903323ff0610130636h642e6c0asefc781e3194d6de@mail.gmail.com> Message-ID: <45322EF7.1020208@ar.media.kyoto-u.ac.jp> Greg Willden wrote: > Hi, > I just tried to checkout numpy and scipy to another machine and got > the following errors: > > $ svn co http://svn.scipy.org/svn/numpy/trunk numpy > svn: REPORT request failed on '/svn/numpy/!svn/vcc/default' > svn: REPORT of '/svn/numpy/!svn/vcc/default': 400 Bad Request > (http://svn.scipy.org) > > $ svn co http://svn.scipy.org/svn/scipy/trunk > scipy > svn: REPORT request failed on '/svn/scipy/!svn/vcc/default' > svn: REPORT of '/svn/scipy/!svn/vcc/default': 400 Bad Request > (http://svn.scipy.org) > > Any ideas? > Greg It may be because you are behind a proxy: http://subversion.tigris.org/faq.html#proxy My advice would be to use a ssh tunnel, if your network configuration enables it; even changing the proxy configuration as explained in the SVN FAQ gives me some strange errors sometimes, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Sun Oct 15 11:25:29 2006 From: aisaac at american.edu (Alan G Isaac) Date: Sun, 15 Oct 2006 11:25:29 -0400 Subject: how to use 'convolve/filter' package? In-Reply-To: References: Message-ID: On Sat, 14 Oct 2006, Nao Suzuki apparently wrote: > Today, I've been trying to take the advantage of filter > packages but none of them work so far. Here is the > attempt to reproduce what's shown in the manual. >>>> from numarray import * >>>> a=[0,0,0,1,0,0,0] >>>> correlate1d(a, [1, 1, 1]) As I understand it (not a numarray user), numarray is superceded by numpy, and this function is just called 'correlate' in numpy. hth, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sun Oct 15 14:35:53 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 15 Oct 2006 12:35:53 -0600 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: On 10/14/06, A. M. Archibald wrote: > > On 14/10/06, Charles R Harris wrote: I don't get the impression that the warnings module is much tested; I > had similar headaches. Turns out to be a rather simple bug (feature?) in warnings, where it shortcircuits the filters by doing a quick look in a registry , which I assume means that the warning has been previously disabled. Commenting out one line makes warnings behave as expected. I also note that clearing the filter doesn't clear the warn once registry, which is probably some sort of bug also waiting to happen. Looking at the warning code leaves me feeling somewhat disappointed: few comments and little discussion of intended behaviour. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Sun Oct 15 15:05:03 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Sun, 15 Oct 2006 16:05:03 -0300 Subject: MPI for Python RC1 Message-ID: For all of you interested in mpi4py, I've uploaded a tarball to PyPI http://www.python.org/pypi/mpi4py/0.4.0rc1 Make sure you have mpicc in your path and then - If you have setuptools, an try $ easy_intall mpi4py - Download the tarball and next $ python setup.py install [--home=$HOME] You should look at 'test/mpi-rev-v1'. I've wrote some tests from the MPI book, chapters 2 and 3 (BibTex reference in README.txt). You should try to run te unittest scripts under '/tests/unittest' Paul: I've tested mpi4py with MPICH1/2, OpenMPI and LAM. Please let me know any issue your have under AIX with your MPI implementation in case you use a vendor MPI. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at mailcan.com Mon Oct 16 02:32:20 2006 From: pgmdevlist at mailcan.com (Pierre GM) Date: Mon, 16 Oct 2006 02:32:20 -0400 Subject: A reimplementation of MaskedArray Message-ID: <200610160232.20888.pgmdevlist@mailcan.com> Folks, I just posted on the scipy/developers zone wiki (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) a reimplementation of the masked_array mopdule, motivated by some problems I ran into while subclassing MaskedArray. The main differences with the initial numpy.core.ma package are that MaskedArray is now a subclass of ndarray and that the _data section can now be any subclass of ndarray (well, it should work in most cases, some tweaking might required here and there). Apart from a couple of issues listed below, the behavior of the new MaskedArray class reproduces the old one. It is quite likely to be significantly slower, though: I was more interested in a clear organization than in performance, so I tended to use wrappers liberally. I'm sure we can improve that rather easily. The new module, along with a test suite and some utilities, are available here: http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedarray.py http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/masked_testutils.py http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_maskedarray.py Please note that it's still a work in progress (even if it seems to work quite OK when I use it). Suggestions, comments, improvements and general feedback are more than welcome ! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Mon Oct 16 06:47:33 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Mon, 16 Oct 2006 12:47:33 +0200 Subject: Usage of NotImplemented in Numexpr Message-ID: <45336345.8020007@carabos.com> Looking at the ``ophelper()`` decorator in the ``expressions`` module of Numexpr, I see the following code is used to check/replace arguments of operators:: for i, x in enumerate(args): if isConstant(x): args[i] = x = ConstantNode(x) if not isinstance(x, ExpressionNode): return NotImplemented This looks like operations on unknown kinds of arguments use the default Python behaviour. However, this yields some strange results: >>> import numpy >>> a = numpy.array([1,1,1]) >>> import numexpr >>> numexpr.evaluate('a < [0,0,0]') array(True, dtype=bool) This is odd because the comparison was not element-wise, but object-wise (between a VariableNode and an -unsupported- python list). Since Numexpr only understands scalar constants, variables and some functions (the last two are expression nodes), it seems more correct to me to simply forbid unsupported objects to avoid surprises, so the previous code may look like this:: for i, x in enumerate(args): if isConstant(x): args[i] = ConstantNode(x) elif not isinstance(x, ExpressionNode): raise TypeError( "unsupported object type: %s", type(x) ) Do you think this is OK, or am I wrong or missing something? Cheers, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From seguridad at banamex.com Mon Oct 16 10:56:35 2006 From: seguridad at banamex.com (Banamex) Date: Mon, 16 Oct 2006 14:56:35 +0000 Subject: Seguridad Banamex Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From andrea.gavana at gmail.com Mon Oct 16 11:44:21 2006 From: andrea.gavana at gmail.com (Andrea Gavana) Date: Mon, 16 Oct 2006 16:44:21 +0100 Subject: F2PY Troubles... Message-ID: Hello NG, I am using the latest Numpy release 1.0rc2 which includes F2PY. I have switched to Python 2.5 so this is the only alternative I have (IIUC). With Python 2.4, I was able to build a very simple fortran extension without problems. My extension contains 4 subroutines that scan a file and do simple operations. Now, attempting to run the second subroutine as: dprops, dwgnames, dunits = readsmspec.readsmspec(smspec, dimens) Prompt a ValueError from Python: File "D:\MyProjects\Carolina\MainPanel.py", line 894, in ReadSMSPECFile dprops, dwgnames, dunits = readsmspec.readsmspec(smspec, dimens) ValueError: data type must provide an itemsize ?!? I have never seen anything like that and googling around didn't give me any answer. The function accepts two inputs: - smspec: a filename, maximum 1000 characters long - dimens: an integer and returns 3 array of chars, each of them with size (8, dimens). Does anyone know what I may be doing wrong? Thank you very much for every pointer. Andrea. "Imagination Is The Only Weapon In The War Against Reality." http://xoomer.virgilio.it/infinity77/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Mon Oct 16 12:26:02 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 16 Oct 2006 09:26:02 -0700 Subject: Usage of NotImplemented in Numexpr In-Reply-To: <45336345.8020007@carabos.com> References: <45336345.8020007@carabos.com> Message-ID: <4533B29A.6010403@ieee.org> Ivan Vilata i Balaguer wrote: > Looking at the ``ophelper()`` decorator in the ``expressions`` module of > Numexpr, I see the following code is used to check/replace arguments of > operators:: > > for i, x in enumerate(args): > if isConstant(x): > args[i] = x = ConstantNode(x) > if not isinstance(x, ExpressionNode): > return NotImplemented > > This looks like operations on unknown kinds of arguments use the default > Python behaviour. However, this yields some strange results: > > >>>> import numpy >>>> a = numpy.array([1,1,1]) >>>> import numexpr >>>> numexpr.evaluate('a < [0,0,0]') >>>> > array(True, dtype=bool) > > This is odd because the comparison was not element-wise, but object-wise > (between a VariableNode and an -unsupported- python list). Since > Numexpr only understands scalar constants, variables and some functions > (the last two are expression nodes), it seems more correct to me to > simply forbid unsupported objects to avoid surprises, so the previous > code may look like this:: > > for i, x in enumerate(args): > if isConstant(x): > args[i] = ConstantNode(x) > elif not isinstance(x, ExpressionNode): > raise TypeError( "unsupported object type: %s", > type(x) ) > > Do you think this is OK, or am I wrong or missing something? That looks right. I'm not entirely happy with this fix; I believe that returning NotImplemented was intentional with the idea that we might someday want to use the NotImplemented machinery. That said, I can't think of a better fix and I don't see us using the NotImplemented machinery anytime soon, so I imagine it should go in. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Mon Oct 16 12:36:49 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Mon, 16 Oct 2006 18:36:49 +0200 Subject: Usage of NotImplemented in Numexpr In-Reply-To: <4533B29A.6010403@ieee.org> References: <45336345.8020007@carabos.com> <4533B29A.6010403@ieee.org> Message-ID: <4533B521.4040000@carabos.com> En/na Tim Hochberg ha escrit:: > Ivan Vilata i Balaguer wrote: >> >> for i, x in enumerate(args): >> if isConstant(x): >> args[i] = ConstantNode(x) >> elif not isinstance(x, ExpressionNode): >> raise TypeError( "unsupported object type: %s", >> type(x) ) >> >> Do you think this is OK, or am I wrong or missing something? > > That looks right. I'm not entirely happy with this fix; I believe that > returning NotImplemented was intentional with the idea that we might > someday want to use the NotImplemented machinery. That said, I can't > think of a better fix and I don't see us using the NotImplemented > machinery anytime soon, so I imagine it should go in. Maybe placing a comment there should be enough for future reference. By the way, I noticed that I splipped a bad string interpolation there, it should be ``"unsupported object type: %s" % type(x)``. Cheers, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Mon Oct 16 13:45:24 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 16 Oct 2006 10:45:24 -0700 Subject: Usage of NotImplemented in Numexpr In-Reply-To: <4533B521.4040000@carabos.com> References: <45336345.8020007@carabos.com> <4533B29A.6010403@ieee.org> <4533B521.4040000@carabos.com> Message-ID: <4533C534.1070903@ieee.org> Ivan Vilata i Balaguer wrote: > En/na Tim Hochberg ha escrit:: > > >> Ivan Vilata i Balaguer wrote: >> >>> for i, x in enumerate(args): >>> if isConstant(x): >>> args[i] = ConstantNode(x) >>> elif not isinstance(x, ExpressionNode): >>> raise TypeError( "unsupported object type: %s", >>> type(x) ) >>> >>> Do you think this is OK, or am I wrong or missing something? >>> >> That looks right. I'm not entirely happy with this fix; I believe that >> returning NotImplemented was intentional with the idea that we might >> someday want to use the NotImplemented machinery. That said, I can't >> think of a better fix and I don't see us using the NotImplemented >> machinery anytime soon, so I imagine it should go in. >> > > Maybe placing a comment there should be enough for future reference. By > the way, I noticed that I splipped a bad string interpolation there, it > should be ``"unsupported object type: %s" % type(x)``. > I went ahead and committed this more or less as is for the time being. I have an idea about how to treat stuff like list and tuple literals. The basic idea is to attempt to convert them to arrays using asarray, then to turn them into pseudo variables. It's all still a little vague, but it should be feasible to have evaluate('a < [1,2,3]') work as it does in numpy. It sounds like work though, so I'm putting it off for now... -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Mon Oct 16 14:25:13 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 16 Oct 2006 12:25:13 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. Message-ID: It seems to me that since the behaviour when copy=0 is to make a copy only if necessary, it should find it necessary and make the downcast. After all, array(a, dtype=single, copy=1) does just that without complaint. Some common code in linalg could be replaced if array and asarray would do that operation. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Mon Oct 16 14:33:43 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 16 Oct 2006 12:33:43 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. In-Reply-To: References: Message-ID: <4533D087.3000803@ieee.org> Charles R Harris wrote: > It seems to me that since the behaviour when copy=0 is to make a copy > only if necessary, it should find it necessary and make the downcast. > After all, array(a, dtype=single, copy=1) does just that without > complaint. Some common code in linalg could be replaced if array and > asarray would do that operation. > Well, the fact that it makes the copy without raising an error is different behavior from Numeric and should be considered an unintended change. We definitely should make this consistent for copy=0 or copy=1. The only question, is do we raise the error in both cases or allow the conversion in both cases. The long-standing behavior is to raise the error on possible-loss conversion and so my opinion is that we should continue with that behavior. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From szhw1359011 at 126.com Mon Oct 16 14:40:33 2006 From: szhw1359011 at 126.com (=?GB2312?B?1cW6wNDL?=) Date: Tue, 17 Oct 2006 02:40:33 +0800 Subject: =?GB2312?B?UkQ7u9i4tA==?= Message-ID: ??????(??/?????! ????????????????,?????:??????????? ??????????????????????????.(????),?? ??????????????! ????????????????????(???????? ?????????????????????????????). ? ? ?: ??? ????: 013590116835 ? ??????????????? E-mail: szhw1359011 at 163.com -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Mon Oct 16 16:20:04 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 16 Oct 2006 14:20:04 -0600 Subject: What does Fortran order mean? Message-ID: Travis, I note that >>> a = arange(6).reshape(2,3,order='F') >>> a array([[0, 1, 2], [3, 4, 5]]) Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a copy, but flat, flatten, and tostring all show the elements in 'C' order. I ask because I wonder if changing the order can be used to prepare arrays for input into the LaPack routines. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Mon Oct 16 16:52:23 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 16 Oct 2006 17:52:23 -0300 Subject: What does Fortran order mean? In-Reply-To: References: Message-ID: On 10/16/06, Charles R Harris wrote: > Travis, > I note that > >>> a = arange(6).reshape(2,3,order='F') > >>> a > array([[0, 1, 2], > [3, 4, 5]]) > > Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a copy, Are you sure? octave:1> a = 0:5 a = 0 1 2 3 4 5 octave:2> reshape(a, 2, 3) ans = 0 2 4 1 3 5 -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ggthydty at usa10.com Mon Oct 16 17:21:22 2006 From: ggthydty at usa10.com (Ideas) Date: Mon, 16 Oct 2006 22:21:22 +0100 Subject: process achieving results. ZOLOFT Message-ID: <000801c6f169$07b8f5e0$97ac9b52@oem09e93460a67> Informed way easily at anytime Surveys Business a opp Toys Make Fast Education Music Contests or Musiccds Gift of? More depression panic disorder ocd stress Ptsd Just Consumers safety. Rheumatoid am Effect Vacation Heres sneak or preview Walt Disney of World before even leave home Sylvan Learning Centers has is proven am. Survey entry win a the new video a Bible Enter chance win set Panasonic vs or Toshiba Portable Choose Player and watch Dvds of anywhere or you would like years Rentals of from! Savers Software Stickers Tape Other Best Site Partners Freaky Freddies fun House or Prison Break Season? Stay informed way am easily at am anytime Surveys Business opp Toys Make is Fast Education Music a Contests Musiccds Gift Ideas Webmaster Affiliate Programs Address Zippostal Code List am Sign in up today freebies? Years Rentals from or lifetime movies Hollywood can am yours Arthritis am in or this youll learn about ways Enbrel is changing am lives many people ra of Parent Resource Download in. Musiccds Gift or Ideas Webmaster Affiliate Programs Address or Zippostal Code List Sign up today freebies address below then click. Movie could be yours ipod with in Complete or brief survey entry win or the new video Bible. Movies is Hollywood can yours or Arthritis in this youll learn about ways Enbrel is changing lives or many people. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: text.gif Type: image/gif Size: 24070 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Mon Oct 16 18:51:50 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 16 Oct 2006 16:51:50 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. In-Reply-To: <4533D087.3000803@ieee.org> References: <4533D087.3000803@ieee.org> Message-ID: On 10/16/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > It seems to me that since the behaviour when copy=0 is to make a copy > > only if necessary, it should find it necessary and make the downcast. > > After all, array(a, dtype=single, copy=1) does just that without > > complaint. Some common code in linalg could be replaced if array and > > asarray would do that operation. > > > Well, the fact that it makes the copy without raising an error is > different behavior from Numeric and should be considered an unintended > change. > > We definitely should make this consistent for copy=0 or copy=1. The > only question, is do we raise the error in both cases or allow the > conversion in both cases. > > The long-standing behavior is to raise the error on possible-loss > conversion and so my opinion is that we should continue with that > behavior. Tradition is tradition. My own preference would be to perform the downcast on the grounds that if I care enough to specify a data type, then that is what I want. But I'm late to this particular party ;) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From michael.sorich at gmail.com Mon Oct 16 22:08:50 2006 From: michael.sorich at gmail.com (Michael Sorich) Date: Tue, 17 Oct 2006 11:38:50 +0930 Subject: A reimplementation of MaskedArray In-Reply-To: <200610160232.20888.pgmdevlist@mailcan.com> References: <200610160232.20888.pgmdevlist@mailcan.com> Message-ID: <16761e100610161908r3bbf383fq4c5caf64f8b805b8@mail.gmail.com> Does this new MA class allow masking of rearray like arrays? The numpy (1.0b5) version does not seem to. e.g. from numpy import * desc = [('name','S30'),('age',int8),('weight',float32)] a = array([('Bill',31,260.0),('Fred', 15, 145.0)], dtype=desc) print a[0] print a['name'] a2 = ma.array([('Bill',31,260.0),('Fred', 15, 145.0)], dtype=desc, mask=ma.nomask) print a2[0] print a2['name'] a3 = ma.array([('Bill',31,260.0),('Fred', 15, 145.0)], dtype=desc, mask=[[True,False,False],[False,False, False]]) print a3[0] print a3['name'] -- output ('Bill', 31, 260.0) [Bill Fred] ('Bill', 31, 260.0) [Bill Fred] Traceback (most recent call last): File "D:\eclipse\Table\scripts\testrecarray.py", line 12, in ? a3 = ma.array([('Bill',31,260.0),('Fred', 15, 145.0)], dtype=desc, mask=[[True,False,False],[False,False, False]]) File "C:\Python23\Lib\site-packages\numpy\core\ma.py", line 592, in __init__ raise MAError, "Mask and data not compatible." numpy.core.ma.MAError: Mask and data not compatible. On 10/16/06, Pierre GM wrote: > Folks, > I just posted on the scipy/developers zone wiki > (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) a reimplementation > of the masked_array mopdule, motivated by some problems I ran into while > subclassing MaskedArray. > > The main differences with the initial numpy.core.ma package are that > MaskedArray is now a subclass of ndarray and that the _data section can now > be any subclass of ndarray (well, it should work in most cases, some tweaking > might required here and there). Apart from a couple of issues listed below, > the behavior of the new MaskedArray class reproduces the old one. It is quite > likely to be significantly slower, though: I was more interested in a clear > organization than in performance, so I tended to use wrappers liberally. I'm > sure we can improve that rather easily. > > The new module, along with a test suite and some utilities, are available > here: > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedarray.py > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/masked_testutils.py > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_maskedarray.py > > Please note that it's still a work in progress (even if it seems to work quite > OK when I use it). Suggestions, comments, improvements and general feedback > are more than welcome ! > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at mailcan.com Mon Oct 16 22:42:58 2006 From: pgmdevlist at mailcan.com (Pierre GM) Date: Mon, 16 Oct 2006 22:42:58 -0400 Subject: A reimplementation of MaskedArray In-Reply-To: <16761e100610161908r3bbf383fq4c5caf64f8b805b8@mail.gmail.com> References: <200610160232.20888.pgmdevlist@mailcan.com> <16761e100610161908r3bbf383fq4c5caf64f8b805b8@mail.gmail.com> Message-ID: <200610162242.58541.pgmdevlist@mailcan.com> On Monday 16 October 2006 22:08, Michael Sorich wrote: > Does this new MA class allow masking of rearray like arrays? Excellent question! Which is to say, I have no idea... I don't use recordarray, so I didn't think about testing them. So, a first test indicates that it doesn't work either. The mask for a3 is understood as having a size 6, when the data is seen as size 2 only (exactly the same problem as the original ma module). I gonna poke around and check whether I can come with a workaround. Just to make it clear: you want to be able to mask some fields in some records, right ? Not mask all the fields of a record ? Thanks for the feedback, I'll keep you posted. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Mon Oct 16 22:53:42 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 16 Oct 2006 20:53:42 -0600 Subject: Subversion Message-ID: Subversion seems to be down. I get the following message: Authentication realm: mpi4py.scipy.org mpi4py? And I am asked for my password, which doesn't work. Scipy.org also seems to be down. You folks having trouble there? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Mon Oct 16 23:14:55 2006 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 16 Oct 2006 22:14:55 -0500 Subject: Subversion In-Reply-To: References: Message-ID: Charles R Harris wrote: > Subversion seems to be down. I get the following message: > > Authentication realm: mpi4py.scipy.org > > > mpi4py? And I am asked for my password, which doesn't work. Scipy.org > also seems to be down. You folks having trouble there? Apparently. I've contacted Jeff Strunk, the sysadmin. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Tue Oct 17 04:03:03 2006 From: faltet at carabos.com (Francesc Altet) Date: Tue, 17 Oct 2006 10:03:03 +0200 Subject: Testing numpy without doing an installation? In-Reply-To: References: <1160748533.3960.5.camel@localhost.localdomain> Message-ID: <200610171003.06141.faltet@carabos.com> A Divendres 13 Octubre 2006 22:20, Lisandro Dalcin va escriure: > On 10/13/06, Francesc Altet wrote: > > Is it possible to test a numpy version directly from the source > > directory without having to install it? > > I usually do: > > $ python setup.py build > $ python setup.py install --home=/tmp > $ export PYTHONPATH=/tmp/lib/python Thanks for your answer Lisandro, but what I want is to completely avoid an installation. The idea is to be able to test the local copy version of numpy in the development directory while doing small changes on it. Having to do the install step slows down the testing phase when doing small changes. Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 04:22:02 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Tue, 17 Oct 2006 10:22:02 +0200 Subject: Testing numpy without doing an installation? In-Reply-To: <200610171003.06141.faltet@carabos.com> References: <1160748533.3960.5.camel@localhost.localdomain> <200610171003.06141.faltet@carabos.com> Message-ID: <20061017082202.GD17998@mentat.za.net> On Tue, Oct 17, 2006 at 10:03:03AM +0200, Francesc Altet wrote: > A Divendres 13 Octubre 2006 22:20, Lisandro Dalcin va escriure: > > On 10/13/06, Francesc Altet wrote: > > > Is it possible to test a numpy version directly from the source > > > directory without having to install it? > > > > I usually do: > > > > $ python setup.py build > > $ python setup.py install --home=/tmp > > $ export PYTHONPATH=/tmp/lib/python > > Thanks for your answer Lisandro, but what I want is to completely avoid an > installation. The idea is to be able to test the local copy version of numpy > in the development directory while doing small changes on it. Having to do > the install step slows down the testing phase when doing small > changes. It would be great if we could get this to work. One problem is that distutils won't build the C-modules in place. Does anyone know of a workaround? At the moment, if you run numpy from the source directory, you see the message In [1]: import numpy Running from numpy source directory. after which you can't access any of the numpy functions. This would be due to this snippet in __init__.py: try: from __config__ import show as show_config except ImportError: show_config = None if show_config is None: import sys as _sys print >> _sys.stderr, 'Running from numpy source directory.' del _sys If we declare set_package_path and restore_path in __init__.py, we can wrap all imports in numpy with set_package_path('../../..') # set path back required depth from numpy import whatever, you, need restore_path() That would take care of things on the python side, at least. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 04:46:57 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Tue, 17 Oct 2006 10:46:57 +0200 Subject: What does Fortran order mean? In-Reply-To: References: Message-ID: <20061017084657.GE17998@mentat.za.net> On Mon, Oct 16, 2006 at 02:20:04PM -0600, Charles R Harris wrote: > Travis, > > I note that > > >>> a = arange(6).reshape(2,3,order='F') > >>> a > array([[0, 1, 2], > [3, 4, 5]]) > > Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a copy, > but flat, flatten, and tostring all show the elements in 'C' order. I ask > because I wonder if changing the order can be used to prepare arrays for input > into the LaPack routines. I think that the calculation of 'a' above is correct, but that the memory representation is incorrect. For example (see attached script -- which is overkill, I know), these commands: x = N.arange(6,dtype=float).reshape((2,3),order='F') catmem(x) x = N.array([[0,1,2],[3,4,5]],order='F',dtype=float) catmem(x) x = N.array(N.arange(6,dtype=float).reshape((2,3)),order='F') catmem(x) x = N.asfortranarray(N.arange(6,dtype=float).reshape((2,3))) catmem(x) yield the following results: 0.000000 1.000000 2.000000 3.000000 4.000000 5.000000 0.000000 3.000000 1.000000 4.000000 2.000000 5.000000 0.000000 3.000000 1.000000 4.000000 2.000000 5.000000 0.000000 3.000000 1.000000 4.000000 2.000000 5.000000 Regards St?fan -------------- next part -------------- A non-text attachment was scrubbed... Name: catmem.c Type: text/x-csrc Size: 134 bytes Desc: not available URL: -------------- next part -------------- from glob import glob env = Environment() env.Replace(CCFLAGS=['-O2','-Wall','-ansi','-pedantic']) env.SharedLibrary('catmem', ['catmem.c']) -------------- next part -------------- A non-text attachment was scrubbed... Name: catmem.py Type: text/x-python Size: 628 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pearu at cens.ioc.ee Tue Oct 17 07:04:11 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Tue, 17 Oct 2006 14:04:11 +0300 (EEST) Subject: Testing numpy without doing an installation? In-Reply-To: <20061017082202.GD17998@mentat.za.net> References: <20061017082202.GD17998@mentat.za.net> Message-ID: On Tue, 17 Oct 2006, Stefan van der Walt wrote: > On Tue, Oct 17, 2006 at 10:03:03AM +0200, Francesc Altet wrote: > > A Divendres 13 Octubre 2006 22:20, Lisandro Dalcin va escriure: > > > On 10/13/06, Francesc Altet wrote: > > > > Is it possible to test a numpy version directly from the source > > > > directory without having to install it? > > > > > > I usually do: > > > > > > $ python setup.py build > > > $ python setup.py install --home=/tmp > > > $ export PYTHONPATH=/tmp/lib/python > > > > Thanks for your answer Lisandro, but what I want is to completely avoid an > > installation. The idea is to be able to test the local copy version of numpy > > in the development directory while doing small changes on it. Having to do > > the install step slows down the testing phase when doing small > > changes. > > It would be great if we could get this to work. One problem is that > distutils won't build the C-modules in place. Does anyone know of a > workaround? Actually numpy.distutils supports in place builds of C modules. However, its rather difficult to make numpy inplace to work for the following reasons: - to build a numpy based C extension, you'll need numpy header files - numpy header files are generated during numpy build So it is a chicken-egg problem. One workaround for testing a numpy subpackage would be installing numpy and then doing inplace build in numpy subpackage directory. For example: cd svn/numpy python setup.py install cd numpy/fft python setup.py build build_ext --inplace Another workaround would be to install only numpy.core (that contains numpy headers) and then doing inplace builds in numpy source directory --- this, however, requires some setup.py modifications. IMO, doing inplace builds has side effects that can easily lead to shooting to a leg. While developing numpy, I would recommend always installing numpy to some place, setting PYTHONPATH accordingly, write unittests, and run them for testing. When test_*.py files are set up properly then you don't need to install everytime you modify unittests, they can be run inplace. Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jstrunk at enthought.com Tue Oct 17 08:20:50 2006 From: jstrunk at enthought.com (Jeff Strunk) Date: Tue, 17 Oct 2006 07:20:50 -0500 Subject: Subversion In-Reply-To: References: Message-ID: <200610170720.50980.jstrunk@enthought.com> This was a problem in Virtualmin when I created the mpi4py virtual server. I reverted the change, and everything seems to work now. Sorry for the inconvenience, Jeff On Monday 16 October 2006 10:14 pm, Robert Kern wrote: > Charles R Harris wrote: > > Subversion seems to be down. I get the following message: > > > > Authentication realm: mpi4py.scipy.org > > > > > > mpi4py? And I am asked for my password, which doesn't work. Scipy.org > > also seems to be down. You folks having trouble there? > > Apparently. I've contacted Jeff Strunk, the sysadmin. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From garyt at bmb.leeds.ac.uk Tue Oct 17 11:00:28 2006 From: garyt at bmb.leeds.ac.uk (gary thompson) Date: Tue, 17 Oct 2006 16:00:28 +0100 Subject: hyper complex numbers Message-ID: what is the best and most efficiednt way to deal with hyper complex numbers in numpy/scipy e.g. for a 1d array we have a complex number containing real and imaginary pairs... conceptually we have ((r,i),(r,i)...(r,i)) however, for a hypercomplex number in 2d we have quartets of numbers e.g. ((rr,ir,ir,ii),rr,ir,ir,ii)....), ((rr,ir,ir,ii),rr,ir,ir,ii)....), . . . ((rr,ir,ir,ii),rr,ir,ir,ii)....), or planes of complex numbers r. ((r,i),(r,i)...(r,i)), ((r,i),(r,i)...(r,i)), . . . ((r,i),(r,i)...(r,i)) i. ((r,i),(r,i)...(r,i)), ((r,i),(r,i)...(r,i)), . . . ((r,i),(r,i)...(r,i)) that is the best way to deal with these in scipy/numpy especially with respect to 2D fourier transforms and data processing regards gary -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Tue Oct 17 12:01:51 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 10:01:51 -0600 Subject: What does Fortran order mean? In-Reply-To: References: Message-ID: <4534FE6F.8060602@ee.byu.edu> Charles R Harris wrote: > Travis, > > I note that > > >>> a = arange(6).reshape(2,3,order='F') > >>> a > array([[0, 1, 2], > [3, 4, 5]]) > > Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a > copy, but flat, flatten, and tostring all show the elements in 'C' > order. I ask because I wonder if changing the order can be used to > prepare arrays for input into the LaPack routines. The order argument to reshape means (how should the big-chunk of data be interpreted when you reshape). So, yes, this should be [[0,2,4],[1,3,5]]. It is a bug that it does not do the right thing in this case. I've committed a fix to SVN and a test for this case. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Tue Oct 17 12:07:19 2006 From: faltet at carabos.com (Francesc Altet) Date: Tue, 17 Oct 2006 18:07:19 +0200 Subject: Accessing data buffers in numpy scalars Message-ID: <200610171807.20392.faltet@carabos.com> Hi, I'm looking for an easy way to access the data area of the numpy scalars no matter its type. I've seen that numpy/arrayscalars.h define a structure for each scalar type, so I'd guess that it will not be possible to find a general way for accessing the data buffer for each type. So, I've decided to look for a workaround and I've devised a couple of possibilities: 1.- Promote the scalar type to a ndarray object and use the regular ndarray C structure to access the data buffer. 2.- Fetch the buffer in scalartype.data and use the buffer protocol in order to access the pointer to data in memory. However, I lack experience in buffer protocol, so suggestions for achieving this are welcome. If there is some other trivial way that I haven't devised (specially if usable from pyrex), please tell me about. TIA, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 17 12:22:21 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 10:22:21 -0600 Subject: Accessing data buffers in numpy scalars In-Reply-To: <200610171807.20392.faltet@carabos.com> References: <200610171807.20392.faltet@carabos.com> Message-ID: <4535033D.5000506@ee.byu.edu> Francesc Altet wrote: >Hi, > >I'm looking for an easy way to access the data area of the numpy scalars no >matter its type. I've seen that numpy/arrayscalars.h define a structure for >each scalar type, so I'd guess that it will not be possible to find a general >way for accessing the data buffer for each type. So, I've decided to look for >a workaround and I've devised a couple of possibilities: > > This problem showed up in writing NumPy several times. One solution might be to use PyArray_ScalarAsCtype(PyObject *scalar, void *ctypeptr) which will copy the data into the area pointed to by ctypeptr (unless you have a "flexible scalar" in which case only a pointer to the data-area will be copied). >2.- Fetch the buffer in scalartype.data and use the buffer protocol in order >to access the pointer to data in memory. However, I lack experience in buffer >protocol, so suggestions for achieving this are welcome. > > This will also work. A read-only buffer protocol is exported by all the scalars. scalar.data will return a buffer object. Or you can use the Python C-API const char *buffer; Py_ssize_t buflen; PyObject_AsReadBuffer(scalar, (const void **)&buffer, &buflen) to retrieve a pointer to the data in buffer and the size of the data in buflen. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Tue Oct 17 12:35:59 2006 From: faltet at carabos.com (Francesc Altet) Date: Tue, 17 Oct 2006 18:35:59 +0200 Subject: Accessing data buffers in numpy scalars In-Reply-To: <4535033D.5000506@ee.byu.edu> References: <200610171807.20392.faltet@carabos.com> <4535033D.5000506@ee.byu.edu> Message-ID: <200610171836.00367.faltet@carabos.com> A Dimarts 17 Octubre 2006 18:22, Travis Oliphant va escriure: > >2.- Fetch the buffer in scalartype.data and use the buffer protocol in > > order to access the pointer to data in memory. However, I lack experience > > in buffer protocol, so suggestions for achieving this are welcome. > > This will also work. A read-only buffer protocol is exported by all > the scalars. > > scalar.data will return a buffer object. > > Or you can use the Python C-API > > const char *buffer; > Py_ssize_t buflen; > > PyObject_AsReadBuffer(scalar, (const void **)&buffer, &buflen) Oh, this one seems pretty easy, and as a plus, you don't have to book memory for copying the data area, so I'll use it. Thanks, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 12:44:10 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Tue, 17 Oct 2006 18:44:10 +0200 Subject: What does Fortran order mean? In-Reply-To: <4534FE6F.8060602@ee.byu.edu> References: <4534FE6F.8060602@ee.byu.edu> Message-ID: <20061017164410.GO17998@mentat.za.net> On Tue, Oct 17, 2006 at 10:01:51AM -0600, Travis Oliphant wrote: > Charles R Harris wrote: > > > Travis, > > > > I note that > > > > >>> a = arange(6).reshape(2,3,order='F') > > >>> a > > array([[0, 1, 2], > > [3, 4, 5]]) > > > > Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a > > copy, but flat, flatten, and tostring all show the elements in 'C' > > order. I ask because I wonder if changing the order can be used to > > prepare arrays for input into the LaPack routines. > > > The order argument to reshape means (how should the big-chunk of data be > interpreted when you reshape). So, yes, this should be > [[0,2,4],[1,3,5]]. It is a bug that it does not do the right thing in > this case. A bit counter-ituitive (I somehow expect reshape to return an array that satisfies all the constraints specified as parameters -- i.e. shape and order in memory), but consistent with Numeric. What confuses me is that, if you call the array constructor, you get In [2]: N.array(N.array([[1,2,3],[4,5,6]]),order='F') Out[2]: array([[1, 2, 3], [4, 5, 6]]) so here 'order' means something else. So then you do x = N.array([[0,1,2],[3,4,5]],order='F') x.reshape((2,3),order='C') and you get array([[0, 1, 2], [3, 4, 5]]) Hmm. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 17 13:01:08 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 11:01:08 -0600 Subject: What does Fortran order mean? In-Reply-To: <20061017164410.GO17998@mentat.za.net> References: <4534FE6F.8060602@ee.byu.edu> <20061017164410.GO17998@mentat.za.net> Message-ID: <45350C54.9010004@ee.byu.edu> Stefan van der Walt wrote: >On Tue, Oct 17, 2006 at 10:01:51AM -0600, Travis Oliphant wrote: > > >>Charles R Harris wrote: >> >> >> >>>Travis, >>> >>>I note that >>> >>> >>> >>>>>>a = arange(6).reshape(2,3,order='F') >>>>>>a >>>>>> >>>>>> >>>array([[0, 1, 2], >>> [3, 4, 5]]) >>> >>>Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a >>>copy, but flat, flatten, and tostring all show the elements in 'C' >>>order. I ask because I wonder if changing the order can be used to >>>prepare arrays for input into the LaPack routines. >>> >>> >>The order argument to reshape means (how should the big-chunk of data be >>interpreted when you reshape). So, yes, this should be >>[[0,2,4],[1,3,5]]. It is a bug that it does not do the right thing in >>this case. >> >> > >A bit counter-ituitive (I somehow expect reshape to return an array >that satisfies all the constraints specified as parameters -- i.e. >shape and order in memory), but consistent with Numeric. > > Perhaps, but that is not what reshape does. When you are reshaping an array, unless you are just adding ones to the shape somewhere or aren't actually changing the shape, then you are implicitly thinking about the array as a 1-d chunk of memory in some order. The default is C-order. The order argument let's you think about it differently. >What confuses me is that, if you call the array constructor, you get > >In [2]: N.array(N.array([[1,2,3],[4,5,6]]),order='F') >Out[2]: >array([[1, 2, 3], > [4, 5, 6]]) > >so here 'order' means something else. > > Not really. I can't imagine what else you would expect here. You have specified the array explicitly. The order argument only specifies how the array will be arranged in memory. It is an implementation detail. The user is allowed some control in order to interface to compiled code more effectively. >So then you do > >x = N.array([[0,1,2],[3,4,5]],order='F') >x.reshape((2,3),order='C') > >and you get > >array([[0, 1, 2], > [3, 4, 5]]) > > > I don't see the problem. The first line gives you an array: [[0,1,2], [3,4,5]] which in memory is laid out as [0,3,1,4,2,5] The second line says reshape into a (2,3) array but this is exactly the same shape as the input array so in fact only a new view of the same data is given to you. The fact that you specify order is inconsequential since you aren't really changing the shape, it doesn't matter. The reshape command is not intended to alter the underlying ordering of an input array. The ordering of an array is an implementation detail that shouldn't matter to the user unless you are trying to interface to compiled code. The require command can help you get the right kind of array for that purpose. In general, arrays can have arbitrary strides. C-order and Fortran-order are just two special cases of that generality. So, you cannot think of any array as just being either in C-order or Fortran-order. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 17 13:13:26 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 11:13:26 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. In-Reply-To: <4533D087.3000803@ieee.org> References: <4533D087.3000803@ieee.org> Message-ID: <45350F36.80003@ee.byu.edu> Travis Oliphant wrote: >Charles R Harris wrote: > > >>It seems to me that since the behaviour when copy=0 is to make a copy >>only if necessary, it should find it necessary and make the downcast. >>After all, array(a, dtype=single, copy=1) does just that without >>complaint. Some common code in linalg could be replaced if array and >>asarray would do that operation. >> >> >> >Well, the fact that it makes the copy without raising an error is >different behavior from Numeric and should be considered an unintended >change. > >We definitely should make this consistent for copy=0 or copy=1. The >only question, is do we raise the error in both cases or allow the >conversion in both cases. > >The long-standing behavior is to raise the error on possible-loss >conversion and so my opinion is that we should continue with that behavior. > > > But, on the other hand, it looks like numarray went the other direction and allows a cast using the array call. Thus import numarray a = numarray.array([1,2,3],'d') numarray.array(a, 'f') works So, I'm willing to go with the numarray behavior in numpy. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Andrew.MacKeith at ABAQUS.com Tue Oct 17 13:28:02 2006 From: Andrew.MacKeith at ABAQUS.com (Andrew MacKeith) Date: Tue, 17 Oct 2006 13:28:02 -0400 Subject: array repr Message-ID: <453512A2.80605@ABAQUS.com> I would like to use the built-in array_repr in numpy, because I need a fast repr that does not contain new line characters. I see no way of doing this without editing the code in numeric.py, and I hate to edit other people's libraries. from numpy import array causes numeric.py to be executed, which includes a call to multiarray.set_string_function(array_repr, 1) If I want to "undo" this, there is no way of doing it. I would like to unset the repr with: numpy.set_string_function(None, 1) but this raises a TypeError because None is not callable. This behavior was the same in Numeric, and in that case I edited the code in Numeric.py but it would be nice if I didn't have to do that. Should I submit a patch for this? Andrew MacKeith ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 13:37:09 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 11:37:09 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. In-Reply-To: <45350F36.80003@ee.byu.edu> References: <4533D087.3000803@ieee.org> <45350F36.80003@ee.byu.edu> Message-ID: On 10/17/06, Travis Oliphant wrote: > > Travis Oliphant wrote: > > >Charles R Harris wrote: > > >The long-standing behavior is to raise the error on possible-loss > >conversion and so my opinion is that we should continue with that > behavior. > > > > > > > But, on the other hand, it looks like numarray went the other direction > and allows a cast using the array call. > > Thus > > import numarray > a = numarray.array([1,2,3],'d') > numarray.array(a, 'f') > > works > > > So, I'm willing to go with the numarray behavior in numpy. > > Thanks for the change, Travis. I'll wait a bit to make sure the API remains stable and then do some cleanups in linalg. Correctness before optimization and all that. I did note that the LaPack interface could throw an error at the fortran level that would issue an informative error message and dump me out of python. Is there anyway you know of to trap this behaviour? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Tue Oct 17 13:46:20 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 11:46:20 -0600 Subject: array repr In-Reply-To: <453512A2.80605@ABAQUS.com> References: <453512A2.80605@ABAQUS.com> Message-ID: <453516EC.5010607@ee.byu.edu> Andrew MacKeith wrote: >I would like to use the built-in array_repr in numpy, because >I need a fast repr that does not contain new line characters. >I see no way of doing this without editing the code in numeric.py, >and I hate to edit other people's libraries. > >from numpy import array >causes numeric.py to be executed, which includes a call to >multiarray.set_string_function(array_repr, 1) > >If I want to "undo" this, there is no way of doing it. > > Why do want to "undo" this? >I would like to unset the repr with: >numpy.set_string_function(None, 1) > > I don't understand what you are trying to do. Why don't you just set the string function to an actual callable. What edits do you propose to make? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Tue Oct 17 13:58:18 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 17 Oct 2006 14:58:18 -0300 Subject: What does Fortran order mean? In-Reply-To: <45350C54.9010004@ee.byu.edu> References: <4534FE6F.8060602@ee.byu.edu> <20061017164410.GO17998@mentat.za.net> <45350C54.9010004@ee.byu.edu> Message-ID: On 10/17/06, Travis Oliphant wrote: > The require command can help you get the right kind of > array for that purpose. > BTW, Travis In [1]: numpy.__version__ Out[1]: '1.0rc2' In [2]: print numpy.require.__doc__ None What is this 'requirements' argument? What should user pass in?. I seen sorce code at numpy.core.numeric.require. It seems this functions expects an iterable producing string. I am right? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Tue Oct 17 14:03:41 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 17 Oct 2006 15:03:41 -0300 Subject: Accessing data buffers in numpy scalars In-Reply-To: <4535033D.5000506@ee.byu.edu> References: <200610171807.20392.faltet@carabos.com> <4535033D.5000506@ee.byu.edu> Message-ID: On 10/17/06, Travis Oliphant wrote: > Or you can use the Python C-API > > const char *buffer; > Py_ssize_t buflen; > > PyObject_AsReadBuffer(scalar, (const void **)&buffer, &buflen) > > to retrieve a pointer to the data in buffer and the size of the data in > buflen. > Travis. Have numpy something similar to a mutable scalar, or the only way is using an array whit only one element? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Andrew.MacKeith at ABAQUS.com Tue Oct 17 14:15:24 2006 From: Andrew.MacKeith at ABAQUS.com (Andrew MacKeith) Date: Tue, 17 Oct 2006 14:15:24 -0400 Subject: array repr In-Reply-To: <453516EC.5010607@ee.byu.edu> References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> Message-ID: <45351DBC.8080502@ABAQUS.com> Travis Oliphant wrote: >Andrew MacKeith wrote: > > > >>I would like to use the built-in array_repr in numpy, because >>I need a fast repr that does not contain new line characters. >>I see no way of doing this without editing the code in numeric.py, >>and I hate to edit other people's libraries. >> >> >> >>from numpy import array > > >>causes numeric.py to be executed, which includes a call to >>multiarray.set_string_function(array_repr, 1) >> >>If I want to "undo" this, there is no way of doing it. >> >> >> >> >Why do want to "undo" this? > > > >>I would like to unset the repr with: >>numpy.set_string_function(None, 1) >> >> >> >> >I don't understand what you are trying to do. Why don't you just set >the string function to an actual callable. > Since arrayobject.c contains a fast repr function, I would like to use that. Why reinvent the wheel. >What edits do you propose to >make? > > If you mean how would I propose to change the code, I would have array_set_string_function recognise None as an acceptable argument and PyArray_SetStringFunction would recognise this to set PyArray_ReprFunction = array_repr; These is the (untested) code. multiarraymodule.c: static PyObject * array_set_string_function(PyObject *dummy, PyObject *args, PyObject *kwds) { PyObject *op; int repr=1; static char *kwlist[] = {"f", "repr", NULL}; if(!PyArg_ParseTupleAndKeywords(args, kwds, "O|i", kwlist, &op, &repr)) return NULL; if (!PyCallable_Check(op) and op != Py_None) { PyErr_SetString(PyExc_TypeError, "Argument must be callable."); return NULL; } PyArray_SetStringFunction(op, repr); Py_INCREF(Py_None); return Py_None; } arrayobject.c: static void PyArray_SetStringFunction(PyObject *op, int repr) { if (repr) { /* Dispose of previous callback */ Py_XDECREF(PyArray_ReprFunction); if (op == Py_None) { PyArray_ReprFunction = array_repr; } else { /* Add a reference to new callback */ Py_XINCREF(op); /* Remember new callback */ PyArray_ReprFunction = op; } } else { /* Dispose of previous callback */ Py_XDECREF(PyArray_StrFunction); if (op == Py_None) { PyArray_StrFunction = array_str; } else { /* Add a reference to new callback */ Py_XINCREF(op); /* Remember new callback */ PyArray_StrFunction = op; } } } >-Travis > > > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion at lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 14:25:18 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 12:25:18 -0600 Subject: Casting In-Reply-To: References: Message-ID: On 10/12/06, Cristian Codorean wrote: > > Hello list, > I got some old code that uses Scientific and Numeric. Recently, these > two packages were updated to the latest versions together with python and my > code started failing. The first major problem I have is when downcasting. > Code that used to work now fails because the array cannot be safely cast > anymore to the required type. If anyone could help me please and direct me > to some documentation about when this change happened (version), why it > happened, is there any other solution but using astype, can numpy help me in > anyway in this situation, because I got really lost ... This seems related to a more recent thread. Travis says that numeric throwing an error on downcasting is traditional, so I am curious as to why the previous code should have worked. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From kortmann at ideaworks.com Tue Oct 17 14:35:25 2006 From: kortmann at ideaworks.com (Kenny Ortmann) Date: Tue, 17 Oct 2006 13:35:25 -0500 Subject: matlab, filter function References: Message-ID: <002101c6f21b$05687b50$0900a8c0@yairlap> just looking for some help, most of the time you guys are good with matlab code, i am trying to use the filter function under this setting y = filter(b,a,X) filters the data in vector X with the filter described by numerator coefficient vector b and denominator coefficient vector a. If a(1) is not equal to 1, filter normalizes the filter coefficients by a(1). If a(1) equals 0, filter returns an error. I pretty much have the same as the example, except my data array is different data = [1:0.2:4]'; windowSize = 5; filter(ones(1,windowSize)/windowSize,1,data) ans = 0.2000 0.4400 0.7200 1.0400 1.4000 1.6000 1.8000 2.0000 2.2000 2.4000 2.6000 2.8000 3.0000 3.2000 3.4000 3.6000 my question is does numpy/scipy have a function for this, if not the algorithm states y(n) = b(1)*x(n) + b(2)*x(n-1) + ... + b(nb+1)*x(n-nb) - a(2)*y(n-1) - ... - a(na+1)*y(n-na)but a is a number not an arrayThanks for the help,~Kenny ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 15:10:55 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 13:10:55 -0600 Subject: dtype always copies In-Reply-To: <20061013142825.GA16246@mentat.za.net> References: <20061013142825.GA16246@mentat.za.net> Message-ID: On 10/13/06, Stefan van der Walt wrote: > > Hi all, > > I've noticed that 'astype' always forces a copy. Is this > behaviour intended? It seems to conflict with 'asarray', that > tries to avoid a copy. > > For example, when wrapping code in ctypes, the following snippet > would have been useful: > > def foo(x): > # ensure x is an array of the right type > x = N.ascontiguousarray(x).astype(N.intc) > > but that will cause a copy, so you'll have to do > > def foo(x): > try: > x = N.ascontiguousarray(x,N.intc) > except: > x = N.ascontiguousarray(x).astype(N.intc) This seems to work now: In [21]: a Out[21]: array([[ 1., 2.], [ 3., 4.]]) In [22]: ascontiguousarray(a, dtype=int) Out[22]: array([[1, 2], [3, 4]]) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Tue Oct 17 15:19:42 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 13:19:42 -0600 Subject: Accessing data buffers in numpy scalars In-Reply-To: References: <200610171807.20392.faltet@carabos.com> <4535033D.5000506@ee.byu.edu> Message-ID: <45352CCE.3060108@ee.byu.edu> Lisandro Dalcin wrote: >On 10/17/06, Travis Oliphant wrote: > > >>Or you can use the Python C-API >> >>const char *buffer; >>Py_ssize_t buflen; >> >>PyObject_AsReadBuffer(scalar, (const void **)&buffer, &buflen) >> >>to retrieve a pointer to the data in buffer and the size of the data in >>buflen. >> >> >> > >Travis. Have numpy something similar to a mutable scalar, or the only >way is using an array whit only one element? > > 0-dim arrays are still mutable scalars. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 17 15:21:37 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 13:21:37 -0600 Subject: array repr In-Reply-To: <45351DBC.8080502@ABAQUS.com> References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> <45351DBC.8080502@ABAQUS.com> Message-ID: <45352D41.2030601@ee.byu.edu> Ah!, I get it. You want to be able to reset to the C-defined array_repr function. The one that gets over-written on import. That makes sense. And is definitely do-able. Please file a ticket. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Andrew.MacKeith at ABAQUS.com Tue Oct 17 15:33:39 2006 From: Andrew.MacKeith at ABAQUS.com (Andrew MacKeith) Date: Tue, 17 Oct 2006 15:33:39 -0400 Subject: array repr In-Reply-To: <45352D41.2030601@ee.byu.edu> References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> <45351DBC.8080502@ABAQUS.com> <45352D41.2030601@ee.byu.edu> Message-ID: <45353013.9000705@ABAQUS.com> Travis Oliphant wrote: >Ah!, I get it. You want to be able to reset to the C-defined >array_repr function. The one that gets over-written on import. That >makes sense. And is definitely do-able. > >Please file a ticket. > > Can you point me to how to file a ticket. Thanks Andrew > > >-Travis > > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion at lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Tue Oct 17 15:48:37 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Tue, 17 Oct 2006 21:48:37 +0200 Subject: recent ubuntu package Message-ID: <45353395.40602@gmx.net> Hi, as suggested on the website I use the kindly provided pre-built (unofficial) ubuntu debs. Recently there is a new one available with version numbe 1.0+~dev3336-0ads1. Apart from the slightly strange +~ thing in there, it very much seems to be based on trac changeset 3336, which is somewhere between rc2 and now, right? And just checking, what are the matplotlib packages that are compatible? The most recent from the same package source seems to be 0.87.5-r2781-0ads2, is that ok with the above numpy? Will the mpl ubuntu package before that break with the latest numpy ubuntu package? Thanks for your help, and for the build effort, Sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 15:51:05 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 13:51:05 -0600 Subject: array repr In-Reply-To: <45353013.9000705@ABAQUS.com> References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> <45351DBC.8080502@ABAQUS.com> <45352D41.2030601@ee.byu.edu> <45353013.9000705@ABAQUS.com> Message-ID: On 10/17/06, Andrew MacKeith wrote: > > Travis Oliphant wrote: > > >Ah!, I get it. You want to be able to reset to the C-defined > >array_repr function. The one that gets over-written on import. That > >makes sense. And is definitely do-able. > > > >Please file a ticket. > > > >** > Can you point me to how to file a ticket. > Thanks Go to http://projects.scipy.org/scipy/numpy/report, login or register (required), then choose new ticket from the menu at the top. BTW, I wish all this was more transparent to the casual visitor to the scipy site. There should be a link on the front page (numpy bugs or something) and a short explanation about needing to register to file a new ticket. Maybe something in the Cookbook would help. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Tue Oct 17 17:20:07 2006 From: david.huard at gmail.com (David Huard) Date: Tue, 17 Oct 2006 17:20:07 -0400 Subject: histogram complete makeover Message-ID: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> Hi all, I'd like to poll the list to see what people want from numpy.histogram(), since I'm currently writing a contender. My main complaints with the current version are: 1. upper outliers are stored in the last bin, while lower outliers are not counted at all, 2. cannot use weights. The new histogram function is well under way (it address these issues and adds an axis keyword), but I want to know what is the preferred behavior regarding the function output, and your willingness to introduce a new behavior that will break some code. Given a number of bins N and range (min, max), histogram constructs linearly spaced bin edges b0 (out-of-range) | b1 | b2 | b3 | .... | bN | bN+1 out-of-range and may return: A. H = array([N_b0, N_b1, ..., N_bN, N_bN+1]) The out-of-range values are the first and last values of the array. The returned array is hence N+2 B. H = array([N_b0 + N_b1, N_b2, ..., N_bN + N_bN+1]) The lower and upper out-of-range values are added to the first and last bin respectively. C. H = array([N_b1, ..., N_bN + N_bN+1]) Current behavior: the upper out-of-range values are added to the last bin. D. H = array([N_b1, N_b2, ..., N_bN]), Lower and upper out-of-range values are given after the histogram array. Ideally, the new function would not break the common usage: H = histogram(x)[0], so this exclude A. B and C are not acceptable in my opinion, so only D remains, with the downsize that the outliers are not returned. A solution might be to add a keyword full_output=False, which when set to True, returns the out-of-range values in a dictionnary. Also, the current function returns -> H, ledges where ledges is the array of left bin edges (N). I propose returning the complete array of edges (N+1), including the rightmost edge. This is a little bit impractical for plotting, as the edges array does not have the same length as the histogram array, but allows the use of user-defined non-uniform bins. Opinions, suggestions ? David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Tue Oct 17 17:55:54 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 15:55:54 -0600 Subject: matlab, filter function In-Reply-To: <002101c6f21b$05687b50$0900a8c0@yairlap> References: <002101c6f21b$05687b50$0900a8c0@yairlap> Message-ID: <4535516A.4030608@ee.byu.edu> Kenny Ortmann wrote: >just looking for some help, most of the time you guys are good with matlab >code, i am trying to use the filter function under this setting > >y = filter(b,a,X) filters the data in vector X with the filter described by >numerator coefficient vector b and denominator coefficient vector a. If a(1) >is not equal to 1, filter normalizes the filter coefficients by a(1). If >a(1) equals 0, filter returns an error. > > There is scipy.signal.lfilter which implements this algorithm. It's doc string is lfilter(b, a, x, axis=-1, zi=None) Filter data along one-dimension with an IIR or FIR filter. Description Filter a data sequence, x, using a digital filter. This works for many fundamental data types (including Object type). The filter is a direct form II transposed implementation of the standard difference equation (see "Algorithm"). Inputs: b -- The numerator coefficient vector in a 1-D sequence. a -- The denominator coefficient vector in a 1-D sequence. If a[0] is not 1, then both a and b are normalized by a[0]. x -- An N-dimensional input array. axis -- The axis of the input data array along which to apply the linear filter. The filter is applied to each subarray along this axis (*Default* = -1) zi -- Initial conditions for the filter delays. It is a vector (or array of vectors for an N-dimensional input) of length max(len(a),len(b)). If zi=None or is not given then initial rest is assumed. SEE signal.lfiltic for more information. Outputs: (y, {zf}) y -- The output of the digital filter. zf -- If zi is None, this is not returned, otherwise, zf holds the final filter delay values. Algorithm: The filter function is implemented as a direct II transposed structure. This means that the filter implements y[n] = b[0]*x[n] + b[1]*x[n-1] + ... + b[nb]*x[n-nb] - a[1]*y[n-1] + ... + a[na]*y[n-na] using the following difference equations: y[m] = b[0]*x[m] + z[0,m-1] z[0,m] = b[1]*x[m] + z[1,m-1] - a[1]*y[m] ... z[n-3,m] = b[n-2]*x[m] + z[n-2,m-1] - a[n-2]*y[m] z[n-2,m] = b[n-1]*x[m] - a[n-1]*y[m] where m is the output sample number and n=max(len(a),len(b)) is the model order. The rational transfer function describing this filter in the z-transform domain is -1 -nb b[0] + b[1]z + ... + b[nb] z Y(z) = ---------------------------------- X(z) -1 -na a[0] + a[1]z + ... + a[na] z ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From shfthdcvrdlf at bakerinc.com.sun.com Tue Oct 17 15:00:01 2006 From: shfthdcvrdlf at bakerinc.com.sun.com (Molly Driscoll) Date: Tue, 17 Oct 2006 22:00:01 +0300 Subject: Significant letter. You must to read. Message-ID: <01c6f237$97f8dcf0$6c822ecf@shfthdcvrdlf> Please read it. Today is the exact day to buy that stock. Please check that note attentively. Here you will find the intimate news about GDKI. Please Read this news. Goldmark Cipher Established Hip Hop Creator Sean Combs A.K.A. Puff Daddy Wednesday this month 18, 11:40 am ET NEW YORK & LOS ANGELES & VANCOUVER, B.C.--(BUSINESS WIRE)-- October 18, 11:40 am ET -- Goldmark Industries, Inc. (PINK SHEETS:GDKI.PK - Reports), is excited to declare that it has signed one of the hottest producers in the hip-hop and r&b industry, Sean Combs A.K.A. Puff Daddy. In an aggressive attempt to stay ahead of the game, the Corporation moved quickly in taking on the already successful and rising famous person. Sean Combs, also well-known as Puff Daddy is the founder of Bad Boys Records marker. By contract the between Goldmark Industries, Inc and Bad Boy Entertainment Inc, Goldmark has to get 78 000 000 $ for the distributing of Sean Combs A.K.A. Puff Daddy fresh album Press Play. Total about from that deal for Goldmark should be about 12 000 000$. After the establishing of that deal Goldmark Industries, Inc will be the second biggest company in the industry of hip-hop & r&b music. Hip Hop famous person Sean "P.Diddy" Combs sees a huge possibility for his company in collaboration with Goldmark inc. Sean "P.Diddy" Combs tells that it is enjoyably to treat with these guys. They as anybody else know entertainment industriousness and precisely know what is necessary for the American spectators. He also emphasizes exclusivity of his fresh album Press Play and tells that the presentation of this album on october 17 will make an effect of the blasted bomb. Today the new album of Puff Daddy Press Play will be on the shelves. TAKE GDKI and you will earn from every sale of that golden disc. Concerning Bad Boy International Entertainment Group: is one of the world's pre-eminent urban entertainment companies, surrounding a wide variety of businesses including recording, music publishing, artist organization, television and film production, recording facility, advertising and advertising, apparel and restaurants. With a collection of businesses whose yearly sales are quickly impending 300$ million annually and an employee base that is 600 strong. Mr. Combs is largely responsible for the pop appeal of urban entertainment. In just eight years, founder and CEO Sean "P.Diddy" Combs has forever changed the entertainment industry by catapulting the music and fashionof urban youth culture into the American mainstream and creating what is measured by experts to be one of the most significant services in entertainment these days. About Goldmark Industries, Inc. (Stock symbol: GDKI.PK) Goldmark Industries is committed to providing the best in all forms of urban entertainment to the 45 Million Hip-Hop consumers in North America. The average North American spends health care & clothing on entertainment than they do on more cash, making entertainment the most attractive industry for investors and advertisers alike. Goldmark Industries is preparing to stand at the forefront of the Hip Hop consumer market, specializing in all aspects of entertainment, including Major Events ,Music ,Feature Films ,Television and Home Video/DVD . The strength of Goldmark Industries is the result of its highly reputable & continuously growing management team. The knowledge and experience that each team member brings consistently supports the growing success of each division at Goldmark Industries. In addition, they are associated with some of the world's leading entertainment companies and top distribution channels worldwide, providing Goldmark Industries with the relationships to continually move forward. P.S We has started to promote GDKI yesterday and the price is up for 16.6% already. Today,october 17 2006, the price will be 100% up. Don't miss that opportunity to buy GDKI for a low price. Just buy it! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Tue Oct 17 21:13:13 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 17 Oct 2006 22:13:13 -0300 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <20061018002445.GP17998@mentat.za.net> References: <20061018002445.GP17998@mentat.za.net> Message-ID: I was surprised by this In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') Out[14]: array([[1, 5], [4, 3], [2, 6]]) In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') Out[15]: array([[1, 2], [3, 4], [5, 6]]) On 10/17/06, Stefan van der Walt wrote: > Hi all, > > Some of you may have seen the interesting thread on Fortran-ordering > earlier. I thought it might be fun to set up a short quiz which tests > your knowledge on the topic. > > If you're up for the challenge, take a look at > > http://mentat.za.net/numpy/quiz > > I won't be held liable for any emotional trauma caused, self-inflicted > wounds or brain-core meltdowns. > > Cheers > St?fan > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From michael.sorich at gmail.com Tue Oct 17 20:19:27 2006 From: michael.sorich at gmail.com (Michael Sorich) Date: Wed, 18 Oct 2006 09:49:27 +0930 Subject: A reimplementation of MaskedArray In-Reply-To: <200610162242.58541.pgmdevlist@mailcan.com> References: <200610160232.20888.pgmdevlist@mailcan.com> <16761e100610161908r3bbf383fq4c5caf64f8b805b8@mail.gmail.com> <200610162242.58541.pgmdevlist@mailcan.com> Message-ID: <16761e100610171719we82588mf8138bdfadce3398@mail.gmail.com> On 10/17/06, Pierre GM wrote: > On Monday 16 October 2006 22:08, Michael Sorich wrote: > > Does this new MA class allow masking of rearray like arrays? > > Excellent question! Which is to say, I have no idea... I don't use > recordarray, so I didn't think about testing them. > > So, a first test indicates that it doesn't work either. The mask for a3 is > understood as having a size 6, when the data is seen as size 2 only (exactly > the same problem as the original ma module). > I gonna poke around and check whether I can come with a workaround. Just to > make it clear: you want to be able to mask some fields in some records, > right ? Not mask all the fields of a record ? Yes, that is correct. It would be preferable to have more control over masking than simply masking an entire record. I view the heterogenouse arrays as 2D arrays in which it should be possible to mask individual values. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 20:24:45 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Wed, 18 Oct 2006 02:24:45 +0200 Subject: The NumPy Fortran-ordering quiz Message-ID: <20061018002445.GP17998@mentat.za.net> Hi all, Some of you may have seen the interesting thread on Fortran-ordering earlier. I thought it might be fun to set up a short quiz which tests your knowledge on the topic. If you're up for the challenge, take a look at http://mentat.za.net/numpy/quiz I won't be held liable for any emotional trauma caused, self-inflicted wounds or brain-core meltdowns. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mrovner at cadence.com Tue Oct 17 20:30:12 2006 From: mrovner at cadence.com (Mike Rovner) Date: Tue, 17 Oct 2006 17:30:12 -0700 Subject: 1.0rc2 strange repeat behavior Message-ID: Hi, I got strange discrepance between 2.4+0.9.8 and 2.5+1.0rc2: model_lib_pool %0 !5019$ python2.5 Python 2.5 (r25:51908, Oct 17 2006, 16:16:21) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-14)] on linux2 Type "help", "copyright", "credits" or "license" for more information. f>>> from numpy import * >>> a=arange(12).reshape(4,3) >>> a array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11]]) >>> a[:,2] array([ 2, 5, 8, 11]) >>> a[:,2].repeat(3) array([2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5]) >>> import numpy >>> numpy.__version__ '1.0rc2' >>> Old version looks right: Python 2.4.2 (#5, Jun 2 2006, 18:33:20) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-20)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from numpy import * >>> a=arange(12).reshape(4,3) >>> a array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11]]) >>> a[:,2] array([ 2, 5, 8, 11]) >>> a[:,2].repeat(3) array([ 2, 2, 2, 5, 5, 5, 8, 8, 8, 11, 11, 11]) >>> import numpy >>> numpy.__version__ '0.9.8' >>> Thanks, Mike ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ndbecker2 at gmail.com Tue Oct 17 20:46:51 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 17 Oct 2006 20:46:51 -0400 Subject: histogram complete makeover References: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> Message-ID: David Huard wrote: > Hi all, > > I'd like to poll the list to see what people want from numpy.histogram(), > since I'm currently writing a contender. > > My main complaints with the current version are: > 1. upper outliers are stored in the last bin, while lower outliers are not > counted at all, > 2. cannot use weights. > > The new histogram function is well under way (it address these issues and > adds an axis keyword), > but I want to know what is the preferred behavior regarding the function > output, and your > willingness to introduce a new behavior that will break some code. > > Given a number of bins N and range (min, max), histogram constructs > linearly spaced bin edges > b0 (out-of-range) | b1 | b2 | b3 | .... | bN | bN+1 out-of-range > and may return: > > A. H = array([N_b0, N_b1, ..., N_bN, N_bN+1]) > The out-of-range values are the first and last values of the array. The > returned array is hence N+2 > > B. H = array([N_b0 + N_b1, N_b2, ..., N_bN + N_bN+1]) > The lower and upper out-of-range values are added to the first and last > bin respectively. > > C. H = array([N_b1, ..., N_bN + N_bN+1]) > Current behavior: the upper out-of-range values are added to the last bin. > > D. H = array([N_b1, N_b2, ..., N_bN]), > Lower and upper out-of-range values are given after the histogram array. > > Ideally, the new function would not break the common usage: H = > histogram(x)[0], so this exclude A. B and C are not acceptable in my > opinion, so only D remains, with the downsize that the outliers are not > returned. A solution might be to add a keyword full_output=False, which > when set to True, returns the out-of-range values in a dictionnary. > > Also, the current function returns -> H, ledges > where ledges is the array of left bin edges (N). > I propose returning the complete array of edges (N+1), including the > rightmost edge. This is a little bit impractical for plotting, as the > edges array does not have the same length as the histogram array, but > allows the use of user-defined non-uniform bins. > > Opinions, suggestions ? > > David I have my own histogram that might interest you. The core is modern c++, with boost::python wrapper. Out-of-bounds behavior is programmable. I'll send it to you if you are interested. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Tue Oct 17 21:30:26 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Wed, 18 Oct 2006 10:30:26 +0900 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: I think the answer to #3 is wrong. >From 1.0rc2 I get: >>> array([1,2,3,4,5,6],order='C').reshape((2,3),order='F') array([[1, 2, 3], [4, 5, 6]]) But the quiz wants me to answer something different. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Andrew.MacKeith at ABAQUS.com Tue Oct 17 21:05:28 2006 From: Andrew.MacKeith at ABAQUS.com (Andrew.MacKeith) Date: Tue, 17 Oct 2006 21:05:28 -0400 Subject: array repr In-Reply-To: References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> <45351DBC.8080502@ABAQUS.com> <45352D41.2030601@ee.byu.edu> <45353013.9000705@ABAQUS.com> Message-ID: <45357DD8.7030508@ABAQUS.com> Charles R Harris wrote: > > > On 10/17/06, Andrew MacKeith > wrote: > > Travis Oliphant wrote: > > >Ah!, I get it. You want to be able to reset to the C-defined > >array_repr function. The one that gets over-written on > import. That > >makes sense. And is definitely do-able. > > > >Please file a ticket. > > > > > Can you point me to how to file a ticket. > Thanks > > > Go to http://projects.scipy.org/scipy/numpy/report > , login or register > (required), then choose new ticket from the menu at the top. > > BTW, I wish all this was more transparent to the casual visitor to the > scipy site. There should be a link on the front page (numpy bugs or > something) and a short explanation about needing to register to file a > new ticket. Maybe something in the Cookbook would help. > > Chuck Thanks for the info. I filed a ticket. Andrew > > >------------------------------------------------------------------------ > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > >------------------------------------------------------------------------ > >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion at lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Tue Oct 17 21:38:13 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 19:38:13 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: <45358585.60907@ieee.org> Lisandro Dalcin wrote: > I was surprised by this > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > Out[14]: > array([[1, 5], > [4, 3], > [2, 6]]) > > In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > Out[15]: > array([[1, 2], > [3, 4], > [5, 6]]) > This is a bug that was recently fixed in SVN. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From djarb at highenergymagic.org Tue Oct 17 19:35:19 2006 From: djarb at highenergymagic.org (Daniel Arbuckle) Date: Tue, 17 Oct 2006 16:35:19 -0700 Subject: Question about indexing arrays with boolean arrays Message-ID: Why does a[b1, b2] not mean the same thing as a[b1][:, b2], when "a" is an array and "b1" and "b2" are appropriately sized arrays of booleans? Thanks ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 21:42:46 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Wed, 18 Oct 2006 03:42:46 +0200 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: <20061018014246.GQ17998@mentat.za.net> On Wed, Oct 18, 2006 at 10:30:26AM +0900, Bill Baxter wrote: > I think the answer to #3 is wrong. > > >From 1.0rc2 I get: > >>> array([1,2,3,4,5,6],order='C').reshape((2,3),order='F') > array([[1, 2, 3], > [4, 5, 6]]) > > But the quiz wants me to answer something different. This recently changed. The quiz is correct for r3348. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 21:47:22 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 19:47:22 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: On 10/17/06, Lisandro Dalcin wrote: > > I was surprised by this > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > Out[14]: > array([[1, 5], > [4, 3], > [2, 6]]) This one still looks wrong. In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > Out[15]: > array([[1, 2], > [3, 4], > [5, 6]]) This one is fixed, In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') Out[3]: array([[1, 4], [2, 5], [3, 6]]) I also don't understand why a copy is returned if 'F' just fiddles with the indices and strides; the underlying data should be the same, just the view changes. FWIW, I think both examples should be returning views. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Tue Oct 17 21:53:11 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 19:53:11 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <20061018002445.GP17998@mentat.za.net> References: <20061018002445.GP17998@mentat.za.net> Message-ID: <45358907.9020509@ieee.org> Stefan van der Walt wrote: > Hi all, > > Some of you may have seen the interesting thread on Fortran-ordering > earlier. I thought it might be fun to set up a short quiz which tests > your knowledge on the topic. > > If you're up for the challenge, take a look at > > http://mentat.za.net/numpy/quiz > > I won't be held liable for any emotional trauma caused, self-inflicted > wounds or brain-core meltdowns. > Cute (especially the comment if you get them all right). I'm not sure if this quiz is a veiled complaint about the rules for Fortran ordering or not ;-) In my mind the Fortran ordering rules are consistent (if not completely bug-free). You just have to get the right idea of what is meant by the order argument when you use it. If you think you are having trouble figuring out the rules, think of the trouble it was to figure out what they should be and then to code them up. Two rules help you pass the quiz with a perfect score: 1) On array construction, the order argument allows you to specify how the array will be organized in memory. This has no effect on what is printed as the user doesn't usually care how the array is stored in memory. So, you can ignore all order= expressions in the array construct for the quiz 2) On reshaping, the order argument specifies how you think the array is organized. Whenever you make a significant reshape you are telling the computer to re-interpret the chunk of data in a different way, it makes a big difference as to how you think about that chunk of data. Do you think of it as organized rows-first (C-order) or columns-first (Fortran-order). The order argument allows you to specify how you think about it and indicates the 1-d indexing order of the array. It also fills in the newly-shaped array in exactly that same order. Semantically, one could technically separate those two concepts and have one order argument that specifies how you think about the input and another that specifies how you think about the output. But, I really didn't want to go there and couldn't see a real advantage to that. So, the single order argument specifies how you think about both. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 17 21:59:41 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 19:59:41 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: <45358A8D.600@ieee.org> Charles R Harris wrote: > > > On 10/17/06, *Lisandro Dalcin* > wrote: > > I was surprised by this > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > Out[14]: > array([[1, 5], > [4, 3], > [2, 6]]) > > > This one still looks wrong. > > In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > Out[15]: > array([[1, 2], > [3, 4], > [5, 6]]) > > > > This one is fixed, > > In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') > Out[3]: > array([[1, 4], > [2, 5], > [3, 6]]) > > I also don't understand why a copy is returned if 'F' just fiddles > with the indices and strides; the underlying data should be the same, > just the view changes. FWIW, I think both examples should be returning > views. You are right, it doesn't need to. My check is not general enough. It can be challenging to come up with a general way to differentiate the view-vs-copy situation and I struggled with it. In this case, it's the fact that while self->nd > 1, the other dimensions are only of shape 1 and so don't really matter. If you could come up with some kind of striding check that would distinguish the two cases, I would appreciate it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 17 22:15:13 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 20:15:13 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: <45358E31.5010909@ieee.org> Charles R Harris wrote: > > > On 10/17/06, *Lisandro Dalcin* > wrote: > > I was surprised by this > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > Out[14]: > array([[1, 5], > [4, 3], > [2, 6]]) > > > This one still looks wrong. Why does this look wrong. What do you want it to be? Perhaps you are thinking about the input array as C-order and the output array as Fortran order. That's not what reshape does. The order argument specifies how you think about both the input and output. Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] and then a Fortran-order based fill of an empty (3,2) array: giving you the result. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 22:21:02 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 20:21:02 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45358A8D.600@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> Message-ID: On 10/17/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > > On 10/17/06, *Lisandro Dalcin* > > wrote: > > > > I was surprised by this > > > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > > Out[14]: > > array([[1, 5], > > [4, 3], > > [2, 6]]) > > > > > > This one still looks wrong. > > > > In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > > Out[15]: > > array([[1, 2], > > [3, 4], > > [5, 6]]) > > > > > > > > This one is fixed, > > > > In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') > > Out[3]: > > array([[1, 4], > > [2, 5], > > [3, 6]]) > > > > I also don't understand why a copy is returned if 'F' just fiddles > > with the indices and strides; the underlying data should be the same, > > just the view changes. FWIW, I think both examples should be returning > > views. > > You are right, it doesn't need to. My check is not general enough. > > It can be challenging to come up with a general way to differentiate the > view-vs-copy situation and I struggled with it. In this case, it's the > fact that while self->nd > 1, the other dimensions are only of shape 1 > and so don't really matter. If you could come up with some kind of > striding check that would distinguish the two cases, I would appreciate > it. I suppose the problem is mostly in discontiguous arrays. Hmmm..., this isn't too different that reshaping the transpose. a.reshape((m,n),order='F' ~ a.reshape((n,m)).T.reshape(m,n) for instance: In [26]: a Out[26]: array([[1, 2, 3], [4, 5, 6]]) In [27]: a.reshape((3,2)).T.reshape((2,3)) Out[27]: array([[1, 3, 5], [2, 4, 6]]) In [28]: a.reshape((2,3), order='F') Out[28]: array([[1, 2, 3], [4, 5, 6]]) Where I actually think the second reshape is correct and the third incorrect. This has the advantage that *all* the steps return views. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From stefan at sun.ac.za Tue Oct 17 22:22:29 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Wed, 18 Oct 2006 04:22:29 +0200 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45358907.9020509@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> Message-ID: <20061018022229.GR17998@mentat.za.net> On Tue, Oct 17, 2006 at 07:53:11PM -0600, Travis Oliphant wrote: > Stefan van der Walt wrote: > > Hi all, > > > > Some of you may have seen the interesting thread on Fortran-ordering > > earlier. I thought it might be fun to set up a short quiz which tests > > your knowledge on the topic. > > > > If you're up for the challenge, take a look at > > > > http://mentat.za.net/numpy/quiz > > > > I won't be held liable for any emotional trauma caused, self-inflicted > > wounds or brain-core meltdowns. > > > > Cute (especially the comment if you get them all right). I'm not sure > if this quiz is a veiled complaint about the rules for Fortran ordering > or not ;-) If it is, it is very tactfully disguised ;) Actually, I was just trying to figure out how Fortran ordering works and thought others might be interested. I reckoned you would be the only one to get 100%, although now you've explained the rules so clearly a couple of other might grasp for the holy grail too. > In my mind the Fortran ordering rules are consistent (if not completely > bug-free). You just have to get the right idea of what is meant by the > order argument when you use it. If you think you are having trouble > figuring out the rules, think of the trouble it was to figure out what > they should be and then to code them up. My sympathies (no, really). > 2) On reshaping, the order argument specifies how you think the array is > organized. Whenever you make a significant reshape you are telling the > computer to re-interpret the chunk of data in a different way, it makes > a big difference as to how you think about that chunk of data. Do you > think of it as organized rows-first (C-order) or columns-first > (Fortran-order). The order argument allows you to specify how you > think about it and indicates the 1-d indexing order of the array. It > also fills in the newly-shaped array in exactly that same order. > Semantically, one could technically separate those two concepts and have > one order argument that specifies how you think about the input and > another that specifies how you think about the output. But, I really > didn't want to go there and couldn't see a real advantage to that. So, > the single order argument specifies how you think about both. Thanks for the detailed overview (we should put it on the wiki). Another thing I'm wondering about: when exactly does reshape need to make copies? One last case, which confuses me still (probably because it is 04:16am): In [41]: x = N.array([[0,1,2],[3,4,5]],order='F') In [42]: x Out[42]: array([[0, 1, 2], [3, 4, 5]]) I assume the data is now stored in memory as [0 3 1 4 2 5] (column-wise) If I now do x.reshape((3,2),order='C') i.e. take that block of memory, assume it is in 'C' order, and make its shape (3,2), I expect [[0 3] [1 4] [2 5]] but get [[1 2] [3 4] [5 6]] I'm obviously missing something trivial -- I'll try again tomorrow. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 22:28:37 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 20:28:37 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45358E31.5010909@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 10/17/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > > On 10/17/06, *Lisandro Dalcin* > > wrote: > > > > I was surprised by this > > > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > > Out[14]: > > array([[1, 5], > > [4, 3], > > [2, 6]]) > > > > > > This one still looks wrong. > > Why does this look wrong. What do you want it to be? Perhaps you are > thinking about the input array as C-order and the output array as > Fortran order. That's not what reshape does. The order argument > specifies how you think about both the input and output. > > Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] > and then a Fortran-order based fill of an empty (3,2) array: giving you > the result. Why a Fortran ravel? I am thinking of it as preserving the memory layout, just messing with how it is addressed. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Oct 17 22:53:52 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 20:53:52 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> Message-ID: On 10/17/06, Charles R Harris wrote: > > > > On 10/17/06, Travis Oliphant wrote: > > > > Charles R Harris wrote: > > > > > > > > > On 10/17/06, *Lisandro Dalcin* > > > wrote: > > > > > > I was surprised by this > > > > > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > > > Out[14]: > > > array([[1, 5], > > > [4, 3], > > > [2, 6]]) > > > > > > > > > This one still looks wrong. > > > > > > In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > > > Out[15]: > > > array([[1, 2], > > > [3, 4], > > > [5, 6]]) > > > > > > > > > > > > This one is fixed, > > > > > > In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') > > > Out[3]: > > > array([[1, 4], > > > [2, 5], > > > [3, 6]]) > > > > > > I also don't understand why a copy is returned if 'F' just fiddles > > > with the indices and strides; the underlying data should be the same, > > > just the view changes. FWIW, I think both examples should be returning > > > > > views. > > > > You are right, it doesn't need to. My check is not general enough. > > > > It can be challenging to come up with a general way to differentiate the > > view-vs-copy situation and I struggled with it. In this case, it's the > > fact that while self->nd > 1, the other dimensions are only of shape 1 > > and so don't really matter. If you could come up with some kind of > > striding check that would distinguish the two cases, I would appreciate > > it. > > > I suppose the problem is mostly in discontiguous arrays. Hmmm..., this > isn't too different that reshaping the transpose. > > a.reshape((m,n),order='F' ~ a.reshape((n,m)).T.reshape(m,n) > Erm, brainfart. Make that a.reshape((m,n),order='F') ~ a.reshape((n,m)).T That's how I think of it, anyway. Actual layout in memory is not affected until something like ascontiguosarray is called. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Tue Oct 17 23:25:33 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Tue, 17 Oct 2006 23:25:33 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 17/10/06, Charles R Harris wrote: > > > On 10/17/06, Travis Oliphant wrote: > > Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] > > and then a Fortran-order based fill of an empty (3,2) array: giving you > > the result. > > Why a Fortran ravel? I am thinking of it as preserving the memory layout, > just messing with how it is addressed. Because, to first order, the memory layout is totally irrelevant to a python user. array([[1,2,3],[4,5,6]],order='C') array([[1,2,3],[4,5,6]],order='F') should be nearly identical to the python user. If the storage order mattered, you'd have to know, every time you used reshape, what order your matrix was stored in (did it come from a transpose operation, for example?). As for why a Fortran ravel rather than a C ravel, that's a simplifying decision. If you like, you can think of reshape as happening in two steps: o1='C' o2='C' A.reshape((6,),order=o1).reshape((2,3),order=o2) The design could have allowed, as Travis Oliphant says, o1!=o2, but explaining that would have been quite difficult (even more than now, I mean). And in any case you can use the code above to achieve that, if you really want. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ckkart at hoc.net Wed Oct 18 00:45:33 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Wed, 18 Oct 2006 04:45:33 +0000 (UTC) Subject: sort(axis=-1) Message-ID: Hi, it seems that -1 as axis parameter is interpreted like in array indexing, i.e. -1 means the last axis rather than meaning the only axis of the flattened representation, like for example with take(). Is that intendend? Christian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 00:35:07 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 22:35:07 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <20061018022229.GR17998@mentat.za.net> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> Message-ID: <4535AEFB.4090202@ieee.org> Stefan van der Walt wrote: > One last case, which confuses me still (probably because it is > 04:16am): > > In [41]: x = N.array([[0,1,2],[3,4,5]],order='F') > > In [42]: x > Out[42]: > array([[0, 1, 2], > [3, 4, 5]]) > > I assume the data is now stored in memory as > > [0 3 1 4 2 5] (column-wise) > > If I now do > > x.reshape((3,2),order='C') > > i.e. take that block of memory, assume it is in 'C' order, and make > its shape (3,2), I expect > > [[0 3] > [1 4] > [2 5]] > > but get > > [[1 2] > [3 4] > [5 6]] > > I'm obviously missing something trivial -- I'll try again tomorrow. > I think I see what is going on and where people are getting tripped up. You have to remember, that to NumPy it doesn't semantically matter what the "ordering" of the array is. There is no guarantee that C- order or Fortran-order is *ever* preserved through an operation. Because, in fact the general memory model of the array has no defined "order". It's defined by the strides array. It just so happens that two special-cases are tracked so that we can call out to compiled routines that expect contiguous arrays more easily. So, your mistake is trying to think that the "block" of memory is [0, 3, 1, 4, 2, 5] when you pass the Fortran-order array to the reshape method. While this is true, and it means that you will save a copy if you passed this off to a Fortran routine, the reshape command does not use this information in determining how to "think-about" the input array. In fact, the reshape method does not allow any way to specify the order of the "input" array (self) separately from the order of the output array. The order argument indicates the defined order of both input and output. You might think that the order of self should be used as the order of the input array. The problem with this is, again, that a general array does not have a defined "order". What should be used as the assumed "order" for an un-strided array? You're left with an unresolved question. To avoid two input order arguments, I just let order indicate the order for both the input and the output arrays. We could provide both, but this seems a bit over-done as the same could be accomplished by separately raveling the input to 1-d and then specifying the order argument on reshape. Please continue to question. All the code needs as much review as it can get. Best regards, -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 00:43:29 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 22:43:29 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> Message-ID: <4535B0F1.1000603@ieee.org> > > You are right, it doesn't need to. My check is not general > enough. > > It can be challenging to come up with a general way to > differentiate the > view-vs-copy situation and I struggled with it. In this case, > it's the > fact that while self->nd > 1, the other dimensions are only of > shape 1 > and so don't really matter. If you could come up with some > kind of > striding check that would distinguish the two cases, I would > appreciate it. > > > I suppose the problem is mostly in discontiguous arrays. Hmmm..., > this isn't too different that reshaping the transpose. > > a.reshape((m,n),order='F' ~ a.reshape((n,m)).T.reshape(m,n) > I just committed a change to the check-for-copy code to SVN. A copy will occur if an actual reshaping is taking place that involves more than just adding or deleting ones from the old shape and if 1) self is not "single-segment". Single-segment means either Fortran-order or C-order. If we don't have a single-segment array, then we have to get a (Fortran or C-) contiguous buffer to do any reshaping (there may be special cases when it's possible, but I haven't figure them out...) 2) self is single-segment but self.squeeze().ndim > 1 and it is in the "wrong" order from what was requested in the order argument (i.e. self is in C-contiguous order but a Fortran-order reshape was requested or self is in F-contiguous order but a C-order reshape was requested). If there are any cases satisfying these rules where a copy does not have to occur then let me know. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 00:56:04 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 22:56:04 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 10/17/06, Charles R Harris wrote: > > > > On 10/17/06, A. M. Archibald wrote: > > > > On 17/10/06, Charles R Harris wrote: > > > > > > > > > On 10/17/06, Travis Oliphant wrote: > > Which doesn't seem to be the case here. I am beginning to wonder if we > really need fortran order, seems that a few well chosen interface routines > would fill the need and avoid much confusion. > For instance, it would be nice if flatten took an order keyword: In [107]: array([[1,2,3],[4,5,6]], dtype=int8, order='F').flatten() Out[107]: array([1, 2, 3, 4, 5, 6], dtype=int8) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From strawman at astraw.com Wed Oct 18 02:10:29 2006 From: strawman at astraw.com (Andrew Straw) Date: Tue, 17 Oct 2006 23:10:29 -0700 Subject: recent ubuntu package In-Reply-To: <45353395.40602@gmx.net> References: <45353395.40602@gmx.net> Message-ID: <4535C555.9060004@astraw.com> Sven Schreiber wrote: > Hi, > as suggested on the website I use the kindly provided pre-built > (unofficial) ubuntu debs. Recently there is a new one available with > version numbe 1.0+~dev3336-0ads1. > > Apart from the slightly strange +~ thing in there, it very much seems to > be based on trac changeset 3336, which is somewhere between rc2 and now, > right? > Yes, it is from svn 3336 with one minor patch. The strange +~ is to make the version numbers sort properly. (I didn't think about this when releasing "1.0b2". > And just checking, what are the matplotlib packages that are compatible? > The most recent from the same package source seems to be > 0.87.5-r2781-0ads2, is that ok with the above numpy? Will the mpl ubuntu > package before that break with the latest numpy ubuntu package? > The matplotlib .deb on my website is working fine for me with the latest numpy .deb there. If there are any recent patches or anything you are missing, please let me know -- it's not really a big deal to update them, although it might take a couple of days for me to find the time. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 00:26:17 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 22:26:17 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 10/17/06, A. M. Archibald wrote: > > On 17/10/06, Charles R Harris wrote: > > > > > > On 10/17/06, Travis Oliphant wrote: > > > > Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] > > > and then a Fortran-order based fill of an empty (3,2) array: giving > you > > > the result. > > > > Why a Fortran ravel? I am thinking of it as preserving the memory > layout, > > just messing with how it is addressed. > > Because, to first order, the memory layout is totally irrelevant to a > python user. > > array([[1,2,3],[4,5,6]],order='C') > array([[1,2,3],[4,5,6]],order='F') > > should be nearly identical to the python user. So what is the point of having a fortran layout if things are not actually layed out in fortran order? As far as I can see, memory layout is the only reason for fortran ordering. Now I can sort of see where the fortran ravel comes from, because the result can be passed to a fortran routine. And it does look like a.ravel('F') is the same as a.T.ravel(). Hmmm. Now I wonder about this: In [62]: array([[1,2,3],[4,5,6]], dtype=int8, order='F').flags Out[62]: CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False Now, either CONTIGUOUS is in error, because it really *is* fortran contiguous (but not c contiguous), or the array cannot be passed to a fortran routine that expects fortran ordering, or CONTIGUOUS refers to C addressing, which we already know is not contiguous, in which case we feel uncertain. Note that knowing the answer matters if I want to call a fortran routine with this by pulling out the data pointer. The fortran routine could be in a library, or maybe in the LaPack wrapper, but whatever, it hasn't been wrapped by f2py that takes care of such details. This also looks fishy: In [93]: asfortranarray(array([[1,2,3],[4,5,6]], dtype=int8)) Out[93]: array([[1, 2, 3], [4, 5, 6]], dtype=int8) In [96]: asfortranarray(array([[1,2,3],[4,5,6]], dtype=int8)).flags Out[96]: CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False because the docstring says: asfortranarray(a, dtype=None) Return 'a' as an array laid out in Fortran-order in memory. Which doesn't seem to be the case here. I am beginning to wonder if we really need fortran order, seems that a few well chosen interface routines would fill the need and avoid much confusion. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Wed Oct 18 01:57:30 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 18 Oct 2006 01:57:30 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45358A8D.600@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> Message-ID: On 17/10/06, Travis Oliphant wrote: > Charles R Harris wrote: > > > > In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') > > Out[3]: > > array([[1, 4], > > [2, 5], > > [3, 6]]) > > > > I also don't understand why a copy is returned if 'F' just fiddles > > with the indices and strides; the underlying data should be the same, > > just the view changes. FWIW, I think both examples should be returning > > views. > > You are right, it doesn't need to. My check is not general enough. > > It can be challenging to come up with a general way to differentiate the > view-vs-copy situation and I struggled with it. In this case, it's the > fact that while self->nd > 1, the other dimensions are only of shape 1 > and so don't really matter. If you could come up with some kind of > striding check that would distinguish the two cases, I would appreciate it. I wrote, just for exercise I guess, a routine that checks to see if a copy is needed for a reshape (and if not, it computes the resulting strides) for an arbitrary array. (It currently only does a C-order reshape, but F-order would be an easy addition.) It does, however, successfully handle arbitrarily strided arrays with arbitrary arrangements of "1" dimensions having arbitrary strides. I think it also correctly handles 0 strides. I don't imagine you'd want to *use* it, but it does provide a complete solution to copy-less reshaping. I'd be happy to help with the C implementation built into numpy, but for the life of me I can't figure out how it works. A. M. Archibald P.S. I haven't written down a *proof* that this implementation never copies unnecessarily, but I'm pretty sure that it doesn't. It has a reshape_restricted that effectively does a ravel() first; this can't cope with things like length (5,2,3) -> length (5,3,2) unless the strides are conveniently equal, so there's a wrapper, reshape(), that breaks the reshape operation up into pieces that can be done by reshape_restricted. -AMA -------------- next part -------------- A non-text attachment was scrubbed... Name: stride.py Type: text/x-python Size: 2234 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Wed Oct 18 03:35:17 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 01:35:17 -0600 Subject: NumPy 1.0rc3 will be out in a few hours Message-ID: <4535D935.2000604@ieee.org> I'm going to release NumPy 1.0rc3 in a few hours. It requires a re-build of extension modules from 1.0rc2. This gives us a chance to start fresh with NumPy 1.0 and also test the non-dying version of the version_checking code. So, it's not all bad. Official NumPy 1.0 will be released on Oct. 24 to many tears of rejoicing... Please file all tickets / requests by this weekend if you hope to have them included in the 1.0 release. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From GRGE at HOTMAIL.COM Wed Oct 18 05:13:16 2006 From: GRGE at HOTMAIL.COM (=?iso-2022-jp?B?R1JHRQ==?=) Date: Wed, 18 Oct 2006 02:13:16 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCPlA0aSROQUdFKCRKPXcbKEI=?= =?iso-2022-jp?b?GyRCJE47UhsoQg==?= Message-ID: ??????????????? http://hlgliu.jeeran.com/egao ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nrvumzt at us.univision.com Wed Oct 18 02:15:40 2006 From: nrvumzt at us.univision.com (enough) Date: Wed, 18 Oct 2006 08:15:40 +0200 Subject: Italy Message-ID: <001101c6f27c$d602ebe0$13b73850@thuis45vfps1la> Decides pull century achieve sets finding someone bankroll is enterprise person Coward big am syndicate boss still running a. Sorted shutup no talks anymore meeting leader threatens is kill pulloff responds will making a grave. Movies past yearsi give stars acting of driving stunts fiction life own report Enjoyable of genial picture am Gomez Pardo. Shipped collateral or Fiat plant Since Italygreat Britain soccer in match being played at same time wants am create diversion is form monumental? Registry Free in ecardsyour Media Your or Accounts is all Product Categories Account in Cart Lists is. Swinger swindler an ingenious huge cache Chinese is be shipped collateral? Beckley is Color Hifi Sound Original recording reissued or English a tapes of Studio Release Date October run a Time Review a based on? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Graham.png Type: image/png Size: 726 bytes Desc: not available URL: From david at ar.media.kyoto-u.ac.jp Wed Oct 18 06:59:16 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 18 Oct 2006 19:59:16 +0900 Subject: questions regarding assignement and copy Message-ID: <45360904.3030703@ar.media.kyoto-u.ac.jp> Hi there, I've just managed to nail down a bug which took me nearly two whole days to find: this is coming from an unexpected (at least from me) behaviour of numpy. I understand that if foo is a numpy array, doing bar = foo makes no copy, and whenever you change the value of foo, you change the value of bar: import numpy as N foo = N.linspace(0, 4, 5) bar = foo print foo[1] bar[1] = -1 print foo[1] => prints 1 and -1. Now, if I do: bar += 1 print bar is foo prints True But if I do bar = bar + 1, then bar is not a copy of foo anymore. Is this intended ? This looks really confusing to me, and I would like to know what the precise rules about copy vs alias are ? Cheers, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Wed Oct 18 07:47:14 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Wed, 18 Oct 2006 13:47:14 +0200 Subject: questions regarding assignement and copy In-Reply-To: <45360904.3030703@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> Message-ID: <45361442.8040400@gmx.net> David Cournapeau schrieb: > Hi there, > > I've just managed to nail down a bug which took me nearly two whole > days to find: this is coming from an unexpected (at least from me) > behaviour of numpy. You have all my sympathy, I tripped over something similar not too long ago, so welcome to the club. > Now, if I do: > > bar += 1 > print bar is foo > > prints True > > But if I do bar = bar + 1, then bar is not a copy of foo anymore. Is > this intended ? This looks really confusing to me, and I would like to > know what the precise rules about copy vs alias are ? > Yes it's intended; as far as I understand the python/numpy syntax, <+> is an operator, and that triggers assignment by copy (even if you do something trivial as bar = +foo, you get a copy, if I'm not mistaken), while <+=> is syntactically not really an operator. AFAIK it's sole purpose is really to _not_ make a copy to save memory (well, it also looks nice ;-). You'll need somebody else to tell you the _precise_ rules, though... -sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Wed Oct 18 08:04:40 2006 From: faltet at carabos.com (Francesc Altet) Date: Wed, 18 Oct 2006 14:04:40 +0200 Subject: Question about boolean types comparisons Message-ID: <1161173081.3974.6.camel@localhost.localdomain> Hi, I've seen that bool type in numpy seems to work more consistently than its python counterpart: In [28]:True+True Out[28]:2 In [29]:numpy.bool_(True)+numpy.bool_(True) Out[29]:True Good! but... In [30]:True > False Out[30]:True In [31]:numpy.bool_(True) > numpy.bool_(False) Out[31]:True Perhaps raising an error saying something like "boolean types cannot be compared" would be nice. Not too important, but worth to notice, IMO. -- >0,0< Francesc Altet | Be careful about using the following code -- V V Carabos Coop. V. | I've only proven that it works, "-" Enjoy Data | I haven't tested it. -- Donald Knuth ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Wed Oct 18 08:04:50 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 18 Oct 2006 21:04:50 +0900 Subject: questions regarding assignement and copy In-Reply-To: <45361442.8040400@gmx.net> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> Message-ID: <45361862.6040801@ar.media.kyoto-u.ac.jp> Sven Schreiber wrote: > > Yes it's intended; as far as I understand the python/numpy syntax, <+> > is an operator, and that triggers assignment by copy (even if you do > something trivial as bar = +foo, you get a copy, if I'm not mistaken), > So basically, whenever you have foo = expr with expr is a numpy expression containing foo, you trigger a copy ? David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Wed Oct 18 09:50:45 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Wed, 18 Oct 2006 15:50:45 +0200 Subject: questions regarding assignement and copy In-Reply-To: <45361862.6040801@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> <45361862.6040801@ar.media.kyoto-u.ac.jp> Message-ID: <45363135.40200@gmx.net> David Cournapeau schrieb: > Sven Schreiber wrote: >> Yes it's intended; as far as I understand the python/numpy syntax, <+> >> is an operator, and that triggers assignment by copy (even if you do >> something trivial as bar = +foo, you get a copy, if I'm not mistaken), >> > So basically, whenever you have > > foo = expr > > with expr is a numpy expression containing foo, you trigger a copy ? > David, I would tend to confirm your conjecture, but given its generality I should not pretend to really know all about that stuff. It could be, for example, that the user may specify inside the expression that she wants a view, not a copy, But I don't know if that's really possible. Anybody else available to help out? -sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 18 10:30:02 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 18 Oct 2006 10:30:02 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 18/10/06, Charles R Harris wrote: > > > On 10/17/06, A. M. Archibald wrote: > > On 17/10/06, Charles R Harris wrote: > > > > > > > > > On 10/17/06, Travis Oliphant wrote: > > > > > > Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] > > > > and then a Fortran-order based fill of an empty (3,2) array: giving > you > > > > the result. > > > > > > Why a Fortran ravel? I am thinking of it as preserving the memory > layout, > > > just messing with how it is addressed. > > > > Because, to first order, the memory layout is totally irrelevant to a > > python user. > > > > array([[1,2,3],[4,5,6]],order='C') > > array([[1,2,3],[4,5,6]],order='F') > > > > should be nearly identical to the python user. > > So what is the point of having a fortran layout if things are not actually > layed out in fortran order? As far as I can see, memory layout is the only > reason for fortran ordering. Now I can sort of see where the fortran ravel > comes from, because the result can be passed to a fortran routine. And it > does look like a.ravel('F') is the same as a.T.ravel(). Hmmm. Now I wonder > about this: Thins are laid out in Fortran order if you request Fortran order upon array creation. You just can't see it, normally. Numpy tries hard to make arrays of all different orders look identical. Of course, the reason I said "to first order" is that when determining when to copy and when not to, or when passing arrays to C or Fortran functions, sometimes it does matter. For example, a typical Fortran linear algebra routine needs its arrays to be in Fortran order in memory. > In [62]: array([[1,2,3],[4,5,6]], dtype=int8, order='F').flags > Out[62]: > CONTIGUOUS : False > FORTRAN : True > OWNDATA : True > WRITEABLE : True > ALIGNED : True > UPDATEIFCOPY : False Unless you're working with C extensions, it doesn't make much sense to ever look at flags. The memory layout does not affect normal array behaviour (including reshape). A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 18 10:55:26 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 18 Oct 2006 10:55:26 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4535B0F1.1000603@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> <4535B0F1.1000603@ieee.org> Message-ID: On 18/10/06, Travis Oliphant wrote: > > I just committed a change to the check-for-copy code to SVN. A copy > will occur if an actual reshaping is taking place that involves more > than just adding or deleting ones from the old shape and if > > 1) self is not "single-segment". Single-segment means either > Fortran-order or C-order. If we don't have a single-segment array, then > we have to get a (Fortran or C-) contiguous buffer to do any reshaping > (there may be special cases when it's possible, but I haven't figure > them out...) Indeed it is often possible to reshape an array that is not contiguous; for example zeros(100)[::10] is not contiguous but can be reshaped to any shape without any copying. More generally, there are all sorts of peculiar arrangements that can be reshaped witout a copy, for example: input has lengths (8,2,3) and strides (39,9,3); output should have lengths (4,4,3,2), so take strides (156,39,6,3) Whether this sort of thing actually *occurs* very often, I don't know... > 2) self is single-segment but self.squeeze().ndim > 1 and it is in the > "wrong" order from what was requested in the order argument (i.e. self > is in C-contiguous order but a Fortran-order reshape was requested or > self is in F-contiguous order but a C-order reshape was requested). > > If there are any cases satisfying these rules where a copy does not have > to occur then let me know. There are. I came up with a general condition for describing exactly when you need to copy an array, but it's difficult to put into words. Here goes: (0) Dimensions of length 1 are an irrelevant annoyance. Conceptually they should be squeezed out before beginning and newaxised back in at the end. (What strides should they have?) Assume that I've done so from now on. (Actual code can just skip them appropriately.) (1) If the array looks like an evenly-strided 1D array that has been reshaped (so strides[i+-1]=lengths[i]*strides[i] for all i) it can always be reshaped without a copy. (For example, zeros(1000)[::10] can be reshaped arbitrarily and as many times as desired without copying.) (2) If the array does *not* look like an evenly-strided 1D array that has been reshaped, you can still reshape it without a copy if it looks like an array of such arrays, each of which can be reshaped separately. (For example, you can view an array of length (8,2,3) that you have to resize to an array of size (4,4,3,2) as two separate pieces: an array of size (2,3) that you have to resize to (3,2) and an array of size (8,) that you need to resize to size (4,4). Each of these pieces separately needs to look like a reshaped 1d array, but there need be no relation between them.) (The condition for when you can pull a smaller resize operation off the end is precisely when you can find a tail segment of each tuple that has the same product; in the case above, 2*3=3*2, so we could stop and do that reshape separately.) (3) If the array cannot be reshaped without a copy using the rules above, it cannot be reshaped without a copy at all. The python code in my previous message actually implements this rule, if you want a less vague description. I should also point out that views are not always more efficient than copies (because of cache concerns, and because a small view can prevent a giant block of memory from being deallocated); nevertheless it should be up to the user to copy when it helps. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 18 11:08:15 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 18 Oct 2006 11:08:15 -0400 Subject: questions regarding assignement and copy In-Reply-To: <45361862.6040801@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> <45361862.6040801@ar.media.kyoto-u.ac.jp> Message-ID: On 18/10/06, David Cournapeau wrote: > Sven Schreiber wrote: > > > > Yes it's intended; as far as I understand the python/numpy syntax, <+> > > is an operator, and that triggers assignment by copy (even if you do > > something trivial as bar = +foo, you get a copy, if I'm not mistaken), > > > So basically, whenever you have > > foo = expr > > with expr is a numpy expression containing foo, you trigger a copy ? No. "=" never copies, but "+" does: when you do "A+B" you produce a new, freshly-allocated array, which you can then store (a reference to) in any variable you like, including A or B. So M = M+1 makes a copy because "M+1" is a new matrix, which you are assigning to M. Unfortunately, this means if you do: M = 2*(M+1) python makes the new matrix M+1, then makes the new matrix 2*(M+1), discards M+1, and sets M to point to 2*(M+1). If you want to avoid all this copying, you can use python's in-place operators: M += 1 M *= 2 This is actually a standard difficulty people have with python, made more obvious because you're working with mutable arrays rather than immutable scalars. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From rays at blue-cove.com Wed Oct 18 12:11:04 2006 From: rays at blue-cove.com (Ray Schumacher) Date: Wed, 18 Oct 2006 09:11:04 -0700 Subject: No forums found for Numerical Python Message-ID: <6.2.3.4.2.20061018090850.05a362e0@blue-cove.com> FYI: at http://numpy.org/, the link "Forums" (http://sourceforge.net/forum/?group_id=1369) gives No forums found for Numerical Python ! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 18 12:38:32 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 10:38:32 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4535AEFB.4090202@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> Message-ID: On 10/17/06, Travis Oliphant wrote: > > Stefan van der Walt wrote: > > One last case, which confuses me still (probably because it is > > 04:16am): > > Please continue to question. All the code needs as much review as it > can get. I am really starting to wonder why we need an order keyword at all except when order matters for interfacing, i.e., in contiguos arrays where one wants fortran contiguous or c contiguous. For output type stuff: tofile tostring ravel flatten These all need the keyword so that they can be used to produce files and arrays to interface to fortran. Right now the fortran interface can be achieved by: a.T.tofile a.T.tostring a.T.ravel a.T.flatten The order keyword is just syntatic sugar that makes the intent clearer. For the input type stuff fromfile fromstring reshape-1d-array (unravel?) Currently, the key operation is reshape, which only needs to return a view in fortran order and doesn't even need to mark the resulting array as fortran order because, well, because it works just fine in numpy as is, it just isn't contiguous. If the other functions took shape and order, reshape wouldn't even need the order keyword. I don't see why the array constructor needs the order keyword, it doesn't *do* anything. For instance a = array([[1,2,3],[4,5,6]], order='F') doesn't produce a fortran contiguous array, it produces the same array as the 'C' form, just sets the fortran flag and marks contiguous as False. What is the use of that? It is just a generic non-contiguous numpy array. And In [131]: ascontiguousarray(array([[1,2,3],[4,5,6]], dtype=int8, order='F')).flags Out[131]: CONTIGUOUS : True FORTRAN : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False Doesn't produce a fortran contiguous array, so what use was the flag? And In [141]: array([1,2,3,4,5,6], dtype=int8).reshape((2,3), order='F').astype(int16).flags Out[141]: CONTIGUOUS : True FORTRAN : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False reorders stuff in memory, so is a bug looking to happen in a fortran interface. mmapped files are the only thing I can think of where one might want vary an operation depending on Fortran ordering because seeking out of order is very expensive. But that means adapting algorithms depending on order type, better I think to just stick to using the small strided dimensions when appropriate. It would be helpful in debugging all this order stuff if it was clear what was supposed to happen in every case. Ravel, for instance, ignores the FORTRAN flag, again begging the question as to why we *have* the flag. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 18 12:49:38 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 10:49:38 -0600 Subject: questions regarding assignement and copy In-Reply-To: <45361862.6040801@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> <45361862.6040801@ar.media.kyoto-u.ac.jp> Message-ID: On 10/18/06, David Cournapeau wrote: > > Sven Schreiber wrote: > > > > Yes it's intended; as far as I understand the python/numpy syntax, <+> > > is an operator, and that triggers assignment by copy (even if you do > > something trivial as bar = +foo, you get a copy, if I'm not mistaken), > > > So basically, whenever you have > > foo = expr > > with expr is a numpy expression containing foo, you trigger a copy ? Yes. Numpy generates a temporary where the expression results are kept, the temporary is then assigned to foo. This can be inefficient, but knowing when you may overwrite data can be complicated. For instance: foo = dot(foo,foo) Can't be done in place because values needed later on in the dot evaluation would be overwritten. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Wed Oct 18 12:55:51 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 18 Oct 2006 11:55:51 -0500 Subject: sort(axis=-1) In-Reply-To: References: Message-ID: Christian Kristukat wrote: > Hi, > > it seems that -1 as axis parameter is interpreted like in array indexing, i.e. > -1 means the last axis rather than meaning the only axis of the flattened > representation, like for example with take(). > Is that intendend? take(axis=-1) also goes over the last axis. axis=None operates over the flattened array. axis=None is the default for take(), but axis=-1 is default for sort() since axis=None doesn't make much sense for an inplace operation and is not permitted. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 13:11:25 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 11:11:25 -0600 Subject: sort(axis=-1) In-Reply-To: References: Message-ID: <4536603D.7070103@ieee.org> Christian Kristukat wrote: > Hi, > > it seems that -1 as axis parameter is interpreted like in array indexing, i.e. > -1 means the last axis rather than meaning the only axis of the flattened > representation, like for example with take(). > Is that intendend? > > I don't understand the question. Yes, sort(axis=-1) sorts along the last dimension. AFAIK, take has the same meaning for axis. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Wed Oct 18 13:12:37 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 18 Oct 2006 12:12:37 -0500 Subject: questions regarding assignement and copy In-Reply-To: <45360904.3030703@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> Message-ID: David Cournapeau wrote: > bar += 1 > print bar is foo > > prints True > > But if I do bar = bar + 1, then bar is not a copy of foo anymore. Is > this intended ? Yes, very much so! If it were otherwise Guido would never have added the augmented assignment operators. From the reference manual: http://docs.python.org/ref/augassign.html """An augmented assignment expression like x += 1 can be rewritten as x = x + 1 to achieve a similar, but not exactly equal effect. In the augmented version, x is only evaluated once. Also, when possible, the actual operation is performed in-place, meaning that rather than creating a new object and assigning that to the target, the old object is modified instead.""" > This looks really confusing to me, and I would like to > know what the precise rules about copy vs alias are ? I'm not going to go through all of the functions in numpy, but here are some (unorganized) rules of thumb: * Any operations with regular operators + - / * // ** ~ ^ & | << >> < > == <=, etc. will put their results into new arrays. If the expression is on the RHS of a regular assignment, just ignore the LHS of the assignment; even if the name on the LHS is the same as a name on the RHS, the expression is evaluated, the new array is created and *then* the new array is assigned to the LHS name. * Augmented assignment operators will operate in-place. * Functions and methods should state that they operate in-place in their docstrings. * The asarray(), asanyarray(), etc. functions will return the input array if it is already suitable. * Slicing, reshaping, transposing and similar structural operations create new array objects, but they are generally views, not copies of the data. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 13:16:17 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 11:16:17 -0600 Subject: questions regarding assignement and copy In-Reply-To: <45361862.6040801@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> <45361862.6040801@ar.media.kyoto-u.ac.jp> Message-ID: <45366161.8000607@ieee.org> David Cournapeau wrote: > Sven Schreiber wrote: > >> Yes it's intended; as far as I understand the python/numpy syntax, <+> >> is an operator, and that triggers assignment by copy (even if you do >> something trivial as bar = +foo, you get a copy, if I'm not mistaken), >> >> > So basically, whenever you have > > foo = expr > > with expr is a numpy expression containing foo, you trigger a copy ? > I think you are better off understanding that "=" is a name binding operation while "+=" calls a special method that allows in-place adding. Thus, bar += foo calls bar.__iadd__(bar, foo) which gives the opportunity to add foo to bar in-place while bar = bar + foo adds bar to foo (which results in a new array that then gets re-bound to the name bar). The '=' sign is not an operator. Perhaps this will help you see the difference. It's good to know what kinds of things trip people up. We can target tutorials and FAQs to those things. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 13:19:08 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 11:19:08 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: <4536620C.6090903@ieee.org> Charles R Harris wrote: > > > On 10/17/06, *Charles R Harris* > wrote: > > > > On 10/17/06, *A. M. Archibald* < peridot.faceted at gmail.com > > wrote: > > On 17/10/06, Charles R Harris > wrote: > > > > > > On 10/17/06, Travis Oliphant < oliphant.travis at ieee.org > > wrote: > > > > > Which doesn't seem to be the case here. I am beginning to wonder > if we really need fortran order, seems that a few well chosen > interface routines would fill the need and avoid much confusion. > > > For instance, it would be nice if flatten took an order keyword: > > In [107]: array([[1,2,3],[4,5,6]], dtype=int8, order='F').flatten() > Out[107]: array([1, 2, 3, 4, 5, 6], dtype=int8) It does take an argument (just not a keyword argument). The general rule I followed (probably inappropriately) was that single-argument methods didn't need keywords. so a.flatten('F') gives you a Fortran-order flattening. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Wed Oct 18 13:15:32 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 18 Oct 2006 12:15:32 -0500 Subject: No forums found for Numerical Python In-Reply-To: <6.2.3.4.2.20061018090850.05a362e0@blue-cove.com> References: <6.2.3.4.2.20061018090850.05a362e0@blue-cove.com> Message-ID: Ray Schumacher wrote: > FYI: > at http://numpy.org/, the link "Forums" ( > http://sourceforge.net/forum/?group_id=1369) gives > > *No forums found for Numerical Python* > > ! Yes, I closed them since people would go there for help, but the people that *could* help don't read the forums. If I knew how to make the Forums link go away on the project page, I would do so. If you have questions about numpy (or even Numeric and numarray), this is the place. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 13:36:02 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 11:36:02 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> Message-ID: <45366602.4090704@ieee.org> > > Currently, the key operation is reshape, which only needs to return a > view in fortran order and doesn't even need to mark the resulting > array as fortran order because, well, because it works just fine in > numpy as is, it just isn't contiguous. If the other functions took > shape and order, reshape wouldn't even need the order keyword. The flag is the there as a quick check for interfacing. The order keyword grew because it was useful to avoid the arbitrariness of C-contiguous order for those who prefer to think of it differently. Remember the .T attribute for .transpose() was a recent addition and sticking .transpose() everywhere is a lot more ugly. But, yes, many uses of the order keyword could be replaced by preceding with .transpose() --- this is not without cost, however. > > I don't see why the array constructor needs the order keyword, it > doesn't *do* anything. For instance > > a = array([[1,2,3],[4,5,6]], order='F') > > doesn't produce a fortran contiguous array, it produces the same array > as the 'C' form, just sets the fortran flag and marks contiguous as > False. What is the use of that? It is just a generic non-contiguous > numpy array. What? You're not understanding something. The order flag definitely does something here. First of all it seems like you are not understanding the meaning of the CONTIGUOUS flag. CONTIGUOUS means "C-order contiguous" while FORTRAN means "FORTRAN-order contiguous". That's why I use the word single-segment to talk about FORTRAN-order or C-contiguous order. For Numeric, CONTIGUOUS always meant C-order contiguous and we are continuing that tradition. All we've done is notice that there is such a think as FORTRAN-order contiguous and copies do not need to be made in all circumstances when you have FORTRAN-order. Look at the difference between: a = array([[1,2,3],[4,5,6]],order='F').data[:] b = array([[1,2,3],[4,5,6]]).data[:] Notice the layout is definitely different between a and b. > And > > In [131]: ascontiguousarray(array([[1,2,3],[4,5,6]], dtype=int8, > order='F')).flags > Out[131]: > CONTIGUOUS : True > FORTRAN : False > OWNDATA : True > WRITEABLE : True > ALIGNED : True > UPDATEIFCOPY : False > > Doesn't produce a fortran contiguous array, so what use was the flag? And Because you requested a C-contiguous array --- that's what contiguous means in NumPy (exactly what it meant in Numeric). > > In [141]: array([1,2,3,4,5,6], dtype=int8).reshape((2,3), > order='F').astype(int16).flags > Out[141]: > CONTIGUOUS : True > FORTRAN : False > OWNDATA : True > WRITEABLE : True > ALIGNED : True > UPDATEIFCOPY : False > > reorders stuff in memory, so is a bug looking to happen in a fortran > interface. Yes, like I said before, all kinds of operations alter the "layout" of data. You can't assume all operations will preserve FORTRAN ordering. FORTRAN-order has meaning beyond how the data is actually set out in memory. Sometimes it indicates how you think it is layed out when you are doing re-shaping operations. > > mmapped files are the only thing I can think of where one might want > vary an operation depending on Fortran ordering because seeking out of > order is very expensive. But that means adapting algorithms depending > on order type, better I think to just stick to using the small strided > dimensions when appropriate. > > It would be helpful in debugging all this order stuff if it was clear > what was supposed to happen in every case. Ravel, for instance, > ignores the FORTRAN flag, again begging the question as to why we > *have* the flag. No it doesn't. Please show your evidence. Look: a = array([[1,2,3],[4,5,6]]) print a.ravel() [1 2 3 4 5 6] print a.ravel('F') [1 4 2 5 3 6] If it's not working in some cases, please report that as a bug. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 13:47:59 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 11:47:59 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45366602.4090704@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> Message-ID: On 10/18/06, Travis Oliphant wrote: > > > > > > Currently, the key operation is reshape, which only needs to return a > > view in fortran order and doesn't even need to mark the resulting > > array as fortran order because, well, because it works just fine in > > numpy as is, it just isn't contiguous. If the other functions took > > shape and order, reshape wouldn't even need the order keyword. > The flag is the there as a quick check for interfacing. The order > keyword grew because it was useful to avoid the arbitrariness of > C-contiguous order for those who prefer to think of it differently. > Remember the .T attribute for .transpose() was a recent addition and > sticking .transpose() everywhere is a lot more ugly. But, yes, many > uses of the order keyword could be replaced by preceding with > .transpose() --- this is not without cost, however. > > > > > I don't see why the array constructor needs the order keyword, it > > doesn't *do* anything. For instance > > > > a = array([[1,2,3],[4,5,6]], order='F') > > > > doesn't produce a fortran contiguous array, it produces the same array > > as the 'C' form, just sets the fortran flag and marks contiguous as > > False. What is the use of that? It is just a generic non-contiguous > > numpy array. > > What? You're not understanding something. The order flag definitely > does something here. First of all it seems like you are not > understanding the meaning of the CONTIGUOUS flag. CONTIGUOUS means > "C-order contiguous" while FORTRAN means "FORTRAN-order contiguous". > That's why I use the word single-segment to talk about FORTRAN-order or > C-contiguous order. For Numeric, CONTIGUOUS always meant C-order > contiguous and we are continuing that tradition. All we've done is > notice that there is such a think as FORTRAN-order contiguous and copies > do not need to be made in all circumstances when you have FORTRAN-order. > > Look at the difference between: > > a = array([[1,2,3],[4,5,6]],order='F').data[:] > > b = array([[1,2,3],[4,5,6]]).data[:] > > Notice the layout is definitely different between a and b. > > > And > > > > In [131]: ascontiguousarray(array([[1,2,3],[4,5,6]], dtype=int8, > > order='F')).flags > > Out[131]: > > CONTIGUOUS : True > > FORTRAN : False > > OWNDATA : True > > WRITEABLE : True > > ALIGNED : True > > UPDATEIFCOPY : False > > > > Doesn't produce a fortran contiguous array, so what use was the flag? > And > > Because you requested a C-contiguous array --- that's what contiguous > means in NumPy (exactly what it meant in Numeric). > > > > > In [141]: array([1,2,3,4,5,6], dtype=int8).reshape((2,3), > > order='F').astype(int16).flags > > Out[141]: > > CONTIGUOUS : True > > FORTRAN : False > > OWNDATA : True > > WRITEABLE : True > > ALIGNED : True > > UPDATEIFCOPY : False > > > > reorders stuff in memory, so is a bug looking to happen in a fortran > > interface. > > Yes, like I said before, all kinds of operations alter the "layout" of > data. You can't assume all operations will preserve FORTRAN ordering. > FORTRAN-order has meaning beyond how the data is actually set out in > memory. Sometimes it indicates how you think it is layed out when you > are doing re-shaping operations. > > > > > mmapped files are the only thing I can think of where one might want > > vary an operation depending on Fortran ordering because seeking out of > > order is very expensive. But that means adapting algorithms depending > > on order type, better I think to just stick to using the small strided > > dimensions when appropriate. > > > > It would be helpful in debugging all this order stuff if it was clear > > what was supposed to happen in every case. Ravel, for instance, > > ignores the FORTRAN flag, again begging the question as to why we > > *have* the flag. > No it doesn't. Please show your evidence. Look: > > a = array([[1,2,3],[4,5,6]]) > > print a.ravel() > [1 2 3 4 5 6] > > print a.ravel('F') > [1 4 2 5 3 6] I'm not talking about the keyword in the ravel call, I'm talking about the flag in a. The question is: do we *need* a fortran flag. I am argueing not, because the only need is for fortran contiguous arrays to pass to fortran function, or translation from fortran contiguous arrays to numpy arrays. What I am saying is that things are unnecessarily complicated. None of the LaPack stuff seems to use the Fortran stuff, they just transpose and copy. I don't even think I want to change that, because it is *clear* what is going on. Interfacing to fortran is all about memory layout, nothing more or less. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Wed Oct 18 13:51:04 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 18 Oct 2006 10:51:04 -0700 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> Message-ID: <45366988.5020204@ieee.org> One thing that may be confusing the issue is that, as I understand it, FORTRAN and CONTIGUOUS together represent three states which I'll call FORTRAN_ORDER, C_ORDER and DISCONTIGUOUS. I periodically wonder if it would be valuable to have a way to query the order directly: the result would be "C", "F" or None, just like the order keyword that is passed in. This might well eliminate sine confusion. However, 99% of the time the order just doesn't matter, so it's probably pointless. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Wed Oct 18 13:58:23 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 18 Oct 2006 10:58:23 -0700 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> Message-ID: <45366B3F.2050103@ieee.org> Charles R Harris wrote: [SNIP] > > I'm not talking about the keyword in the ravel call, I'm talking about > the flag in a. The question is: do we *need* a fortran flag. I am > argueing not, because the only need is for fortran contiguous arrays > to pass to fortran function, or translation from fortran contiguous > arrays to numpy arrays. What I am saying is that things are > unnecessarily complicated. None of the LaPack stuff seems to use the > Fortran stuff, they just transpose and copy. I don't even think I want > to change that, because it is *clear* what is going on. Interfacing to > fortran is all about memory layout, nothing more or less. > Chuck, There are two things here. One is the order keyword and one is the FORTRAN flag. The latter is mainly an optimization for use at the C-level so that one doesn't have to check whether a given array is in contiguous FORTRAN order by examining the strides, in the same way that the CONTIGUOUS flag allows you to skip examining the strides when you need a contiguous C-order matrix. I believe it is the former that you are objecting to, but it would help if you could specify whether you are talking about the order keyword or whether you are talking about the FORTRAN flag. I'll also note that the order keyword could probably have been used to fix a performance problem someone was having a few weeks ago. We ended up transposing the data, but the individual felt that obscured the intent of the algorithm. I believe the same effect could probably have been been achieved without re jiggering the algorithm by using the order parameter. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From erin.sheldon at gmail.com Wed Oct 18 14:00:20 2006 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Wed, 18 Oct 2006 14:00:20 -0400 Subject: histogram complete makeover In-Reply-To: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> References: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> Message-ID: <331116dc0610181100k474af71akac41e54db7b7bb41@mail.gmail.com> On 10/17/06, David Huard wrote: > Hi all, > > I'd like to poll the list to see what people want from numpy.histogram(), > since I'm currently writing a contender. > > My main complaints with the current version are: > 1. upper outliers are stored in the last bin, while lower outliers are not > counted at all, > 2. cannot use weights. > > The new histogram function is well under way (it address these issues and > adds an axis keyword), > but I want to know what is the preferred behavior regarding the function > output, and your > willingness to introduce a new behavior that will break some code. > > Given a number of bins N and range (min, max), histogram constructs linearly > spaced bin edges > b0 (out-of-range) | b1 | b2 | b3 | .... | bN | bN+1 out-of-range > and may return: > > A. H = array([N_b0, N_b1, ..., N_bN, N_bN+1]) > The out-of-range values are the first and last values of the array. The > returned array is hence N+2 > > B. H = array([N_b0 + N_b1, N_b2, ..., N_bN + N_bN+1]) > The lower and upper out-of-range values are added to the first and last bin > respectively. > > C. H = array([N_b1, ..., N_bN + N_bN+1]) > Current behavior: the upper out-of-range values are added to the last bin. > > D. H = array([N_b1, N_b2, ..., N_bN]), > Lower and upper out-of-range values are given after the histogram array. > > Ideally, the new function would not break the common usage: H = > histogram(x)[0], so this exclude A. B and C are not acceptable in my > opinion, so only D remains, with the downsize that the outliers are not > returned. A solution might be to add a keyword full_output=False, which when > set to True, returns the out-of-range values in a dictionnary. > > Also, the current function returns -> H, ledges > where ledges is the array of left bin edges (N). > I propose returning the complete array of edges (N+1), including the > rightmost edge. This is a little bit impractical for plotting, as the edges > array does not have the same length as the histogram array, but allows the > use of user-defined non-uniform bins. > > Opinions, suggestions ? I dislike the current behavior. I don't want the histogram to count anything outside the range I specify. It would also be nice to allow specification of a binsize which would be used if number of bins wasn't sent. Personally, since I don't have any code yet that uses histogram, I feel like edges could be returned in a keyword. Perhaps in a dictionary with other useful items, such as bin middles, mean of the data in bins and other statistics, or whatever, which would only be calculated if the keyword dict was sent. Hopefully Google and sourceforge are playing nice and you will see this within a day of sending. Erin ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 14:08:07 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 12:08:07 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45366B3F.2050103@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <45366B3F.2050103@ieee.org> Message-ID: On 10/18/06, Tim Hochberg wrote: > > Charles R Harris wrote: > > [SNIP] > > > > I'm not talking about the keyword in the ravel call, I'm talking about > > the flag in a. The question is: do we *need* a fortran flag. I am > > argueing not, because the only need is for fortran contiguous arrays > > to pass to fortran function, or translation from fortran contiguous > > arrays to numpy arrays. What I am saying is that things are > > unnecessarily complicated. None of the LaPack stuff seems to use the > > Fortran stuff, they just transpose and copy. I don't even think I want > > to change that, because it is *clear* what is going on. Interfacing to > > fortran is all about memory layout, nothing more or less. > > > > Chuck, > > There are two things here. One is the order keyword and one is the > FORTRAN flag. The latter is mainly an optimization for use at the > C-level so that one doesn't have to check whether a given array is in > contiguous FORTRAN order by examining the strides, in the same way that > the CONTIGUOUS flag allows you to skip examining the strides when you > need a contiguous C-order matrix. That sounds like the two flags should be named f-contiguous and c-contiguous. Then they would be orthogonal and one could have all four combinations. Is that the case now? Perhaps I am misunderstanding the meaning of the flags. I believe it is the former that you > are objecting to, but it would help if you could specify whether you are > talking about the order keyword or whether you are talking about the > FORTRAN flag. Both. I was argueing against the FORTRAN flag, and of limiting the order keyword to those cases where f or c contiguous arrays were the output or input. I'll also note that the order keyword could probably have been used to > fix a performance problem someone was having a few weeks ago. We ended > up transposing the data, but the individual felt that obscured the > intent of the algorithm. I believe the same effect could probably have > been been achieved without re jiggering the algorithm by using the order > parameter. Some more details would be helpful. It would be good to know what problem the order keyword should solve. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Wed Oct 18 14:27:38 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 18 Oct 2006 11:27:38 -0700 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <45366B3F.2050103@ieee.org> Message-ID: <4536721A.4050104@ieee.org> Charles R Harris wrote: > > > On 10/18/06, *Tim Hochberg* > wrote: > > Charles R Harris wrote: > > [SNIP] > > > > I'm not talking about the keyword in the ravel call, I'm talking > about > > the flag in a. The question is: do we *need* a fortran flag. I am > > argueing not, because the only need is for fortran contiguous > arrays > > to pass to fortran function, or translation from fortran contiguous > > arrays to numpy arrays. What I am saying is that things are > > unnecessarily complicated. None of the LaPack stuff seems to use > the > > Fortran stuff, they just transpose and copy. I don't even think > I want > > to change that, because it is *clear* what is going on. > Interfacing to > > fortran is all about memory layout, nothing more or less. > > > > Chuck, > > There are two things here. One is the order keyword and one is the > FORTRAN flag. The latter is mainly an optimization for use at the > C-level so that one doesn't have to check whether a given array is in > contiguous FORTRAN order by examining the strides, in the same way > that > the CONTIGUOUS flag allows you to skip examining the strides when you > need a contiguous C-order matrix. > > > That sounds like the two flags should be named f-contiguous and > c-contiguous. Then they would be orthogonal and one could have all > four combinations. Is that the case now? Perhaps I am misunderstanding > the meaning of the flags. That is the case now. The flag names simply mirror their values in C. Why they have those names in something of a historical accident I believe. Take a look at this: >>> array([1,2,3,4]).flags CONTIGUOUS : True FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False >>> array([1,2,3,4])[::2].flags CONTIGUOUS : False FORTRAN : False OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False >>> array([[1,2],[3,4]]).flags CONTIGUOUS : True FORTRAN : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False >>> array([[1,2],[3,4]], order='F').flags CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False See, all four combinations. I guess my previous post was wrong -- there really are four combinations not three like I said, but they are C-order, Fortran-order, Both and neither. I forgot what the flags signified and had to play with them for a bit to remember. > I believe it is the former that you > are objecting to, but it would help if you could specify whether > you are > talking about the order keyword or whether you are talking about the > FORTRAN flag. > > > Both. I was argueing against the FORTRAN flag, and of limiting the > order keyword to those cases where f or c contiguous arrays were the > output or input. > > I'll also note that the order keyword could probably have been used to > fix a performance problem someone was having a few weeks ago. We > ended > up transposing the data, but the individual felt that obscured the > intent of the algorithm. I believe the same effect could probably have > been been achieved without re jiggering the algorithm by using the > order > parameter. > > > Some more details would be helpful. It would be good to know what > problem the order keyword should solve. > Well, in general, memory layout can be important for performance not just for interfacing with Fortran. You can do this with suitable applications of transpose, but using the order flag is probably clearer. Particularly, if you are trying to match a textbook algorithm, its nice to have the axes in the same places. I'm just moving over from numarray which didn't have the equivalent of the order flag as far as I know, so I don't have experience with this at this point though. Here is one of the posts in questions: > David Cournapeau wrote: > Hi, > > I was wondering if there was any way to speed up the following code: > > y = N.zeros((n, K)) > for i in range(K): > y[:, i] = gauss_den(data, mu[i, :], va[i, :]) > > Where K is of order 1e1, n of order 1e5. Normally, gauss_den is a > quite expensive function, but the profiler tells me that the indexing > y[:,i] takes almost as much time as the gauss_den computation (which > computes n exp !). To see if the profiler is "right", i replaces with > the (non valid) following function: > > y = N.zeros((n, K)) > for i in range(K): > yt = gauss_den(data, mu[i, :], va[i, :]) > return y > > Where more than 99% of the code is spent inside gauss_den. > > I guess the problem is coming from the fact that y being C order, y[:, > i] needs accessing data in a non 'linear' way. Is there a way to speed > this up ? I did something like this: > > y = N.zeros((K, n)) > for i in range(K): > y[i] = gauss_den(data, mu[i, :], va[i, :]) > return y.T > > which works, but I don't like it very much. I believe that the same efficiency as the last could have been achieved using something like: y = N.zeros((n,K), order='F') for i in range(K): y[:,i] = gauss_den(data, mu[i, :], va[i, :]) return y This probably would have made the original poster happier. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Wed Oct 18 14:29:17 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 18 Oct 2006 11:29:17 -0700 Subject: histogram complete makeover In-Reply-To: <331116dc0610181100k474af71akac41e54db7b7bb41@mail.gmail.com> References: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> <331116dc0610181100k474af71akac41e54db7b7bb41@mail.gmail.com> Message-ID: <4536727D.4020308@ieee.org> My $0.02: If histogram is going to get a makeover, particularly one that makes it more complex than at present, it should probably be moved to SciPy. Failing that, it should be moved to a submodule of numpy with similar statistical tools. Preferably with consistent interfaces for all of the functions. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 15:10:16 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 13:10:16 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536721A.4050104@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <45366B3F.2050103@ieee.org> <4536721A.4050104@ieee.org> Message-ID: On 10/18/06, Tim Hochberg wrote: > > Charles R Harris wrote: > > > > > > On 10/18/06, *Tim Hochberg* > > wrote: > > > > Charles R Harris wrote: > > > > [SNIP] > > > > > > I'm not talking about the keyword in the ravel call, I'm talking > > about > > > the flag in a. The question is: do we *need* a fortran flag. I am > > > argueing not, because the only need is for fortran contiguous > > arrays > > > to pass to fortran function, or translation from fortran > contiguous > > > arrays to numpy arrays. What I am saying is that things are > > > unnecessarily complicated. None of the LaPack stuff seems to use > > the > > > Fortran stuff, they just transpose and copy. I don't even think > > I want > > > to change that, because it is *clear* what is going on. > > Interfacing to > > > fortran is all about memory layout, nothing more or less. > > > > > > > Chuck, > > > > There are two things here. One is the order keyword and one is the > > FORTRAN flag. The latter is mainly an optimization for use at the > > C-level so that one doesn't have to check whether a given array is > in > > contiguous FORTRAN order by examining the strides, in the same way > > that > > the CONTIGUOUS flag allows you to skip examining the strides when > you > > need a contiguous C-order matrix. > > > > > > That sounds like the two flags should be named f-contiguous and > > c-contiguous. Then they would be orthogonal and one could have all > > four combinations. Is that the case now? Perhaps I am misunderstanding > > the meaning of the flags. > That is the case now. The flag names simply mirror their values in C. > Why they have those names in something of a historical accident I > believe. Take a look at this: OK, that is good. I no longer have any objection to the flags, I just wish the names were more descriptive of what they mean. In fact, it looks like the following sort of construction will be useful in the linalg module. In [17]:a = array([[1,2],[3,4]], dtype=int) In [18]:b = array(a, dtype=double, order='f') In [19]:b.flags Out[19]: CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False I've been a pain in the a** because I really want to know what is going on down in the boiler room. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From chanley at stsci.edu Wed Oct 18 15:10:08 2006 From: chanley at stsci.edu (Christopher Hanley) Date: Wed, 18 Oct 2006 15:10:08 -0400 Subject: scalar object bug or feature? Message-ID: <45367C10.40606@stsci.edu> Greetings, I can't decide if the following is a bug or feature. Numpy scalars are allowed to overflow silently while numarray upcasts to a larger python type. I guess my biggest problem is that the overflow occurs silently. In any case, is this known and expected behavior? Thanks, Chris NUMARRAY example: In [1]: import numarray as n In [2]: a = n.array([2200,14000],type=n.UInt32) In [3]: a0= a[0] In [4]: a1 = a[1] In [5]: adiff = a0-a1 In [6]: print a0,a1,adiff 2200 14000 -11800 In [7]: print type(a0),type(a1),type(adiff) NUMPY example: In [1]: import numpy as n In [2]: a = n.array([2200,14000],dtype=n.uint32) In [3]: a0= a[0] In [4]: a1 = a[1] In [5]: adiff = a0-a1 In [6]: print a0,a1,adiff 2200 14000 4294955496 In [7]: print type(a0),type(a1),type(adiff) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Wed Oct 18 15:31:43 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Wed, 18 Oct 2006 12:31:43 -0700 Subject: scalar object bug or feature? In-Reply-To: <45367C10.40606@stsci.edu> References: <45367C10.40606@stsci.edu> Message-ID: On 10/18/06, Christopher Hanley wrote: > I can't decide if the following is a bug or feature. Numpy scalars are > allowed to overflow silently while numarray upcasts to a larger python > type. I guess my biggest problem is that the overflow occurs silently. I had that problem yesterday, which was difficult to diagnose for a new user like me. Here's an example: >> x = zeros((300, 300)) >> x = x > 1 # False >> x = 1 - x # ones >> y = x.T * x >> y[0,0] 44 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 14:33:49 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 12:33:49 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45366988.5020204@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> Message-ID: <4536738D.7070701@ieee.org> Tim Hochberg wrote: > One thing that may be confusing the issue is that, as I understand it, > FORTRAN and CONTIGUOUS together represent three states which I'll call > FORTRAN_ORDER, C_ORDER and DISCONTIGUOUS. Yep, that's what they mean. CONTIGUOUS is the name Numeric gave it and it meant C-order contiguous. We have kept the same meaning. All we've done is selected out from the class of arrays that Numeric called DISTCONTIGUOUS, arrays that are FORTRAN-order (and so still single-segment), but discontiguous in the sense that Numeric had. > I periodically wonder if it > would be valuable to have a way to query the order directly: the result > would be "C", "F" or None, just like the order keyword that is passed > in. You an do it with the flags a.flags.contiguous a.flags.fortran Discontiguous is when both of these are false. Note that for a.ndim < 2, both a.flags.contiguous and a.flags.fortran are true if one of them is true. This is all explained in the first chapters of my book. You have to understand CONTIGUOUS == C-order contiguous and FORTRAN == Fortran-order contiguous. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 14:28:40 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 12:28:40 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> Message-ID: <45367258.8070503@ieee.org> > > I'm not talking about the keyword in the ravel call, I'm talking about > the flag in a. Ah. Yes, I see. I misunderstood. Of course ravel ignores the FORTRAN flag (actually it doesn't because if a copy is not necessary it doesn't make one). The key is that the Python user doesn't need to care about the array flag unless they are interfacing to compiled code. That's the point of the flag. It's actually redundant because it could be checked every time it's needed. But, right now, it's kept updated so that the check is simple. The same is true with the C-CONTIGUOUS flag (called contiguous). > The question is: do we *need* a fortran flag. No, you don't *need* the flag. But, it saves copying data to check it (look how many times ISFORTRAN is called in the code). Without the flag all of those cases would need to do a strides-check which is done in the UpdateFlags code. > I am argueing not, because the only need is for fortran contiguous > arrays to pass to fortran function, or translation from fortran > contiguous arrays to numpy arrays. What I am saying is that things are > unnecessarily complicated. I disagree. It's actually not that complicated. The FORTRAN flag gives us a lot more flexibility when it comes to copying data or not. I think part of the complication is that you are misunderstanding some of the terms and the purposes of the keywords. > None of the LaPack stuff seems to use the Fortran stuff, they just > transpose and copy. It doesn't now only because I haven't had time to go through and change it, but it should. Look at scipy's LaPack interface. It (through f2py) uses the FORTRAN stuff extensively (much was borrowed from there in the first place). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 14:31:18 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 12:31:18 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> Message-ID: <453672F6.9070307@ieee.org> > > I'm not talking about the keyword in the ravel call, I'm talking about > the flag in a. Ah. Yes, I see. I misunderstood. Of course ravel ignores the FORTRAN flag (actually it doesn't because if a copy is not necessary it doesn't make one). The key is that the Python user doesn't need to care about the array flag unless they are interfacing to compiled code. That's the point of the flag. It's actually redundant because it could be checked every time it's needed. But, right now, it's kept updated so that the check is simple. The same is true with the C-CONTIGUOUS flag (called contiguous). > The question is: do we *need* a fortran flag. No, you don't *need* the flag. But, it saves copying data to check it (look how many times ISFORTRAN is called in the code). Without the flag all of those cases would need to do a strides-check which is done in the UpdateFlags code. > I am argueing not, because the only need is for fortran contiguous > arrays to pass to fortran function, or translation from fortran > contiguous arrays to numpy arrays. What I am saying is that things are > unnecessarily complicated. I disagree. It's actually not that complicated. Even if it was compilcated to implement, the point is that it is now done. There is no sense ripping it out (that would be a huge pain and for what purpose?) The FORTRAN flag gives us a lot more flexibility when it comes to copying data or not. I think part of the complication is that you are misunderstanding some of the terms and the purposes of the keywords. > None of the LaPack stuff seems to use the Fortran stuff, they just > transpose and copy. It doesn't now only because I haven't had time to go through and change it, but it should. Look at scipy's LaPack interface. It (through f2py) uses the FORTRAN stuff extensively (much was borrowed from there in the first place). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Wed Oct 18 16:25:30 2006 From: aisaac at american.edu (Alan G Isaac) Date: Wed, 18 Oct 2006 16:25:30 -0400 Subject: scalar object bug or feature? In-Reply-To: References: <45367C10.40606@stsci.edu> Message-ID: On Wed, 18 Oct 2006, Keith Goodman apparently wrote: > Here's an example: >>> x = zeros((300, 300)) >>> x = x > 1 # False >>> x = 1 - x # ones >>> y = x.T * x >>> y[0,0] > 44 Here's a simpler (?) example: >>> x=numpy.random.rand(300,1)>0 >>> x.sum() 300 >>> sum(x) array([44], dtype=int8) >>> x=numpy.random.rand(300)>0 >>> sum(x) 300 Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 16:51:12 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 14:51:12 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536738D.7070701@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> Message-ID: Travis, On 10/18/06, Travis Oliphant wrote: > > Tim Hochberg wrote: > > One thing that may be confusing the issue is that, as I understand it, > > FORTRAN and CONTIGUOUS together represent three states which I'll call > > FORTRAN_ORDER, C_ORDER and DISCONTIGUOUS. > > Yep, that's what they mean. CONTIGUOUS is the name Numeric gave it and > it meant C-order contiguous. We have kept the same meaning. All we've > done is selected out from the class of arrays that Numeric called > DISTCONTIGUOUS, arrays that are FORTRAN-order (and so still > single-segment), but discontiguous in the sense that Numeric had. > > > I periodically wonder if it > > would be valuable to have a way to query the order directly: the result > > would be "C", "F" or None, just like the order keyword that is passed > > in. > You an do it with the flags > > a.flags.contiguous > a.flags.fortran > > Discontiguous is when both of these are false. Note that for a.ndim < > 2, both a.flags.contiguous and a.flags.fortran are true if one of them > is true. > > This is all explained in the first chapters of my book. You have to > understand CONTIGUOUS == C-order contiguous and FORTRAN == Fortran-order > contiguous. Could we make a few changes ;) For printing the flags I would suggest using C-Contiguous and F-Contiguous so folks don't have to read the book. And at the c level define alternates, i.e, #define c-contiguous contiguous or whatever. That way backward compatibility would be maintained but more descriptive names would be available. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 18 17:03:32 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 15:03:32 -0600 Subject: scalar object bug or feature? In-Reply-To: References: <45367C10.40606@stsci.edu> Message-ID: On 10/18/06, Alan G Isaac wrote: > > On Wed, 18 Oct 2006, Keith Goodman apparently wrote: Here's a simpler (?) example: > >>> x=numpy.random.rand(300,1)>0 > >>> x.sum() > 300 > >>> sum(x) > array([44], dtype=int8) > >>> x=numpy.random.rand(300)>0 > >>> sum(x) > 300 > > Alan Isaac Hmmm, I think sum(x) and x.sum() should behave the same. Note that In [12]:sum(x, dtype=int) Out[12]:300 I think sum should stick to the modular arithmetic unless specified otherwise. But in any case sum(x) and x.sum() should do the same thing. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gnurser at googlemail.com Wed Oct 18 17:05:19 2006 From: gnurser at googlemail.com (George Nurser) Date: Wed, 18 Oct 2006 22:05:19 +0100 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <453672F6.9070307@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <453672F6.9070307@ieee.org> Message-ID: <1d1e6ea70610181405x2732a584hbb975ea76883f3e9@mail.gmail.com> > > None of the LaPack stuff seems to use the Fortran stuff, they just > > transpose and copy. You've got me worried here. I have assumed that when you start with a c-contiguous array, a, with say,a.shape = (m,n), if you use the transpose as an argument to a fortran routine which requires an mxn size array, then no copying is required. This seems to work for me -- the transpose *does* have fortran order. Also, in f2py, if I use -DF2PY_REPORT_ON_ARRAY_COPY=1 I receive no alert of any copy. Apologies if these are simply confused ravings. George Nurser. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 17:48:22 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 15:48:22 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <1d1e6ea70610181405x2732a584hbb975ea76883f3e9@mail.gmail.com> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <453672F6.9070307@ieee.org> <1d1e6ea70610181405x2732a584hbb975ea76883f3e9@mail.gmail.com> Message-ID: On 10/18/06, George Nurser wrote: > > > > None of the LaPack stuff seems to use the Fortran stuff, they just > > > transpose and copy. > > You've got me worried here. I have assumed that when you start with a > c-contiguous array, a, with say,a.shape = (m,n), if you use the > transpose as an argument to a fortran routine which requires an mxn > size array, then no copying is required. > > This seems to work for me -- the transpose *does* have fortran order. Nope. The result is an (n,m) array in fortran order, not an (m,n) array in fortran order. In [52]:a = array([[1,2,3],[4,5,6]]) In [53]:a.transpose().flags Out[53]: CONTIGUOUS : False FORTRAN : True OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False In [54]:a.transpose().shape Out[54]:(3, 2) Looks like what you want is either fastcopyandtranspose or the order flag. In [56]:fastCopyAndTranspose(a).flags Out[56]: CONTIGUOUS : True FORTRAN : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False which is a (n,m) array in c order, i.e., an (m,n) array in fortran order. Or In [57]:array(a, order='F').flags Out[57]: CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False which is a (m,n) array in fortran order. Also, in f2py, if I use -DF2PY_REPORT_ON_ARRAY_COPY=1 I receive no > alert of any copy. f2py takes care of the ordering, which is one reason why it is so useful. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 18 18:06:32 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 18 Oct 2006 16:06:32 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> Message-ID: <4536A568.4090907@ee.byu.edu> Charles R Harris wrote: > > Could we make a few changes ;) > > For printing the flags I would suggest using C-Contiguous and > F-Contiguous so folks don't have to read the book. And at the c level > define alternates, i.e, #define c-contiguous contiguous or whatever. > That way backward compatibility would be maintained but more > descriptive names would be available. Printing the flags is not intended for the casual user. So, I'd like to keep consistent with C-level names and the names that are printed. CONTIGUOUS is the old name Numeric used. It always meant C-CONTIGUOUS and so that meaning is preserved. FORTRAN is the new one flag and it means FORTRAN CONTIGUOUS. So, you want something like? #define NPY_C_CONTIGUOUS NPY_CONTIGUOUS #define NPY_F_CONTIGUOUS NPY_FORTRAN and to have C_CONTIGUOUS and F_CONTIGUOUS print for the flags description? I'm not opposed to it, but I don't really see the need. It's just a semantic question. Given the history of CONTIGUOUS in Numeric I thought it was clear that CONTIGUOUS always meant C-contiguous. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 18:19:36 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 16:19:36 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536A568.4090907@ee.byu.edu> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> Message-ID: On 10/18/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > Could we make a few changes ;) > > > > For printing the flags I would suggest using C-Contiguous and > > F-Contiguous so folks don't have to read the book. And at the c level > > define alternates, i.e, #define c-contiguous contiguous or whatever. > > That way backward compatibility would be maintained but more > > descriptive names would be available. > > Printing the flags is not intended for the casual user. So, I'd like to > keep consistent with C-level names and the names that are printed. > > CONTIGUOUS is the old name Numeric used. It always meant C-CONTIGUOUS > and so that meaning is preserved. FORTRAN is the new one flag and it > means FORTRAN CONTIGUOUS. > > So, you want something like? > > #define NPY_C_CONTIGUOUS NPY_CONTIGUOUS > #define NPY_F_CONTIGUOUS NPY_FORTRAN > > and to have C_CONTIGUOUS and F_CONTIGUOUS print for the flags description? Yes, I think that would be more informative. I'm not opposed to it, but I don't really see the need. It's just a > semantic question. Given the history of CONTIGUOUS in Numeric I thought > it was clear that CONTIGUOUS always meant C-contiguous. Well, I knew that for numeric, but it was a good deal less obvious in combo with the order keyword. For instance, contiguous could change its meaning to match up with FORTRAN, so that FORTRAN=True and CONTIGUOUS=True meant Fortran contiguous, which was sort of what I was thinking. Explicit never hurts. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gnurser at googlemail.com Wed Oct 18 18:21:19 2006 From: gnurser at googlemail.com (George Nurser) Date: Wed, 18 Oct 2006 23:21:19 +0100 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <453672F6.9070307@ieee.org> <1d1e6ea70610181405x2732a584hbb975ea76883f3e9@mail.gmail.com> Message-ID: <1d1e6ea70610181521u3eb3120fv9870f73c6a8e0602@mail.gmail.com> On 18/10/06, Charles R Harris wrote: > > > On 10/18/06, George Nurser wrote: > > > > None of the LaPack stuff seems to use the Fortran stuff, they just > > > > transpose and copy. > > > > You've got me worried here. I have assumed that when you start with a > > c-contiguous array, a, with say, a.shape = (m,n), if you use the > > transpose as an argument to a fortran routine which requires an mxn > > size array, then no copying is required. > > > > This seems to work for me -- the transpose *does* have fortran order. > > Nope. The result is an (n,m) array in fortran order, not an (m,n) array in > fortran order. Presumably that's because it's a view of the original array. >> > Also, in f2py, if I use -DF2PY_REPORT_ON_ARRAY_COPY=1 I receive no > > alert of any copy. > > f2py takes care of the ordering, which is one reason why it is so useful. Yes, when I first used it, I assumed that the fortran routine had to use an nxm array. But f2py is clever enough to make the above work. George. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 18 18:23:17 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 18 Oct 2006 16:23:17 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> Message-ID: <4536A955.5060504@ee.byu.edu> Charles R Harris wrote: > Well, I knew that for numeric, but it was a good deal less obvious in > combo with the order keyword. For instance, contiguous could change > its meaning to match up with FORTRAN, so that FORTRAN=True and > CONTIGUOUS=True meant Fortran contiguous, which was sort of what I was > thinking. Explicit never hurts. Ahh, so you were confused by looking at flags for 1-d arrays (where indeed both CONTIGUOUS and FORTRAN can be True --- 1-d arrays that are CONTIGUOUS are also FORTRAN CONTIGUOUS). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 18:44:23 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 16:44:23 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536A955.5060504@ee.byu.edu> References: <20061018002445.GP17998@mentat.za.net> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> <4536A955.5060504@ee.byu.edu> Message-ID: On 10/18/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > Well, I knew that for numeric, but it was a good deal less obvious in > > combo with the order keyword. For instance, contiguous could change > > its meaning to match up with FORTRAN, so that FORTRAN=True and > > CONTIGUOUS=True meant Fortran contiguous, which was sort of what I was > > thinking. Explicit never hurts. > > > Ahh, so you were confused by looking at flags for 1-d arrays (where More confused by what seemed to make sense to me. Tim was the one who actually ran the experiment to see what was going on, and he wasn't sure what FORTRAN meant either. Now if I had accessed the full set of offsets, strides, and counts, it would all have become clear. Numpy tries to hide the nasty details, which is good until you really have to know what it happening underneath. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 18 18:55:06 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 18 Oct 2006 16:55:06 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> <4536A955.5060504@ee.byu.edu> Message-ID: <4536B0CA.9070402@ee.byu.edu> Charles R Harris wrote: > > > On 10/18/06, *Travis Oliphant* > wrote: > > Charles R Harris wrote: > > > Well, I knew that for numeric, but it was a good deal less > obvious in > > combo with the order keyword. For instance, contiguous could change > > its meaning to match up with FORTRAN, so that FORTRAN=True and > > CONTIGUOUS=True meant Fortran contiguous, which was sort of what > I was > > thinking. Explicit never hurts. > > > Ahh, so you were confused by looking at flags for 1-d arrays (where > > > More confused by what seemed to make sense to me. Tim was the one who > actually ran the experiment to see what was going on, and he wasn't > sure what FORTRAN meant either. Now if I had accessed the full set of > offsets, strides, and counts, it would all have become clear. Numpy > tries to hide the nasty details, which is good until you really have > to know what it happening underneath. I've done as you requested. I've added F_CONTIGUOUS as an alias to FORTRAN and C_CONTIGUOUS as an alias to CONTIGUOUS. These are the names that show up when you print the flags and you can use them whenever you used the other names. The other names are still everywhere in the code, but perhaps the existence of these aliases will help people understand what is meant better. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 19:05:13 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 17:05:13 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536B0CA.9070402@ee.byu.edu> References: <20061018002445.GP17998@mentat.za.net> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> <4536A955.5060504@ee.byu.edu> <4536B0CA.9070402@ee.byu.edu> Message-ID: On 10/18/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > > > On 10/18/06, *Travis Oliphant* > > wrote: > > > > Charles R Harris wrote: > > > > > Well, I knew that for numeric, but it was a good deal less > > obvious in > > > combo with the order keyword. For instance, contiguous could > change > > > its meaning to match up with FORTRAN, so that FORTRAN=True and > > > CONTIGUOUS=True meant Fortran contiguous, which was sort of what > > I was > > > thinking. Explicit never hurts. > > > > > > Ahh, so you were confused by looking at flags for 1-d arrays (where > > > > > > More confused by what seemed to make sense to me. Tim was the one who > > actually ran the experiment to see what was going on, and he wasn't > > sure what FORTRAN meant either. Now if I had accessed the full set of > > offsets, strides, and counts, it would all have become clear. Numpy > > tries to hide the nasty details, which is good until you really have > > to know what it happening underneath. > > I've done as you requested. I've added F_CONTIGUOUS as an alias to > FORTRAN and C_CONTIGUOUS as an alias to CONTIGUOUS. These are the names > that show up when you print the flags and you can use them whenever you > used the other names. The other names are still everywhere in the code, > but perhaps the existence of these aliases will help people understand > what is meant better. Thanks, Travis. I really appreciate your willingness to make such modifications. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From michael.sorich at gmail.com Wed Oct 18 20:33:06 2006 From: michael.sorich at gmail.com (Michael Sorich) Date: Thu, 19 Oct 2006 10:03:06 +0930 Subject: Question about indexing arrays with boolean arrays In-Reply-To: References: Message-ID: <16761e100610181733g760c71bev441e439dfbea4458@mail.gmail.com> On 10/18/06, Daniel Arbuckle wrote: > Why does a[b1, b2] not mean the same thing as a[b1][:, b2], when "a" > is an array and "b1" and "b2" are appropriately sized arrays of > booleans? >From my previous experience with R I am used to a[b1, b2] being equivalent to a[b1][:, b2], so this is a little strange to me as well. In numpy I think you need to use the function ix_ to get the cross product. e.g. a[ix_(b1,b2)] is equivalent to a[b1][:, b2] a[b1,b2] raises an error about a type mismatch. I have no idea what this syntax is supposed to do in numpy. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Wed Oct 18 20:29:15 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 19 Oct 2006 02:29:15 +0200 Subject: adding an attribute to an nd-array Message-ID: <20061019002914.GA20609@mentat.za.net> A quick question on extending numpy arrays: is it possible to easily add an attribute to an ndarray? With Python-defined classes one can do class X(object): pass x = X() x.foo = 'bar' but with ndarrays you get x = N.array([1,2,3]) x.foo = 'bar' AttributeError: 'numpy.ndarray' object has no attribute 'foo' Is there an easy way around this (without writing a C extension)? Thanks for any advice. St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ckkart at hoc.net Wed Oct 18 21:10:06 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Thu, 19 Oct 2006 01:10:06 +0000 (UTC) Subject: sort(axis=-1) References: <4536603D.7070103@ieee.org> Message-ID: Travis Oliphant ieee.org> writes: > > Christian Kristukat wrote: > > Hi, > > > > it seems that -1 as axis parameter is interpreted like in array indexing, > > -1 means the last axis rather than meaning the only axis of the flattened > > representation, like for example with take(). > > Is that intendend? > > > > > > I don't understand the question. Yes, sort(axis=-1) sorts along the > last dimension. > > AFAIK, take has the same meaning for axis. Oh sure, I mixed that up with axis=None as Robert pointed out. However I don't understand why axis=None wouldn't make sense for an inplace operation. Anyway, it's not important. Sorry for the noise. Christian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Wed Oct 18 21:17:49 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Wed, 18 Oct 2006 21:17:49 -0400 Subject: adding an attribute to an nd-array In-Reply-To: <20061019002914.GA20609@mentat.za.net> References: <20061019002914.GA20609@mentat.za.net> Message-ID: <200610182117.49777.pgmdevlist@gmail.com> On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: > A quick question on extending numpy arrays: is it possible to easily > add an attribute to an ndarray? It might be easier to create a subclass: pleasehave a look here: http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_subclasses.py That's a tiny example of subclassing ndarrays, with some extra attributes. (BTW, I'm not sure that's the most obvious place where to look: if it turns out to be useful, I'll put it on the scipy wiki) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From xaagujzjtrz at txteam.com Thu Oct 19 03:49:28 2006 From: xaagujzjtrz at txteam.com (News Catch) Date: Thu, 19 Oct 2006 15:49:28 +0800 Subject: CA tel: Message-ID: <000801c6f353$1a9a1ae0$ed3308dd@4a21589f39a64fc> Barbados Bermuda Boston Cabo Lucaslos Cabos Cancun Cayman Islands Grand Chicago Cozumel Daytona a Beach of Disney Orlando. Spaces Blogger or Wordpress Drupal Technorati or Flickr a Yiyi sun cd Quinnipiac University Official Athletic dd Varsity Hockeyice Bandcrazy. Blogger Wordpress Drupal Technorati or Flickr a Yiyi sun cd Quinnipiac University or Official am Athletic dd Varsity. Accused historical abuse charges is made brief appearance court bus is driver found guilty dragging trapped or boy Nelson. Suffering going way peaceably suddenly is come him swear slavishly obey everything dictate consider good plan decide law a Secondly hand part dispose ofwe forcibly opposing orders Thirdly of elect others? Series off Mastering app choice easy Visual of Quickstart Guides a They feature clear helpful screen am shotsmore Computing in About or Order. Reloded in reload trailers script is vcd trailor or dot max payne cow or kazaa revolution am preview of find complete guide in file or sharing software a members area Plus get access pp tutorials. Stuff my Homeabout copy Fairfax Limited material this page has am protection copyright rights reserved Search web Headlines of Alert Overstayer jailed drug is haul African who. Easy Visual Quickstart Guides a They of feature am clear helpful screen shotsmore a Computing or About or Order Government. Textbooks Childrens Dvds Music Toys Items Browse Bookswhats am Soonbampn Annexsave. Tampa bay Tickets Tonights Gameend Allentowns Everyday Heroes ea Tech Neededfree Pocono of Race or Football is Employment Designer. Tours in Search Quickfind Quickfind Maritime Museums a Racing or Booking Types Hotels Performs Visitor Beatles Nightlife am Walks is Guided Trade exhibition aims. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Pickup:.gif Type: image/gif Size: 12248 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david at ar.media.kyoto-u.ac.jp Thu Oct 19 06:25:52 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 19 Oct 2006 19:25:52 +0900 Subject: Conditional tests for test classes derived from NumpyTest Message-ID: <453752B0.3080308@ar.media.kyoto-u.ac.jp> Hi, Is there a simple way to execute tests inherited from NumpyTest conditionally ? I have something like that for now: class test_c_implementation(NumpyTestCase): def _check(self, level, decimal = 12): try: from foo import bar Y = bar() except ImportError, inst: print "Error while importing bar, not tested" print " -> (Import error was %s)" % inst And all checking functions of the class call _check. The problem is that if there are 10 check functions, there are 10 errors reported, which is a bit stupid, cheers, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Thu Oct 19 06:33:43 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Thu, 19 Oct 2006 13:33:43 +0300 (EEST) Subject: Conditional tests for test classes derived from NumpyTest In-Reply-To: <453752B0.3080308@ar.media.kyoto-u.ac.jp> References: <453752B0.3080308@ar.media.kyoto-u.ac.jp> Message-ID: On Thu, 19 Oct 2006, David Cournapeau wrote: > Hi, > > Is there a simple way to execute tests inherited from NumpyTest > conditionally ? I have something like that for now: > > class test_c_implementation(NumpyTestCase): > def _check(self, level, decimal = 12): > try: > from foo import bar > Y = bar() > except ImportError, inst: > print "Error while importing bar, not tested" > print " -> (Import error was %s)" % inst > > And all checking functions of the class call _check. The problem is > that if there are 10 check functions, there are 10 errors reported, > which is a bit stupid, One approach is demonstrated in scipy/Lib/lib/blas/tests/test_fblas.py. In your case: class base_test_c_implementation: def check_bar(self, level, decimal=12): Y = bar() try: from foo import bar class test_c_implementation(base_test_c_implementation,NumpyTestCase): pass except ImportError, inst: print "Error while importing bar, not tested" print " -> (Import error was %s)" % inst Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 19 07:20:32 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 19 Oct 2006 13:20:32 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <200610182117.49777.pgmdevlist@gmail.com> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> Message-ID: <20061019112032.GE20609@mentat.za.net> On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: > > A quick question on extending numpy arrays: is it possible to easily > > add an attribute to an ndarray? > > It might be easier to create a subclass: pleasehave a look here: > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_subclasses.py > That's a tiny example of subclassing ndarrays, with some extra attributes. > (BTW, I'm not sure that's the most obvious place where to look: if it turns > out to be useful, I'll put it on the scipy wiki) Thanks very much, Pierre. If I understand correctly, the following should work: import numpy as N class InfoArray(N.ndarray): def __new__(info_arr_cls,arr,info={}): info_arr_cls.info = info return N.array(arr).view(info_arr_cls) When does __array_finalize__ get called, and is it always necessary to specify it? Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Thu Oct 19 07:59:08 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 19 Oct 2006 13:59:08 +0200 Subject: Product of an empty sequence Message-ID: <4537688C.5070908@carabos.com> Today I was surprised by this:: >>> import numpy >>> numpy.__version__ '1.0.dev3341' >>> numpy.prod(()) 1.0 Wouldn't be 0.0 a more natural answer? I understand that 1 is neutral on product operations, but I still can't see why 1.0 is a better answer. Sorry for the clueless question, but I have searched lists, docstrings and web and found nothing about this. (BTW, the definitions of ``prod()`` and ``product()`` are still identical as noted by Sebastian Haase some time ago.) :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From rudolphv at gmail.com Thu Oct 19 08:35:33 2006 From: rudolphv at gmail.com (Rudolph van der Merwe) Date: Thu, 19 Oct 2006 14:35:33 +0200 Subject: Unit test error with numpy rc3 Message-ID: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> I get the following error with RC3 on a RHE Linux box: Python 2.4.3 (#4, Mar 31 2006, 12:12:43) [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.0rc3' >>> numpy.test() Found 5 tests for numpy.distutils.misc_util Found 3 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 32 tests for numpy.linalg Found 13 tests for numpy.core.umath Found 4 tests for numpy.core.scalarmath Found 9 tests for numpy.lib.arraysetops Found 42 tests for numpy.lib.type_check Found 183 tests for numpy.core.multiarray Found 3 tests for numpy.fft.helper Found 36 tests for numpy.core.ma Found 12 tests for numpy.lib.twodim_base Found 10 tests for numpy.core.defmatrix Found 1 tests for numpy.lib.ufunclike Found 4 tests for numpy.ctypeslib Found 41 tests for numpy.lib.function_base Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 26 tests for numpy.core.numeric Found 4 tests for numpy.lib.index_tricks Found 47 tests for numpy.lib.shape_base Found 0 tests for __main__ .......................................................................................................Warning: invalid value encountered in divide ..Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ..Warning: invalid value encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ..Warning: invalid value encountered in divide ..Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ........Warning: invalid value encountered in divide .Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide ........................................................................................................................................................................................................Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide .....................Warning: invalid value encountered in sqrt Warning: invalid value encountered in log Warning: invalid value encountered in log10 ..Warning: invalid value encountered in sqrt Warning: invalid value encountered in sqrt Warning: divide by zero encountered in log Warning: divide by zero encountered in log Warning: divide by zero encountered in log10 Warning: divide by zero encountered in log10 Warning: invalid value encountered in arcsin Warning: invalid value encountered in arcsin Warning: invalid value encountered in arccos Warning: invalid value encountered in arccos Warning: invalid value encountered in arccosh Warning: invalid value encountered in arccosh Warning: divide by zero encountered in arctanh Warning: divide by zero encountered in arctanh Warning: invalid value encountered in divide Warning: invalid value encountered in true_divide Warning: invalid value encountered in floor_divide Warning: invalid value encountered in remainder Warning: invalid value encountered in fmod ............................................................................................................................................................... ---------------------------------------------------------------------- Ran 516 tests in 0.368s OK >>> -- Rudolph van der Merwe KAT (Karoo Array Telescope) / www.kat.ac.za ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Thu Oct 19 09:04:12 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Thu, 19 Oct 2006 15:04:12 +0200 Subject: recent ubuntu package In-Reply-To: <4535C555.9060004@astraw.com> References: <45353395.40602@gmx.net> <4535C555.9060004@astraw.com> Message-ID: <453777CC.6070804@gmx.net> Andrew Straw schrieb: > The matplotlib .deb on my website is working fine for me with the latest > numpy .deb there. If there are any recent patches or anything you are > missing, please let me know -- it's not really a big deal to update > them, although it might take a couple of days for me to find the time. > > Thanks for your answer. No, I'm not aware of any new features I would use, so that's ok for me. It just wasn't obvious to me why the builds are from in-between-releases devel versions, that's why I asked. After numpy 1.0 is released, are you planning to put up only official releases or also devel snapshots? thanks, sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Thu Oct 19 09:07:51 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 19 Oct 2006 09:07:51 -0400 Subject: Product of an empty sequence In-Reply-To: <4537688C.5070908@carabos.com> References: <4537688C.5070908@carabos.com> Message-ID: On Thu, 19 Oct 2006, Ivan Vilata i Balaguer apparently wrote: >>>> numpy.prod(()) > 1.0 > Wouldn't be 0.0 a more natural answer? I think it must be 1 or a TypeError. E.g., http://docs.python.org/lib/built-in-funcs.html#reduce reduce(function, sequence[, initializer]) Apply function of two arguments cumulatively to the items of sequence, from left to right, so as to reduce the sequence to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the sequence. If the optional initializer is present, it is placed before the items of the sequence in the calculation, and serves as a default when the sequence is empty. If initializer is not given and sequence contains only one item, the first item is returned. hth, Alan Isaac (just a user) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 09:32:30 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 06:32:30 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> Message-ID: <45377E6E.7040006@ieee.org> Rudolph van der Merwe wrote: > I get the following error with RC3 on a RHE Linux box: > > Python 2.4.3 (#4, Mar 31 2006, 12:12:43) > [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>>> import numpy >>>> numpy.__version__ >>>> > '1.0rc3' > [SNIP lots of warnings] Ah! I see that Travis has turned up the default warning level. Excellent! This will require some tweaking of the tests, but that shouldn't be a big deal. If I have time I will try to work on that later today. For the moment, try this: >>> olderr = numpy.seterr(all='ignore') >>> numpy.test() >>> numpy.seterr(*olderr) This should run the tests with the warnings suppressed. On a visual studio build, I'm getting a real failure though: FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", line 219, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError The code in question is dependent on the spelling of INF(or at least the *length* of the spelling) on a given platform which is why it's failing. I suspect that the correct test is: str(a)[1:-1] == str(a[0]) However, I'm not entirely sure what this is testing, so I'm reluctant to check that in. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 19 10:25:36 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 19 Oct 2006 08:25:36 -0600 Subject: adding an attribute to an nd-array In-Reply-To: <20061019112032.GE20609@mentat.za.net> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> Message-ID: <45378AE0.1080003@ieee.org> Stefan van der Walt wrote: > On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > >> On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: >> >>> A quick question on extending numpy arrays: is it possible to easily >>> add an attribute to an ndarray? >>> >> It might be easier to create a subclass: pleasehave a look here: >> http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_subclasses.py >> That's a tiny example of subclassing ndarrays, with some extra attributes. >> (BTW, I'm not sure that's the most obvious place where to look: if it turns >> out to be useful, I'll put it on the scipy wiki) >> > > Thanks very much, Pierre. > > If I understand correctly, the following should work: > > import numpy as N > > class InfoArray(N.ndarray): > def __new__(info_arr_cls,arr,info={}): > info_arr_cls.info = info > return N.array(arr).view(info_arr_cls) > > When does __array_finalize__ get called, and is it always necessary to > specify it? > It gets called whenever a new array is created. No, it is not necessary to specify it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 19 10:27:20 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 19 Oct 2006 08:27:20 -0600 Subject: Product of an empty sequence In-Reply-To: <4537688C.5070908@carabos.com> References: <4537688C.5070908@carabos.com> Message-ID: <45378B48.9030105@ieee.org> Ivan Vilata i Balaguer wrote: > Today I was surprised by this:: > > >>>> import numpy >>>> numpy.__version__ >>>> > '1.0.dev3341' > >>>> numpy.prod(()) >>>> > 1.0 > > Wouldn't be 0.0 a more natural answer? I understand that 1 is neutral > on product operations, but I still can't see why 1.0 is a better answer. > It's the identity for the multiplication ufunc. > Sorry for the clueless question, but I have searched lists, docstrings > and web and found nothing about this. > > (BTW, the definitions of ``prod()`` and ``product()`` are still > identical as noted by Sebastian Haase some time ago.) > Is this a problem? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 19 10:29:26 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 19 Oct 2006 08:29:26 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <45377E6E.7040006@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> Message-ID: <45378BC6.70609@ieee.org> Tim Hochberg wrote: > Rudolph van der Merwe wrote: > >> I get the following error with RC3 on a RHE Linux box: >> >> Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >> [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >> >>>>> import numpy >>>>> numpy.__version__ >>>>> >>>>> >> '1.0rc3' >> >> > > On a visual studio build, I'm getting a real failure though: > > FAIL: Ticket #112 > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", > line 219, in check_longfloat_repr > assert(str(a)[1:9] == str(a[0])[:8]) > AssertionError > > The code in question is dependent on the spelling of INF(or at least the > *length* of the spelling) on a given platform which is why it's failing. > Actually, you shouldn't be getting an INF at all. This is what the test is designed to test for (so I guess it's working). The test was actually written wrong and was never failing because previously keyword arguments to ufuncs were ignored. Can you show us what 'a' is on your platform. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From strawman at astraw.com Thu Oct 19 10:48:24 2006 From: strawman at astraw.com (Andrew Straw) Date: Thu, 19 Oct 2006 07:48:24 -0700 Subject: recent ubuntu package In-Reply-To: <453777CC.6070804@gmx.net> References: <45353395.40602@gmx.net> <4535C555.9060004@astraw.com> <453777CC.6070804@gmx.net> Message-ID: <45379038.9080902@astraw.com> Sven Schreiber wrote: > Andrew Straw schrieb: > > >> The matplotlib .deb on my website is working fine for me with the latest >> numpy .deb there. If there are any recent patches or anything you are >> missing, please let me know -- it's not really a big deal to update >> them, although it might take a couple of days for me to find the time. >> >> >> > > Thanks for your answer. No, I'm not aware of any new features I would > use, so that's ok for me. It just wasn't obvious to me why the builds > are from in-between-releases devel versions, that's why I asked. After > numpy 1.0 is released, are you planning to put up only official releases > or also devel snapshots? > With numpy, I currently use the most recent SVN version I can because I want to help Travis testing for the 1.0 release. With that in mind, I keep matplotlib and scipy relatively up-to-date as well in case there have been any changes that those two have tracked. I'm not sure what I'll do after the release of numpy 1.0. I do not intend to be on the "bleeding edge" -- I use these repositories for a number of machines at work. But if the leading edge isn't bleeding (as currently), it's quite possible I'll keep updating them, particularly if features come out I'm interested in. This would be a good reason for me to keep updating scipy and matplotlib, even if I keep numpy more stable. -Andrew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 11:04:55 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 08:04:55 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <45378BC6.70609@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> Message-ID: <45379417.4060601@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > >> Rudolph van der Merwe wrote: >> >> >>> I get the following error with RC3 on a RHE Linux box: >>> >>> Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >>> [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> >>> >>>>>> import numpy >>>>>> numpy.__version__ >>>>>> >>>>>> >>>>>> >>> '1.0rc3' >>> >>> >>> >> On a visual studio build, I'm getting a real failure though: >> >> FAIL: Ticket #112 >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", >> line 219, in check_longfloat_repr >> assert(str(a)[1:9] == str(a[0])[:8]) >> AssertionError >> >> The code in question is dependent on the spelling of INF(or at least the >> *length* of the spelling) on a given platform which is why it's failing. >> >> > > Actually, you shouldn't be getting an INF at all. This is what the > test is designed to test for (so I guess it's working). The test was > actually written wrong and was never failing because previously keyword > arguments to ufuncs were ignored. > > Can you show us what 'a' is on your platform. Sure: >>> import numpy as N >>> a = N.exp(N.array([1000],dtype=N.longfloat)) Warning: overflow encountered in exp >>> a array([1.#INF], dtype=float64) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 11:08:50 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 08:08:50 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <45378BC6.70609@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> Message-ID: <45379502.1040707@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > >> Rudolph van der Merwe wrote: >> >> >>> I get the following error with RC3 on a RHE Linux box: >>> >>> Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >>> [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> >>> >>>>>> import numpy >>>>>> numpy.__version__ >>>>>> >>>>>> >>>>>> >>> '1.0rc3' >>> >>> >>> >> On a visual studio build, I'm getting a real failure though: >> >> FAIL: Ticket #112 >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", >> line 219, in check_longfloat_repr >> assert(str(a)[1:9] == str(a[0])[:8]) >> AssertionError >> >> The code in question is dependent on the spelling of INF(or at least the >> *length* of the spelling) on a given platform which is why it's failing. >> >> > > Actually, you shouldn't be getting an INF at all. This is what the > test is designed to test for (so I guess it's working). The test was > actually written wrong and was never failing because previously keyword > arguments to ufuncs were ignored. > > Can you show us what 'a' is on your platform. > I expect that the problem is related to this: >>> N.array([1000],dtype=N.float).dtype dtype('float64') >>> N.array([1000],dtype=N.longfloat).dtype dtype('float64') longfloat appears to just be an alias for float under VS. -tim > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 19 11:10:05 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 19 Oct 2006 09:10:05 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <45379417.4060601@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <45379417.4060601@ieee.org> Message-ID: <4537954D.1060301@ee.byu.edu> Tim Hochberg wrote: >Travis Oliphant wrote: > > >>Tim Hochberg wrote: >> >> >> >>>Rudolph van der Merwe wrote: >>> >>> >>> >>> >>>>I get the following error with RC3 on a RHE Linux box: >>>> >>>>Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >>>>[GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >>>>Type "help", "copyright", "credits" or "license" for more information. >>>> >>>> >>>> >>>> >>>> >>>>>>>import numpy >>>>>>>numpy.__version__ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>'1.0rc3' >>>> >>>> >>>> >>>> >>>> >>>On a visual studio build, I'm getting a real failure though: >>> >>> FAIL: Ticket #112 >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File >>> "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", >>> line 219, in check_longfloat_repr >>> assert(str(a)[1:9] == str(a[0])[:8]) >>> AssertionError >>> >>>The code in question is dependent on the spelling of INF(or at least the >>>*length* of the spelling) on a given platform which is why it's failing. >>> >>> >>> >>> >>Actually, you shouldn't be getting an INF at all. This is what the >>test is designed to test for (so I guess it's working). The test was >>actually written wrong and was never failing because previously keyword >>arguments to ufuncs were ignored. >> >>Can you show us what 'a' is on your platform. >> >> >Sure: > > >>> import numpy as N > >>> a = N.exp(N.array([1000],dtype=N.longfloat)) >Warning: overflow encountered in exp > >>> a >array([1.#INF], dtype=float64) > > O.K. We need to modify the test in case to check and see that the size of longfloat isn't the same as double. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Thu Oct 19 11:39:15 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 19 Oct 2006 17:39:15 +0200 Subject: Product of an empty sequence In-Reply-To: <45378B48.9030105@ieee.org> References: <4537688C.5070908@carabos.com> <45378B48.9030105@ieee.org> Message-ID: <45379C23.4050008@carabos.com> En/na Travis Oliphant ha escrit:: > Ivan Vilata i Balaguer wrote: >> Wouldn't be 0.0 a more natural answer? I understand that 1 is neutral >> on product operations, but I still can't see why 1.0 is a better answer. > > It's the identity for the multiplication ufunc. OK, I guess that means it's something like an implicit ``initializer`` (as pointed by Alan) to ufunc reductions. Thanks! :: >> (BTW, the definitions of ``prod()`` and ``product()`` are still >> identical as noted by Sebastian Haase some time ago.) >> > Is this a problem? No, but it also caught my eye. :) :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 19 11:45:02 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 19 Oct 2006 09:45:02 -0600 Subject: adding an attribute to an nd-array In-Reply-To: <20061019112032.GE20609@mentat.za.net> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> Message-ID: <45379D7E.30609@ee.byu.edu> Stefan van der Walt wrote: >On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > > >>On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: >> >> >>>A quick question on extending numpy arrays: is it possible to easily >>>add an attribute to an ndarray? >>> >>> >>It might be easier to create a subclass: pleasehave a look here: >>http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_subclasses.py >>That's a tiny example of subclassing ndarrays, with some extra attributes. >>(BTW, I'm not sure that's the most obvious place where to look: if it turns >>out to be useful, I'll put it on the scipy wiki) >> >> > >Thanks very much, Pierre. > >If I understand correctly, the following should work: > >import numpy as N > >class InfoArray(N.ndarray): > def __new__(info_arr_cls,arr,info={}): > info_arr_cls.info = info > return N.array(arr).view(info_arr_cls) > >When does __array_finalize__ get called, and is it always necessary to >specify it? > > Actually something as simple as class InfoArray(N.ndarray): pass will allow you to add attributes to InfoArray. I just learned about how to allow built-ins to have attributes assigned to their instances. It's actually pretty easy because of Python support for it --- but it comes at a cost. You have to add a dictionary to the PyArrayObject structure, create that dictionary when the ndarray is allocated, and set the tp_dictoffset in the TypeObject structure to its location in PyArrayObject. It takes 4 lines of code with the cost of creating a new dictionary for every ndarray. I don't think the extra bytes for every ndarray object are worth it, given how easy it is to sub-class and create your own ndarray that can have attributes attached. What are others opinions. -Travis P.S. Here is the patch that adds it: Index: numpy/core/include/numpy/ndarrayobject.h =================================================================== --- numpy/core/include/numpy/ndarrayobject.h (revision 3366) +++ numpy/core/include/numpy/ndarrayobject.h (working copy) @@ -1172,6 +1172,7 @@ PyArray_Descr *descr; /* Pointer to type structure */ int flags; /* Flags describing array -- see below*/ PyObject *weakreflist; /* For weakreferences */ + PyObject *instancedict; /* For instance attributes */ } PyArrayObject; #define NPY_AO PyArrayObject Index: numpy/core/src/arrayobject.c =================================================================== --- numpy/core/src/arrayobject.c (revision 3366) +++ numpy/core/src/arrayobject.c (working copy) @@ -1906,6 +1906,8 @@ if (self->weakreflist != NULL) PyObject_ClearWeakRefs((PyObject *)self); + Py_DECREF(self->instancedict); + if(self->base) { /* UPDATEIFCOPY means that base points to an array that should be updated with the contents @@ -5305,6 +5307,7 @@ self->descr = descr; self->base = (PyObject *)NULL; self->weakreflist = (PyObject *)NULL; + self->instancedict = PyDict_New(); if (nd > 0) { self->dimensions = PyDimMem_NEW(2*nd); @@ -6689,7 +6692,7 @@ 0, /* tp_dict */ 0, /* tp_descr_get */ 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ + offsetof(PyArrayObject, instancedict), /* tp_dictoffset */ (initproc)0, /* tp_init */ array_alloc, /* tp_alloc */ (newfunc)array_new, /* tp_new */ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 19 11:53:05 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 19 Oct 2006 09:53:05 -0600 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: On 10/19/06, Travis Oliphant wrote: > > Stefan van der Walt wrote: > > >On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > > > > > >>On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: > >> I don't think the extra bytes for every ndarray object are worth it, > given how easy it is to sub-class and create your own ndarray that can > have attributes attached. What are others opinions. I'd be more inclined to worry about speed. One of the drawbacks of numarray was the time it took to create arrays. Since I often use many small arrays, numpy was a big improvement in that area. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Thu Oct 19 11:54:42 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 19 Oct 2006 10:54:42 -0500 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: Travis Oliphant wrote: > I just learned about how to allow built-ins to have attributes assigned > to their instances. It's actually pretty easy because of Python > support for it --- but it comes at a cost. You have to add a dictionary > to the PyArrayObject structure, create that dictionary when the ndarray > is allocated, and set the tp_dictoffset in the TypeObject structure to > its location in PyArrayObject. It takes 4 lines of code with the cost of > creating a new dictionary for every ndarray. > > I don't think the extra bytes for every ndarray object are worth it, > given how easy it is to sub-class and create your own ndarray that can > have attributes attached. What are others opinions. I'd say leave it off. Many uses of that feature will require custom __array_finalize__ methods anyways. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david.huard at gmail.com Thu Oct 19 12:23:08 2006 From: david.huard at gmail.com (David Huard) Date: Thu, 19 Oct 2006 12:23:08 -0400 Subject: Dimension reduction Message-ID: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Hi, Is there an elegant way to reduce an array but conserve the reduced dimension ? Currently, >>> a = random.random((10,10,10)) >>> a.sum(1).shape (10,10) but i'd like to keep (10,1,10) so I can do a/a.sum(1) directly. Thanks, David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Thu Oct 19 12:39:36 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 19 Oct 2006 11:39:36 -0500 Subject: Dimension reduction In-Reply-To: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> References: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Message-ID: David Huard wrote: > Hi, > > Is there an elegant way to reduce an array but conserve the reduced > dimension ? > > Currently, > >>> a = random.random((10,10,10)) > >>> a.sum(1).shape > (10,10) > > but i'd like to keep (10,1,10) so I can do a/a.sum(1) directly. def nonreducing_reducer(reducing_func, arr, axis): reduced = reducing_func(arr, axis=axis) shape = list(reduced.shape) axis = axis % len(arr.shape) shape.insert(axis, 1) reduced.shape = tuple(shape) return reduced I think. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 12:03:10 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 09:03:10 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <4537954D.1060301@ee.byu.edu> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <45379417.4060601@ieee.org> <4537954D.1060301@ee.byu.edu> Message-ID: <4537A1BE.9000606@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > > >> Travis Oliphant wrote: >> >> >> >>> Tim Hochberg wrote: >>> >>> >>> >>> >>>> Rudolph van der Merwe wrote: >>>> >>>> >>>> >>>> >>>> >>>>> I get the following error with RC3 on a RHE Linux box: >>>>> >>>>> Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >>>>> [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >>>>> Type "help", "copyright", "credits" or "license" for more information. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>>>> import numpy >>>>>>>> numpy.__version__ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>> '1.0rc3' >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> On a visual studio build, I'm getting a real failure though: >>>> >>>> FAIL: Ticket #112 >>>> ---------------------------------------------------------------------- >>>> Traceback (most recent call last): >>>> File >>>> "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", >>>> line 219, in check_longfloat_repr >>>> assert(str(a)[1:9] == str(a[0])[:8]) >>>> AssertionError >>>> >>>> The code in question is dependent on the spelling of INF(or at least the >>>> *length* of the spelling) on a given platform which is why it's failing. >>>> >>>> >>>> >>>> >>>> >>> Actually, you shouldn't be getting an INF at all. This is what the >>> test is designed to test for (so I guess it's working). The test was >>> actually written wrong and was never failing because previously keyword >>> arguments to ufuncs were ignored. >>> >>> Can you show us what 'a' is on your platform. >>> >>> >>> >> Sure: >> >> >>>>> import numpy as N >>>>> a = N.exp(N.array([1000],dtype=N.longfloat)) >>>>> >> Warning: overflow encountered in exp >> >>>>> a >>>>> >> array([1.#INF], dtype=float64) >> >> >> > O.K. We need to modify the test in case to check and see that the size > of longfloat isn't the same as double. > How about: def check_longfloat_repr(self,level=rlevel): """Ticket #112""" if N.dtype(N.longfloat) != N.dtype(N.float): a = N.exp(N.array([1000],dtype=N.longfloat)) assert(str(a)[1:9] == str(a[0])[:8]) That seems to work here. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 12:52:03 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 09:52:03 -0700 Subject: Dimension reduction In-Reply-To: References: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Message-ID: <4537AD33.30306@ieee.org> Robert Kern wrote: > David Huard wrote: > >> Hi, >> >> Is there an elegant way to reduce an array but conserve the reduced >> dimension ? >> >> Currently, >> >>> a = random.random((10,10,10)) >> >>> a.sum(1).shape >> (10,10) >> >> but i'd like to keep (10,1,10) so I can do a/a.sum(1) directly. >> > > def nonreducing_reducer(reducing_func, arr, axis): > reduced = reducing_func(arr, axis=axis) > shape = list(reduced.shape) > axis = axis % len(arr.shape) > shape.insert(axis, 1) > reduced.shape = tuple(shape) > return reduced > > > I think. > > Alternatively (untested): def nonreducing_reducer(reducing_func, arr, axis): return reducing_func(arr.swapaxis(0, axis), axis=0)[newaxis].swapaxis(0, axis) Adding some vertical whitespace probably wouldn't hurt for readability I suppose. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 19 12:54:52 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 19 Oct 2006 10:54:52 -0600 Subject: Dimension reduction In-Reply-To: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> References: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Message-ID: On 10/19/06, David Huard wrote: > > Hi, > > Is there an elegant way to reduce an array but conserve the reduced > dimension ? > > Currently, > >>> a = random.random((10,10,10)) > >>> a.sum(1).shape > (10,10) > > but i'd like to keep (10,1,10) so I can do a/a.sum(1) directly. In [8]: a.sum(1)[:,newaxis,:].shape Out[8]: (10, 1, 10) Don't know if this is as universal as you want, but it works for this. Robert's answer is more general. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Thu Oct 19 13:01:37 2006 From: david.huard at gmail.com (David Huard) Date: Thu, 19 Oct 2006 13:01:37 -0400 Subject: Dimension reduction In-Reply-To: References: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Message-ID: <91cf711d0610191001jd4c2193n478ff61f76e7acb0@mail.gmail.com> 2006/10/19, Robert Kern : > def nonreducing_reducer(reducing_func, arr, axis): > reduced = reducing_func(arr, axis=axis) > shape = list(reduced.shape) > axis = axis % len(arr.shape) > shape.insert(axis, 1) > reduced.shape = tuple(shape) > return reduced Cute ! Here is another one: 1dfunc = lambda x: atleast_1d(reducing_function(x)) apply_along_axis(1dfunc, axis, arr) Is it something people often need ? Would it be worth, (for numpy 1.1 ?), to consider complex axis arguments as a wish to preserve the rank of the array? >>> a.shape (n,m,o,p) >>> a.sum(axis=1j) (n,1,o,p) >>> a.sum(axis=1) (n,o,p) Thanks, David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Thu Oct 19 13:08:37 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 10:08:37 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <4537954D.1060301@ee.byu.edu> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <45379417.4060601@ieee.org> <4537954D.1060301@ee.byu.edu> Message-ID: <4537B115.8010207@ieee.org> [CHOP] OK, I've checked in changes to suppress all the warnings in the test suite. I tried to be as targeted as possible so that any regressions from the current state in terms of warnings should show up. I suspect that there may be some issues with regards to masked arrays issuing spurious warnings. I'm not a ma user, so I didn't take the time to investigate this, but someone who cares should probably look at this and consider how to resolve it. Also with regard to the recent addition of the errcall argument to errstate, it appears that if errcall is *not* specified, self.errcall will be set to None. This means that any existing error callback will be removed when entering the block (and replaced when you leave). I don't think that's the desired behavior. Instead, if errcall is not specified, I believe we should leave the value of errcall alone. Does that sound right, or am I missing something? This would result in something like: def __init__(self, **kwargs): if 'errcall' in kwargs: self.errcall = kwargs.pop('errcall') else: self.errcall = geterrcall() self.kwargs = kwargs This still allows you to set the errcall to None for the block by using: with errstate(errcall=None): # do stuff Hmm. Should that be 'errcall' or just 'call'? 'errstate(errcall...)' seems somewhat redundant. Finally, and this is a moot point anyway assuming the above rewrite, is there a reason to use: if 'errcall' in kwargs: self.errcall = kwargs.pop('errcall') else: self.errcall = geterrcall() rather than simply: self.errcall = kwargs.pop('errcall', None) ? regards, -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Thu Oct 19 14:09:37 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 19 Oct 2006 14:09:37 -0400 Subject: indexing Message-ID: John J. worked up short set of NumPy Indexing Examples that I think supplements what is at and so I made a couple edits and added them: Please edit as appropriate. Thank you, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jkgruet at sandia.gov Thu Oct 19 15:14:52 2006 From: jkgruet at sandia.gov (James K. Gruetzner) Date: Thu, 19 Oct 2006 13:14:52 -0600 Subject: numpy - scipy version hell Message-ID: <200610191314.54482.jkgruet@sandia.gov> I'm using Fedora core 5, that insect-laden wonder, which includes python 2.4.3. I'm trying to install numpy (scipy-core) and scipy (the rest). I can't seem to get around package conflicts. I've tried two methods, one in which numpy fails and one in which scipy fails. METHOD 1 (numpy fails) Download numpy (numpy-1.0r3); untarball and install. Test by running python and importing numpy. Looks OK, but notes "running from numpy source directory". So, test by running python from a different directory and importing numpy. Get the following error: >>> import numpy Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.4/site-packages/numpy/__init__.py", line 40, in ? import linalg File "/usr/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, in ? from linalg import * File "/usr/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 25, in ? from numpy.linalg import lapack_lite ImportError: /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: s_cat (The other method, where scipy fails, I'm asking for help on the scipy-user list.)) I suspect that there's a bit of out-of-sync with the packages, but am not sure. Bottom line question: how do I get both installed at once? Thanks! James -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Thu Oct 19 15:29:26 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 19 Oct 2006 14:29:26 -0500 Subject: numpy - scipy version hell In-Reply-To: <200610191314.54482.jkgruet@sandia.gov> References: <200610191314.54482.jkgruet@sandia.gov> Message-ID: James K. Gruetzner wrote: > I'm using Fedora core 5, that insect-laden wonder, which includes python > 2.4.3. I'm trying to install numpy (scipy-core) and scipy (the rest). I > can't seem to get around package conflicts. I've tried two methods, one in > which numpy fails and one in which scipy fails. > > METHOD 1 (numpy fails) > Download numpy (numpy-1.0r3); untarball and install. > Test by running python and importing numpy. Looks OK, but notes "running from > numpy source directory". So, test by running python from a different > directory and importing numpy. Get the following error: > >>>> import numpy > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/lib/python2.4/site-packages/numpy/__init__.py", line 40, in ? > import linalg > File "/usr/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, > in ? > from linalg import * > File "/usr/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 25, > in ? > from numpy.linalg import lapack_lite > ImportError: /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: > undefined symbol: s_cat It looks like you linked against a FORTRAN LAPACK, but didn't manage to link the FORTRAN runtime library libg2c. Can you give us the output of your build? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tgrav at mac.com Thu Oct 19 15:54:21 2006 From: tgrav at mac.com (Tommy Grav) Date: Thu, 19 Oct 2006 15:54:21 -0400 Subject: Installation problem Message-ID: <08626899-A3C2-4963-9B07-E2936BAFF9F9@mac.com> I am on a Mac OS X and are trying to install numpy/scipy/matplotlib ActivePython 2.4.3 Build 11 (ActiveState Software Inc.) based on Python 2.4.3 (#1, Apr 3 2006, 18:07:18) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.1.2881' >>> import scipy >>> scipy.__version__ '0.5.0.2095' >>> import pylab RuntimeError: module compiled against version 1000002 of C-API but this version of numpy is 1000000 The import of the numpy version of the nxutils module, _nsnxutils, failed. This is is either because numpy was unavailable when matplotlib was compiled, because a dependency of _nsnxutils could not be satisfied, or because the build flag for this module was turned off in setup.py. If it appears that _nsnxutils was not built, make sure you have a working copy of numpy and then re-install matplotlib. Otherwise, the following traceback gives more details: Traceback (most recent call last): File "", line 1, in ? File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/pylab.py", line 1, in ? from matplotlib.pylab import * File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/matplotlib/pylab.py", line 199, in ? import mlab #so I can override hist, psd, etc... File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/matplotlib/mlab.py", line 64, in ? import nxutils File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/matplotlib/nxutils.py", line 17, in ? from matplotlib._ns_nxutils import * ImportError: numpy.core.multiarray failed to import >>> Anyone know what the problem is? Cheers Tommy tgrav at mac.com http://homepage.mac.com/tgrav/ "Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction" -- Albert Einstein -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 19 16:39:55 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 19 Oct 2006 14:39:55 -0600 Subject: Installation problem In-Reply-To: <08626899-A3C2-4963-9B07-E2936BAFF9F9@mac.com> References: <08626899-A3C2-4963-9B07-E2936BAFF9F9@mac.com> Message-ID: On 10/19/06, Tommy Grav wrote: > > I am on a Mac OS X and are trying to install numpy/scipy/matplotlib > > ActivePython 2.4.3 Build 11 (ActiveState Software Inc.) based on > Python 2.4.3 (#1, Apr 3 2006, 18:07:18) > [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > >>> numpy.__version__ > '1.1.2881' > >>> import scipy > >>> scipy.__version__ > '0.5.0.2095' > >>> import pylab > RuntimeError: module compiled against version 1000002 of C-API but this > version of numpy is 1000000 > You need a later version of numpy. The latest is 1000006 so you will probably need to also recompile both matplotlib and scipy against numpy latest. This is a recent check that Travis put in to track C-API changes. If you don't compile your own I don't know what you can do except wait. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Thu Oct 19 16:46:48 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 19 Oct 2006 17:46:48 -0300 Subject: adding an attribute to an nd-array In-Reply-To: References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: On 10/19/06, Robert Kern wrote: > Travis Oliphant wrote: > > I don't think the extra bytes for every ndarray object are worth it, > > given how easy it is to sub-class and create your own ndarray that can > > have attributes attached. What are others opinions. > > I'd say leave it off. Many uses of that feature will require custom > __array_finalize__ methods anyways. > I second Travis and Robert. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Thu Oct 19 16:59:49 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 19 Oct 2006 13:59:49 -0700 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: <4537E745.7010403@noaa.gov> Travis Oliphant wrote: > Actually something as simple as > > class InfoArray(N.ndarray): > pass > > will allow you to add attributes to InfoArray. Well, sure, but how the heck do you initialize it? >>> class InfoArray(N.ndarray): ... pass ... >>> InfoArray((1,2,3)) InfoArray([[[ 6.61855173e-306, nan, 4.94708449e+173], [ 3.32457336e-309, 1.08825491e+175, 2.12468326e-314]]]) InfoArray(N.array((1,2,3))) InfoArray([[[ 2.17854722e-305, 1.90979621e-313, 1.90979621e-313], [ 1.90979621e-313, 1.90979621e-313, 1.03977794e-312]]]) I, for one, would like a very easy to subclass version of ndarray, maybe one that had a default constructor like numpy.array(). oh, and: Charles R Harris wrote: > I'd be more inclined to worry about speed. One of the drawbacks of > numarray was the time it took to create arrays. Since I often use many > small arrays, numpy was a big improvement in that area. +1 keep basic arrays simple and lean. Wasn't there one a UserArray class or something? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Thu Oct 19 17:00:09 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 19 Oct 2006 14:00:09 -0700 Subject: questions regarding assignement and copy In-Reply-To: References: <45360904.3030703@ar.media.kyoto-u.ac.jp> Message-ID: <4537E759.3050606@noaa.gov> Robert Kern wrote: > From the reference manual: > > http://docs.python.org/ref/augassign.html > > """An augmented assignment expression like x += 1 can be rewritten as x = x + 1 > to achieve a similar, but not exactly equal effect. In the augmented version, x > is only evaluated once. Also, when possible, the actual operation is performed > in-place, meaning that rather than creating a new object and assigning that to > the target, the old object is modified instead.""" I've always thought this was a mistake -- it is a source of weird errors and bugs. AFAIC, the augmented assignments should ONLY work with mutable objects, and ALWAYS do the operation in place. But then you couldn't write: i = 1 i += 1 Because python numbers are immutable. I think the problem here is that augmented assignment is solving two distinct problems: in place operations and nice syntax for incrementing. Oh well, it's not that big a deal, and usually the confusion causes errors that show up right away. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 17:34:57 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 14:34:57 -0700 Subject: adding an attribute to an nd-array In-Reply-To: <4537E745.7010403@noaa.gov> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <4537E745.7010403@noaa.gov> Message-ID: <4537EF81.3090402@ieee.org> Christopher Barker wrote: > Travis Oliphant wrote: > >> Actually something as simple as >> >> class InfoArray(N.ndarray): >> pass >> >> will allow you to add attributes to InfoArray. >> > > Well, sure, but how the heck do you initialize it? > > >>> class InfoArray(N.ndarray): > ... pass > ... > >>> InfoArray((1,2,3)) > InfoArray([[[ 6.61855173e-306, nan, 4.94708449e+173], > [ 3.32457336e-309, 1.08825491e+175, 2.12468326e-314]]]) > > InfoArray(N.array((1,2,3))) > InfoArray([[[ 2.17854722e-305, 1.90979621e-313, 1.90979621e-313], > [ 1.90979621e-313, 1.90979621e-313, 1.03977794e-312]]]) > > I, for one, would like a very easy to subclass version of ndarray, maybe > one that had a default constructor like numpy.array(). > > Back when I was working on basearray, I had an idea for this. Basically, it was to to move the ndarray constructor[1] over to basearray and make the ndarray constructor work just like array does now[2]. Thus array(args) and ndarray(args) would behave the same[2 again]. For the full power/complexity of the current ndarray constructor, you would instead use basearray(). I believe (and I did some experiments on this at one point), that this would allow straightforward inheritance from ndarray. I've been away from this for several months now, so sadly I forget many of the details. -tim [1] I know, I know, it's really __new__, but it gets used like a constructor so I'll call it one for the time being. [2] Modulo the copy arg which would disappear if I had anything to with it. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From markusro at element.fkp.physik.tu-darmstadt.de Thu Oct 19 17:56:05 2006 From: markusro at element.fkp.physik.tu-darmstadt.de (Markus Rosenstihl) Date: Thu, 19 Oct 2006 23:56:05 +0200 Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 Message-ID: <5228203bb5a6003c571d52a5e94ef370@element.fkp.physik.tu-darmstadt.de> Hi! I try to compile numpy rc3 on Panther and get following errors. (I start build with "python2.3 setup.py build" to be sure to use the python shipped with OS X. I din't manage to compile Python2.5 either yet with similar errors) Does anynbody has an Idea? gcc-3.3 XCode 1.5 November gcc updater is installed regards Markus python2.3 setup.py build ... compile options: '-Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src -Inumpy/core/include -Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core -Inumpy/core/src -Inumpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/ python2.3 -c' gcc: numpy/core/src/multiarraymodule.c In file included from numpy/core/src/arrayobject.c:511, from numpy/core/src/multiarraymodule.c:63: numpy/core/src/arraytypes.inc.src: In function `LONGDOUBLE_scan': numpy/core/src/arraytypes.inc.src:883: warning: long double format, npy_longdouble arg (arg 3) gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/ multiarraymodule.o -o build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so ld: Undefined symbols: _fstatvfs referenced from Python expected to be defined in libSystem _lchown referenced from Python expected to be defined in libSystem _statvfs referenced from Python expected to be defined in libSystem ld: Undefined symbols: _fstatvfs referenced from Python expected to be defined in libSystem _lchown referenced from Python expected to be defined in libSystem _statvfs referenced from Python expected to be defined in libSystem error: Command "gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/ multiarraymodule.o -o build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so" failed with exit status 1 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mrosenstihl at macnews.de Thu Oct 19 17:55:37 2006 From: mrosenstihl at macnews.de (Markus Rosenstihl) Date: Thu, 19 Oct 2006 23:55:37 +0200 Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 Message-ID: <2370f9608954a8ab2caf96b20612a349@macnews.de> Hi! I try to compile numpy rc3 on Panther and get following errors. (I start build with "python2.3 setup.py build" to be sure to use the python shipped with OS X. I din't manage to compile Python2.5 either yet with similar errors) Does anynbody has an Idea? gcc-3.3 XCode 1.5 November gcc updater is installed regards Markus python2.3 setup.py build ... compile options: '-Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src -Inumpy/core/include -Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core -Inumpy/core/src -Inumpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/ python2.3 -c' gcc: numpy/core/src/multiarraymodule.c In file included from numpy/core/src/arrayobject.c:511, from numpy/core/src/multiarraymodule.c:63: numpy/core/src/arraytypes.inc.src: In function `LONGDOUBLE_scan': numpy/core/src/arraytypes.inc.src:883: warning: long double format, npy_longdouble arg (arg 3) gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/ multiarraymodule.o -o build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so ld: Undefined symbols: _fstatvfs referenced from Python expected to be defined in libSystem _lchown referenced from Python expected to be defined in libSystem _statvfs referenced from Python expected to be defined in libSystem ld: Undefined symbols: _fstatvfs referenced from Python expected to be defined in libSystem _lchown referenced from Python expected to be defined in libSystem _statvfs referenced from Python expected to be defined in libSystem error: Command "gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/ multiarraymodule.o -o build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so" failed with exit status 1 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2427 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 19 19:00:38 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 19 Oct 2006 17:00:38 -0600 Subject: adding an attribute to an nd-array In-Reply-To: <4537E745.7010403@noaa.gov> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <4537E745.7010403@noaa.gov> Message-ID: <45380396.7040300@ee.byu.edu> Christopher Barker wrote: >Travis Oliphant wrote: > > >>Actually something as simple as >> >>class InfoArray(N.ndarray): >> pass >> >>will allow you to add attributes to InfoArray. >> >> > >Well, sure, but how the heck do you initialize it? > > You use the same constructor as ndarray has. numpy.info(numpy.ndarray) If you want an array-like function that produces the array, you do array(obj).view(InfoArray) or wrap that up in your own function. There are many over-lapping ways to construct ndarray's. You can use all of these ways to construct instances of your own type by getting the ndarray and using the .view() method. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 19 19:09:44 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 20 Oct 2006 01:09:44 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <4537E745.7010403@noaa.gov> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <4537E745.7010403@noaa.gov> Message-ID: <20061019230944.GB10843@mentat.za.net> On Thu, Oct 19, 2006 at 01:59:49PM -0700, Christopher Barker wrote: > Travis Oliphant wrote: > > Actually something as simple as > > > > class InfoArray(N.ndarray): > > pass > > > > will allow you to add attributes to InfoArray. > > Well, sure, but how the heck do you initialize it? Looks like x = N.array([1,2,3]) x.view(InfoArray) works. Thanks to everyone for the useful feedback! Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 19 20:19:37 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 20 Oct 2006 02:19:37 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: <20061020001937.GC10843@mentat.za.net> On Thu, Oct 19, 2006 at 09:45:02AM -0600, Travis Oliphant wrote: > Stefan van der Walt wrote: > > >If I understand correctly, the following should work: > > > >import numpy as N > > > >class InfoArray(N.ndarray): > > def __new__(info_arr_cls,arr,info={}): > > info_arr_cls.info = info > > return N.array(arr).view(info_arr_cls) One has to be careful of this approach. It ads *the same* information to all arrays, i.e. In [2]: a = ImageInfo(N.array([1,2,3]),{1:1}) In [3]: b = ImageInfo(N.array([1,2,3]),{1:2}) In [4]: a Out[4]: ImageInfo([1, 2, 3]) In [5]: b Out[5]: ImageInfo([1, 2, 3]) In [6]: a.info Out[6]: {1: 2} In [7]: b.info Out[7]: {1: 2} Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From parlar at gmail.com Thu Oct 19 20:53:32 2006 From: parlar at gmail.com (Jay Parlar) Date: Thu, 19 Oct 2006 20:53:32 -0400 Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 Message-ID: > Hi! > I try to compile numpy rc3 on Panther and get following errors. > (I start build with "python2.3 setup.py build" to be sure to use the > python shipped with OS X. I din't manage to compile Python2.5 either > yet with similar errors) > Does anynbody has an Idea? > gcc-3.3 > XCode 1.5 > November gcc updater is installed > I couldn't get numpy building with Python 2.5 on 10.3.9 (although I had different compile errors). The solution that ended up working for me was Python 2.4. There's a bug in the released version of Python 2.5 that's preventing it from working with numpy, should be fixed in the next release. You can find a .dmg for Python 2.4 here: http://pythonmac.org/packages/py24-fat/index.html Jay P. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Thu Oct 19 21:06:23 2006 From: pgmdevlist at gmail.com (P GM) Date: Thu, 19 Oct 2006 21:06:23 -0400 Subject: Stefan van der Walt Message-ID: <777651ce0610191806s3b394699s9ad7072094757595@mail.gmail.com> > > >class InfoArray(N.ndarray): > > > def __new__(info_arr_cls,arr,info={}): > > > info_arr_cls.info = info > > > return N.array(arr).view(info_arr_cls) > > One has to be careful of this approach. It ads *the same* information > to all arrays, i.e. Indeed. That's basically why you have to edit your __array_finalize__ . class InfoArray(N.ndarray): def __new__(info_arr_cls,arr,info={}): info_arr_cls._info = info return N.array(arr).view(info_arr_cls) def __array_finalize__(self, obj): if hasattr(obj,'info'): self.info = obj.info else: self.info = self._info return OK, so you end up w/ two attributes 'info' and '_info', the latter having the info you want, the latter playing a temporary placeholder. That looks a bit overkill, but that works pretty nice. a = InfoArray(N.array([1,2,3]),{1:1}) b = InfoArray(N.array([1,2,3]),{1:2}) assert a.info=={1:1} assert b.info=={1:2} assert (a+1).info==a.info assert (b-2).info==b.info -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Thu Oct 19 21:03:57 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Thu, 19 Oct 2006 21:03:57 -0400 Subject: adding an attribute to an nd-array In-Reply-To: <20061020001937.GC10843@mentat.za.net> References: <20061019002914.GA20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <20061020001937.GC10843@mentat.za.net> Message-ID: <200610192103.58265.pgmdevlist@gmail.com> > > >class InfoArray(N.ndarray): > > > def __new__(info_arr_cls,arr,info={}): > > > info_arr_cls.info = info > > > return N.array(arr).view(info_arr_cls) > > One has to be careful of this approach. It ads *the same* information > to all arrays, i.e. Indeed. That's basically why you have to edit your __array_finalize__ . class InfoArray(N.ndarray): def __new__(info_arr_cls,arr,info={}): info_arr_cls._info = info return N.array(arr).view(info_arr_cls) def __array_finalize__(self, obj): if hasattr(obj,'info'): self.info = obj.info else: self.info = self._info return OK, so you end up w/ two attributes 'info' and '_info', the latter having the info you want, the latter playing a temporary placeholder. That looks a bit overkill, but that works pretty nice. a = InfoArray(N.array([1,2,3]),{1:1}) b = InfoArray(N.array([1,2,3]),{1:2}) assert a.info=={1:1} assert b.info=={1:2} assert (a+1).info==a.info assert (b-2).info==b.info ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 20 03:01:46 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 20 Oct 2006 03:01:46 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4535B0F1.1000603@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> <4535B0F1.1000603@ieee.org> Message-ID: On 18/10/06, Travis Oliphant wrote: > If there are any cases satisfying these rules where a copy does not have > to occur then let me know. For example, zeros((4,4))[:,1].reshape((2,2)) need not be copied. I filed a bug in trac and supplied a patch to multiarray.c that avoids copies in PyArray_NewShape unless absolutely necessary. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Fri Oct 20 03:31:03 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 20 Oct 2006 01:31:03 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> <4535B0F1.1000603@ieee.org> Message-ID: <45387B37.8080303@ee.byu.edu> A. M. Archibald wrote: >On 18/10/06, Travis Oliphant wrote: > > > >>If there are any cases satisfying these rules where a copy does not have >>to occur then let me know. >> >> > >For example, zeros((4,4))[:,1].reshape((2,2)) need not be copied. > >I filed a bug in trac and supplied a patch to multiarray.c that avoids >copies in PyArray_NewShape unless absolutely necessary. > > > Very, very nice. Thanks. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Fri Oct 20 04:47:05 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 20 Oct 2006 10:47:05 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <200610192103.58265.pgmdevlist@gmail.com> References: <20061019002914.GA20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <20061020001937.GC10843@mentat.za.net> <200610192103.58265.pgmdevlist@gmail.com> Message-ID: <20061020084705.GD10843@mentat.za.net> On Thu, Oct 19, 2006 at 09:03:57PM -0400, Pierre GM wrote: > Indeed. That's basically why you have to edit your __array_finalize__ . > > class InfoArray(N.ndarray): > def __new__(info_arr_cls,arr,info={}): > info_arr_cls._info = info > return N.array(arr).view(info_arr_cls) > def __array_finalize__(self, obj): > if hasattr(obj,'info'): > self.info = obj.info > else: > self.info = self._info > return > > OK, so you end up w/ two attributes 'info' and '_info', the latter having the > info you want, the latter playing a temporary placeholder. That looks a bit > overkill, but that works pretty nice. Is there any reason why one can't simply do class InfoArray(N.ndarray): def __new__(info_arr_cls,arr,info={}): x = N.array(arr).view(info_arr_cls) x.info = info return x def __array_finalize__(self, obj): if hasattr(obj,'info'): self.info = obj.info return Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Sebastien.Bardeau at obs.u-bordeaux1.fr Fri Oct 20 05:42:26 2006 From: Sebastien.Bardeau at obs.u-bordeaux1.fr (Sebastien Bardeau) Date: Fri, 20 Oct 2006 11:42:26 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? Message-ID: <45389A02.30901@obs.u-bordeaux1.fr> Hi! I am confused with Numpy behavior with its scalar or 0-d arrays objects: >>> numpy.__version__ '1.0rc2' >>> a = numpy.array((1,2,3)) >>> b = a[:2] >>> b += 1 >>> b array([2, 3]) >>> a array([2, 3, 3]) >>> type(b) To this point all is ok for me: subarrays share (by default) memory with their parent array. But: >>> c = a[2] >>> c += 1 >>> c 4 >>> a array([2, 3, 3]) >>> type(c) >>> id(c) 169457808 >>> c += 1 >>> id(c) 169737448 That's really confusing, because slices (from __getslice__ method) are not copies (they share memory), and items (single elements from __getitem__ ) are copies to one of the scalar objects provided by Numpy. I can understand that numpy.scalars do not provide inplace operations (like Python standard scalars, they are immutable), so I'd like to use 0-d Numpy.ndarrays. But: >>> d = numpy.array(a[2],copy=False) >>> d += 1 >>> d array(4) >>> a array([2, 3, 3]) >>> type(d) >>> d.shape () >>> id(d) 169621280 >>> d += 1 >>> id(d) 169621280 This is not a solution because d is a copy since construction time... My question is: is there a way to get a single element of an array into a 0-d array which shares memory with its parent array? Thx for your help, Sebastien ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From EAGF at HOTMAIL.COM Fri Oct 20 06:54:15 2006 From: EAGF at HOTMAIL.COM (=?iso-2022-jp?B?RUFHRg==?=) Date: Fri, 20 Oct 2006 03:54:15 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCJCQkRCRHJGIkSSQzJEcbKEI=?= =?iso-2022-jp?b?GyRCJGIlJCUrJDk9UDJxJCRFNzlxGyhC?= Message-ID: ?????????????????????????? http://carmastorra.com/ika/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Fri Oct 20 06:55:57 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 20 Oct 2006 12:55:57 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <45389A02.30901@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> Message-ID: <200610201255.57968.faltet@carabos.com> A Divendres 20 Octubre 2006 11:42, Sebastien Bardeau va escriure: [snip] > I can understand that numpy.scalars do not provide inplace operations > (like Python standard scalars, they are immutable), so I'd like to use > > 0-d Numpy.ndarrays. But: > >>> d = numpy.array(a[2],copy=False) > >>> d += 1 > >>> d > > array(4) > > >>> a > > array([2, 3, 3]) > > >>> type(d) > > > > >>> d.shape > > () > > >>> id(d) > > 169621280 > > >>> d += 1 > >>> id(d) > > 169621280 > > This is not a solution because d is a copy since construction time... > My question is: is there a way to get a single element of an array into > a 0-d array which shares memory with its parent array? One possible solution (there can be more) is using ndarray: In [47]: a=numpy.array([1,2,3], dtype="i4") In [48]: n=1 # the position that you want to share In [49]: b=numpy.ndarray(buffer=a[n:n+1], shape=(), dtype="i4") In [50]: a Out[50]: array([1, 2, 3]) In [51]: b Out[51]: array(2) In [52]: b += 1 In [53]: b Out[53]: array(3) In [54]: a Out[54]: array([1, 3, 3]) Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From markusro at element.fkp.physik.tu-darmstadt.de Fri Oct 20 07:13:49 2006 From: markusro at element.fkp.physik.tu-darmstadt.de (Markus Rosenstihl) Date: Fri, 20 Oct 2006 13:13:49 +0200 Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 In-Reply-To: References: Message-ID: <59a9095b29d3fa277f2e853a35590c72@element.fkp.physik.tu-darmstadt.de> Am 20.10.2006 um 02:53 schrieb Jay Parlar: >> Hi! >> I try to compile numpy rc3 on Panther and get following errors. >> (I start build with "python2.3 setup.py build" to be sure to use the >> python shipped with OS X. I din't manage to compile Python2.5 either >> yet with similar errors) >> Does anynbody has an Idea? >> gcc-3.3 >> XCode 1.5 >> November gcc updater is installed >> > > I couldn't get numpy building with Python 2.5 on 10.3.9 (although I > had different compile errors). The solution that ended up working for > me was Python 2.4. There's a bug in the released version of Python 2.5 > that's preventing it from working with numpy, should be fixed in the > next release. > > You can find a .dmg for Python 2.4 here: > http://pythonmac.org/packages/py24-fat/index.html > > Jay P. > I have that installed already but i get some bus errors with that. Furthermore it is built with gcc4 and i need to compile an extra module(pytables) and I fear that will not work, hence I try to compile myself. Python 2.5 dosent't compile either (libSystemStubs is only on Tiger). The linking works when i remove the -lSystemStubs and it compiled clean. Numpy rc3 wass also compiling now with python 2.5, but the tests failed: Python 2.5 (r25:51908, Oct 20 2006, 11:40:08) [GCC 3.3 20030304 (Apple Computer, Inc. build 1671)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.test(10) Found 5 tests for numpy.distutils.misc_util Found 4 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 32 tests for numpy.linalg Found 13 tests for numpy.core.umath Found 4 tests for numpy.core.scalarmath Found 9 tests for numpy.lib.arraysetops Found 42 tests for numpy.lib.type_check Found 183 tests for numpy.core.multiarray Found 3 tests for numpy.fft.helper Found 36 tests for numpy.core.ma Found 1 tests for numpy.lib.ufunclike Found 12 tests for numpy.lib.twodim_base Found 10 tests for numpy.core.defmatrix Found 4 tests for numpy.ctypeslib Found 41 tests for numpy.lib.function_base Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 28 tests for numpy.core.numeric Found 4 tests for numpy.lib.index_tricks Found 47 tests for numpy.lib.shape_base Found 0 tests for __main__ ........................................................................ ................................Warning: invalid value encountered in divide ..Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ..Warning: invalid value encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ..Warning: invalid value encountered in divide ..Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ........Warning: invalid value encountered in divide .Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide ........................................................................ ..............................................Warning: overflow encountered in exp F....................................................................... ..........Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide .....................Warning: invalid value encountered in sqrt Warning: invalid value encountered in log Warning: invalid value encountered in log10 ..Warning: invalid value encountered in sqrt Warning: invalid value encountered in sqrt Warning: divide by zero encountered in log Warning: divide by zero encountered in log Warning: divide by zero encountered in log10 Warning: divide by zero encountered in log10 Warning: invalid value encountered in arcsin Warning: invalid value encountered in arcsin Warning: invalid value encountered in arccos Warning: invalid value encountered in arccos Warning: invalid value encountered in arccosh Warning: invalid value encountered in arccosh Warning: divide by zero encountered in arctanh Warning: divide by zero encountered in arctanh Warning: invalid value encountered in divide Warning: invalid value encountered in true_divide Warning: invalid value encountered in floor_divide Warning: invalid value encountered in remainder Warning: invalid value encountered in fmod ........................................................................ ........................................................................ ................. ====================================================================== FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- packages/numpy/core/tests/test_regression.py", line 219, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError ---------------------------------------------------------------------- Ran 519 tests in 3.760s FAILED (failures=1) >>> regards Markus ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gael.varoquaux at normalesup.org Fri Oct 20 07:28:52 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 20 Oct 2006 13:28:52 +0200 Subject: Helper function to "unroll" a array Message-ID: <20061020112848.GB3555@clipper.ens.fr> Hi, There is an operation I do a lot, I would call it "unrolling" a array. The best way to describe it is probably to give the code: def unroll(M): """ Flattens the array M and returns a 2D array with the first columns being the indices of M, and the last column the flatten M. """ return hstack((indices(M.shape).reshape(-1,M.ndim),M.reshape(-1,1))) Example: >>> M array([[ 0.73530097, 0.3553424 , 0.3719772 ], [ 0.83353373, 0.74622133, 0.14748905], [ 0.72023762, 0.32306969, 0.19142366]]) >>> unroll(M) array([[ 0. , 0. , 0.73530097], [ 0. , 1. , 0.3553424 ], [ 1. , 1. , 0.3719772 ], [ 2. , 2. , 0.83353373], [ 2. , 0. , 0.74622133], [ 1. , 2. , 0.14748905], [ 0. , 1. , 0.72023762], [ 2. , 0. , 0.32306969], [ 1. , 2. , 0.19142366]]) The docstring sucks. The function is trivial (when you know numpy a bit). Maybe this function already exists in numpy, if so I couldn't find it. Elsewhere I propose it for inclusion. Cheers, Ga?l ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Fri Oct 20 09:22:33 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 20 Oct 2006 15:22:33 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <45389A02.30901@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> Message-ID: <20061020132232.GE10843@mentat.za.net> On Fri, Oct 20, 2006 at 11:42:26AM +0200, Sebastien Bardeau wrote: > >>> a = numpy.array((1,2,3)) > >>> b = a[:2] Here you index by a slice. > >>> c = a[2] Whereas here you index by a scalar. So you want to do b = a[[2]] b += 1 or in the general case b = a[slice(2,3)] b += 1 Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Sebastien.Bardeau at obs.u-bordeaux1.fr Fri Oct 20 09:25:49 2006 From: Sebastien.Bardeau at obs.u-bordeaux1.fr (Sebastien Bardeau) Date: Fri, 20 Oct 2006 15:25:49 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <200610201255.57968.faltet@carabos.com> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> Message-ID: <4538CE5D.5000903@obs.u-bordeaux1.fr> > One possible solution (there can be more) is using ndarray: > > In [47]: a=numpy.array([1,2,3], dtype="i4") > In [48]: n=1 # the position that you want to share > In [49]: b=numpy.ndarray(buffer=a[n:n+1], shape=(), dtype="i4") > Ok thanks. Actually that was also the solution I found. But this is much more complicated when arrays are N dimensional with N>1, and above all if user asks for a slice in one or more dimension. Here is how I redefine the __getitem__ method for my arrays. Remember that the goal is to return a 0-d array rather than a numpy.scalar when I extract a single element out of a N-dimensional (N>=1) array: def __getitem__(self,index): # Index may be either an int or a tuple # Index length: if type(index) == int: # A single element through first dimension ilen = 1 index = (index,) # A tuple else: ilen = len(index) # Array rank: arank = len(self.shape) # Check if there is a slice: for i in index: if type(i) == slice: hasslice = True else: hasslice = False # Array is already a 0-d array: if arank == 0 and index == (0,): return self[()] elif arank == 0: raise IndexError, "0-d array has only one element at index 0." # This will return a single element as a 0-d array: elif arank == ilen and hasslice: # This ugly thing returns a numpy 0-D array AND NOT a numpy scalar! # (Numpy scalars do not share their data with the parent array) newindex = list(index) newindex[0] = slice(index[0],index[0]+1,None) newindex = tuple(newindex) return self[newindex].reshape(()) # This will return a n-D subarray (n>=1): else: return self[index] Well... I do not think this is very nice. Someone has another idea? My question in my first post was: is there a way to get a single element of an array into a 0-d array which shares memory with its parent array? Sebastien ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 20 09:44:58 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 06:44:58 -0700 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <200610201255.57968.faltet@carabos.com> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> Message-ID: <4538D2DA.9010604@ieee.org> Francesc Altet wrote: > A Divendres 20 Octubre 2006 11:42, Sebastien Bardeau va escriure: > [snip] > >> I can understand that numpy.scalars do not provide inplace operations >> (like Python standard scalars, they are immutable), so I'd like to use >> >> 0-d Numpy.ndarrays. But: >> >>> d = numpy.array(a[2],copy=False) >> >>> d += 1 >> >>> d >> >> array(4) >> >> >>> a >> >> array([2, 3, 3]) >> >> >>> type(d) >> >> >> >> >>> d.shape >> >> () >> >> >>> id(d) >> >> 169621280 >> >> >>> d += 1 >> >>> id(d) >> >> 169621280 >> >> This is not a solution because d is a copy since construction time... >> My question is: is there a way to get a single element of an array into >> a 0-d array which shares memory with its parent array? >> > > One possible solution (there can be more) is using ndarray: [SNIP] Here's a slightly more concise version of the same idea: b = a[n:n+1].reshape([]) -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Sebastien.Bardeau at obs.u-bordeaux1.fr Fri Oct 20 09:49:15 2006 From: Sebastien.Bardeau at obs.u-bordeaux1.fr (Sebastien Bardeau) Date: Fri, 20 Oct 2006 15:49:15 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <4538CE5D.5000903@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> <4538CE5D.5000903@obs.u-bordeaux1.fr> Message-ID: <4538D3DB.6040707@obs.u-bordeaux1.fr> Ooops sorry there was two mistakes with the 'hasslice' flag. This seems now to work for me. def __getitem__(self,index): # Index may be either an int or a tuple # Index length: if type(index) == int: # A single element through first dimension ilen = 1 index = (index,) # A tuple else: ilen = len(index) # Array rank: arank = len(self.shape) # Check if there is a slice: hasslice = False for i in index: if type(i) == slice: hasslice = True # Array is already a 0-d array: if arank == 0 and index == (0,): return self elif arank == 0: raise IndexError, "0-d array has only one element at index 0." # This will return a single element as a 0-d array: elif arank == ilen and not hasslice: # This ugly thing returns a numpy 0-D array AND NOT a numpy scalar! # (Numpy scalars do not share their data with the parent array) newindex = list(index) newindex[0] = slice(index[0],index[0]+1,None) newindex = tuple(newindex) return self[newindex].reshape(()) # This will return a n-D subarray (n>=1): else: return self[index] Sebastien Bardeau wrote: >> One possible solution (there can be more) is using ndarray: >> >> In [47]: a=numpy.array([1,2,3], dtype="i4") >> In [48]: n=1 # the position that you want to share >> In [49]: b=numpy.ndarray(buffer=a[n:n+1], shape=(), dtype="i4") >> >> > Ok thanks. Actually that was also the solution I found. But this is much > more complicated when arrays are N dimensional with N>1, and above all > if user asks for a slice in one or more dimension. Here is how I > redefine the __getitem__ method for my arrays. Remember that the goal is > to return a 0-d array rather than a numpy.scalar when I extract a single > element out of a N-dimensional (N>=1) array: > > def __getitem__(self,index): # Index may be either an int or a tuple > # Index length: > if type(index) == int: # A single element through first dimension > ilen = 1 > index = (index,) # A tuple > else: > ilen = len(index) > # Array rank: > arank = len(self.shape) > # Check if there is a slice: > for i in index: > if type(i) == slice: > hasslice = True > else: > hasslice = False > # Array is already a 0-d array: > if arank == 0 and index == (0,): > return self[()] > elif arank == 0: > raise IndexError, "0-d array has only one element at index 0." > # This will return a single element as a 0-d array: > elif arank == ilen and hasslice: > # This ugly thing returns a numpy 0-D array AND NOT a numpy scalar! > # (Numpy scalars do not share their data with the parent array) > newindex = list(index) > newindex[0] = slice(index[0],index[0]+1,None) > newindex = tuple(newindex) > return self[newindex].reshape(()) > # This will return a n-D subarray (n>=1): > else: > return self[index] > > Well... I do not think this is very nice. Someone has another idea? My > question in my first post was: is there a way to get a single element of > an array into > a 0-d array which shares memory with its parent array? > > Sebastien > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > -- ------------------------- Sebastien Bardeau L3AB - CNRS UMR 5804 2 rue de l'observatoire BP 89 F - 33270 Floirac Tel: (+33) 5 57 77 61 46 ------------------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 20 10:17:32 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 07:17:32 -0700 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <4538D3DB.6040707@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> <4538CE5D.5000903@obs.u-bordeaux1.fr> <4538D3DB.6040707@obs.u-bordeaux1.fr> Message-ID: <4538DA7C.5020806@ieee.org> Sebastien Bardeau wrote: > Ooops sorry there was two mistakes with the 'hasslice' flag. This seems > now to work for me. > > [SNIP code] That looks overly complicated. I believe that this (minimally tested in a slightly different setting) or some variation should work: return self[...,newaxis][index].reshape(self[index].shape) -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david.huard at gmail.com Fri Oct 20 13:47:28 2006 From: david.huard at gmail.com (David Huard) Date: Fri, 20 Oct 2006 13:47:28 -0400 Subject: histogram complete makeover In-Reply-To: <4536727D.4020308@ieee.org> References: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> <331116dc0610181100k474af71akac41e54db7b7bb41@mail.gmail.com> <4536727D.4020308@ieee.org> Message-ID: <91cf711d0610201047oaa1e566sb735e6771529ab8e@mail.gmail.com> Thanks for the comments, Here is the code for the new histogram, tests included. I'll wait for comments or suggestions before submitting a patch (numpy / scipy) ? Cheers David 2006/10/18, Tim Hochberg : > > > My $0.02: > > If histogram is going to get a makeover, particularly one that makes it > more complex than at present, it should probably be moved to SciPy. > Failing that, it should be moved to a submodule of numpy with similar > statistical tools. Preferably with consistent interfaces for all of the > functions. > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: histogram1d.py Type: text/x-python Size: 6489 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From josh8912 at yahoo.com Fri Oct 20 16:27:14 2006 From: josh8912 at yahoo.com (JJ) Date: Fri, 20 Oct 2006 13:27:14 -0700 (PDT) Subject: slicing suggestion Message-ID: <20061020202714.12071.qmail@web51702.mail.yahoo.com> Hello. I have a suggestion that might make slicing using matrices more user-friendly. I often have a matrix of row or column numbers that I wish to use as a slice. If K was a matrix of row numbers (nx1) and M was a nxm matrix, then I would use ans = M[K.A.ravel(),:] to obtain the matrix I want. It turns out that I use .A.ravel() quite a lot in my code, as I usually work with matrices rather than arrays. My suggestion is to create a new attribute, such as .AR, so that the following could be used: M[K.AR,:]. I believe this would be more concise, easier to read, and well used. If slices are made in both directions of the matrix, then the .A.ravel() becomes even more unwieldy. Does anyone else like this idea? John __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Fri Oct 20 16:37:51 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Fri, 20 Oct 2006 13:37:51 -0700 Subject: slicing suggestion In-Reply-To: <20061020202714.12071.qmail@web51702.mail.yahoo.com> References: <20061020202714.12071.qmail@web51702.mail.yahoo.com> Message-ID: On 10/20/06, JJ wrote: > My suggestion is to > create a new attribute, such as .AR, so that the > following could be used: M[K.AR,:] It would be even better if M[K,:] worked. Would such a patch be accepted? (Not that I know how to make it.) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sebzur at pin.if.uz.zgora.pl Fri Oct 20 17:33:47 2006 From: sebzur at pin.if.uz.zgora.pl (=?ISO-8859-2?Q?Sebastian_=AFurek?=) Date: Fri, 20 Oct 2006 23:33:47 +0200 Subject: Model and experiment fitting. Message-ID: Hi! This is probably a silly question but I'm getting confused with a certain problem: a comparison between experimental data points (2D points set) and a model (2D points set - no analytical form). The physical model produces (by a sophisticated simulations done by an external program) some 2D points data and one of my task is to compare those calculated data with an experimental one. The experimental and modeled data have form of 2D curves, build of n 2D-points, i.e.: expDat=[[x1,x2,x3,..xn],[y1,y2,y3,...,yn]] simDat=[[X1,X2,X3,...,Xn],[Y1,Y2,Y3,...,Yn]] The task of determining, let's say, a root mean squarred error (RMSe) is trivial if x1==X1, x2==X2, etc. In general, which is a common situation xk differs from Xk (k=0..n) and one may not simply compare succeeding Yk and yk (k=0..n) to determine the goodness-of-fit. The distance h=Xk-X(k-1) is constant, but similar distance m(k)=xk-x(k-1) depends on k-th point and is not a constant value, although the data array lengths for simulation and experiment are the same. My first idea was to do some interpolations to obtain the missing points, but I held it 'by a hand' (which, BTW gave quite rewarding results) and I suppose, there's some i.g. numpy method to do it for me, isn't it? I suppose to do something like: gfit(expDat,simDat,'measure_type') which I hope will return the number determining the goodness-of-fit (mean squarred error, root mean squarred error,...) of two sets of discrete 2D data points. Is there something like that in any numerical python modules (numpy, pylab) I could use? I can imagine, I can fit the data with some polynomial or whatever, and than compare the fitted data, but my goal is to operate on as raw data as it's possible. Thanks for your comments! Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Fri Oct 20 18:14:45 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 20 Oct 2006 17:14:45 -0500 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: Sebastian ?urek wrote: > Hi! > > This is probably a silly question but I'm getting confused with a > certain problem: a comparison between experimental data points (2D > points set) and a model (2D points set - no analytical form). > > The physical model produces (by a sophisticated simulations done by an > external program) some 2D points data and one of my task is to compare > those calculated data with an experimental one. > > The experimental and modeled data have form of 2D curves, build of n > 2D-points, i.e.: > > expDat=[[x1,x2,x3,..xn],[y1,y2,y3,...,yn]] > simDat=[[X1,X2,X3,...,Xn],[Y1,Y2,Y3,...,Yn]] > > The task of determining, let's say, a root mean squarred error (RMSe) > is trivial if x1==X1, x2==X2, etc. > > In general, which is a common situation xk differs from Xk (k=0..n) and > one may not simply compare succeeding Yk and yk (k=0..n) to determine > the goodness-of-fit. The distance h=Xk-X(k-1) is constant, but similar > distance m(k)=xk-x(k-1) depends on k-th point and is not a constant > value, although the data array lengths for simulation and experiment are > the same. Your description is a bit vague. Do you mean that you have some model function f that maps X values to Y values? f(x) -> y If that is the case, is there some reason that you cannot run your simulation using the same X points as your experimental data? OTOH, is there some other independent variable (say Z) that *is* common between your experimental and simulated data? f(z) -> (x, y) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Fri Oct 20 19:12:50 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 20 Oct 2006 17:12:50 -0600 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <4538CE5D.5000903@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> <4538CE5D.5000903@obs.u-bordeaux1.fr> Message-ID: <453957F2.8070507@ee.byu.edu> Sebastien Bardeau wrote: >>One possible solution (there can be more) is using ndarray: >> >>In [47]: a=numpy.array([1,2,3], dtype="i4") >>In [48]: n=1 # the position that you want to share >>In [49]: b=numpy.ndarray(buffer=a[n:n+1], shape=(), dtype="i4") >> >> >> >Ok thanks. Actually that was also the solution I found. But this is much >more complicated when arrays are N dimensional with N>1, and above all >if user asks for a slice in one or more dimension. Here is how I >redefine the __getitem__ method for my arrays. Remember that the goal is >to return a 0-d array rather than a numpy.scalar when I extract a single >element out of a N-dimensional (N>=1) array: > > How about this. To get the i,j,k,l element a[i:i+1,j:j+1,k:k+1,l:l+1].squeeze() -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 20 19:28:27 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 20 Oct 2006 19:28:27 -0400 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: On 20/10/06, Sebastian ?urek wrote: > Is there something like that in any numerical python modules (numpy, > pylab) I could use? In scipy there are some very convenient spline fitting tools which will allow you to fit a nice smooth spline through the simulation data points (or near, if they have some uncertainty); you can then easily look at the RMS difference in the y values. You can also, less easily, look at the distance from the curve allowing for some uncertainty in the x values. I suppose you could also fit a curve through the experimental points and compare the two curves in some way. > I can imagine, I can fit the data with some polynomial or whatever, > and than compare the fitted data, but my goal is to operate on > as raw data as it's possible. If you want to avoid using an a priori model, Numerical Recipes discuss some possible approaches ("Do two-dimensional distributions differ?" at http://www.nrbook.com/a/bookcpdf.html is one) but it's not clear how to turn the problem you describe into a solvable one - some assumption about how the models vary between sampled x values appears to be necessary, and that amounts to interpolation. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 20 22:42:22 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 20:42:22 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc Message-ID: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> Hi, i am running numpy on aix compiling with xlc. Revision 1.0rc2 works fine and passes all tests. But 1.0rc3 and more recent give the following on import: Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in add Warning: invalid value encountered in not_equal Warning: invalid value encountered in absolute Warning: invalid value encountered in less Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in equal Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in add Warning: invalid value encountered in not_equal Warning: invalid value encountered in absolute Warning: invalid value encountered in less Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in equal Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply [lots more of this] The odd thing is that all tests pass. I have looked, but can't find where this Warning is coming from in the code. Any thoughts on where this is coming from? What can I do to help debug this? I am not sure what revision introduced this issue. Thanks Brian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 20 23:10:15 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 20:10:15 -0700 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> Message-ID: <45398F97.2030509@ieee.org> Brian Granger wrote: > Hi, > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > fine and passes all tests. But 1.0rc3 and more recent give the > following on import: > > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in add > Warning: invalid value encountered in not_equal > Warning: invalid value encountered in absolute > Warning: invalid value encountered in less > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in equal > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in add > Warning: invalid value encountered in not_equal > Warning: invalid value encountered in absolute > Warning: invalid value encountered in less > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in equal > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > [lots more of this] > > The odd thing is that all tests pass. I have looked, but can't find > where this Warning is coming from in the code. Any thoughts on where > this is coming from? What can I do to help debug this? I am not sure > what revision introduced this issue. > The reason that you are seeing this now is that the default error state has been tightened up. There were some issues with tests failing as a result of this, but I believe I fixed those already and you're seeing this on import, not when running the tests correct? The first thing to do is figure out where the invalids are occurring, and the natural way to do that is to set the error state to raise, but you can't set the error state till you import it, so that's not going to help here. I think the first thing that I would try is to throw in a seterr(all='raise', under='ignore') right after the call to _setdef in numeric.py. If you're lucky, this will point out where the invalids are popping up. As a sanity check, you could instead make this seterr(all='ignore'), which should make all the warnings go away, but won't tell you anything about why there are warnings to begin with. Regards, -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 20 23:32:56 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 21:32:56 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45398F97.2030509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> Message-ID: <6ce0ac130610202032p4ea33e2agc64edb5e13e4f2c6@mail.gmail.com> Here is the traceback that I got: In [1]: import numpy --------------------------------------------------------------------------- exceptions.FloatingPointError Traceback (most recent call last) /u2/granger/ /usr/common/homes/g/granger/usr/local/lib/python/numpy/__init__.py 36 import core 37 from core import * ---> 38 import lib 39 from lib import * 40 import linalg /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/__init__.py 3 from numpy.version import version as __version__ 4 ----> 5 from type_check import * 6 from index_tricks import * 7 from function_base import * /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/type_check.py 8 import numpy.core.numeric as _nx 9 from numpy.core.numeric import asarray, array, isnan, obj2sctype, zeros ---> 10 from ufunclike import isneginf, isposinf 11 12 _typecodes_by_elsize = 'GDFgdfQqLlIiHhBb?' /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/ufunclike.py 44 return y 45 ---> 46 _log2 = umath.log(2) 47 def log2(x, y=None): 48 """Returns the base 2 logarithm of x FloatingPointError: invalid value encountered in log Obviously because I am having the error raised, I only get the first one. Hmmm. Brian On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Hi, > > > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > > fine and passes all tests. But 1.0rc3 and more recent give the > > following on import: > > > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > [lots more of this] > > > > The odd thing is that all tests pass. I have looked, but can't find > > where this Warning is coming from in the code. Any thoughts on where > > this is coming from? What can I do to help debug this? I am not sure > > what revision introduced this issue. > > > The reason that you are seeing this now is that the default error state > has been tightened up. There were some issues with tests failing as a > result of this, but I believe I fixed those already and you're seeing > this on import, not when running the tests correct? The first thing to > do is figure out where the invalids are occurring, and the natural way > to do that is to set the error state to raise, but you can't set the > error state till you import it, so that's not going to help here. > > I think the first thing that I would try is to throw in a > seterr(all='raise', under='ignore') right after the call to _setdef in > numeric.py. If you're lucky, this will point out where the invalids are > popping up. As a sanity check, you could instead make this > seterr(all='ignore'), which should make all the warnings go away, but > won't tell you anything about why there are warnings to begin with. > > Regards, > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 20 23:43:43 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 21:43:43 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45398F97.2030509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> Message-ID: <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> Also, when I use seterr(all='ignore') the the tests fail: ====================================================================== FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", line 219, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError ---------------------------------------------------------------------- Ran 516 tests in 0.823s FAILED (failures=1) Thanks for helping out on this. On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Hi, > > > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > > fine and passes all tests. But 1.0rc3 and more recent give the > > following on import: > > > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > [lots more of this] > > > > The odd thing is that all tests pass. I have looked, but can't find > > where this Warning is coming from in the code. Any thoughts on where > > this is coming from? What can I do to help debug this? I am not sure > > what revision introduced this issue. > > > The reason that you are seeing this now is that the default error state > has been tightened up. There were some issues with tests failing as a > result of this, but I believe I fixed those already and you're seeing > this on import, not when running the tests correct? The first thing to > do is figure out where the invalids are occurring, and the natural way > to do that is to set the error state to raise, but you can't set the > error state till you import it, so that's not going to help here. > > I think the first thing that I would try is to throw in a > seterr(all='raise', under='ignore') right after the call to _setdef in > numeric.py. If you're lucky, this will point out where the invalids are > popping up. As a sanity check, you could instead make this > seterr(all='ignore'), which should make all the warnings go away, but > won't tell you anything about why there are warnings to begin with. > > Regards, > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 20 23:49:36 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 21:49:36 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45398F97.2030509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> Message-ID: <6ce0ac130610202049w6dd1e84dp29651624f1917c22@mail.gmail.com> When I set seterr(all='warn') I see the following: In [1]: import numpy /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/ufunclike.py:46: RuntimeWarning: invalid value encountered in log _log2 = umath.log(2) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/scimath.py:19: RuntimeWarning: invalid value encountered in log _ln2 = nx.log(2.0) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:64: RuntimeWarning: invalid value encountered in add two = one + one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:65: RuntimeWarning: invalid value encountered in subtract zero = one - one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:71: RuntimeWarning: invalid value encountered in add a = a + a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:72: RuntimeWarning: invalid value encountered in add temp = a + one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:73: RuntimeWarning: invalid value encountered in subtract temp1 = temp - a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:74: RuntimeWarning: invalid value encountered in subtract if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:74: RuntimeWarning: invalid value encountered in not_equal if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:80: RuntimeWarning: invalid value encountered in add b = b + b /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:81: RuntimeWarning: invalid value encountered in add temp = a + b /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:82: RuntimeWarning: invalid value encountered in subtract itemp = int_conv(temp-a) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:83: RuntimeWarning: invalid value encountered in not_equal if any(itemp != 0): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:95: RuntimeWarning: invalid value encountered in multiply b = b * beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:96: RuntimeWarning: invalid value encountered in add temp = b + one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:97: RuntimeWarning: invalid value encountered in subtract temp1 = temp - b /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:98: RuntimeWarning: invalid value encountered in subtract if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:98: RuntimeWarning: invalid value encountered in not_equal if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:103: RuntimeWarning: invalid value encountered in divide betah = beta / two /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:106: RuntimeWarning: invalid value encountered in add a = a + a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:107: RuntimeWarning: invalid value encountered in add temp = a + one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:108: RuntimeWarning: invalid value encountered in subtract temp1 = temp - a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:109: RuntimeWarning: invalid value encountered in subtract if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:109: RuntimeWarning: invalid value encountered in not_equal if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:113: RuntimeWarning: invalid value encountered in add temp = a + betah /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:115: RuntimeWarning: invalid value encountered in subtract if any(temp-a != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:115: RuntimeWarning: invalid value encountered in not_equal if any(temp-a != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:117: RuntimeWarning: invalid value encountered in add tempa = a + beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:118: RuntimeWarning: invalid value encountered in add temp = tempa + betah /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:119: RuntimeWarning: invalid value encountered in subtract if irnd==0 and any(temp-tempa != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:119: RuntimeWarning: invalid value encountered in not_equal if irnd==0 and any(temp-tempa != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:124: RuntimeWarning: invalid value encountered in divide betain = one / beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:127: RuntimeWarning: invalid value encountered in multiply a = a * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:130: RuntimeWarning: invalid value encountered in subtract temp = one - a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:131: RuntimeWarning: invalid value encountered in subtract if any(temp-one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:131: RuntimeWarning: invalid value encountered in not_equal if any(temp-one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:133: RuntimeWarning: invalid value encountered in multiply a = a * beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:149: RuntimeWarning: invalid value encountered in add temp = one + a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:150: RuntimeWarning: invalid value encountered in subtract if any(temp-one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:150: RuntimeWarning: invalid value encountered in not_equal if any(temp-one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:152: RuntimeWarning: invalid value encountered in multiply a = a * beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:160: RuntimeWarning: invalid value encountered in add temp = one + eps /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:168: RuntimeWarning: invalid value encountered in add t = one + eps /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:172: RuntimeWarning: invalid value encountered in multiply z = y*y /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:173: RuntimeWarning: invalid value encountered in multiply a = z*one # Check here for underflow /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:174: RuntimeWarning: invalid value encountered in multiply temp = z*t /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:175: RuntimeWarning: invalid value encountered in add if any(a+a == zero) or any(abs(z)>=y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:175: RuntimeWarning: invalid value encountered in equal if any(a+a == zero) or any(abs(z)>=y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:175: RuntimeWarning: invalid value encountered in absolute if any(a+a == zero) or any(abs(z)>=y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:175: RuntimeWarning: invalid value encountered in greater_equal if any(a+a == zero) or any(abs(z)>=y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:177: RuntimeWarning: invalid value encountered in multiply temp1 = temp * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:178: RuntimeWarning: invalid value encountered in multiply if any(temp1*beta == z): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:178: RuntimeWarning: invalid value encountered in equal if any(temp1*beta == z): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:174: RuntimeWarning: underflow encountered in multiply temp = z*t /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:184: RuntimeWarning: invalid value encountered in not_equal if ibeta != 10: /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:198: RuntimeWarning: invalid value encountered in multiply y = y * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:199: RuntimeWarning: invalid value encountered in multiply a = y * one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:200: RuntimeWarning: invalid value encountered in multiply temp = y * t /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:201: RuntimeWarning: invalid value encountered in add if any(a+a != zero) and any(abs(y) < xmin): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:201: RuntimeWarning: invalid value encountered in not_equal if any(a+a != zero) and any(abs(y) < xmin): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:201: RuntimeWarning: invalid value encountered in absolute if any(a+a != zero) and any(abs(y) < xmin): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:201: RuntimeWarning: invalid value encountered in less if any(a+a != zero) and any(abs(y) < xmin): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:203: RuntimeWarning: invalid value encountered in multiply temp1 = temp * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:204: RuntimeWarning: invalid value encountered in multiply if any(temp1*beta == y) and any(temp != y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:204: RuntimeWarning: invalid value encountered in equal if any(temp1*beta == y) and any(temp != y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:203: RuntimeWarning: underflow encountered in multiply temp1 = temp * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:204: RuntimeWarning: invalid value encountered in not_equal if any(temp1*beta == y) and any(temp != y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:215: RuntimeWarning: invalid value encountered in not_equal if mx <= k + k - 3 and ibeta != 10: /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:223: RuntimeWarning: invalid value encountered in equal if ibeta == 2 and not i: /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:227: RuntimeWarning: invalid value encountered in not_equal if any(a != y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:229: RuntimeWarning: invalid value encountered in subtract xmax = one - epsneg /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:230: RuntimeWarning: invalid value encountered in multiply if any(xmax*one != xmax): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:230: RuntimeWarning: invalid value encountered in not_equal if any(xmax*one != xmax): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:232: RuntimeWarning: invalid value encountered in multiply xmax = xmax / (xmin*beta*beta*beta) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:232: RuntimeWarning: invalid value encountered in divide xmax = xmax / (xmin*beta*beta*beta) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:235: RuntimeWarning: invalid value encountered in equal if ibeta==2: /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:236: RuntimeWarning: invalid value encountered in add xmax = xmax + xmax /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:266: RuntimeWarning: invalid value encountered in add ten = two + two + two + two + two /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:267: RuntimeWarning: invalid value encountered in power resolution = ten ** (-self.precision) On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Hi, > > > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > > fine and passes all tests. But 1.0rc3 and more recent give the > > following on import: > > > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > [lots more of this] > > > > The odd thing is that all tests pass. I have looked, but can't find > > where this Warning is coming from in the code. Any thoughts on where > > this is coming from? What can I do to help debug this? I am not sure > > what revision introduced this issue. > > > The reason that you are seeing this now is that the default error state > has been tightened up. There were some issues with tests failing as a > result of this, but I believe I fixed those already and you're seeing > this on import, not when running the tests correct? The first thing to > do is figure out where the invalids are occurring, and the natural way > to do that is to set the error state to raise, but you can't set the > error state till you import it, so that's not going to help here. > > I think the first thing that I would try is to throw in a > seterr(all='raise', under='ignore') right after the call to _setdef in > numeric.py. If you're lucky, this will point out where the invalids are > popping up. As a sanity check, you could instead make this > seterr(all='ignore'), which should make all the warnings go away, but > won't tell you anything about why there are warnings to begin with. > > Regards, > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 20 23:55:49 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 20:55:49 -0700 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> Message-ID: <45399A45.6050509@ieee.org> Brian Granger wrote: > Also, when I use seterr(all='ignore') the the tests fail: > > ====================================================================== > FAIL: Ticket #112 > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", > line 219, in check_longfloat_repr > assert(str(a)[1:9] == str(a[0])[:8]) > AssertionError > > ---------------------------------------------------------------------- > Ran 516 tests in 0.823s > > FAILED (failures=1) > > Thanks for helping out on this. > How recent is your version? I just a problem that was causing this same failure yesterday -- if you checkout is older than that, you may want to get the most recent stuff from SVN and see if that fixes this. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Sat Oct 21 00:00:34 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 22:00:34 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45399A45.6050509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> <45399A45.6050509@ieee.org> Message-ID: <6ce0ac130610202100q1b13c2efm2e50fc3b32243447@mail.gmail.com> I have been doing these recent tests with 1.0rc3. I am building from trunk right now and we will see how that goes. Thanks for your help. Brian On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Also, when I use seterr(all='ignore') the the tests fail: > > > > ====================================================================== > > FAIL: Ticket #112 > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", > > line 219, in check_longfloat_repr > > assert(str(a)[1:9] == str(a[0])[:8]) > > AssertionError > > > > ---------------------------------------------------------------------- > > Ran 516 tests in 0.823s > > > > FAILED (failures=1) > > > > Thanks for helping out on this. > > > How recent is your version? I just a problem that was causing this same > failure yesterday -- if you checkout is older than that, you may want to > get the most recent stuff from SVN and see if that fixes this. > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Sat Oct 21 00:13:07 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 21:13:07 -0700 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610202049w6dd1e84dp29651624f1917c22@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202049w6dd1e84dp29651624f1917c22@mail.gmail.com> Message-ID: <45399E53.9080109@ieee.org> Brian Granger wrote: > When I set seterr(all='warn') I see the following: > > In [1]: import numpy > /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/ufunclike.py:46: > RuntimeWarning: invalid value encountered in log > _log2 = umath.log(2) > /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/scimath.py:19: > RuntimeWarning: invalid value encountered in log > _ln2 = nx.log(2.0) > [etc, etc] Wow! That looks pretty bad. What do you get if you try just "numpy.log(2)" or "numpy.log(2.0)"? Is it producing sane results for scalars at all? I suppose another possibility is that the error reporting is broken on AIX for some reason. Hmmm. I'm betting that is is. The macro UFUNC_CHECK_STATUS is very platform dependent. There is a version from AIX (ufuncobject.h line 301), but perhaps it's broken on your particular configuration and as a result is spitting out all kinds of bogus errors. This is only coming to light now because the default error checking level got cranked up. I gotta call it a night and I'll be out tomorrow, so I won't be much more help, but here's something that you might look into: have you compiled numarray sucessfully? If you haven't you might want to try it. It uses the same default error checking that numpy is now using. If you have, you might want to look for the equivalent of UFUNC_CHECK_STATUS (it might even have the same name) and splice it into numpy and see if it fixes your problems. Of course, if numpy.log(2) is spitting out something bogus, there's something much worse going on, but I suspect you would have noticed that by now. Good luck, -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Sat Oct 21 00:13:21 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 22:13:21 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45399A45.6050509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> <45399A45.6050509@ieee.org> Message-ID: <6ce0ac130610202113u3d550aabld8157dac915c19db@mail.gmail.com> Tim, I just tried everything with r3375. I set seterr(all='warn') and the tests passed. But all the floating point warning are still there. With seterr(all='ignore') the warnings go away and all the tests pass. should I worry about the warnings? thanks Brian On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Also, when I use seterr(all='ignore') the the tests fail: > > > > ====================================================================== > > FAIL: Ticket #112 > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", > > line 219, in check_longfloat_repr > > assert(str(a)[1:9] == str(a[0])[:8]) > > AssertionError > > > > ---------------------------------------------------------------------- > > Ran 516 tests in 0.823s > > > > FAILED (failures=1) > > > > Thanks for helping out on this. > > > How recent is your version? I just a problem that was causing this same > failure yesterday -- if you checkout is older than that, you may want to > get the most recent stuff from SVN and see if that fixes this. > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Sat Oct 21 00:21:07 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 21:21:07 -0700 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610202113u3d550aabld8157dac915c19db@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> <45399A45.6050509@ieee.org> <6ce0ac130610202113u3d550aabld8157dac915c19db@mail.gmail.com> Message-ID: <4539A033.9050800@ieee.org> Brian Granger wrote: > Tim, > > I just tried everything with r3375. I set seterr(all='warn') and the > tests passed. But all the floating point warning are still there. > With seterr(all='ignore') the warnings go away and all the tests pass. > should I worry about the warnings? > Maybe. I just sent you some email on this. But my guess is that the code that checks for FP errors is broken on your particular system. Mainly I suspect this because I think you would have noticed by now if everything was as broken as the warnings seem to indicate. Assuming that's the case, and this will probably become clear if you test a bunch of computations that give correct (and non INF/NAN) results, but still spit out warnings, you have two choices: try to fix the warnings code or disable the warnings. The former would be preferable since then you could actually use the warnings code, but it may be a pain in the neck unless you can find some place to steal the relevant code from. -tim > thanks > > Brian > > > > On 10/20/06, Tim Hochberg wrote: > >> Brian Granger wrote: >> >>> Also, when I use seterr(all='ignore') the the tests fail: >>> >>> ====================================================================== >>> FAIL: Ticket #112 >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", >>> line 219, in check_longfloat_repr >>> assert(str(a)[1:9] == str(a[0])[:8]) >>> AssertionError >>> >>> ---------------------------------------------------------------------- >>> Ran 516 tests in 0.823s >>> >>> FAILED (failures=1) >>> >>> Thanks for helping out on this. >>> >>> >> How recent is your version? I just a problem that was causing this same >> failure yesterday -- if you checkout is older than that, you may want to >> get the most recent stuff from SVN and see if that fixes this. >> >> -tim >> >> >> ------------------------------------------------------------------------- >> Using Tomcat but need to do more? Need to support web services, security? >> Get stuff done quickly with pre-integrated technology to make your job easier >> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Sat Oct 21 01:01:42 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 23:01:42 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45399E53.9080109@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202049w6dd1e84dp29651624f1917c22@mail.gmail.com> <45399E53.9080109@ieee.org> Message-ID: <6ce0ac130610202201i406c8ea3r3e90cb519547a@mail.gmail.com> Thanks, I will investigate more on these things and get back to you early in the week. But for now numpy seems to be functioning pretty normally (log(2) gives the correct answer). thanks again. It would be great to figure this stuff out before 1.0, but we might not have time. Brian On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > When I set seterr(all='warn') I see the following: > > > > In [1]: import numpy > > /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/ufunclike.py:46: > > RuntimeWarning: invalid value encountered in log > > _log2 = umath.log(2) > > /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/scimath.py:19: > > RuntimeWarning: invalid value encountered in log > > _ln2 = nx.log(2.0) > > > [etc, etc] > > Wow! That looks pretty bad. What do you get if you try just > "numpy.log(2)" or "numpy.log(2.0)"? Is it producing sane results for > scalars at all? I suppose another possibility is that the error > reporting is broken on AIX for some reason. > > Hmmm. > > I'm betting that is is. The macro UFUNC_CHECK_STATUS is very platform > dependent. There is a version from AIX (ufuncobject.h line 301), but > perhaps it's broken on your particular configuration and as a result is > spitting out all kinds of bogus errors. This is only coming to light now > because the default error checking level got cranked up. > > I gotta call it a night and I'll be out tomorrow, so I won't be much > more help, but here's something that you might look into: have you > compiled numarray sucessfully? If you haven't you might want to try it. > It uses the same default error checking that numpy is now using. If you > have, you might want to look for the equivalent of UFUNC_CHECK_STATUS > (it might even have the same name) and splice it into numpy and see if > it fixes your problems. > > Of course, if numpy.log(2) is spitting out something bogus, there's > something much worse going on, but I suspect you would have noticed that > by now. > > Good luck, > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Sat Oct 21 02:07:01 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 21 Oct 2006 00:07:01 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> Message-ID: <4539B905.3010705@ieee.org> Brian Granger wrote: > Hi, > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > fine and passes all tests. But 1.0rc3 and more recent give the > following on import: > Most likely the error-detection code is not working on your platform. The platform dependent stuff is not that difficult. I tried to implement something for AIX, but very likely got it wrong (and don't have a platform to test it on). It is the UFUNC_CHECK_STATUS that must be implemented. Perhaps, we can do a simple check and disable the error modes: seterr(all='ignore') will work and "turn-off" error-detection on your platform. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sebzur at pin.if.uz.zgora.pl Sat Oct 21 09:41:45 2006 From: sebzur at pin.if.uz.zgora.pl (=?UTF-8?B?U2ViYXN0aWFuIMW7dXJlaw==?=) Date: Sat, 21 Oct 2006 15:41:45 +0200 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: Robert Kern napisa?(a): > Your description is a bit vague. Possibly by my weak English... I'll try to make myself clearer now. Do you mean that you have some model function f > that maps X values to Y values? > > f(x) -> y > My model is quantum energy operator - spin hamiltonian (SH) with some additional assumption about so called 'line shape', 'line widths',etc. It describes various electron interactions, visible in electron paramagnetic resonance (EPR, ESR) experiment. The simplest SH can be written in a form: H = m B g S (1) where m is a constant (bohr magneton), B is magnetic field (my x-variable), g is so called 'zeeman matrix' and S is total spin angular momentum operator. Summing it all together: the simple model is parametrized by: - line shape, - line width, - zeeman matrix (3x3 diagonal matrix - the spatial dependence), - total spin S. After SH (1) diagonalization one can obtain so called 'resonance fields' and 'resonance intensities'. After a convolution with appropriate line shape function which is parametrized by the line width one can finally get the simulated EPR spectrum (simDat=[[X1,...,Xn],[Y1,...,Yn]]). This is a roughly, schematic description, appropriate to EPR spectra of monocrystals. In my situation the problem is more sophisticated - I have polycrystaline (powders) data, and to obtain a simulated EPR powder spectrum I need to sum up the EPR spectra of monocrystals that come from many possible spatial orientations, and the resultant spectrum is an envelope of all the monocrystals spectra. There's no simple model function that maps X -> Y. > If that is the case, is there some reason that you cannot run your simulation > using the same X points as your experimental data? > I can only demand a X range and number of X values within the range, there's no possibility to find the Y(X) for a specified X. These limitations on one hand come from the external program I'm using to simulate the EPR spectra, on the other are a result of spatial averaging of EPR data for powders, where a lot of interpolations are involved. > OTOH, is there some other independent variable (say Z) that *is* common between > your experimental and simulated data? > > f(z) -> (x, y) > This is probably the situation I'm in. These other variables are my model parameters, namely: line shape-width, zeeman matrix... and they're commen between the experiment and the simulation. To make it clear. I've already solved the problem by a simple linear interpolation of simulated points within the narrow neighborhood of experimental data point. The simulation points are uniformly distributed along the X-range, with a density I'm able to tune. It all works quite well but I'm founding it as a 'brute-force' method and I wonder, if there's any more sophisticated and maybe already incorporated into any Python module method? Anyway, it looks like it's impossible to compare two discrete 2D data sets without any interpolations included... :] A. M. Archibald has proposed spline fitting, which I'll try. I'll also look at the Numerical Recipes discussion he has proposed. Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From veomagxa at powerhill.com Sat Oct 21 09:54:45 2006 From: veomagxa at powerhill.com (India:) Date: Sat, 21 Oct 2006 21:54:45 +0800 Subject: browser we Message-ID: <000f01c6f518$777a93e0$955ea2dc@SUPER7ABF2EC2B> Cold addition less division or comfy futonsa lot or fuss or water in featurethe a rules ride lots. Todays events Advanced is Local in Nation World Business in Editorials a Opinion Columnists Sports in Comics Games Living Pacific Northwest Magazine or Travel Outdoors Real am estate Obituaries Special Alerts? Other Packages Related depends recommends suggests libc arm or hppa mk mips mipsel powerpcgnu Library Shared sparclibc in amd libc alphagnu ia a Download available of filessize. Boots Qayoure only one pain brain a Getting Startedno is excuse mom woes Microsofts Halo film is. Confident un action Iran is Shiites in vs Shiites in Iraq rift of parade west flock Jerusalem of brave is Israeli takes harder line North in leaders Darfur careful blame or either dogs fury? Offense a facing tough testso far flaws grow baseball cities Larry Stoneworld a Series brink Movies is. Somali women swimming Sunday am expanding canal births Japan adoption agencies of ask. Swimming Sunday expanding of canal births Japan a adoption agencies in ask why! Straggle Detroit Seriesmore is ap read emailed articles air Kuow Hear columnist Danny Westneat a review weeks Weekday is Fridays am is. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Groz.gif Type: image/gif Size: 10621 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From sebzur at pin.if.uz.zgora.pl Sat Oct 21 10:00:27 2006 From: sebzur at pin.if.uz.zgora.pl (=?ISO-8859-2?Q?Sebastian_=AFurek?=) Date: Sat, 21 Oct 2006 16:00:27 +0200 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: A. M. Archibald napisa?(a): > > In scipy there are some very convenient spline fitting tools which > will allow you to fit a nice smooth spline through the simulation data > points (or near, if they have some uncertainty); you can then easily > look at the RMS difference in the y values. You can also, less easily, > look at the distance from the curve allowing for some uncertainty in > the x values. > I'll try a spline fitting. I've already made some linear interpolations (see Robert Kern answer) which works well enough to use it. I'm working on a genetic algorithms application to the model parameters optimalization problem and this RMSe comparison serves me as 'fitness function'. This 'fitness function' is important element in whole procedure, so I'm trying to found the best solution to obtain it. > I suppose you could also fit a curve through the experimental points > and compare the two curves in some way. > Well, I can do it, indeed. But every single fitting procedure implicate some additional error, so when it comes to fit, I must use it very cautiously. The simulated data-points fitting should be the only acceptable fitting procedure, I guess. > If you want to avoid using an a priori model, Numerical Recipes > discuss some possible approaches ("Do two-dimensional distributions > differ?" at http://www.nrbook.com/a/bookcpdf.html is one) but it's not > clear how to turn the problem you describe into a solvable one - some > assumption about how the models vary between sampled x values appears > to be necessary, and that amounts to interpolation. > I'll look to this NR discussion. Thank You for these comments! Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 21 11:51:56 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 21 Oct 2006 09:51:56 -0600 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: On 10/21/06, Sebastian ?urek wrote: > > Robert Kern napisa?(a): > > > To make it clear. > > I've already solved the problem by a simple linear interpolation of > simulated points within the narrow neighborhood of experimental data > point. The simulation points are uniformly distributed along the > X-range, with a density I'm able to tune. It all works quite well but > I'm founding it as a 'brute-force' method and I wonder, if there's any > more sophisticated and maybe already incorporated into any Python module > method? > > Anyway, it looks like it's impossible to compare two discrete 2D data > sets without any interpolations included... :] I note that interpolation can be seen as a linear map from the data points to the interpolation points, so a lot of standard tools to be used. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From sebzur at pin.if.uz.zgora.pl Sat Oct 21 14:47:05 2006 From: sebzur at pin.if.uz.zgora.pl (=?UTF-8?B?U2ViYXN0aWFuIMW7dXJlaw==?=) Date: Sat, 21 Oct 2006 20:47:05 +0200 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: The problem seemed to be solved, by the A. M. Archibald clue. I've used splines to fit the simulation data. After that, I can easily find any Y(X) point, for all X in range (x_min,x_max) where x_min and x_max are the experiment independent variable. The experimental data stay untouched. Sorry for all the confusion I've made. Thanks a lot to all of You! Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nadavh at visionsense.com Sat Oct 21 15:05:21 2006 From: nadavh at visionsense.com (Nadav Horesh) Date: Sat, 21 Oct 2006 21:05:21 +0200 Subject: Model and experiment fitting. Message-ID: <07C6A61102C94148B8104D42DE95F7E8C8F0F7@exchange2k.envision.co.il> 1. If at least one of your data sets to be interpulated is on a grid, you can use numpy.ndimage.map function for fast interpolation for 2d (in fact for any dimensional) dataset. 2. Isn't there an analytic expression to average the expectration values of SH over all possible orientations between B and the crystal axis? My experience shows that some analytic work can save 99% of simulation time. Nadav -----Original Message----- From: numpy-discussion-bounces at lists.sourceforge.net on behalf of Sebastian Zurek Sent: Sat 21-Oct-06 15:41 To: numpy-discussion at lists.sourceforge.net Cc: Subject: Re: [Numpy-discussion] Model and experiment fitting. Robert Kern napisal(a): > Your description is a bit vague. Possibly by my weak English... I'll try to make myself clearer now. Do you mean that you have some model function f > that maps X values to Y values? > > f(x) -> y > My model is quantum energy operator - spin hamiltonian (SH) with some additional assumption about so called 'line shape', 'line widths',etc. It describes various electron interactions, visible in electron paramagnetic resonance (EPR, ESR) experiment. The simplest SH can be written in a form: H = m B g S (1) where m is a constant (bohr magneton), B is magnetic field (my x-variable), g is so called 'zeeman matrix' and S is total spin angular momentum operator. Summing it all together: the simple model is parametrized by: - line shape, - line width, - zeeman matrix (3x3 diagonal matrix - the spatial dependence), - total spin S. After SH (1) diagonalization one can obtain so called 'resonance fields' and 'resonance intensities'. After a convolution with appropriate line shape function which is parametrized by the line width one can finally get the simulated EPR spectrum (simDat=[[X1,...,Xn],[Y1,...,Yn]]). This is a roughly, schematic description, appropriate to EPR spectra of monocrystals. In my situation the problem is more sophisticated - I have polycrystaline (powders) data, and to obtain a simulated EPR powder spectrum I need to sum up the EPR spectra of monocrystals that come from many possible spatial orientations, and the resultant spectrum is an envelope of all the monocrystals spectra. There's no simple model function that maps X -> Y. > If that is the case, is there some reason that you cannot run your simulation > using the same X points as your experimental data? > I can only demand a X range and number of X values within the range, there's no possibility to find the Y(X) for a specified X. These limitations on one hand come from the external program I'm using to simulate the EPR spectra, on the other are a result of spatial averaging of EPR data for powders, where a lot of interpolations are involved. > OTOH, is there some other independent variable (say Z) that *is* common between > your experimental and simulated data? > > f(z) -> (x, y) > This is probably the situation I'm in. These other variables are my model parameters, namely: line shape-width, zeeman matrix... and they're commen between the experiment and the simulation. To make it clear. I've already solved the problem by a simple linear interpolation of simulated points within the narrow neighborhood of experimental data point. The simulation points are uniformly distributed along the X-range, with a density I'm able to tune. It all works quite well but I'm founding it as a 'brute-force' method and I wonder, if there's any more sophisticated and maybe already incorporated into any Python module method? Anyway, it looks like it's impossible to compare two discrete 2D data sets without any interpolations included... :] A. M. Archibald has proposed spline fitting, which I'll try. I'll also look at the Numerical Recipes discussion he has proposed. Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 5129 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From kwgoodman at gmail.com Sat Oct 21 17:05:42 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Sat, 21 Oct 2006 14:05:42 -0700 Subject: some work on arpack In-Reply-To: References: Message-ID: On 8/15/06, David Grant wrote: > My idea is (if I have time) to write an eigs-like function in python > that will only perform a subset of what Matlab's eigs does for. It > will, for example, compute a certain number of eigenvalues and > eigenvectors for a real, sparse, symmetric matrix (the case I'm > interested in)... I hope that this subset-of-matlab's-eigs function > will not be too hard to write. Then more functionality can be added on > to eigs.py later... Does this make sense? Did you, or anybody else on the list, have any luck making a numpy version of eigs? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Sat Oct 21 19:26:46 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Sat, 21 Oct 2006 20:26:46 -0300 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <453957F2.8070507@ee.byu.edu> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> <4538CE5D.5000903@obs.u-bordeaux1.fr> <453957F2.8070507@ee.byu.edu> Message-ID: On 10/20/06, Travis Oliphant wrote: > > How about this. To get the i,j,k,l element > > a[i:i+1,j:j+1,k:k+1,l:l+1].squeeze() > > -Travis I think all this can be condensed in a method call or similar mehcanism, natively provided by ndarray type. Or should this be seen as a special use case? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Sat Oct 21 19:36:21 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 22 Oct 2006 08:36:21 +0900 Subject: some work on arpack In-Reply-To: References: Message-ID: Here's something I've been wondering: is it ok to port Matlab functions over to python? If so, then it's maybe an afternoon's work to get eigs working, given eigs.m and python Arpack wrappers. I'm 99% sure it would *not* be kosher to copy-paste their code directly, but it's not a copy paste job because of the differences between Python and .m. So it seems a grayer area to me. --bb On 10/22/06, Keith Goodman wrote: > On 8/15/06, David Grant wrote: > > > My idea is (if I have time) to write an eigs-like function in python > > that will only perform a subset of what Matlab's eigs does for. It > > will, for example, compute a certain number of eigenvalues and > > eigenvectors for a real, sparse, symmetric matrix (the case I'm > > interested in)... I hope that this subset-of-matlab's-eigs function > > will not be too hard to write. Then more functionality can be added on > > to eigs.py later... Does this make sense? > > Did you, or anybody else on the list, have any luck making a numpy > version of eigs? > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 21 19:44:48 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 21 Oct 2006 17:44:48 -0600 Subject: some work on arpack In-Reply-To: References: Message-ID: On 10/21/06, Bill Baxter wrote: > > Here's something I've been wondering: is it ok to port Matlab > functions over to python? If so, then it's maybe an afternoon's work > to get eigs working, given eigs.m and python Arpack wrappers. > > I'm 99% sure it would *not* be kosher to copy-paste their code > directly, but it's not a copy paste job because of the differences > between Python and .m. So it seems a grayer area to me. > > --bb I haven't the faintest idea. What did the EULA say? What about a copyright? This sounds like a job for a lawyer, but I would guess it would be a bad idea unless the code is from a third party with a free license. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Sat Oct 21 20:27:39 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 22 Oct 2006 09:27:39 +0900 Subject: some work on arpack In-Reply-To: References: Message-ID: On 10/22/06, Charles R Harris wrote: > On 10/21/06, Bill Baxter wrote: > > Here's something I've been wondering: is it ok to port Matlab > > functions over to python? If so, then it's maybe an afternoon's work > > to get eigs working, given eigs.m and python Arpack wrappers. > > [...] > > I haven't the faintest idea. What did the EULA say? What about a copyright? > This sounds like a job for a lawyer, but I would guess it would be a bad > idea unless the code is from a third party with a free license. Gee, there's an idea. :-) ---- from License.txt ---- "Except as expressly provided by this Agreement, including the attached Addendum, Licensee may not adapt, translate, or convert "M-files", "MDL-files" or "P-code" contained in the Programs in order to create software, a principal purpose of which is to perform the same or similar functions as Programs licensed by MathWorks or which is intended to replace any component of the Programs. The Licensee may not incorporate or use "M-files", "P-code", source code, or any other part of the Programs in or as part of another computer program without the consent of MathWorks. A Licensed User may modify pieces of MathWorks' code for the Licensed User's own use. A Licensed User may share such modified code with others provided each recipient is also a Licensed User for the original form of the code. " ------------------------- So. If you're a Matlab licensee it would be ok to use it in your own stuff, but putting it in Numpy is definitely out of the question. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From AGED at HOTMAIL.COM Sun Oct 22 01:41:58 2006 From: AGED at HOTMAIL.COM (=?iso-2022-jp?B?QUdFRA==?=) Date: Sat, 21 Oct 2006 22:41:58 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCJDMkcyRLJEEkTxsoQg==?= Message-ID: ?????????????????? http://11487.com/shi/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Sun Oct 22 05:43:01 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Sun, 22 Oct 2006 11:43:01 +0200 Subject: some work on arpack In-Reply-To: References: Message-ID: <20061022094301.GO10843@mentat.za.net> On Sun, Oct 22, 2006 at 09:27:39AM +0900, Bill Baxter wrote: > On 10/22/06, Charles R Harris wrote: > > On 10/21/06, Bill Baxter wrote: > > > Here's something I've been wondering: is it ok to port Matlab > > > functions over to python? If so, then it's maybe an afternoon's work > > > to get eigs working, given eigs.m and python Arpack wrappers. > > > [...] > > > > I haven't the faintest idea. What did the EULA say? What about a copyright? > > This sounds like a job for a lawyer, but I would guess it would be a bad > > idea unless the code is from a third party with a free license. > > Gee, there's an idea. :-) > > ---- from License.txt ---- [...] > ------------------------- > > So. If you're a Matlab licensee it would be ok to use it in your own > stuff, but putting it in Numpy is definitely out of the question. A lot of the code in the Octave Forge is licensed under the BSD license, so you can use that. Unfortunately, the main Octave code is released under the GPL. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sebzur at pin.if.uz.zgora.pl Sun Oct 22 06:39:24 2006 From: sebzur at pin.if.uz.zgora.pl (=?UTF-8?B?U2ViYXN0aWFuIMW7dXJlaw==?=) Date: Sun, 22 Oct 2006 12:39:24 +0200 Subject: Model and experiment fitting. In-Reply-To: <07C6A61102C94148B8104D42DE95F7E8C8F0F7@exchange2k.envision.co.il> References: <07C6A61102C94148B8104D42DE95F7E8C8F0F7@exchange2k.envision.co.il> Message-ID: Nadav Horesh napisa?(a): > 1. If at least one of your data sets to be interpulated is on a grid, > you can use numpy.ndimage.map function for fast interpolation for 2d (in fact for any dimensional) dataset. I've already used a splines to interpolate a missing simulated points. That procedure works great and is very fast. But I'll check the numpy.ndimage - I haven't used it, yet. > 2. Isn't there an analytic expression to average the expectration values of SH over all possible orientations > between B and the crystal axis? My experience shows that some analytic work can save 99% of simulation time. Well, the simulations are already very fast. The time consumption is approximately ~0.3s for a single powder spectrum (2.8GHz Pentium D). The calculations are held by an external, very fine EPR spectra simulation tool. The author must have incorporated into it a lot of rationalizations, but this is a binary tool (unfortunately) and I do not know, what exactly sits inside of it... All I know, is that the orientations are represented by a grid (with an increment step tunable by a user). From library documentation: "After having computed the spectrum for a number of orientations specified, the simulation function interpolates these spectra for additional orientations before summing up all spectra." The interpolation is accomplish with a splines. Thank you for your comment, best regards Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From hagberg at lanl.gov Sun Oct 22 10:08:51 2006 From: hagberg at lanl.gov (Aric Hagberg) Date: Sun, 22 Oct 2006 08:08:51 -0600 Subject: some work on arpack In-Reply-To: References: Message-ID: <20061022140850.GR10447@t7.lanl.gov> On Sat, Oct 21, 2006 at 02:05:42PM -0700, Keith Goodman wrote: > Did you, or anybody else on the list, have any luck making a numpy > version of eigs? I made a start at an ARPACK wrapper, see http://projects.scipy.org/scipy/scipy/ticket/231 and the short thread at scipy-dev http://thread.gmane.org/gmane.comp.python.scientific.devel/5166/focus=5175 In addition to the wrapper there is a Python interface (and some tests). I don't know if the interface is like "eigs" - I don't use Matlab. It will give you a few eigenvalues and eigenvectors for the standard eigenproblem (Ax=lx) for any type of A (symmetric/nonsymmetric, real/complex, single/double, sparse/nonsparse). The generalized and shifted modes are not implemented. I need to find some time (or some help) to get it finished. Regards, Aric ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Sun Oct 22 10:20:36 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sun, 22 Oct 2006 16:20:36 +0200 Subject: some work on arpack In-Reply-To: <20061022140850.GR10447@t7.lanl.gov> References: <20061022140850.GR10447@t7.lanl.gov> Message-ID: <000c01c6f5e5$3eadc7d0$0a83a8c0@ratbert> Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Aric Hagberg > Sent: Sunday, October 22, 2006 4:09 PM > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] some work on arpack > > On Sat, Oct 21, 2006 at 02:05:42PM -0700, Keith Goodman wrote: > > Did you, or anybody else on the list, have any luck making a numpy > > version of eigs? > > I made a start at an ARPACK wrapper, see > http://projects.scipy.org/scipy/scipy/ticket/231 > and the short thread at scipy-dev > http://thread.gmane.org/gmane.comp.python.scientific.devel/5166/focus=5175 > > In addition to the wrapper there is a Python interface (and some tests). > I don't know if the interface is like "eigs" - I don't use Matlab. If you're interested in making an interface that looks like MATLAB's, you can take a peek at: > > It will give you a few eigenvalues and eigenvectors for the standard > eigenproblem (Ax=lx) for any type of A (symmetric/nonsymmetric, > real/complex, > single/double, sparse/nonsparse). > > The generalized and shifted modes are not implemented. > I need to find some time (or some help) to get it finished. > > Regards, > Aric > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Sun Oct 22 10:21:40 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sun, 22 Oct 2006 16:21:40 +0200 Subject: some work on arpack In-Reply-To: <20061022140850.GR10447@t7.lanl.gov> References: <20061022140850.GR10447@t7.lanl.gov> Message-ID: <000d01c6f5e5$64551010$0a83a8c0@ratbert> Argh. > > In addition to the wrapper there is a Python interface (and some tests). > I don't know if the interface is like "eigs" - I don't use Matlab. http://www.mathworks.de/access/helpdesk/help/techdoc/ref/index.html?/access/ helpdesk/help/techdoc/ref/eigs.html should give you some idea of how MATLAB's eigs works. Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Sun Oct 22 11:34:58 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Sun, 22 Oct 2006 08:34:58 -0700 Subject: some work on arpack In-Reply-To: <20061022140850.GR10447@t7.lanl.gov> References: <20061022140850.GR10447@t7.lanl.gov> Message-ID: On 10/22/06, Aric Hagberg wrote: > On Sat, Oct 21, 2006 at 02:05:42PM -0700, Keith Goodman wrote: > > Did you, or anybody else on the list, have any luck making a numpy > > version of eigs? > > I made a start at an ARPACK wrapper, see > http://projects.scipy.org/scipy/scipy/ticket/231 > and the short thread at scipy-dev > http://thread.gmane.org/gmane.comp.python.scientific.devel/5166/focus=5175 That looks very promising! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Sun Oct 22 13:10:55 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Mon, 23 Oct 2006 02:10:55 +0900 Subject: [Numpy-discussion] some work on arpack In-Reply-To: <20061022140850.GR10447@t7.lanl.gov> References: <20061022140850.GR10447@t7.lanl.gov> Message-ID: Don't know if this is any use, but to me (not knowing Fortran nearly so well as C++) this looks pretty useful: http://www.caam.rice.edu/software/ARPACK/arpack++.html http://www.ime.unicamp.br/~chico/arpack++/ It provides a nice high-level interface on top of ARPACK. I could see it being useful on a number of levels: 1) actually use its wrappers directly instead of calling the f2py'ed fortran code - at least as a stop gap measure to cover the holes ("shifted modes" etc) - this could potentially even be faster since the reverse-communication interface requires some iteration, and this way the loops would run in compiled C++ instead of python. 2) use it's code just as a reference to help write more wrappers to the fortran code 3) if nothing else, it has pretty decent documentation. For instance it includes a nice table of the amount of storage you need to reserve for various different calling modes. (It seems ARPACK's main documentation is not freely available, so I'm not sure what's in it, but if you have that then ARPACK++'s docs may not be much help). As far as I can tell, it must be under the same licensing terms as ARPACK itself. It doesn't actually specify anything as far as I could see. But it seems to be linked prominently from the ARPACK website and appears as an offshoot project. --bb On 10/22/06, Aric Hagberg wrote: > On Sat, Oct 21, 2006 at 02:05:42PM -0700, Keith Goodman wrote: > > Did you, or anybody else on the list, have any luck making a numpy > > version of eigs? > > I made a start at an ARPACK wrapper, see > http://projects.scipy.org/scipy/scipy/ticket/231 > and the short thread at scipy-dev > http://thread.gmane.org/gmane.comp.python.scientific.devel/5166/focus=5175 > > In addition to the wrapper there is a Python interface (and some tests). > I don't know if the interface is like "eigs" - I don't use Matlab. > > It will give you a few eigenvalues and eigenvectors for the standard > eigenproblem (Ax=lx) for any type of A (symmetric/nonsymmetric, real/complex, > single/double, sparse/nonsparse). > > The generalized and shifted modes are not implemented. > I need to find some time (or some help) to get it finished. > > Regards, > Aric From simon at arrowtheory.com Sun Oct 22 17:38:21 2006 From: simon at arrowtheory.com (Simon Burton) Date: Sun, 22 Oct 2006 14:38:21 -0700 Subject: PyDX - announcement Message-ID: <20061022143821.3b085ca4.simon@arrowtheory.com> Apologies if this is too off-topic for these lists, but I hope some people here find this interesting! PyDX - first public announcement ================================ Overview -------- PyDX is a package for working with calculus (differential geometry), arbitrary precision arithmetic (using gmpy), and interval arithmetic. It provides: * multivariate automatic differentiation (to arbitrary order) * Tensor objects for computations in differential geometry * Interval scalars (based on libMPFI) for calculating rigorous bounds on arithmetic operations (validated numerics). * Arbitrary order validated ODE solver. PyDX uses lazy computation techniques to greatly enhance performance of the resulting functions. This code grew out of a research project at The Australian National University, Department of Physics, which involved computing bounds on geodesics in relativistic space-time's. Documentation ------------- http://gr.anu.edu.au/svn/people/sdburton/pydx/doc/user-guide.html http://gr.anu.edu.au/svn/people/sdburton/pydx/doc/api/index.html Subversion Repository --------------------- http://gr.anu.edu.au/svn/people/sdburton/pydx/ Simon Burton October 22 2006 The Australian National University From ellisonbg.net at gmail.com Sun Oct 22 17:37:46 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sun, 22 Oct 2006 15:37:46 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <4539B905.3010705@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <4539B905.3010705@ieee.org> Message-ID: <6ce0ac130610221437i66118e04p33d64f0616dd1665@mail.gmail.com> > Most likely the error-detection code is not working on your platform. > The platform dependent stuff is not that difficult. I tried to > implement something for AIX, but very likely got it wrong (and don't > have a platform to test it on). It is the UFUNC_CHECK_STATUS that must > be implemented. Perhaps, we can do a simple check and disable the > error modes: I looked at the UFUNC_CHECK_STATUS implementation for AIX and put in some print statements. The the fpstatus returned to fp_read_flag is always indicating an FP_INVALID so UFUNC_CHECK_STATUS return 8. But the code looks fine to me. It seems a little odd that it would always indicate this flag. Where else can I look to try to see what it goiing on? Do you think this is worth worrying about, or should we just have use seterr(all='ignore')? I am willing to try to hunt this down, but I don't know much about the internals of numpy. Thanks Brian > seterr(all='ignore') > > will work and "turn-off" error-detection on your platform. > > -Travis > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Sun Oct 22 18:19:55 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sun, 22 Oct 2006 16:19:55 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <4539B905.3010705@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <4539B905.3010705@ieee.org> Message-ID: <6ce0ac130610221519q42a4eb4odc119aab8e4a6bee@mail.gmail.com> So, I have figured out the problem and have a solution. I have submitted a ticket for this: http://projects.scipy.org/scipy/numpy/ticket/362 That describes the problem and solution. Thanks for everyones ideas on this. Brian On 10/21/06, Travis Oliphant wrote: > Brian Granger wrote: > > Hi, > > > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > > fine and passes all tests. But 1.0rc3 and more recent give the > > following on import: > > > > Most likely the error-detection code is not working on your platform. > The platform dependent stuff is not that difficult. I tried to > implement something for AIX, but very likely got it wrong (and don't > have a platform to test it on). It is the UFUNC_CHECK_STATUS that must > be implemented. Perhaps, we can do a simple check and disable the > error modes: > > seterr(all='ignore') > > will work and "turn-off" error-detection on your platform. > > -Travis > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From humufr at yahoo.fr Sun Oct 22 18:45:42 2006 From: humufr at yahoo.fr (humufr at yahoo.fr) Date: Sun, 22 Oct 2006 18:45:42 -0400 Subject: bug? Message-ID: <200610221845.42894.humufr@yahoo.fr> Hello, the docstring for compress in numpy give this help(numpy.compress) compress(condition, m, axis=None, out=None) compress(condition, x, axis=None) = those elements of x corresponding to those elements of condition that are "true". condition must be the same size as the given dimension of x. So (but perhaps I can misundertand the help due to my english) I don't undersand the following error, for me a and c array does have the same dimension and size. So someone can explain me the result please? Thanks, N. In [86]: a = numpy.arange(9) In [87]: a = numpy.arange(9).reshape(3,3) In [88]: c = numpy.ones(9).reshape(3,3) In [89]: numpy.compress(c,a) --------------------------------------------------------------------------- exceptions.ValueError Traceback (most recent call last) /home/gruel/tmp/Astro/FATBOY/ /home/gruel/usr/lib/python2.4/site-packages/numpy/core/fromnumeric.py in compress(condition, m, axis, out) 353 except AttributeError: 354 return _wrapit(m, 'compress', condition, axis, out) --> 355 return compress(condition, axis, out) 356 357 def clip(m, m_min, m_max): ValueError: condition must be 1-d array ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Sun Oct 22 19:12:57 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 22 Oct 2006 18:12:57 -0500 Subject: bug? In-Reply-To: <200610221845.42894.humufr@yahoo.fr> References: <200610221845.42894.humufr@yahoo.fr> Message-ID: humufr at yahoo.fr wrote: > Hello, > > the docstring for compress in numpy give this > > help(numpy.compress) > > compress(condition, m, axis=None, out=None) > compress(condition, x, axis=None) = those elements of x corresponding > to those elements of condition that are "true". condition must be the > same size as the given dimension of x. > > > So (but perhaps I can misundertand the help due to my english) I don't > undersand the following error, for me a and c array does have the same > dimension and size. So someone can explain me the result please? The docstring is a bit underspecified. The condition array *must* be a 1D array with the same size *as the given axis* of the other array (using the convention that axis=None implies operating over the flattened array). There's simply no valid interpretation of this, for example: compress(array([[1, 0, 0], [1, 1, 0]]), arange(6).reshape(2,3)) since numpy arrays cannot be "ragged". -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Sun Oct 22 20:47:10 2006 From: fullung at gmail.com (Albert Strasheim) Date: Mon, 23 Oct 2006 02:47:10 +0200 Subject: Strange results when sorting array with fields Message-ID: <002001c6f63c$c63c0250$0a83a8c0@ratbert> Hello all I'm trying to sort an array with two fields, but I'm getting a result that doesn't seem to make sense. What I tried (first attempt): I have two 2-D arrays. I would like to sort one based on the sort of the other. I managed to do this with argsort. However, the fancy indexing required to get the sorted array using what argsort returned was very slow. I followed this example: http://www.scipy.org/Numpy_Example_List#head-9f8656795227e3c43e849c6c0435eee b32afd722 What I tried (second attempt): I created an array with two fields. I think/hope/expected that sorting the array would sort it on the first field in the dtype and then on the second. This is *much* faster than the fancy indexing approach. Code: import numpy as N print N.__version__ print dt = N.dtype([('aaa', N.float64), ('bbb', N.uint8)]) x = N.empty((2, 2), dt) xa = x[dt.names[0]] xa[:] = [[1.,2], [3.,4.]] xb = x[dt.names[1]] xb[:] = [[1, 2], [3, 4]] print 'before sort:' print x print x.sort(kind='quicksort') # other kinds not supported print 'after sort:' print x Output on my system: 1.0.dev3376 before sort: [[(1.0, 1) (2.0, 2)] [(3.0, 3) (4.0, 4)]] after sort: [[(2.0, 2) (1.0, 1)] [(4.0, 4) (3.0, 3)]] The already sorted array has been unsorted in some way. Any thoughts? Thanks! Regards, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jfishman at bowdoin.edu Sun Oct 22 22:05:40 2006 From: jfishman at bowdoin.edu (Jeremy R. Fishman) Date: Sun, 22 Oct 2006 22:05:40 -0400 Subject: Cross compilation Message-ID: <453C2374.5000208@bowdoin.edu> Hi, I was wondering if anyone can give me some advice on how to go about cross-compiling NumPy. I have been searching around and can't find any support in distutils for cross compilation. Is there some way I can still compile Numerical Python using a mipsel-linux compiler, on say a Cygwin host? The end result would be to use NumPy with an embedded Python interpreter running on a mipsel host. In specific, The OPEN-R system by Sony for use on Aibo robotic dogs. I need to figure out how to get setup to use the mipsel compiler, not how to set up the compiler. Thanks, Jeremy Fishman -------------- next part -------------- A non-text attachment was scrubbed... Name: jfishman.vcf Type: text/x-vcard Size: 158 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Sun Oct 22 22:44:07 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 22 Oct 2006 21:44:07 -0500 Subject: Cross compilation In-Reply-To: <453C2374.5000208@bowdoin.edu> References: <453C2374.5000208@bowdoin.edu> Message-ID: Jeremy R. Fishman wrote: > Hi, I was wondering if anyone can give me some advice on how to go about > cross-compiling NumPy. I have been searching around and can't find any > support in distutils for cross compilation. Is there some way I can > still compile Numerical Python using a mipsel-linux compiler, on say a > Cygwin host? I'm afraid that distutils really does not support cross-compilation. numpy adds some more complications in that it tries to configure itself by compiling and running small programs. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sun Oct 22 23:22:20 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 22 Oct 2006 21:22:20 -0600 Subject: Strange results when sorting array with fields In-Reply-To: <002001c6f63c$c63c0250$0a83a8c0@ratbert> References: <002001c6f63c$c63c0250$0a83a8c0@ratbert> Message-ID: On 10/22/06, Albert Strasheim wrote: > > Hello all > > I'm trying to sort an array with two fields, but I'm getting a result that > doesn't seem to make sense. > > What I tried (first attempt): I have two 2-D arrays. I would like to sort > one based on the sort of the other. I managed to do this with argsort. > However, the fancy indexing required to get the sorted array using what > argsort returned was very slow. I followed this example: It is certainly awkward. I am going to add a function to do this after 1.0comes out. Sounds like you need it now. http://www.scipy.org/Numpy_Example_List#head-9f8656795227e3c43e849c6c0435eee > b32afd722 > > What I tried (second attempt): I created an array with two fields. I > think/hope/expected that sorting the array would sort it on the first > field > in the dtype and then on the second. This is *much* faster than the fancy > indexing approach. I believe it sorts on the two fields together as one big binary blob. Output on my system: > > 1.0.dev3376 > > before sort: > [[(1.0, 1) (2.0, 2)] > [(3.0, 3) (4.0, 4)]] > > after sort: > [[(2.0, 2) (1.0, 1)] > [(4.0, 4) (3.0, 3)]] > > The already sorted array has been unsorted in some way. Any thoughts? I suspect something to do with the binary representation of the floats. Probably depends on big/little endian also. The floats 1.0, 2.0, and 4.0all have zero mantissas while 3.0 has a one. The sort behaves differently if uint8 is used for both fields. In [3]: dt = dtype([('aaa', uint8), ('bbb', uint8)]) In [4]: x = empty((2, 2), dt) In [5]: xa = x[dt.names[0]] In [6]: xa[:] = [[1,2], [3,4]] In [7]: xb = x[dt.names[1]] In [8]: xb[:] = [[1,2], [3,4]] In [9]: x.sort() In [11]: x Out[11]: array([[(1, 1), (2, 2)], [(3, 3), (4, 4)]], dtype=[('aaa', '|u1'), ('bbb', '|u1')]) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Mon Oct 23 00:24:41 2006 From: pgmdevlist at gmail.com (P GM) Date: Mon, 23 Oct 2006 00:24:41 -0400 Subject: A reimplementation of MaskedArray + MaskedRecords Message-ID: <777651ce0610222124r40697b0eyaf4a7c107ea3e624@mail.gmail.com> Folks, I updated the alternative implementation of MaskedArray on the wiki, mainly to correct a couple of bugs. (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) In addition, I attached another file, maskedrecordarray, which introduce a new class, MaskedRecord, as a subclass of recarray and MaskedArray. An instance of this class accepts a recarray as data, and uses two masks: the 'recordmask' has as many entries as records in the array, each entry with the same fields as a record, but of boolean types, indicating whether a field is masked or not; an entry is flagged as masked in the 'mask' array if at least one field is masked. The 'mask' object is introduced mostly for compatibilty with MaskedArray, only 'recordmask' is really useful. A few examples in the file should give you an idea of what can be done. In particular, you can define a new maskedrecord array as simply as ; a = masked_record([('Alan',29,200.), ('Bill',31,260.0)], dtype=[('name','S30'),('age',int_),('weight',float_)], mask=[(1,0,0), (0,0,0)]) Note that maskedrecordarray is still quite experimental. As I'm not a regular user of records, I don't really know what should be implemented... The file can be accessed at http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedrecordarray.py Once again, I need your comments and suggestions ! Thanks. Pierre -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From lfriedri at imtek.de Mon Oct 23 02:25:11 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Mon, 23 Oct 2006 08:25:11 +0200 Subject: memory position of numpy arrays In-Reply-To: <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> Message-ID: <1161584711.6785.12.camel@localhost> Hello all, to interact with some hardware (data retrieval), I use the following scheme (Windows, Python 2.4): * in Python, I create a numpy array as a buffer * I pass this array to a self written .dll using ctypes * the C-code in the .dll passes the pointer to the buffer to the API of the hardware; then the API starts writing data to my buffer * from python I can use a helper function of the .dll to know which part of the buffer is safe to be read at the moment; so I can copy this part to a different numpy array and work with the data This works quite well, but today I got a blue screen, reporting some paging-problem. It occured after doing some other stuff like moving some windows on the screen and importing pylab, and I heard the harddisk working hard, so I assume that the part of memory my buffer is in, was paged to the harddisk. This is a problem, since the hardware driver will continuously try to write to this specific memory location I gave to it. My primary question is, how to avoid a numpy-array being paged. In the ideal case there would be a flag to set, that makes sure that this array is always at this position in physical memory. Of course I am also interested in other people's work on hardware access. Do you think the above discribed way is a good one? To me it was the best way because I could do as much as possible in Python and keep may C-coded .dll very small. Is anyone doing similar things? Thanks for every comment Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From strawman at astraw.com Mon Oct 23 03:45:19 2006 From: strawman at astraw.com (Andrew Straw) Date: Mon, 23 Oct 2006 00:45:19 -0700 Subject: memory position of numpy arrays In-Reply-To: <1161584711.6785.12.camel@localhost> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> Message-ID: <453C730F.4090000@astraw.com> It sounds like your hardware drivers may be buggy -- you should only get segfaults, not (the Windows equivalent of) kernel panics, when your userspace code accesses wrong memory. But if you have buggy hardware drivers, I suppose it's possible that locking the memory will help. This wouldn't be the domain of numpy, however. In Linux, this is acheived with a system call such as mlock() or mlockall(). You'll have to figure out the appropriate call in Windows. Thinking about it, if your hardware drivers require the memory to be locked, they should do it themselves. However, I'm not convinced this is the real issue. It seems at least equally likely that your hardware drivers were developed with particular pattern of timing when accessing the buffers, but now you may be attempting to hold a buffer longer (preventing the driver writing to it) than the developer ever tested. It shouldn't blue-screen, but it does... I think it quite likely that you have some buggy hardware drivers. What hardware is it? -Andrew Lars Friedrich wrote: > Hello all, > > to interact with some hardware (data retrieval), I use the following > scheme (Windows, Python 2.4): > > * in Python, I create a numpy array as a buffer > * I pass this array to a self written .dll using ctypes > * the C-code in the .dll passes the pointer to the buffer to the API of > the hardware; then the API starts writing data to my buffer > * from python I can use a helper function of the .dll to know which part > of the buffer is safe to be read at the moment; so I can copy this part > to a different numpy array and work with the data > > This works quite well, but today I got a blue screen, reporting some > paging-problem. It occured after doing some other stuff like moving some > windows on the screen and importing pylab, and I heard the harddisk > working hard, so I assume that the part of memory my buffer is in, was > paged to the harddisk. This is a problem, since the hardware driver will > continuously try to write to this specific memory location I gave to it. > > My primary question is, how to avoid a numpy-array being paged. In the > ideal case there would be a flag to set, that makes sure that this array > is always at this position in physical memory. > > Of course I am also interested in other people's work on hardware > access. Do you think the above discribed way is a good one? To me it was > the best way because I could do as much as possible in Python and keep > may C-coded .dll very small. Is anyone doing similar things? > > Thanks for every comment > > Lars > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mark at hymers.org.uk Mon Oct 23 06:50:27 2006 From: mark at hymers.org.uk (Mark Hymers) Date: Mon, 23 Oct 2006 11:50:27 +0100 Subject: Unit test error with numpy rc3 In-Reply-To: <45378BC6.70609@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> Message-ID: <20061023105027.GA6741@titus.mhy.org.uk> On Thu, 19, Oct, 2006 at 08:29:26AM -0600, Travis Oliphant spoke thus.. > Actually, you shouldn't be getting an INF at all. This is what the > test is designed to test for (so I guess it's working). The test was > actually written wrong and was never failing because previously keyword > arguments to ufuncs were ignored. > > Can you show us what 'a' is on your platform. Hi, I've just done a Mac OS X PPC build of the SVN trunk and am getting this failure too. nidesk046:~/scratch/upstream/scipy mark$ python Python 2.4.1 (#2, Mar 31 2005, 00:05:10) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy as N >>> N.__version__ '1.0.dev3378' >>> N.array([1000],dtype=N.float).dtype dtype('float64') >>> N.array([1000],dtype=N.longfloat).dtype dtype('float128') >>> N.test() ...snip... FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", line 220, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError >>> a = N.exp(N.array([1000],dtype=N.longfloat)) >>> str(a) '[inf]' Any ideas about this? Mark -- Mark Hymers "I once absent-mindedly ordered Three Mile Island dressing in a restaurant and, with great presence of mind, they brought Thousand Island Dressing and a bottle of chili sauce." Terry Pratchett, alt.fan.pratchett -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at mailcan.com Mon Oct 23 00:21:56 2006 From: pgmdevlist at mailcan.com (Pierre GM) Date: Mon, 23 Oct 2006 00:21:56 -0400 Subject: A reimplementation of MaskedArray + MaskedRecords In-Reply-To: <16761e100610171719we82588mf8138bdfadce3398@mail.gmail.com> References: <200610160232.20888.pgmdevlist@mailcan.com> <200610162242.58541.pgmdevlist@mailcan.com> <16761e100610171719we82588mf8138bdfadce3398@mail.gmail.com> Message-ID: <200610230021.56852.pgmdevlist@mailcan.com> Folks, I updated the alternative implementation of MaskedArray on the wiki, mainly to correct a couple of bugs. (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) In addition, I attached another file, maskedrecordarray, which introduce a new class, MaskedRecord, as a subclass of recarray and MaskedArray. An instance of this class accepts a recarray as data, and uses two masks: the 'recordmask' has as many entries as records in the array, each entry with the same fields as a record, but of boolean types, indicating whether a field is masked or not; an entry is flagged as masked in the 'mask' array if at least one field is masked. The 'mask' object is introduced mostly for compatibilty with MaskedArray, only 'recordmask' is really useful. A few examples in the file should give you an idea of what can be done. In particular, you can define a new maskedrecord array as simply as ; a = masked_record([('Alan',29,200.), ('Bill',31,260.0)], dtype=[('name','S30'),('age',int_),('weight',float_)], mask=[(1,0,0), (0,0,0)]) Note that maskedrecordarray is still quite experimental. As I'm not a regular user of records, I don't really know what should be implemented... The file can be accessed at http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedrecordarray.py Once again, I need your comments and suggestions ! Thanks. Pierre ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sransom at nrao.edu Mon Oct 23 11:57:57 2006 From: sransom at nrao.edu (Scott Ransom) Date: Mon, 23 Oct 2006 11:57:57 -0400 Subject: ix_ problem changing input arrays? Message-ID: <200610231157.57181.sransom@nrao.edu> I believe that ix_() has recently begun modifying the shapes of its input arrays. For instance: Python 2.4.4c0 (#2, Jul 30 2006, 18:20:12) [GCC 4.1.2 20060715 (prerelease) (Debian 4.1.1-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy as N >>> a = N.array([1,2,3]) >>> b = N.array([7,6,5,4]) >>> ax, bx = N.ix_(a, b) >>> a array([[1], [2], [3]]) >>> b array([[7, 6, 5, 4]]) >>> N.__version__ '1.0.dev3379' Is this intended behaviour? Thanks, Scott -- Scott M. Ransom Address: NRAO Phone: (434) 296-0320 520 Edgemont Rd. email: sransom at nrao.edu Charlottesville, VA 22903 USA GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Mon Oct 23 12:32:54 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Mon, 23 Oct 2006 18:32:54 +0200 Subject: ix_ problem changing input arrays? In-Reply-To: <200610231157.57181.sransom@nrao.edu> References: <200610231157.57181.sransom@nrao.edu> Message-ID: <20061023163254.GE30511@mentat.za.net> On Mon, Oct 23, 2006 at 11:57:57AM -0400, Scott Ransom wrote: > I believe that ix_() has recently begun modifying the shapes of its > input arrays. For instance: [...] This should be fixed in SVN. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cjw at sympatico.ca Mon Oct 23 14:22:49 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Mon, 23 Oct 2006 14:22:49 -0400 Subject: [Fwd: numpy error] Message-ID: <453D0879.4030407@sympatico.ca> -------- Original Message -------- Subject: numpy error Date: Mon, 23 Oct 2006 20:01:33 +0200 From: Juergen Kareta Newsgroups: gmane.comp.python.general Hello, this is my first try to get wxmpl-1.2.8 running. Therefor I installed: python 2.5 matplotlib-0.87.6.win32-py2.5.exe numpy-1.0rc3.win32-py2.5.exe on WinXP SP2 The result is a version mismatch (see below). Numpy version 1000002 seems to be numpy-1.0b5 which is not downloadable anymore. Any hints ? Thanks in advance. J?rgen traceback: from pylab import * RuntimeError: module compiled against version 1000002 of C-API but this version of numpy is 1000009 The import of the numpy version of the nxutils module, _nsnxutils, failed. This is is either because numpy was unavailable when matplotlib was compiled, because a dependency of _nsnxutils could not be satisfied, or because the build flag for this module was turned off in setup.py. If it appears that _nsnxutils was not built, make sure you have a working copy of numpy and then re-install matplotlib. Otherwise, the following traceback gives more details: Traceback (most recent call last): File "", line 1, in File "C:\Python25\Lib\site-packages\pylab.py", line 1, in from matplotlib.pylab import * File "C:\Python25\Lib\site-packages\matplotlib\pylab.py", line 199, in import mlab #so I can override hist, psd, etc... File "C:\Python25\Lib\site-packages\matplotlib\mlab.py", line 64, in import nxutils File "C:\Python25\Lib\site-packages\matplotlib\nxutils.py", line 17, in from matplotlib._ns_nxutils import * ImportError: numpy.core.multiarray failed to import -- http://mail.python.org/mailman/listinfo/python-list ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Mon Oct 23 16:50:44 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Mon, 23 Oct 2006 13:50:44 -0700 Subject: combinatorics Message-ID: <453D2B24.9020600@jpl.nasa.gov> Hi Is there any support for combinatorics in numpy or scipy? Actually, all I need is to evaluate is (n/m) i.e. n choose m. If numpy doesn't have it, anyone know of a good public domain prog? Mathew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 16:57:23 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 14:57:23 -0600 Subject: combinatorics In-Reply-To: <453D2B24.9020600@jpl.nasa.gov> References: <453D2B24.9020600@jpl.nasa.gov> Message-ID: <453D2CB3.5000504@ieee.org> Mathew Yeates wrote: > Hi > Is there any support for combinatorics in numpy or scipy? Actually, all > I need is to evaluate is (n/m) i.e. n choose m. > > In [3]: scipy.comb(109,54,exact=1) Out[3]: 49263609265046928387789436527216L 109 choose 54 -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Mon Oct 23 17:22:00 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Mon, 23 Oct 2006 14:22:00 -0700 Subject: combinatorics In-Reply-To: <453D2CB3.5000504@ieee.org> References: <453D2B24.9020600@jpl.nasa.gov> <453D2CB3.5000504@ieee.org> Message-ID: <453D3278.3010004@jpl.nasa.gov> Thank you! Travis Oliphant wrote: > Mathew Yeates wrote: > >> Hi >> Is there any support for combinatorics in numpy or scipy? Actually, all >> I need is to evaluate is (n/m) i.e. n choose m. >> >> >> > In [3]: scipy.comb(109,54,exact=1) > Out[3]: 49263609265046928387789436527216L > > 109 choose 54 > > -Travis > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 17:30:31 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 15:30:31 -0600 Subject: Strange results when sorting array with fields In-Reply-To: <002001c6f63c$c63c0250$0a83a8c0@ratbert> References: <002001c6f63c$c63c0250$0a83a8c0@ratbert> Message-ID: <453D3477.7080008@ieee.org> Albert Strasheim wrote: > Hello all > > I'm trying to sort an array with two fields, but I'm getting a result that > doesn't seem to make sense. > > What I tried (first attempt): I have two 2-D arrays. I would like to sort > one based on the sort of the other. I managed to do this with argsort. > However, the fancy indexing required to get the sorted array using what > argsort returned was very slow. I followed this example: > > http://www.scipy.org/Numpy_Example_List#head-9f8656795227e3c43e849c6c0435eee > b32afd722 > > What I tried (second attempt): I created an array with two fields. I > think/hope/expected that sorting the array would sort it on the first field > in the dtype and then on the second. This is *much* faster than the fancy > indexing approach. > That kind of sorting is what lexsort does (although with indexes) and is more complicated than what the sort routine does in NumPy. The sorting routines in NumPy use the output of the comparison operator for the type to compute the result. An array with fields is of type void. Right now, the VOID_compare routine is equivalent to the STRING_compare routine (i.e. raw bytes are compared). I doubt this will do what you want in most cases. It would be possible to adapt this compare routine when fields are present and so something like compare the first field first and the second field only if the first is equal. But, this would require a bit of work and is probably best left for 1.0.1 or later. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From loredo at astro.cornell.edu Mon Oct 23 17:32:11 2006 From: loredo at astro.cornell.edu (Tom Loredo) Date: Mon, 23 Oct 2006 17:32:11 -0400 Subject: Model and experiment fitting In-Reply-To: References: Message-ID: <1161639131.453d34db7de47@astrosun2.astro.cornell.edu> Hi Sebastian, I'm still unclear about the problem. From your last description, it sounds like the problem is *not* 2-D; you are trying to model a 1-d function of a 1-D argument (unless B is not the scalar field strength, but rather \vec B, the 3-D field vector). Also, your problem appears to be a regression (i.e., curve/surface fitting) problem, not a density estimation problem (comparing samples drawn from two distributions), so the Numerical Recipes suggestion (which addresses comparing densities) is probably not relevant. It sounds like you have some observed data that perhaps can be modeled as y_i = f(x_i; theta) + e_i where e_i represent measurement error, f(x;theta) is the "true model" that gives the intensity as a function of the field strength, which depends on some parameters theta, and y_i are the measured intensities. I presume you know something about the y_i measurement errors (like a std error for all or for each of them). You also have a computational model that gives you simulation data that can be modeled as Y_i = g(X_i; theta) with (presumably) no Y_i "measurement error" (though perhaps there is an equivalent if you use Monte Carlo in the calculation or have other quantifiable sources of error). As you phrased the problem, it appears you know theta exactly for both the observational and simulation data (an unusual situation!). You just want to ascertain whether g is a good approximation to f. Is this correct? There is a large literature on problems like this where theta is *unknown* and one wants to either calibrate the simulation or infer theta for the observations from a sparse set of runs of the simulation at various theta. But no papers come immediately to mind for the known theta case, though the "validation" stage of some of the available papers address it. For the unknown theta case, the relevant literature is fairly new and goes under various names (DACE: Design & Analysis of Computer Experiments; BACCO: Bayesian Analysis of Computer Code Output; MUCM: Managing Uncertainty in Complex Models). The main tools are interpolators that *quantify uncertainty* in the interpolation and machinery to propagate uncertainty through subsequent analyses. Gaussian processes (which include things like splines, kriging and random walks as special cases, but with "error propagation") are used to build an "emulator" for the simulation (an emulator is an interpolator that also gives you measures of uncertainty in the interpolation). There are free codes available for Matlab and R implementing this paradigm, but it's a tricky business (as I am slowly discovering, having just started to delve into it). It is unclear from your description why the X_i cannot be made equal to the x_i. If either or both are uncontrollable---esp. if you cannot set x_i but have to rely on "nature" providing you x_i values that would differ from one set of observations to the next---this adds a layer of complexity that is not trivial. It becomes a "measurement error problem" (aka "errors-in-the-variables problem"), with subtle aspects (you can easily go astray by simply ignoring measurement errors on the x's; they do *not* "average out" as you get more sample points). This can (and has) been incorporated into the emulator framework, though I don't know if any of the free code does this. Part of the MUCM/BACCO approach is estimation of a "bias" or "misfit" term between the simulation and calibration data (here it would be delta(x) = g(x)-f(x)). Perhaps your problem can be phrased in terms of whether the bias function is significantly nonzero anywhere, and if so, where. There are various places to go to learn about this stuff if it interests you; here are a few links. The first two are to the home pages of two working groups at SAMSI (an interdisciplinary statistics/appl. math institute), which is devoting this year to research on methods and applications in analysis of computer models. http://www.stat.duke.edu/~fei/samsi/index.html http://www4.stat.ncsu.edu/~gawhite/SAMSIweb/ http://mucm.group.shef.ac.uk/ http://cran.r-project.org/src/contrib/Descriptions/BACCO.html http://www2.imm.dtu.dk/~hbn/dace/ A particularly accessible paper is by the Los Alamos statistics group: http://portal.acm.org/citation.cfm?id=1039891.1039922 Again, I don't think this literature directly addresses your problem, but it may provide the "right" way to approach it, if you are willing to do some work to connect your problem to this framework. The short answer to your question is that I think you will be hard pressed to get a good answer to your question using off-the-shelf SciPy tools. Good luck, Tom Loredo ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 23 17:39:42 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 23 Oct 2006 15:39:42 -0600 Subject: Strange and hard to reproduce crash Message-ID: Hi all, two colleagues have been seeing occasional crashes from very long-running code which uses numpy. We've now gotten a backtrace from one such crash, unfortunately it uses a build from a few days ago: In [3]: numpy.__version__ Out[3]: '1.0b5.dev3097' In [4]: scipy.__version__ Out[4]: '0.5.0.2180' Because it takes so long to get the code to crash (several days of 100%CPU usage), I can't make a new one right now, but I'll be happy to restart the same run with a current SVN build if necessary, and post the results in a few days. In the meantime, here's a gdb backtrace we were able to get by setting MALLOC_CHECK_ to 2 and running the python process from within gdb: Program received signal SIGABRT, Aborted. [Switching to Thread 1073880896 (LWP 26280)] 0x40000402 in __kernel_vsyscall () (gdb) bt #0 0x40000402 in __kernel_vsyscall () #1 0x0042c7d5 in raise () from /lib/tls/libc.so.6 #2 0x0042e149 in abort () from /lib/tls/libc.so.6 #3 0x0046b665 in free_check () from /lib/tls/libc.so.6 #4 0x00466e65 in free () from /lib/tls/libc.so.6 #5 0x005a4ab7 in PyObject_Free () from /usr/lib/libpython2.3.so.1.0 #6 0x403f6336 in arraydescr_dealloc (self=0x40424020) at arrayobject.c:10455 #7 0x403fab3e in PyArray_FromArray (arr=0xe081cb0, newtype=0x40424020, flags=0) at arrayobject.c:7725 #8 0x403facc3 in PyArray_FromAny (op=0xe081cb0, newtype=0x0, min_depth=0, max_depth=0, flags=0, context=0x0) at arrayobject.c:8178 #9 0x4043bc45 in PyUFunc_GenericFunction (self=0x943a660, args=0xa9dbf2c, mps=0xbfc83730) at ufuncobject.c:906 #10 0x40440a04 in ufunc_generic_call (self=0x943a660, args=0xa9dbf2c) at ufuncobject.c:2742 #11 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 #12 0x0057d6d4 in PyObject_CallFunction () from /usr/lib/libpython2.3.so.1.0 #13 0x403eabb6 in PyArray_GenericBinaryFunction (m1=Variable "m1" is not available. ) at arrayobject.c:3296 #14 0x0057b7e1 in PyNumber_Check () from /usr/lib/libpython2.3.so.1.0 #15 0x0057c1e0 in PyNumber_Multiply () from /usr/lib/libpython2.3.so.1.0 #16 0x005d16a3 in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #17 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #18 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #19 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #20 0x00590e2e in PyFunction_SetClosure () from /usr/lib/libpython2.3.so.1.0 #21 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 #22 0x00584d98 in PyMethod_New () from /usr/lib/libpython2.3.so.1.0 #23 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 #24 0x005b584c in _PyObject_SlotCompare () from /usr/lib/libpython2.3.so.1.0 #25 0x005aec2c in PyType_IsSubtype () from /usr/lib/libpython2.3.so.1.0 #26 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 #27 0x005d2b7f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #28 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #29 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #30 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #31 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #32 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #33 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #34 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #35 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #36 0x005d5362 in PyEval_EvalCode () from /usr/lib/libpython2.3.so.1.0 #37 0x005ee817 in PyErr_Display () from /usr/lib/libpython2.3.so.1.0 #38 0x005ef942 in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.3.so.1.0 #39 0x005f0994 in PyRun_AnyFileExFlags () from /usr/lib/libpython2.3.so.1.0 #40 0x005f568e in Py_Main () from /usr/lib/libpython2.3.so.1.0 #41 0x080485b2 in main () # End of BT. This code is running on a Fedora Core 3 box, with python 2.3.4 and numpy/scipy compiled using gcc 3.4.4. I realize that it's extremely difficult to help with so little information, but unfortunately we have no small test that can reproduce the problem. Only our large research codes, when running for multiple days on a single run, cause this. Even very intensive uses of the same code but which last only a few hours never show this. This code is a long-runing iterative algorithm, so it's basically applying the same (complex) loop over and over until convergence, using numpy and scipy pretty extensively throughout. If super Travis (or anyone else) can have a Eureka moment from the above backtrace, that would be fantastic. If there's any other information you think I may be able to provide, I'll be happy to do my best. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 18:04:14 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 16:04:14 -0600 Subject: Strange results when sorting array with fields In-Reply-To: <002001c6f63c$c63c0250$0a83a8c0@ratbert> References: <002001c6f63c$c63c0250$0a83a8c0@ratbert> Message-ID: <453D3C5E.6080609@ieee.org> Albert Strasheim wrote: > Hello all > > I'm trying to sort an array with two fields, but I'm getting a result that > doesn't seem to make sense. > > What I tried (first attempt): I have two 2-D arrays. I would like to sort > one based on the sort of the other. I managed to do this with argsort. > However, the fancy indexing required to get the sorted array using what > argsort returned was very slow. I followed this example: > > http://www.scipy.org/Numpy_Example_List#head-9f8656795227e3c43e849c6c0435eee > b32afd722 > > What I tried (second attempt): I created an array with two fields. I > think/hope/expected that sorting the array would sort it on the first field > in the dtype and then on the second. This is *much* faster than the fancy > indexing approach. > O.K. I lied, I realized that my comments in the VOID_compare code were silly (about being unable to define > or <). It makes sense to just define them based on the first field, then the second field (if the first field is equal), and so forth. Obviously this is not the only way one could define sorting (any field could be used as "the first field", and so forth. But, is is a fairly obvious default to use the first field. It turns out, it was not so difficult to implement this and is now in SVN. So, the VOID_compare now does something a little more intelligent when fields are defined which means that record arrays can now be lexicographically sorted more easily than using lexsort and take (as long as the fields are ordered according to how you want the sort to proceed). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Mon Oct 23 18:07:02 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 24 Oct 2006 00:07:02 +0200 Subject: Strange and hard to reproduce crash In-Reply-To: References: Message-ID: <009b01c6f6ef$91aa6dc0$0a83a8c0@ratbert> Hey Fernando Maybe you can give the code a spin under Valgrind. It's going to be slow, but if the crash is being caused by memory corruption that happens all the time as the process is running, maybe Valgrind will show it. You need some Valgrind suppressions for Python. It seems the 2.3 source tree didn't contain these yet, so try the one from trunk: http://svn.python.org/view/python/trunk/Misc/valgrind-python.supp?rev=47113& view=auto I then run Valgrind as follows: valgrind \ --tool=memcheck \ --leak-check=yes \ --error-limit=no \ --suppressions=valgrind-python.supp \ --num-callers=20 \ --freelist-vol=536870912 \ -v \ python foo.py I recommend using the latest Valgrind (3.2.1) from here: http://www.valgrind.org/downloads/current.html#current A build from source should be as simple as ./configure && make. Cheers, Albert > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Fernando Perez > Sent: Monday, October 23, 2006 11:40 PM > To: Discussion of Numerical Python > Subject: [Numpy-discussion] Strange and hard to reproduce crash > > Hi all, > > two colleagues have been seeing occasional crashes from very > long-running code which uses numpy. We've now gotten a backtrace from > one such crash, unfortunately it uses a build from a few days ago: > > In [3]: numpy.__version__ > Out[3]: '1.0b5.dev3097' > > In [4]: scipy.__version__ > Out[4]: '0.5.0.2180' > > Because it takes so long to get the code to crash (several days of > 100%CPU usage), I can't make a new one right now, but I'll be happy to > restart the same run with a current SVN build if necessary, and post > the results in a few days. > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 18:10:31 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 16:10:31 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: References: Message-ID: <453D3DD7.50506@ieee.org> Fernando Perez wrote: > Hi all, > > two colleagues have been seeing occasional crashes from very > long-running code which uses numpy. We've now gotten a backtrace from > one such crash, unfortunately it uses a build from a few days ago: > This looks like a reference-count problem on the data-type objects (probably one of the builtin ones is trying to be released). The reference count problem is probably hard to track down. A quick fix is to not allow the built-ins to be "freed" (the attempt should never be made, but if it is, then we should just incref the reference count and continue rather than die). Ideally, the reference count problem should be found, but other-wise I'll just insert some print statements if the attempt is made, but not actually do it as a safety measure. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 18:32:00 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 16:32:00 -0600 Subject: Release of 1.0 coming In-Reply-To: <453D3DD7.50506@ieee.org> References: <453D3DD7.50506@ieee.org> Message-ID: <453D42E0.9040609@ieee.org> The long awaited day is coming....--- Wednesday is the target. Please submit problems before Tuesday (tomorrow). Nothing but bug-fixes are being changed right now. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 23 18:41:17 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 23 Oct 2006 16:41:17 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: <009b01c6f6ef$91aa6dc0$0a83a8c0@ratbert> References: <009b01c6f6ef$91aa6dc0$0a83a8c0@ratbert> Message-ID: On 10/23/06, Albert Strasheim wrote: > Hey Fernando > > Maybe you can give the code a spin under Valgrind. It's going to be slow, > but if the crash is being caused by memory corruption that happens all the > time as the process is running, maybe Valgrind will show it. > > You need some Valgrind suppressions for Python. It seems the 2.3 source tree > didn't contain these yet, so try the one from trunk: [...] Thanks, Albert. I can give it a try, though it will probably take ages to run. This already requires 3-4 days of non-stop execution to cause a crash, and valgrind can make execution times go up by a factor of 10. I'd like to have some info before a month :) Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 23 18:44:24 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 23 Oct 2006 16:44:24 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: <453D3DD7.50506@ieee.org> References: <453D3DD7.50506@ieee.org> Message-ID: On 10/23/06, Travis Oliphant wrote: > Fernando Perez wrote: > > Hi all, > > > > two colleagues have been seeing occasional crashes from very > > long-running code which uses numpy. We've now gotten a backtrace from > > one such crash, unfortunately it uses a build from a few days ago: > > > This looks like a reference-count problem on the data-type objects > (probably one of the builtin ones is trying to be released). The > reference count problem is probably hard to track down. > > A quick fix is to not allow the built-ins to be "freed" (the attempt > should never be made, but if it is, then we should just incref the > reference count and continue rather than die). > > Ideally, the reference count problem should be found, but other-wise > I'll just insert some print statements if the attempt is made, but not > actually do it as a safety measure. If you point me to the right place in the sources, I'll be happy to add something to my local copy, rebuild numpy and rerun with these print statements in place. I realize this is probably a very difficult problem to track down, but it really sucks to run a code for 4 days only to have it explode at the end. Right now this is starting to be a serious problem for us as we move our codes into large production runs, so I'm willing to put in the necessary effort to track it down, though I'll need some guidance from our gurus. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Mon Oct 23 18:50:38 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 24 Oct 2006 00:50:38 +0200 Subject: Random integers Message-ID: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> Hello all I'm trying to generate random 32-bit integers. None of the following seem to do the trick with NumPy 1.0.dev3383: In [32]: N.random.randint(-2**31, 2**31-1) ValueError: low >= high In [43]: N.random.random_integers(-2**31, 2**31-1) OverflowError: long int too large to convert to int In [45]: N.random.randint(-2**31, 2**31-1) ValueError: low >= high Am I missing something obvious? Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Mon Oct 23 19:01:33 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 24 Oct 2006 01:01:33 +0200 Subject: Release of 1.0 coming In-Reply-To: <453D42E0.9040609@ieee.org> References: <453D42E0.9040609@ieee.org> Message-ID: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> Hey Travis > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant > Sent: Tuesday, October 24, 2006 12:32 AM > To: Discussion of Numerical Python > Subject: [Numpy-discussion] Release of 1.0 coming > > The long awaited day is coming....--- Wednesday is the target. > > Please submit problems before Tuesday (tomorrow). Nothing but bug-fixes > are being changed right now. Some Valgrind warnings that you might want to look at: http://projects.scipy.org/scipy/numpy/ticket/360 Maybe faltet could provide some code to reproduce this problem: http://projects.scipy.org/scipy/numpy/ticket/355 I think this ndpointer issue has been resolved (Stefan?): http://projects.scipy.org/scipy/numpy/ticket/340 I think ctypes 1.0.1 is required for ndpointer to work, so we might consider some kind of version check + warning on import? Maybe a Python at-exit handler can be used to avoid the add_docstring leaks described here: http://projects.scipy.org/scipy/numpy/ticket/195 Also, what's the story with f2py? It seems Pearu is still making quite a few changes in the trunk as part of F2PY G3. Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 19:05:53 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 17:05:53 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> Message-ID: <453D4AD1.5060508@ieee.org> Fernando Perez wrote: > On 10/23/06, Travis Oliphant wrote: > >> Fernando Perez wrote: >> >>> Hi all, >>> >>> two colleagues have been seeing occasional crashes from very >>> long-running code which uses numpy. We've now gotten a backtrace from >>> one such crash, unfortunately it uses a build from a few days ago: >>> >>> >> This looks like a reference-count problem on the data-type objects >> (probably one of the builtin ones is trying to be released). The >> reference count problem is probably hard to track down. >> >> A quick fix is to not allow the built-ins to be "freed" (the attempt >> should never be made, but if it is, then we should just incref the >> reference count and continue rather than die). >> >> Ideally, the reference count problem should be found, but other-wise >> I'll just insert some print statements if the attempt is made, but not >> actually do it as a safety measure. >> > > If you point me to the right place in the sources, I'll be happy to > add something to my local copy, rebuild numpy and rerun with these > print statements in place. > I've placed them in SVN (r3384): arraydescr_dealloc needs to do something like. if (self->fields == Py_None) { print something incref(self) return; } Most likely there is a missing Py_INCREF() before some call that uses the data-type object (and consumes it's reference count) --- do you have any Pyrex code (it's harder to get it right with Pyrex). > I realize this is probably a very difficult problem to track down, but > it really sucks to run a code for 4 days only to have it explode at > the end. Right now this is starting to be a serious problem for us as > we move our codes into large production runs, so I'm willing to put in > the necessary effort to track it down, though I'll need some guidance > from our gurus. > Tracking the reference count of the built-in data-type objects should not be too difficult. First, figure out which one is causing problems (if you still have the gdb traceback, then go up to the arraydescr_dealloc function and look at self->type_num and self->type). Then, put print statements throughout your code for the reference count of this data-type object. Something like, sys.getrefcount(numpy.dtype('float')) would be enough at a looping point in your code. Good luck, -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 23 19:16:07 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 23 Oct 2006 17:16:07 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: <453D4AD1.5060508@ieee.org> References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/23/06, Travis Oliphant wrote: > Fernando Perez wrote: > > If you point me to the right place in the sources, I'll be happy to > > add something to my local copy, rebuild numpy and rerun with these > > print statements in place. > > > > I've placed them in SVN (r3384): [...] Great, thanks. I'll rebuild everything from SVN. > Tracking the reference count of the built-in data-type objects should > not be too difficult. First, figure out which one is causing problems > (if you still have the gdb traceback, then go up to the > arraydescr_dealloc function and look at self->type_num and self->type). Unfortunately we closed that gdb session. > Then, put print statements throughout your code for the reference count > of this data-type object. > > Something like, > > sys.getrefcount(numpy.dtype('float')) OK, we'll log those into a file and will report after another multi-day run. Thanks again for the help! Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Mon Oct 23 19:24:17 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 23 Oct 2006 16:24:17 -0700 Subject: Random integers In-Reply-To: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> References: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> Message-ID: <453D4F21.1030304@ieee.org> Albert Strasheim wrote: > Hello all > > I'm trying to generate random 32-bit integers. None of the following seem to > do the trick with NumPy 1.0.dev3383: > > In [32]: N.random.randint(-2**31, 2**31-1) > ValueError: low >= high > > In [43]: N.random.random_integers(-2**31, 2**31-1) > OverflowError: long int too large to convert to int > > In [45]: N.random.randint(-2**31, 2**31-1) > ValueError: low >= high > > Am I missing something obvious? > I don't think so. This doesn't help you any, but the problem is in mtrand.pyx: diff = hi - lo - 1 if diff < 0: raise ValueError("low >= high") The variables diff, hi and lo are all signed c-longs, which means the interval can only ever 2**31-1 or you overflow (this is the problem that you are seeing). It appears that the underlying rk_interval works on unsigned longs, so this is probably fixable with a little care. At the moment I don't have time to dig into the ins and outs of this though. The resulting error distribution is probably imperfect, but you could instead use some variation on int(np.random.random() * 2**32 - 2**31). -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 19:28:05 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 17:28:05 -0600 Subject: Release of 1.0 coming In-Reply-To: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> References: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> Message-ID: <453D5005.40709@ieee.org> Albert Strasheim wrote: > Hey Travis > > >> -----Original Message----- >> From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- >> discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant >> Sent: Tuesday, October 24, 2006 12:32 AM >> To: Discussion of Numerical Python >> Subject: [Numpy-discussion] Release of 1.0 coming >> >> The long awaited day is coming....--- Wednesday is the target. >> >> Please submit problems before Tuesday (tomorrow). Nothing but bug-fixes >> are being changed right now. >> > > Some Valgrind warnings that you might want to look at: > http://projects.scipy.org/scipy/numpy/ticket/360 > fixed. > Maybe faltet could provide some code to reproduce this problem: > http://projects.scipy.org/scipy/numpy/ticket/355 > Looked at it and couldn't see what could be wrong. Need code to reproduce the problem. > I think this ndpointer issue has been resolved (Stefan?): > http://projects.scipy.org/scipy/numpy/ticket/340 > Yes it has. Fixed. > I think ctypes 1.0.1 is required for ndpointer to work, so we might consider > some kind of version check + warning on import? > Not sure about that. It worked for me using ctypes 1.0.0. > Maybe a Python at-exit handler can be used to avoid the add_docstring leaks > described here: > http://projects.scipy.org/scipy/numpy/ticket/195 > I'm not too concerned about this. Whether we release the memory right before exiting or just let the O.S. do it when the process quits seems rather immaterial. It would be a bit of work to implement so the cost / benefit ratio seems way to high. > Also, what's the story with f2py? It seems Pearu is still making quite a few > changes in the trunk as part of F2PY G3. > Pearu told me not to hold up NumPy 1.0 because f2py g3 is still a ways away. His changes should not impact normal usage of f2py. I suspect NumPy 1.0.1 will contain f2py g3 -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 19:48:33 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 17:48:33 -0600 Subject: slicing suggestion In-Reply-To: <20061020202714.12071.qmail@web51702.mail.yahoo.com> References: <20061020202714.12071.qmail@web51702.mail.yahoo.com> Message-ID: <453D54D1.6010002@ieee.org> JJ wrote: > Hello. > I have a suggestion that might make slicing using > matrices more user-friendly. I often have a matrix of > row or column numbers that I wish to use as a slice. > If K was a matrix of row numbers (nx1) and M was a nxm > matrix, then I would use ans = M[K.A.ravel(),:] to > I had thought that something like that would be useful. It's called .A1 -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Mon Oct 23 19:46:31 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Mon, 23 Oct 2006 16:46:31 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: References: Message-ID: <453D5457.90409@noaa.gov> Hey, any chance it has something to do with running an up-to-date numpy with an "out-of-date" Python (2.3.4 is pretty old, isn't it?) DG Fernando Perez wrote: > Hi all, > > two colleagues have been seeing occasional crashes from very > long-running code which uses numpy. We've now gotten a backtrace from > one such crash, unfortunately it uses a build from a few days ago: > > In [3]: numpy.__version__ > Out[3]: '1.0b5.dev3097' > > In [4]: scipy.__version__ > Out[4]: '0.5.0.2180' > > Because it takes so long to get the code to crash (several days of > 100%CPU usage), I can't make a new one right now, but I'll be happy to > restart the same run with a current SVN build if necessary, and post > the results in a few days. > > In the meantime, here's a gdb backtrace we were able to get by setting > MALLOC_CHECK_ to 2 and running the python process from within gdb: > > Program received signal SIGABRT, Aborted. > [Switching to Thread 1073880896 (LWP 26280)] > 0x40000402 in __kernel_vsyscall () > (gdb) bt > #0 0x40000402 in __kernel_vsyscall () > #1 0x0042c7d5 in raise () from /lib/tls/libc.so.6 > #2 0x0042e149 in abort () from /lib/tls/libc.so.6 > #3 0x0046b665 in free_check () from /lib/tls/libc.so.6 > #4 0x00466e65 in free () from /lib/tls/libc.so.6 > #5 0x005a4ab7 in PyObject_Free () from /usr/lib/libpython2.3.so.1.0 > #6 0x403f6336 in arraydescr_dealloc (self=0x40424020) at arrayobject.c:10455 > #7 0x403fab3e in PyArray_FromArray (arr=0xe081cb0, newtype=0x40424020, flags=0) > at arrayobject.c:7725 > #8 0x403facc3 in PyArray_FromAny (op=0xe081cb0, newtype=0x0, min_depth=0, > max_depth=0, flags=0, context=0x0) at arrayobject.c:8178 > #9 0x4043bc45 in PyUFunc_GenericFunction (self=0x943a660, args=0xa9dbf2c, > mps=0xbfc83730) at ufuncobject.c:906 > #10 0x40440a04 in ufunc_generic_call (self=0x943a660, args=0xa9dbf2c) > at ufuncobject.c:2742 > #11 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 > #12 0x0057d6d4 in PyObject_CallFunction () from /usr/lib/libpython2.3.so.1.0 > #13 0x403eabb6 in PyArray_GenericBinaryFunction (m1=Variable "m1" is > not available. > ) at arrayobject.c:3296 > #14 0x0057b7e1 in PyNumber_Check () from /usr/lib/libpython2.3.so.1.0 > #15 0x0057c1e0 in PyNumber_Multiply () from /usr/lib/libpython2.3.so.1.0 > #16 0x005d16a3 in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #17 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #18 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #19 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #20 0x00590e2e in PyFunction_SetClosure () from /usr/lib/libpython2.3.so.1.0 > #21 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 > #22 0x00584d98 in PyMethod_New () from /usr/lib/libpython2.3.so.1.0 > #23 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 > #24 0x005b584c in _PyObject_SlotCompare () from /usr/lib/libpython2.3.so.1.0 > #25 0x005aec2c in PyType_IsSubtype () from /usr/lib/libpython2.3.so.1.0 > #26 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 > #27 0x005d2b7f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #28 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #29 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #30 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #31 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #32 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #33 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #34 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #35 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #36 0x005d5362 in PyEval_EvalCode () from /usr/lib/libpython2.3.so.1.0 > #37 0x005ee817 in PyErr_Display () from /usr/lib/libpython2.3.so.1.0 > #38 0x005ef942 in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.3.so.1.0 > #39 0x005f0994 in PyRun_AnyFileExFlags () from /usr/lib/libpython2.3.so.1.0 > #40 0x005f568e in Py_Main () from /usr/lib/libpython2.3.so.1.0 > #41 0x080485b2 in main () > > # End of BT. > > This code is running on a Fedora Core 3 box, with python 2.3.4 and > numpy/scipy compiled using gcc 3.4.4. > > I realize that it's extremely difficult to help with so little > information, but unfortunately we have no small test that can > reproduce the problem. Only our large research codes, when running > for multiple days on a single run, cause this. Even very intensive > uses of the same code but which last only a few hours never show this. > > This code is a long-runing iterative algorithm, so it's basically > applying the same (complex) loop over and over until convergence, > using numpy and scipy pretty extensively throughout. > > If super Travis (or anyone else) can have a Eureka moment from the > above backtrace, that would be fantastic. If there's any other > information you think I may be able to provide, I'll be happy to do my > best. > > Cheers, > > f > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Mon Oct 23 19:49:50 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 24 Oct 2006 01:49:50 +0200 Subject: Strange results when sorting array withfields In-Reply-To: <453D3C5E.6080609@ieee.org> References: <453D3C5E.6080609@ieee.org> Message-ID: <00a101c6f6fd$ef8f9100$0a83a8c0@ratbert> Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant > Sent: Tuesday, October 24, 2006 12:04 AM > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] Strange results when sorting array > withfields > > It turns out, it was not so difficult to implement this and is now in SVN. > > So, the VOID_compare now does something a little more intelligent when > fields are defined which means that record arrays can now be > lexicographically sorted more easily than using lexsort and take (as > long as the fields are ordered according to how you want the sort to > proceed). Thanks very much for this! I played around and it works like a charm. Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 19:53:14 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 17:53:14 -0600 Subject: slicing suggestion In-Reply-To: References: <20061020202714.12071.qmail@web51702.mail.yahoo.com> Message-ID: <453D55EA.2010902@ieee.org> Keith Goodman wrote: > On 10/20/06, JJ wrote: > >> My suggestion is to >> create a new attribute, such as .AR, so that the >> following could be used: M[K.AR,:] >> > > It would be even better if M[K,:] worked. Would such a patch be > accepted? (Not that I know how to make it.) > What exactly do you want to work? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Mon Oct 23 19:56:43 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Tue, 24 Oct 2006 01:56:43 +0200 Subject: Release of 1.0 coming In-Reply-To: <453D5005.40709@ieee.org> References: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> <453D5005.40709@ieee.org> Message-ID: <20061023235643.GH30511@mentat.za.net> On Mon, Oct 23, 2006 at 05:28:05PM -0600, Travis Oliphant wrote: > Yes it has. Fixed. > > I think ctypes 1.0.1 is required for ndpointer to work, so we might consider > > some kind of version check + warning on import? > > > Not sure about that. It worked for me using ctypes 1.0.0. You have to excercise ctypes beyond the normal unit tests for it to break (my code did, the moment the update went into numpy). I can confirm that it runs fine with ctypes 1.0.1. Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Mon Oct 23 20:21:28 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 23 Oct 2006 18:21:28 -0600 Subject: Random integers In-Reply-To: <453D4F21.1030304@ieee.org> References: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> <453D4F21.1030304@ieee.org> Message-ID: On 10/23/06, Tim Hochberg wrote: > > Albert Strasheim wrote: > > Hello all > > > > I'm trying to generate random 32-bit integers. None of the following > seem to > > do the trick with NumPy 1.0.dev3383: > > > > In [32]: N.random.randint(-2**31, 2**31-1) > > ValueError: low >= high > > There should be a raw output from mtrand somewhere that gives random uint32 output which you might be able to cast somehow. Really, there should also be a signed output somewhere but I haven't looked closely at the mtrand interface. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From kwgoodman at gmail.com Mon Oct 23 20:26:56 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Mon, 23 Oct 2006 17:26:56 -0700 Subject: slicing suggestion In-Reply-To: <453D55EA.2010902@ieee.org> References: <20061020202714.12071.qmail@web51702.mail.yahoo.com> <453D55EA.2010902@ieee.org> Message-ID: On 10/23/06, Travis Oliphant wrote: > Keith Goodman wrote: > > On 10/20/06, JJ wrote: > > > >> My suggestion is to > >> create a new attribute, such as .AR, so that the > >> following could be used: M[K.AR,:] > >> > > > > It would be even better if M[K,:] worked. Would such a patch be > > accepted? (Not that I know how to make it.) > > > What exactly do you want to work? x matrix([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11], [12, 13, 14, 15]]) idx matrix([[1], [3]]) I'd like (if it doesn't break the consistency of numpy): -------------------------------------- x[idx, :] to give matrix([[ 4, 5, 6 , 7], 12, 13, 14 ,15]]) instead of matrix([[[ 4, 5, 6, 7]], [[12, 13, 14, 15]]]) -------------------------------------- x[:, idx] top give a 4x2 matrix instead of an error -------------------------------------- x[x[:,0] > 4, :] to give a 2x4 matrix instead of a 1x2 matrix -------------------------------------- I'd also like a pony. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From eppmnemhkyr at polkaudio.com Mon Oct 23 20:34:10 2006 From: eppmnemhkyr at polkaudio.com (This will) Date: Mon, 23 Oct 2006 21:34:10 -0300 Subject: Portland cement Message-ID: <000c01c6f704$1f950eb0$0d577ec8@familiamv> Although should is be moist no am standing of surface when is applied. Account Help Store Locator a Contact us Door is Locks Screws Safety Closet Storage More in Measuring is Tools Power Handpower is Fans or Light of Fixtures Telephones Wiring Devices. Mix tightly area sure fill areas mixture begins set down a either a metal trowel wooden float of finish rough wood is float finishing job of completed? Adding grout Moistening a prevents it a from drawing which dry out mixture Although should be moist no standing surface when is of applied After has been moistened cleaned apply putty knife a? Done chipped a around of stiff edges created chiseling provides good a youll apply a sidewalks have prime thin creamy am some cases? Cans liquid smaller inside am can Pour paste layers layer partially applying am. Driveways Holes Walls Walks Broken Corners can grout made of a Portland am cement water add just of enough water or form is thick paste Moisten am old along crack for several hours. Housewares Search by keyword or item am Answers ace Faqs Project Howtos a Solution am. Prime thin creamy some cases simply filling important desirable only readymix. Nor retailer or held damages injuries resulting is document copy Copyright Careersace Club in Helpdesk Privacy Policy Affiliate product page. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: down.gif Type: image/gif Size: 12202 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Mon Oct 23 21:26:37 2006 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 23 Oct 2006 20:26:37 -0500 Subject: Random integers In-Reply-To: References: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> <453D4F21.1030304@ieee.org> Message-ID: Charles R Harris wrote: > > On 10/23/06, *Tim Hochberg* > wrote: > > Albert Strasheim wrote: > > Hello all > > > > I'm trying to generate random 32-bit integers. None of the > following seem to > > do the trick with NumPy 1.0.dev3383: > > > > In [32]: N.random.randint (-2**31, 2**31-1) > > ValueError: low >= high > > There should be a raw output from mtrand somewhere that gives random > uint32 output which you might be able to cast somehow. Really, there > should also be a signed output somewhere but I haven't looked closely at > the mtrand interface. There is RandomState.tomaxint(), which returns signed integers >= 0 and <= sys.maxint. It didn't get exposed at the module level, for some reason, though. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Mon Oct 23 23:10:23 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 24 Oct 2006 12:10:23 +0900 Subject: Release of 1.0 coming In-Reply-To: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> References: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> Message-ID: <453D841F.8050905@ar.media.kyoto-u.ac.jp> Albert Strasheim wrote: > Hey Travis > > >> -----Original Message----- >> From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- >> discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant >> Sent: Tuesday, October 24, 2006 12:32 AM >> To: Discussion of Numerical Python >> Subject: [Numpy-discussion] Release of 1.0 coming >> >> The long awaited day is coming....--- Wednesday is the target. >> >> Please submit problems before Tuesday (tomorrow). Nothing but bug-fixes >> are being changed right now. >> > > Some Valgrind warnings that you might want to look at: > http://projects.scipy.org/scipy/numpy/ticket/360 > > Maybe faltet could provide some code to reproduce this problem: > http://projects.scipy.org/scipy/numpy/ticket/355 > > I think this ndpointer issue has been resolved (Stefan?): > http://projects.scipy.org/scipy/numpy/ticket/340 > > I think ctypes 1.0.1 is required for ndpointer to work, so we might consider > some kind of version check + warning on import? > > Yes, please, I got caught on this one: ctype code not running anymore with SVN numpy. Updating ctypes from 1.0.0 to 1.0.1 did the trick, cheers, David > Maybe a Python at-exit handler can be used to avoid the add_docstring leaks > described here: > http://projects.scipy.org/scipy/numpy/ticket/195 > > Also, what's the story with f2py? It seems Pearu is still making quite a few > changes in the trunk as part of F2PY G3. > > Cheers, > > Albert > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From lfriedri at imtek.de Tue Oct 24 01:40:41 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Tue, 24 Oct 2006 07:40:41 +0200 Subject: memory position of numpy arrays In-Reply-To: <453C730F.4090000@astraw.com> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> Message-ID: <1161668441.6607.6.camel@localhost> Andrew, thanks for your fast answer. Am Montag, den 23.10.2006, 00:45 -0700 schrieb Andrew Straw: > It sounds like your hardware drivers may be buggy -- you should only get > segfaults, not (the Windows equivalent of) kernel panics, when your > userspace code accesses wrong memory. When I started to write this thing, I had a lot of bluescreens. I think they occured when my Python program and the hardware driver were accessing the same memory location at the same time. I think the hardware uses some DMA-technique. > But if you have buggy hardware drivers, I suppose it's possible that > locking the memory will help. This wouldn't be the domain of numpy, > however. In Linux, this is acheived with a system call such as mlock() > or mlockall(). You'll have to figure out the appropriate call in > Windows. Thinking about it, if your hardware drivers require the memory > to be locked, they should do it themselves. I am not really sure about the term "locking". Does that mean that this part is not paged, or that this part is not accessed by two entities at the same time? Or both? Would I do these mlock()-calls in my C-Dll? If Yes, what would happen, if I tried to access the numpy-array from python, during the time it is locked? > However, I'm not convinced > this is the real issue. It seems at least equally likely that your > hardware drivers were developed with particular pattern of timing when > accessing the buffers, but now you may be attempting to hold a buffer > longer (preventing the driver writing to it) than the developer ever > tested. It shouldn't blue-screen, but it does... > I think it quite likely that you have some buggy hardware drivers. What > hardware is it? It is a camera with its own PCI-frame-grabber card. I am using it in "continuous-acquisition-mode" so I suppose, the driver is ready for *long* uses... Anyway, if the bluescreens continue I will have to switch to the "single-frame-acquisition-mode" and prepare a single buffer for every frame to grab. Of course, there would be a different way: I could allocate the buffer in C, in the dll. After data retrievel using the hardware API I could then copy the data to some numpy-array, still using C. But this would make my C-coded dll longer, thus harder to maintain. Is my way a common way? I mean, letting python/numpy do the memory allocation by creating a numpy-array with zeros in and passing its memory location to the hardware-API? Thanks Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Tue Oct 24 02:21:21 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Tue, 24 Oct 2006 02:21:21 -0400 Subject: memory position of numpy arrays In-Reply-To: <1161668441.6607.6.camel@localhost> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> Message-ID: On 24/10/06, Lars Friedrich wrote: > I am not really sure about the term "locking". Does that mean that this > part is not paged, or that this part is not accessed by two entities at > the same time? Or both? There are two kinds of locking, and really, you probably want both. But mlock() just ensures that the virtual memory stays in actual RAM. > Is my way a common way? I mean, letting python/numpy do the memory > allocation by creating a numpy-array with zeros in and passing its > memory location to the hardware-API? It's not necessary to do it this way. I think a more usual approach would be to create the buffer however is convenient in your C code, then provide its address to numpy. You can then use the ndarray function from python to tell it how to interpret that buffer as an array. Since the C code is creating the buffer, you can make sure it is in a special locked area of memory, ensure that the garbage collector never comes calling for it, or whatever you like. If you're having problems with driver stability, though, you may be safest having your C code copy the buffer into a numpy array in one shot - then you have complete control over when and how the DMA memory is accessed. (In C, I'm afraid, but for this sort of thing C is well-suited.) A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Tue Oct 24 02:29:21 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 24 Oct 2006 15:29:21 +0900 Subject: memory position of numpy arrays In-Reply-To: <1161668441.6607.6.camel@localhost> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> Message-ID: <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> Lars Friedrich wrote: > Andrew, > > thanks for your fast answer. > > Am Montag, den 23.10.2006, 00:45 -0700 schrieb Andrew Straw: > >> It sounds like your hardware drivers may be buggy -- you should only get >> segfaults, not (the Windows equivalent of) kernel panics, when your >> userspace code accesses wrong memory. >> > > When I started to write this thing, I had a lot of bluescreens. I think > they occured when my Python program and the hardware driver were > accessing the same memory location at the same time. I think the > hardware uses some DMA-technique. > I don't know anything about your device, but a driver directly accessing a memory buffer from a userland program sounds like a bug to me. I am far from being knowledgeable about os programming, but kernel and user space are two different adress spaces, so I don't see how a user-land program could access directly memory from kernel-space (the buffer from your hardware). There has to be copy somewhere, or fancier methods of sharing data; otherwise, this is really unsafe (using data from userland in kernel land is sure to cause problems....). >> But if you have buggy hardware drivers, I suppose it's possible that >> locking the memory will help. This wouldn't be the domain of numpy, >> however. In Linux, this is acheived with a system call such as mlock() >> or mlockall(). You'll have to figure out the appropriate call in >> Windows. Thinking about it, if your hardware drivers require the memory >> to be locked, they should do it themselves. >> > > I am not really sure about the term "locking". Does that mean that this > part is not paged, or that this part is not accessed by two entities at > the same time? Or both? > > Would I do these mlock()-calls in my C-Dll? If Yes, what would happen, > if I tried to access the numpy-array from python, during the time it is > locked? > > In this context, this means preventing the memory manager from putting the corresponding pages to the hard-drive. This is actually part of POSIX (I cannot remember which one), not linux-specific. Windows has a similar API, but I remember having seen somewhere that it is not reliable (ie it can swap pages out); I cannot find the corresponding API right now. > > It is a camera with its own PCI-frame-grabber card. I am using it in > "continuous-acquisition-mode" so I suppose, the driver is ready for > *long* uses... Anyway, if the bluescreens continue I will have to switch > to the "single-frame-acquisition-mode" and prepare a single buffer for > every frame to grab. > > Of course, there would be a different way: I could allocate the buffer > in C, in the dll. After data retrievel using the hardware API I could > then copy the data to some numpy-array, still using C. But this would > make my C-coded dll longer, thus harder to maintain. > I don't understand this either: how allocating in C changes the problem ? David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From michael.sorich at gmail.com Tue Oct 24 02:50:19 2006 From: michael.sorich at gmail.com (Michael Sorich) Date: Tue, 24 Oct 2006 16:20:19 +0930 Subject: A reimplementation of MaskedArray In-Reply-To: <200610160232.20888.pgmdevlist@mailcan.com> References: <200610160232.20888.pgmdevlist@mailcan.com> Message-ID: <16761e100610232350u28329c5bp7944ac325030d0ba@mail.gmail.com> I am currently running numpy rc2 (I haven't tried your reimplementation yet as I am still using python 2.3). I am wondering whether the new maskedarray is able to handle construction of arrays from masked scalar values (not sure if this is the correct term). I ran across a situation recently when I was picking individual values from a masked array, collecting them in a list and then subsequently constructing an array with these values. This does not work if any of the values choosen are masked. See example below On a more general note I am interested to find out whether there are any other languages that handle masked/missing data well and if so how this is done. My only experience is with R, which I have found to be quite good (there is a special value NA this signifies a masked value - this can be mixed in with non-masked values when defining an array). from numpy import * a = ma.array([1,2,3], mask=[True, False, False]) print a[0], type(a[0]) print a[1], type(a[1]) print list(a) a = ma.array(list(a)) -- output -- -- 2 [array(data = 999999, mask = True, fill_value=999999) , 2, 3] C:\Python23\Lib\site-packages\numpy\core\ma.py:604: UserWarning: Cannot automatically convert masked array to numeric because data is masked in one or more locations. warnings.warn("Cannot automatically convert masked array to "\ Traceback (most recent call last): File "D:\eclipse\Table\scripts\testrecarray.py", line 23, in ? a = ma.array(list(a)) File "C:\Python23\Lib\site-packages\numpy\core\ma.py", line 562, in __init__ c = numeric.array(data, dtype=tc, copy=True, order=order) TypeError: an integer is required On 10/16/06, Pierre GM wrote: > Folks, > I just posted on the scipy/developers zone wiki > (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) a reimplementation > of the masked_array mopdule, motivated by some problems I ran into while > subclassing MaskedArray. > > The main differences with the initial numpy.core.ma package are that > MaskedArray is now a subclass of ndarray and that the _data section can now > be any subclass of ndarray (well, it should work in most cases, some tweaking > might required here and there). Apart from a couple of issues listed below, > the behavior of the new MaskedArray class reproduces the old one. It is quite > likely to be significantly slower, though: I was more interested in a clear > organization than in performance, so I tended to use wrappers liberally. I'm > sure we can improve that rather easily. > > The new module, along with a test suite and some utilities, are available > here: > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedarray.py > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/masked_testutils.py > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_maskedarray.py > > Please note that it's still a work in progress (even if it seems to work quite > OK when I use it). Suggestions, comments, improvements and general feedback > are more than welcome ! > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mark at hymers.org.uk Tue Oct 24 05:46:37 2006 From: mark at hymers.org.uk (Mark Hymers) Date: Tue, 24 Oct 2006 10:46:37 +0100 Subject: Unit test error with numpy rc3 In-Reply-To: <20061023105027.GA6741@titus.mhy.org.uk> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> Message-ID: <20061024094637.GA11601@titus.mhy.org.uk> On Mon, 23, Oct, 2006 at 11:50:27AM +0100, Mark Hymers spoke thus.. > Hi, > > I've just done a Mac OS X PPC build of the SVN trunk and am getting this > failure too. > FAIL: Ticket #112 I've just been looking into this a bit further (though I may be heading down the wrong road) and come across this which doesn't exactly look right. Again this is on a PPC Mac OS X 10.4 install: In [1]: import numpy In [2]: numpy.__version__ Out[2]: '1.0.dev3390' In [3]: print numpy.finfo(numpy.float32).min, numpy.finfo(numpy.float32).max, numpy.finfo(numpy.float32).eps -3.40282346639e+38 3.40282346639e+38 1.19209289551e-07 In [4]: print numpy.finfo(numpy.float64).min, numpy.finfo(numpy.float64).max, numpy.finfo(numpy.float64).eps -1.79769313486e+308 1.79769313486e+308 2.22044604925e-16 In [5]: print numpy.finfo(numpy.float128).min, numpy.finfo(numpy.float128).max, numpy.finfo(numpy.float128).eps Warning: overflow encountered in add Warning: invalid value encountered in subtract Warning: invalid value encountered in subtract Warning: overflow encountered in add Warning: invalid value encountered in subtract Warning: invalid value encountered in subtract 9223372034707292160.0 -9223372034707292160.0 1.38178697010200053743e-76 Anyone got any comments /thoughts on this? Should I file it as a bug? I just tested this on an x86 Linux box (running Debian though that should be irrelevant) and numpy.float128 doesn't exist on x86 Linux but float96 does gives: >>> print numpy.finfo(numpy.float96).min, numpy.finfo(numpy.float96).max, numpy.finfo(numpy.float96).eps -1.18973149535723176502e+4932 1.18973149535723176502e+4932 1.08420217248550443401e-19 which seems right. Any ideas? Cheers, Mark -- Mark Hymers "The relationship between journalists and politicians has often been likened to that between a dog and a lamp post, although I have never worked out who is supposed to be which." Nick Assinder, BBC Online Political Correspondent -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tobbe at tobbe.nu Tue Oct 24 07:00:53 2006 From: tobbe at tobbe.nu (Tobias Bengtsson) Date: Tue, 24 Oct 2006 13:00:53 +0200 Subject: LPC, durbin-levinson, covariance, autocorrelation and lattice method? Message-ID: <453DF265.8080509@tobbe.nu> Hi I wish to do some Linear Predictive Coding, using durbin-levinson, covariance, autocorrelation or lattice method algoritms. However I don't know anything on digital signal processing, neither am I a star at math. I've ported http://www.phon.ucl.ac.uk/courses/spsci/dsp/lpc.html to python, but I couldn't get it to work, that was ported from fortran. Does scipy/numpy contain the means to help me? please reply directly to me as I'm not on the list. kind regards, Tobias -------------- next part -------------- A non-text attachment was scrubbed... Name: tobbe.vcf Type: text/x-vcard Size: 260 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Oct 24 09:48:35 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 24 Oct 2006 07:48:35 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <20061024094637.GA11601@titus.mhy.org.uk> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <20061024094637.GA11601@titus.mhy.org.uk> Message-ID: On 10/24/06, Mark Hymers wrote: > > On Mon, 23, Oct, 2006 at 11:50:27AM +0100, Mark Hymers spoke thus.. > > Hi, > > > > I've just done a Mac OS X PPC build of the SVN trunk and am getting this > > failure too. > > > FAIL: Ticket #112 > > > I've just been looking into this a bit further (though I may be heading > down the wrong road) and come across this which doesn't exactly look > right. Again this is on a PPC Mac OS X 10.4 install: > > In [1]: import numpy > > In [2]: numpy.__version__ > Out[2]: '1.0.dev3390' > > In [3]: print numpy.finfo(numpy.float32).min, numpy.finfo(numpy.float32).max, > numpy.finfo(numpy.float32).eps > -3.40282346639e+38 3.40282346639e+38 1.19209289551e-07 > > In [4]: print numpy.finfo(numpy.float64).min, numpy.finfo(numpy.float64).max, > numpy.finfo(numpy.float64).eps > -1.79769313486e+308 1.79769313486e+308 2.22044604925e-16 > > In [5]: print numpy.finfo(numpy.float128).min, numpy.finfo(numpy.float128).max, > numpy.finfo(numpy.float128).eps > Warning: overflow encountered in add > Warning: invalid value encountered in subtract > Warning: invalid value encountered in subtract > Warning: overflow encountered in add > Warning: invalid value encountered in subtract > Warning: invalid value encountered in subtract > 9223372034707292160.0 -9223372034707292160.0 1.38178697010200053743e-76 These aren't right. Are you running on a 64 bit system? Currently float80, float96, and float128 are all float80 with different alignments, so show up as float96 on 32 bit machines and float128 on 64 bit machines. This is probably set by a combination of compiler defaults and machine architecture. It sounds like something is getting misidentified, but I don't know much about that. Anyone got any comments /thoughts on this? Should I file it as a bug? > I just tested this on an x86 Linux box (running Debian though that > should be irrelevant) and numpy.float128 doesn't exist on x86 Linux > but float96 does gives: My impression is that it is a bug. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Oct 24 09:57:34 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 24 Oct 2006 07:57:34 -0600 Subject: LPC, durbin-levinson, covariance, autocorrelation and lattice method? In-Reply-To: <453DF265.8080509@tobbe.nu> References: <453DF265.8080509@tobbe.nu> Message-ID: On 10/24/06, Tobias Bengtsson wrote: > > Hi > > I wish to do some Linear Predictive Coding, using durbin-levinson, > covariance, autocorrelation or lattice method algoritms. > However I don't know anything on digital signal processing, neither am I > a star at math. > > I've ported http://www.phon.ucl.ac.uk/courses/spsci/dsp/lpc.html to > python, but I couldn't get it to work, that was ported from fortran. Quick and dirty: make a matrix A whose column are the data (d) delayed by 1, 2, 3... units respectively, then use least squares to solve the equation Ax=d. Durbin-Levinson, nee Levinson, is just an efficient way to do this using knowledge of the special structure of the problem. Does scipy/numpy contain the means to help me? Don't know. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dguhbwbow at rgpt.com Tue Oct 24 10:03:43 2006 From: dguhbwbow at rgpt.com (Rootsraquo File) Date: Tue, 24 Oct 2006 16:03:43 +0200 Subject: control Message-ID: <000701c6f775$376de0e0$290e9754@WORKSTATION3> Namibia Nauru or Nepal or Antilles Caledonia Nicaragua Niger or Nigeria Niue Norfolk North Korea Norway Oman Pakistan Palau in Panama Papua or Paraguay Peru Pitcairn of. Uta Terranigma a Tetris Tokimeki Memorial lots classes studying choices say a Incredible agree is tunes selection. Netsasoft Arabic jun in Dangerous sep of apr Boby xeik Chinese Simplified sundoit Oohoo jul Alleykat John Lgoosens Stefan oct delosfarsi am aa Bullitt Galixte of de Ezcom sie German. Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium am Belize Benin. Now still might be searching special having or brilliant a idea what could perfect avatar? Js displays only error messages show css errors problem much appearing difficult looking. Dvdrw dl Dvdram bdr Bdre Dbre dl udf toma Undelete a! Path width of height altpowered border successor highly of popular opensource bulletin system or Rewritten ground flexible secure feature. El Salvador Equatorial Guinea Eritrea Estonia Ethiopia of Falkland Malvinas Faroe Fiji Finland France French Polynesia a Southern Gabon is. Convertors generally or written maintained Group guarantee status no bad which occur risk provided enquires directed relevant of author is. Links unknowns or Photopaul National Opinion am Business Technology am Jobs is Real Estate Cars Dating classified in adsitemap Privacy Conditions. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: SE.gif Type: image/gif Size: 13327 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From markusro at element.fkp.physik.tu-darmstadt.de Tue Oct 24 10:54:03 2006 From: markusro at element.fkp.physik.tu-darmstadt.de (markusro at element.fkp.physik.tu-darmstadt.de) Date: Tue, 24 Oct 2006 16:54:03 +0200 (CEST) Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 In-Reply-To: <59a9095b29d3fa277f2e853a35590c72@element.fkp.physik.tu-darmstadt.de> References: <59a9095b29d3fa277f2e853a35590c72@element.fkp.physik.tu-darmstadt.de> Message-ID: <60193.163.1.38.43.1161701643.squirrel@element.fkp.physik.tu-darmstadt.de> > > Am 20.10.2006 um 02:53 schrieb Jay Parlar: > >>> Hi! >>> I try to compile numpy rc3 on Panther and get following errors. >>> (I start build with "python2.3 setup.py build" to be sure to use the >>> python shipped with OS X. I din't manage to compile Python2.5 either >>> yet with similar errors) >>> Does anynbody has an Idea? >>> gcc-3.3 >>> XCode 1.5 >>> November gcc updater is installed >>> >> >> I couldn't get numpy building with Python 2.5 on 10.3.9 (although I >> had different compile errors). The solution that ended up working for >> me was Python 2.4. There's a bug in the released version of Python 2.5 >> that's preventing it from working with numpy, should be fixed in the >> next release. >> >> You can find a .dmg for Python 2.4 here: >> http://pythonmac.org/packages/py24-fat/index.html >> >> Jay P. >> > > > I have that installed already but i get some bus errors with that. > Furthermore it is built with gcc4 > and i need to compile an extra module(pytables) and I fear that will > not work, hence I try to compile myself. Python 2.5 dosent't compile > either (libSystemStubs is only on Tiger). I have now everything compileing nicely: Python 2.5 (removed the -llibSystemStubs from the Makefile) the BusError when using iPython was still present; the reason is an unpatched readline 5.1. This problem was solved by libreadline-5.2. numpy rc3 compiled, too but numpy.test() still fails with them same error (see previous post) regards Markus ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From schut at sarvision.nl Tue Oct 24 10:58:24 2006 From: schut at sarvision.nl (Vincent Schut) Date: Tue, 24 Oct 2006 16:58:24 +0200 Subject: indexing with arrays of indexes and += Message-ID: <453E2A10.3080803@sarvision.nl> It is clear to me that the numpy += operator in combination with the use of arrays of indexes, as is explained in the Tentative Numpy Tutorial (http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e), the limitation being that indexes that appear more than 1 time in the indexes-array will get incremented only once. Does anybody know a way to work around this? I am using this to fill up a custom nd-histogram, and obviously each bin should be able to get incremented more than once. Looping over the entire array and incrementing each bin succesively takes waaay to long (these are pretty large arrays, like 4000x2000 items, or even larger) Cheers, Vincent ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 24 12:31:30 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 24 Oct 2006 10:31:30 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <20061023105027.GA6741@titus.mhy.org.uk> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> Message-ID: <453E3FE2.8080905@ieee.org> Mark Hymers wrote: > On Thu, 19, Oct, 2006 at 08:29:26AM -0600, Travis Oliphant spoke thus.. > >> Actually, you shouldn't be getting an INF at all. This is what the >> test is designed to test for (so I guess it's working). The test was >> actually written wrong and was never failing because previously keyword >> arguments to ufuncs were ignored. >> >> Can you show us what 'a' is on your platform. >> > > Hi, > > I've just done a Mac OS X PPC build of the SVN trunk and am getting this > failure too. > I thought we had this fixed. The basic problem is that the longfloat type is not very cross-platform. The functionality depends on your C-compiler / platform when a long double is specified as the type. I suspect it doesn't work well with all compilers. In particular gcc 3.3 on apple I know has trouble with the long double type. Bascially, the "failure" is a failure of the platform. The best we can do in NumPy is not run the test or print something instead of raising an error. > nidesk046:~/scratch/upstream/scipy mark$ python > Python 2.4.1 (#2, Mar 31 2005, 00:05:10) > [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>>> import numpy as N >>>> N.__version__ >>>> > '1.0.dev3378' > >>>> N.array([1000],dtype=N.float).dtype >>>> > dtype('float64') > >>>> N.array([1000],dtype=N.longfloat).dtype >>>> > dtype('float128') > Please show us what N.array([1000],dtype=N.longfloat).itemsize is? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gnata at obs.univ-lyon1.fr Tue Oct 24 12:31:33 2006 From: gnata at obs.univ-lyon1.fr (Xavier Gnata) Date: Tue, 24 Oct 2006 18:31:33 +0200 Subject: A way to compute a 2d histogram Message-ID: <453E3FE5.1080201@obs.univ-lyon1.fr> Hi, I have a set of 3 1D large arrays. The first 2 one stand for the coordinates of particules and the last one for their masses. I would like to be able to plot this data ie to compute a 2D histogram summing the masses in each bin. I cannot find a way to do that without any loop on the indices resulting too a very slow function. I'm looking for an elegant way to do that with numpy (or scipy??) function. For instance, scipy.histogram2d cannot do the job because it only counts the number of samples in each bin. There is no way to deal with weights. Xavier. -- ############################################ Xavier Gnata CRAL - Observatoire de Lyon 9, avenue Charles Andr? 69561 Saint Genis Laval cedex Phone: +33 4 78 86 85 28 Fax: +33 4 78 86 83 86 E-mail: gnata at obs.univ-lyon1.fr ############################################ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Seguridad at santander.com.mx Tue Oct 24 12:46:29 2006 From: Seguridad at santander.com.mx (Santander Serfin) Date: Tue, 24 Oct 2006 16:46:29 +0000 Subject: Nueva Seguridad Santander Serfin Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Tue Oct 24 13:02:26 2006 From: david.huard at gmail.com (David Huard) Date: Tue, 24 Oct 2006 13:02:26 -0400 Subject: A way to compute a 2d histogram In-Reply-To: <453E3FE5.1080201@obs.univ-lyon1.fr> References: <453E3FE5.1080201@obs.univ-lyon1.fr> Message-ID: <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> Hi Xavier, You could tweak histogram2d to do what you want, or you could give me a couple of days and I'll do it and let you know. If you want to help, you could write a test using your particular application and data. David 2006/10/24, Xavier Gnata : > > Hi, > > I have a set of 3 1D large arrays. > The first 2 one stand for the coordinates of particules and the last one > for their masses. > I would like to be able to plot this data ie to compute a 2D histogram > summing the masses in each bin. > I cannot find a way to do that without any loop on the indices resulting > too a very slow function. > > I'm looking for an elegant way to do that with numpy (or scipy??) > function. > > For instance, scipy.histogram2d cannot do the job because it only counts > the number of samples in each bin. > There is no way to deal with weights. > > Xavier. > > > -- > ############################################ > Xavier Gnata > CRAL - Observatoire de Lyon > 9, avenue Charles Andr? > 69561 Saint Genis Laval cedex > Phone: +33 4 78 86 85 28 > Fax: +33 4 78 86 83 86 > E-mail: gnata at obs.univ-lyon1.fr > ############################################ > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From strawman at astraw.com Tue Oct 24 13:10:22 2006 From: strawman at astraw.com (Andrew Straw) Date: Tue, 24 Oct 2006 10:10:22 -0700 Subject: memory position of numpy arrays In-Reply-To: <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> Message-ID: <453E48FE.9050907@astraw.com> David Cournapeau wrote: > I don't know anything about your device, but a driver directly accessing > a memory buffer from a userland program sounds like a bug to me. David, DMA memory (yes, I know thats an example of RAS Syndrome, apologies) allows hardware to fill a chunk of RAM and then hand it over to a userspace program. In my experience, RAM used for this purpose must be pre-allocated, usually in a ring-buffer type arrangement. So this is normal operating procedure for something like a frame grabber and not a bug at all. However, the fact that Lars is able to elicit blue screens from a user-mode program indicates driver bugs to me. It's likely, however, that once he gets his program operating within the bounds of what the developers tested, it'll work fine. Lars, for now I suggest doing what AM Archibald suggests and doing a memcpy to copy your framebuffers immediately into non-DMA memory and then hand that DMA memory back to the hardware driver. (Typically these drivers have function calls that indicate whether you "own" that part of memory -- this is, confusingly also called "locking" in the thread sense, as opposed to "locking" in the memory page sense.) Finally, whether you allocate memory in C or in numpy makes little difference. But if you want to use numpy, empty() will be presumably faster than zeros(). And it will have the advantage of doing memory management via Python's standard ref-counting. Cheers! Andrew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mark at hymers.org.uk Tue Oct 24 13:39:34 2006 From: mark at hymers.org.uk (Mark Hymers) Date: Tue, 24 Oct 2006 18:39:34 +0100 Subject: Unit test error with numpy rc3 In-Reply-To: <453E3FE2.8080905@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <453E3FE2.8080905@ieee.org> Message-ID: <20061024173934.GC13725@titus.mhy.org.uk> On Tue, 24, Oct, 2006 at 10:31:30AM -0600, Travis Oliphant spoke thus.. > The basic problem is that the longfloat type is not very > cross-platform. The functionality depends on your C-compiler / > platform when a long double is specified as the type. I suspect it > doesn't work well with all compilers. In particular gcc 3.3 on apple I > know has trouble with the long double type. Ah. I've just realised that we're using a MacPython build compiled with gcc-3.3 but our default compiler is 4.0.1. This could be causing an issue. Let me try building the extension with gcc-3.3 and, if I can find one, upgrading python to a version compiled with 4.0 (I really don't want to have to do a framework python build from source though). > Please show us what N.array([1000],dtype=N.longfloat).itemsize is? Python 2.4.1 (#2, Mar 31 2005, 00:05:10) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.0.dev3390' >>> numpy.array([1000],dtype=numpy.longfloat).itemsize 16 Cheers, Mark -- Mark Hymers "I once absent-mindedly ordered Three Mile Island dressing in a restaurant and, with great presence of mind, they brought Thousand Island Dressing and a bottle of chili sauce." Terry Pratchett, alt.fan.pratchett -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From Seguridad at santander.com.mx Tue Oct 24 13:46:16 2006 From: Seguridad at santander.com.mx (Santander Serfin) Date: Tue, 24 Oct 2006 19:46:16 +0200 Subject: Nueva Seguridad Santander Serfin Message-ID: <7f31547eb656c1e4485acdf559b673f8@www.arcoshop.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From mark at hymers.org.uk Tue Oct 24 13:50:10 2006 From: mark at hymers.org.uk (Mark Hymers) Date: Tue, 24 Oct 2006 18:50:10 +0100 Subject: Unit test error with numpy rc3 In-Reply-To: <453E3FE2.8080905@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <453E3FE2.8080905@ieee.org> Message-ID: <20061024175010.GA13791@titus.mhy.org.uk> On Tue, 24, Oct, 2006 at 10:31:30AM -0600, Travis Oliphant spoke thus.. > The basic problem is that the longfloat type is not very > cross-platform. The functionality depends on your C-compiler / > platform when a long double is specified as the type. I suspect it > doesn't work well with all compilers. In particular gcc 3.3 on apple I > know has trouble with the long double type. Apologies, I'm a complete moron. If you compile python with gcc-3.3 and the module with 4.0, things break. If I recompile numpy with gcc-3.3, you just don't get a float80,96 or 128 type which as far as I'm concerned is fine; certainly much better than it breaking. This is all on Mac OS X 10.4, I haven't had time to look at what would happen with a python and module compiled with 4.0. Sigh, the sooner we move these machines to Debian the better :-) Cheers, Mark -- Mark Hymers "I told you I was ill" The epitaph of Spike Milligan (1918-2002) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Tue Oct 24 14:08:40 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Tue, 24 Oct 2006 14:08:40 -0400 Subject: A reimplementation of MaskedArray In-Reply-To: <16761e100610232350u28329c5bp7944ac325030d0ba@mail.gmail.com> References: <200610160232.20888.pgmdevlist@mailcan.com> <16761e100610232350u28329c5bp7944ac325030d0ba@mail.gmail.com> Message-ID: <200610241408.40415.pgmdevlist@gmail.com> On Tuesday 24 October 2006 02:50, Michael Sorich wrote: > I am currently running numpy rc2 (I haven't tried your > reimplementation yet as I am still using python 2.3). I am wondering > whether the new maskedarray is able to handle construction of arrays > from masked scalar values (not sure if this is the correct term). The answer is no, unfortunately: both the new and old implementations fail at the same point, raising a TypeError: lists are processed through numpy.core.numeric.array, which doesn't handle that. But there's a workaround: creating an empty masked array, and filling it by hand: b = list(a) A = MA.array(N.empty(len(b))) for (k,v) in enumerate(b): A[k] = v should work in most cases. I guess I could plug something along those lines in the new implementation... But there's a problem anyway: if you don't precise a type at the creation of the empty array, the type will be determined automatically, which may be a problem if you mix numbers and strings: the maskedarray is detected as a string in that case. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Tue Oct 24 14:16:19 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Tue, 24 Oct 2006 11:16:19 -0700 Subject: eigh: eigenvalue order Message-ID: Is there any guarantee on the order of eigenvalues (and eigenvectors) returned by numpy.linalg.eigh? If I want to make sure the eigenvalues are in ascending order of magnitude should I sort them myself? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nwagner at iam.uni-stuttgart.de Tue Oct 24 14:26:30 2006 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Tue, 24 Oct 2006 20:26:30 +0200 Subject: eigh: eigenvalue order In-Reply-To: References: Message-ID: On Tue, 24 Oct 2006 11:16:19 -0700 "Keith Goodman" wrote: > Is there any guarantee on the order of eigenvalues (and >eigenvectors) > returned by numpy.linalg.eigh? > > If I want to make sure the eigenvalues are in ascending >order of > magnitude should I sort them myself? > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web >services, security? > Get stuff done quickly with pre-integrated technology to >make your job easier > Download IBM WebSphere Application Server v.1.0.1 based >on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion AFAIK eigh is based on dsyevd.f http://www.netlib.org/lapack/double/dsyevd.f. (real symmetric) http://www.netlib.org/lapack/complex16/zheevd.f (complex Hermitian). If INFO = 0, the eigenvalues are given in ascending order. Please correct me if I get it wrong. Nils ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cookedm at physics.mcmaster.ca Tue Oct 24 14:54:08 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Tue, 24 Oct 2006 14:54:08 -0400 Subject: [Numpy-discussion] PyDX - announcement In-Reply-To: <20061022143821.3b085ca4.simon@arrowtheory.com> References: <20061022143821.3b085ca4.simon@arrowtheory.com> Message-ID: <0294D4F3-4E20-4BCB-A53E-5D2FDB9895C9@physics.mcmaster.ca> On Oct 22, 2006, at 17:38 , Simon Burton wrote: > > Apologies if this is too off-topic for these lists, but > I hope some people here find this interesting! It looks very nifty. Interval arithmetic for rigourous bounds is something that should be used more often, I think. You should add it to the topical software list at http:// www.scipy.org/Topical_Software. [and it's on topic, btw; announcements of Python numerical software is always welcome here]. > PyDX - first public announcement > ================================ > > Overview > -------- > > PyDX is a package for working with calculus (differential > geometry), > arbitrary precision arithmetic (using gmpy), and interval arithmetic. > > It provides: > * multivariate automatic differentiation (to arbitrary order) > * Tensor objects for computations in differential geometry > * Interval scalars (based on libMPFI) for calculating rigorous > bounds > on arithmetic operations (validated numerics). > * Arbitrary order validated ODE solver. > > PyDX uses lazy computation techniques to greatly enhance > performance of > the resulting functions. > > This code grew out of a research project at The Australian National > University, > Department of Physics, which involved computing bounds on geodesics in > relativistic space-time's. -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From david.huard at gmail.com Tue Oct 24 16:40:22 2006 From: david.huard at gmail.com (David Huard) Date: Tue, 24 Oct 2006 16:40:22 -0400 Subject: A way to compute a 2d histogram In-Reply-To: <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> Message-ID: <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> Xavier, Here is the patch against svn. Please report any bug. I haven't had the time to test it extensively, something that should be done before commiting the patch to the repo. I'd appreciate your feedback. David 2006/10/24, David Huard : > > Hi Xavier, > > You could tweak histogram2d to do what you want, or you could give me a > couple of days and I'll do it and let you know. If you want to help, you > could write a test using your particular application and data. > > David > > > > > > > > > > > > 2006/10/24, Xavier Gnata : > > > > Hi, > > > > I have a set of 3 1D large arrays. > > The first 2 one stand for the coordinates of particules and the last one > > for their masses. > > I would like to be able to plot this data ie to compute a 2D histogram > > summing the masses in each bin. > > I cannot find a way to do that without any loop on the indices resulting > > too a very slow function. > > > > I'm looking for an elegant way to do that with numpy (or scipy??) > > function. > > > > For instance, scipy.histogram2d cannot do the job because it only counts > > the number of samples in each bin. > > There is no way to deal with weights. > > > > Xavier. > > > > > > -- > > ############################################ > > Xavier Gnata > > CRAL - Observatoire de Lyon > > 9, avenue Charles Andr? > > 69561 Saint Genis Laval cedex > > Phone: +33 4 78 86 85 28 > > Fax: +33 4 78 86 83 86 > > E-mail: gnata at obs.univ-lyon1.fr > > ############################################ > > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, > > security? > > Get stuff done quickly with pre-integrated technology to make your job > > easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache > > Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: weighted_histograms.patch Type: text/x-patch Size: 12123 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Tue Oct 24 16:46:54 2006 From: david.huard at gmail.com (David Huard) Date: Tue, 24 Oct 2006 16:46:54 -0400 Subject: indexing with arrays of indexes and += In-Reply-To: <453E2A10.3080803@sarvision.nl> References: <453E2A10.3080803@sarvision.nl> Message-ID: <91cf711d0610241346h2e2bcf69v1c39ce35c2df3c5f@mail.gmail.com> 2006/10/24, Vincent Schut : > > It is clear to me that the numpy += operator in combination with the use > of arrays of indexes, as is explained in the Tentative Numpy Tutorial > ( > http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e > ), > the limitation being that indexes that appear more than 1 time in the > indexes-array will get incremented only once. > > Does anybody know a way to work around this? > I am using this to fill up a custom nd-histogram, and obviously each bin > should be able to get incremented more than once. Looping over the > entire array and incrementing each bin succesively takes waaay to long > (these are pretty large arrays, like 4000x2000 items, or even larger) > > I don't know the answer to the first question, but I'd like to ask if you tried histogramdd ? If its lacking some features, i'd be willing to implement them. David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From trey21 at 163.com Tue Oct 24 22:06:35 2006 From: trey21 at 163.com (=?GB2312?B?IjEw1MIyOC0yOS/Jz7qjIg==?=) Date: Wed, 25 Oct 2006 02:06:35 +0000 (UTC) Subject: =?GB2312?B?VElFLS3Uy9PDSUW8vMr1zOG438ir1LHJ+rL60KfCyg==?= Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david at ar.media.kyoto-u.ac.jp Tue Oct 24 22:28:38 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 25 Oct 2006 11:28:38 +0900 Subject: memory position of numpy arrays In-Reply-To: <453E48FE.9050907@astraw.com> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> Message-ID: <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> Andrew Straw wrote: > David Cournapeau wrote: > >> I don't know anything about your device, but a driver directly accessing >> a memory buffer from a userland program sounds like a bug to me. >> > David, DMA memory (yes, I know thats an example of RAS Syndrome, > apologies) allows hardware to fill a chunk of RAM and then hand it over > to a userspace program. In my experience, RAM used for this purpose must > be pre-allocated, usually in a ring-buffer type arrangement. So this is > normal operating procedure for something like a frame grabber and not a > bug at all. > What I understood from former emails was that the user is allocating a memory buffer, and that it gives this memory buffer to the hardware. In this sense, I don't see how it is possible to avoid kernel panic or equivalents. If on the contrary, the driver gives you the memory buffer, then, ok, by eg a mmap-like call, you can access directly the device memory, but within a range fixed by the driver, which is valid if the driver is not buggy. That's why I don't understand the paging problem and why allocating anything from C or python would change anything (I think windows can page out kernel, contrary to linux, but I doubt it can page out DMA areas), because the user does not allocate anything in this scenario. But again, this is just what I would think from "common sense", and I have never done any system programming, so I may just miss something, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mforbes at phys.washington.edu Wed Oct 25 01:25:02 2006 From: mforbes at phys.washington.edu (Michael McNeil Forbes) Date: Tue, 24 Oct 2006 22:25:02 -0700 Subject: array + where return type: scalar vs. array Message-ID: Could someone please explain the semantics of the following. >>> from numpy import * >>> a1 = array([1,2,3]) >>> a2 = array([[1,2,3]]) >>> a1[where(a1==3)],a2[where(a2==3)] (array([3]), 3) Why are 1-dimensional fundamentally different than N-dimensional arrays in this regard? When there is a single match, N-d arrays always return a scalar whereas 1-d arrays return an array. Is this a bug? Thanks, Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From DTRR at HOTMAIL.COM Wed Oct 25 02:09:37 2006 From: DTRR at HOTMAIL.COM (=?iso-2022-jp?B?RFRSUg==?=) Date: Tue, 24 Oct 2006 23:09:37 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCMCZFNyVVJSElcyUvJWkbKEI=?= =?iso-2022-jp?b?GyRCJVZIL0ItNS1HMCU5JVolNyVjJWsbKEI=?= Message-ID: ??????????? ????????????????????? http://jiruan.com/ccckkk/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 25 02:20:13 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 25 Oct 2006 00:20:13 -0600 Subject: array + where return type: scalar vs. array In-Reply-To: References: Message-ID: <453F021D.2050008@ieee.org> Michael McNeil Forbes wrote: > Could someone please explain the semantics of the following. > > >>>> from numpy import * >>>> a1 = array([1,2,3]) >>>> a2 = array([[1,2,3]]) >>>> a1[where(a1==3)],a2[where(a2==3)] >>>> > (array([3]), 3) > > Why are 1-dimensional fundamentally different than N-dimensional arrays > in this regard? When there is a single match, N-d arrays always return > a scalar whereas 1-d arrays return an array. > > Is this a bug? > Yes, it's a bug. The optimization for a[0,2] was being called in this case because the arrays with 1 element were being interpreted as integers. I beefed-up the conversion check and fixed this in SVN. Thanks for the check. Now both have the same shape. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From schut at sarvision.nl Wed Oct 25 03:18:54 2006 From: schut at sarvision.nl (Vincent Schut) Date: Wed, 25 Oct 2006 09:18:54 +0200 Subject: indexing with arrays of indexes and += In-Reply-To: <91cf711d0610241346h2e2bcf69v1c39ce35c2df3c5f@mail.gmail.com> References: <453E2A10.3080803@sarvision.nl> <91cf711d0610241346h2e2bcf69v1c39ce35c2df3c5f@mail.gmail.com> Message-ID: <453F0FDE.9050302@sarvision.nl> David Huard wrote: > 2006/10/24, Vincent Schut >: > > It is clear to me that the numpy += operator in combination with > the use > of arrays of indexes, as is explained in the Tentative Numpy Tutorial > ( > http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e), > the limitation being that indexes that appear more than 1 time in the > indexes-array will get incremented only once. > > Does anybody know a way to work around this? > > > I am using this to fill up a custom nd-histogram, and obviously > each bin > should be able to get incremented more than once. Looping over the > entire array and incrementing each bin succesively takes waaay to long > (these are pretty large arrays, like 4000x2000 items, or even larger) > > > I don't know the answer to the first question, but I'd like to ask if > you tried histogramdd ? If its lacking some features, i'd be willing > to implement them. > > David No, haven't tried that, but actually my nd-histogram is a bit special. Firstly, it is not a histogram in the usual sample counting sense, but I need it to give me the average of values that are inserted in a certain bin (so actually I'm summing and counting, and finally divide the sums by the counts). Then, the bin number a sample goes into is not determined by the position of the sample in the array, but by the value of cells with the same position in other arrays. Lastly, not only these arrays are large, but there are many, and I need the final average, so I need to be able to update my histogram many times in a row. I think these needs are a bit too peculiar to have them added to histogramdd. I have already got my own nd-averagingHistogram class which does exactly what I want, only point it that it is slow because of the loop. So that's why I asked here if anyone knows a workaround. Vincent. > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From eisele at dfki.de Wed Oct 25 04:53:49 2006 From: eisele at dfki.de (Andreas Eisele) Date: Wed, 25 Oct 2006 10:53:49 +0200 Subject: can this be made faster? Message-ID: <453F261D.7080002@dfki.de> Recently, there were several requests and discussions on this list about how to increment an array a in cells pointed to from a second integer array b (optionally by values from a third array c), such as: > Yes, that'd be > a[b] += c > > On 10/8/06, Daniel Mahler wrote: > > Is there a 'loop free' way to do this in Numeric > > > > for i in arange(l): > > a[b[i]]+=c[i] > > > > where l == len(b) == len(c) > > > > thanks > > Daniel > > or > It is clear to me that the numpy += operator in combination with the use > of arrays of indexes, as is explained in the Tentative Numpy Tutorial > > (http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e), > > the limitation being that indexes that appear more than 1 time in the > indexes-array will get incremented only once. > > Does anybody know a way to work around this? > > I am using this to fill up a custom nd-histogram, and obviously each bin > should be able to get incremented more than once. Looping over the > entire array and incrementing each bin succesively takes waaay to long > (these are pretty large arrays, like 4000x2000 items, or even larger) I just came across a function that seems to provide the solution to both requests, which is called bincount. The first usecase could be written as a += bincount(b,c) (assuming a has already the right dimension, otherwise a = bincount(b,c) would create an array with the minimal required size), the second case is even simpler: counts = bincount(index) On my machine, this does 20M counting operations per second, which is _much_ faster than anything that could be done in an explicit for loop. Hope this helps, Andreas ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 25 06:33:55 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 25 Oct 2006 06:33:55 -0400 Subject: vectorize pitfall Message-ID: Hi, Vectorize is a very handy function, but it has at least one pitfall: def f(x): if 1.3 References: <453F261D.7080002@dfki.de> Message-ID: <453F4573.8090701@sarvision.nl> Andreas Eisele wrote: > Recently, there were several requests and discussions on this list about > how to > increment an array a in cells pointed to from a second integer array b > (optionally by > values from a third array c), such as: > > >> Yes, that'd be >> a[b] += c >> >> On 10/8/06, Daniel Mahler wrote: >> > Is there a 'loop free' way to do this in Numeric >> > >> > for i in arange(l): >> > a[b[i]]+=c[i] >> > >> > where l == len(b) == len(c) >> > >> > thanks >> > Daniel >> >> >> > or > >> It is clear to me that the numpy += operator in combination with the use >> of arrays of indexes, as is explained in the Tentative Numpy Tutorial >> >> (http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e), >> >> the limitation being that indexes that appear more than 1 time in the >> indexes-array will get incremented only once. >> >> Does anybody know a way to work around this? >> >> I am using this to fill up a custom nd-histogram, and obviously each bin >> should be able to get incremented more than once. Looping over the >> entire array and incrementing each bin succesively takes waaay to long >> (these are pretty large arrays, like 4000x2000 items, or even larger) >> > I just came across a function that seems to provide the solution to both > requests, > which is called bincount. > > The first usecase could be written as > > a += bincount(b,c) > > (assuming a has already the right dimension, otherwise a = bincount(b,c) > would create an > array with the minimal required size), the second case is even simpler: > > counts = bincount(index) > > On my machine, this does 20M counting operations per second, which is _much_ > faster than anything that could be done in an explicit for loop. > > Hope this helps, > > Andreas > Andreas, thanks for this tip! And thanks to this, I stumbled across the related function 'digitize' which is also very useful for me. Now the only problem left is that bincount has no way to deal with nd-histograms (where you have multiple index arrays, that together point to a bin in a multi-dimensional histogram 'grid'). Anyone any ideas about that? Only thing I can think of is to create a unique number for each possible nd index combination and use that as an indermediate step in bincount... Would probably work ok and fast enough, but gets picky when you use lots of bins (so your unique numbers will need to be very large). In my case (something like 10x10x10 bins) it would however be OK I guess. Anyway thanks a lot for sharing this. VS > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 25 07:39:41 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 25 Oct 2006 05:39:41 -0600 Subject: vectorize pitfall In-Reply-To: References: Message-ID: <453F4CFD.2050404@ieee.org> A. M. Archibald wrote: > Hi, > > Vectorize is a very handy function, but it has at least one pitfall: > > def f(x): > if 1.3 return sqrt(x) > else: > return 0 > > Now vectorize(f)(2)=1.41421356237 but vectorize(f)(array([1,2]))=array([0,1]). > > The problem is that, when given an array as input, vectorize feeds in > the first element, looks at the return type, and returns an array of > that type - and I didn't put a "." after the zero. > > This should perhaps be in the docstring of vectorize, since I can't > see any way to work around it, but it can easily lead to > difficult-to-find bugs. It may seem like an artificial example, but it > came up with a function I was working on. But it's confusing > behaviour. > > vectorize appears to support an "otypes" argument, but it doesn't take > standard numpy type objects, and it doesn't do anything obvious. > It takes old "type-codes" as one big string so you say vectorize(f,otypes='d') This should be modernized to handle a list of dtype objects. I've fixed vectorize in SVN. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 25 07:44:37 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 25 Oct 2006 07:44:37 -0400 Subject: vectorize pitfall In-Reply-To: <453F4CFD.2050404@ieee.org> References: <453F4CFD.2050404@ieee.org> Message-ID: On 25/10/06, Travis Oliphant wrote: > It takes old "type-codes" as one big string so you say > > vectorize(f,otypes='d') > > This should be modernized to handle a list of dtype objects. > > I've fixed vectorize in SVN. Thanks! A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nicolas.champavert at obs.univ-lyon1.fr Wed Oct 25 07:56:04 2006 From: nicolas.champavert at obs.univ-lyon1.fr (Nicolas Champavert) Date: Wed, 25 Oct 2006 13:56:04 +0200 Subject: A way to compute a 2d histogram In-Reply-To: <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> Message-ID: <453F50D4.4080209@obs.univ-lyon1.fr> Hi, there is a problem when using this patch of numpy.histogram2d. I've tried: a=pylab.rand(10) numpy.histogram2d(a,a) and there is an exception (exceptions.AttributeError) in numpy.histogramdd function called by numpy.histogram2d. But numpy.histogram2d(a,a,bins=[10,10]) works. So I've tried to use directly numpy.histogramdd function: numpy.histogramdd([a,a]) and it works. I've looked at the histogram2d code: try: N = len(bins) except TypeError: N = 1 bins = [bins] I think the last line (bins = [bins]) have to be deleted because if you do so, numpy.histogram2d(a,a) works. Or maybe the good think to do is: try: N = len(bins) except TypeError: N = 1 bins = 2*[bins] Nicolas David Huard a ?crit : > Xavier, > Here is the patch against svn. Please report any bug. I haven't had > the time to test it extensively, something that should be done before > commiting the patch to the repo. I'd appreciate your feedback. > > David > > 2006/10/24, David Huard >: > > Hi Xavier, > > You could tweak histogram2d to do what you want, or you could give > me a couple of days and I'll do it and let you know. If you want > to help, you could write a test using your particular application > and data. > > David > > > > > > > > > > > > 2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr > >: > > Hi, > > I have a set of 3 1D large arrays. > The first 2 one stand for the coordinates of particules and > the last one > for their masses. > I would like to be able to plot this data ie to compute a 2D > histogram > summing the masses in each bin. > I cannot find a way to do that without any loop on the indices > resulting > too a very slow function. > > I'm looking for an elegant way to do that with numpy (or > scipy??) function. > > For instance, scipy.histogram2d cannot do the job because it > only counts > the number of samples in each bin. > There is no way to deal with weights. > > Xavier. > > > -- > ############################################ > Xavier Gnata > CRAL - Observatoire de Lyon > 9, avenue Charles Andr? > 69561 Saint Genis Laval cedex > Phone: +33 4 78 86 85 28 > Fax: +33 4 78 86 83 86 > E-mail: gnata at obs.univ-lyon1.fr > ############################################ > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web > services, security? > Get stuff done quickly with pre-integrated technology to make > your job easier > Download IBM WebSphere Application Server v.1.0.1 based on > Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nicolas.champavert at obs.univ-lyon1.fr Wed Oct 25 08:25:42 2006 From: nicolas.champavert at obs.univ-lyon1.fr (Nicolas Champavert) Date: Wed, 25 Oct 2006 14:25:42 +0200 Subject: A way to compute a 2d histogram In-Reply-To: <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> Message-ID: <453F57C6.2020101@obs.univ-lyon1.fr> Hi, it would be great if you could add the weight option in the 1D histogram too. Nicolas David Huard a ?crit : > Xavier, > Here is the patch against svn. Please report any bug. I haven't had > the time to test it extensively, something that should be done before > commiting the patch to the repo. I'd appreciate your feedback. > > David > > 2006/10/24, David Huard >: > > Hi Xavier, > > You could tweak histogram2d to do what you want, or you could give > me a couple of days and I'll do it and let you know. If you want > to help, you could write a test using your particular application > and data. > > David > > > 2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr > >: > > Hi, > > I have a set of 3 1D large arrays. > The first 2 one stand for the coordinates of particules and > the last one > for their masses. > I would like to be able to plot this data ie to compute a 2D > histogram > summing the masses in each bin. > I cannot find a way to do that without any loop on the indices > resulting > too a very slow function. > > I'm looking for an elegant way to do that with numpy (or > scipy??) function. > > For instance, scipy.histogram2d cannot do the job because it > only counts > the number of samples in each bin. > There is no way to deal with weights. > > Xavier. > > > -- > ############################################ > Xavier Gnata > CRAL - Observatoire de Lyon > 9, avenue Charles Andr? > 69561 Saint Genis Laval cedex > Phone: +33 4 78 86 85 28 > Fax: +33 4 78 86 83 86 > E-mail: gnata at obs.univ-lyon1.fr > ############################################ > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web > services, security? > Get stuff done quickly with pre-integrated technology to make > your job easier > Download IBM WebSphere Application Server v.1.0.1 based on > Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From thlbnie at pcsdnfs1.eq.gs.com Wed Oct 25 09:06:32 2006 From: thlbnie at pcsdnfs1.eq.gs.com (locking drag) Date: Wed, 25 Oct 2006 15:06:32 +0200 Subject: classgrect Message-ID: <001301c6f836$6471f470$312fbf51@Andreas2> Fricke include tom Petty Jayhawks Gary Louris himself of Byrds mr ed. Fricke include tom Petty Jayhawks Gary Louris himself of Byrds mr ed. Messages this a dictfiltc elcido cc such directory Error make in utils or problem crypt routine libc operating linkers in tendto strict boneheaded than behere alleviate Makefiles. Stop a encoded remaining stem foo foot footle fubar is fub of grunt compresses xdawg magic. Headlining tour Sour Pearson news tip Anything happen attended inside info artists thats somebody personally releasing youd covered remain completely in anonymous unless am given. Days in Ready Raconteurs Infinite Mixtape Working Nuclear am City Troubled son Commercial Byen Invade Regina Spektor is Begins Spiral Stairs a Talks of Pavement Reissue Reunion Rumors Announces Puppets thu Dosh Taps. Ondemand streams a mtv well charged meeting or Peppers sexy Justin Timberlake a anyone floats am musicvideo boat helper Cristin Frodella collages famous Picasa Virginia Civil Archive Search pyramids Egypt hearing. Material reduce runtime encouraged explorethe afforded or commands sortand comm enable is equivalent amatter sorted of crackedand thenew print? Compile timechange srclibdes timeto a thereand modify Makefile match require instance of value Ccvariable in compiler gnu gccrather withmake am. Landmark jazz album by Carla Bley Emilys late father or Paul Haines a When approaches pharmacy things a take turn surreal or patrons am line game human or dominoes seriously piece aptly rides woozy. Child federal bonuses teachers tuition fees fouryear schools of Sicom or Rankings mlb playoff Techs dip cost breaks in Soldier Double or. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: presented.gif Type: image/gif Size: 6944 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Wed Oct 25 09:13:43 2006 From: david.huard at gmail.com (David Huard) Date: Wed, 25 Oct 2006 09:13:43 -0400 Subject: A way to compute a 2d histogram In-Reply-To: <453F57C6.2020101@obs.univ-lyon1.fr> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> <453F57C6.2020101@obs.univ-lyon1.fr> Message-ID: <91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com> Nicolas, thanks for the bug report, I fooled around with argument passing and should have checked every case. You'll find the histogram function that deals with weights on the numpy trac ticket 189, I'm waiting for some hints as to where the histogram function should reside (numpy.histogram, numpy.stats.histogram, ...) before submitting a patch . Salut, David 2006/10/25, Nicolas Champavert : > > Hi, > > it would be great if you could add the weight option in the 1D > histogram too. > > Nicolas > > David Huard a ?crit : > > Xavier, > > Here is the patch against svn. Please report any bug. I haven't had > > the time to test it extensively, something that should be done before > > commiting the patch to the repo. I'd appreciate your feedback. > > > > David > > > > 2006/10/24, David Huard > >: > > > > Hi Xavier, > > > > You could tweak histogram2d to do what you want, or you could give > > me a couple of days and I'll do it and let you know. If you want > > to help, you could write a test using your particular application > > and data. > > > > David > > > > > > 2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr > > >: > > > > Hi, > > > > I have a set of 3 1D large arrays. > > The first 2 one stand for the coordinates of particules and > > the last one > > for their masses. > > I would like to be able to plot this data ie to compute a 2D > > histogram > > summing the masses in each bin. > > I cannot find a way to do that without any loop on the indices > > resulting > > too a very slow function. > > > > I'm looking for an elegant way to do that with numpy (or > > scipy??) function. > > > > For instance, scipy.histogram2d cannot do the job because it > > only counts > > the number of samples in each bin. > > There is no way to deal with weights. > > > > Xavier. > > > > > > -- > > ############################################ > > Xavier Gnata > > CRAL - Observatoire de Lyon > > 9, avenue Charles Andr? > > 69561 Saint Genis Laval cedex > > Phone: +33 4 78 86 85 28 > > Fax: +33 4 78 86 83 86 > > E-mail: gnata at obs.univ-lyon1.fr > > ############################################ > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web > > services, security? > > Get stuff done quickly with pre-integrated technology to make > > your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on > > Apache Geronimo > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > < > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642> > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fullung at gmail.com Wed Oct 25 09:16:20 2006 From: fullung at gmail.com (Albert Strasheim) Date: Wed, 25 Oct 2006 15:16:20 +0200 Subject: numpy.distutils Intel Visual Fortran detection Message-ID: <20061025131619.GA22073@dogbert.sdsl.sun.ac.za> Hello all I'm trying to get SciPy compiling with MSVC and the Intel Visual Fortran compiler, version 9.1. I'm running into problems with the version detection code failing. The current version_cmd in intel.py is: 'version_cmd' : [fc_exe, "-FI -V -c %(fname)s.f -o %(fname)s.o" \ % {'fname':dummy_fortran_file()}], Even when running inside a Command Prompt with the necessary environment variables for the compiler set up, this command causes the following error: LINK : fatal error LNK1104: cannot open file 'ifconsol.lib' Changing version_cmd to to pass /help /quiet instead of trying to compile a dummy file does the trick. (/quiet so that the compiler doesn't warn about it being called ifort now, causing the regular expression that detects the version to fail.) Can we fix this? Is it important to try to compile the dummy file? Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david.huard at gmail.com Wed Oct 25 09:25:50 2006 From: david.huard at gmail.com (David Huard) Date: Wed, 25 Oct 2006 09:25:50 -0400 Subject: indexing with arrays of indexes and += In-Reply-To: <453F0FDE.9050302@sarvision.nl> References: <453E2A10.3080803@sarvision.nl> <91cf711d0610241346h2e2bcf69v1c39ce35c2df3c5f@mail.gmail.com> <453F0FDE.9050302@sarvision.nl> Message-ID: <91cf711d0610250625g79695bb2jc8e8c17ff0e6f791@mail.gmail.com> [snip] Vincent, I may be wrong, but it looks like you could use a weighted histogramdd (see other thread where I sent a patch to do so). You run it once to find the number of sample in each bin, then again with weights=value, normed=False to find the total weight in each bin, then you can divide each bin by the number of samples to find the average. N, d = histogramdd(cells) W, d = histogramdd(cells, weights = value) average = W/N David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Wed Oct 25 09:44:42 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Wed, 25 Oct 2006 10:44:42 -0300 Subject: LAPACK warning on FC2 Message-ID: FYI, I got the following warning building numpy-1.0 from sources on a FC2 box, using distribution provided blas and lapack rpm's. $ rpm -q blas blas-3.0-24.1 $ rpm -q lapack lapack-3.0-24.1 $ python setup.py build > build.log Running from numpy source directory. /u/dalcinl/Build/Python/numpy-1.0/numpy/distutils/system_info.py:934: UserWarning: ********************************************************************* Lapack library (from ATLAS) is probably incomplete: size of /usr/lib/liblapack.so is 3617k (expected >4000k) Follow the instructions in the KNOWN PROBLEMS section of the file numpy/INSTALL.txt. ********************************************************************* warnings.warn(message) My lapack shared library is only 3.6MB, But I believe it is complete. Am I right? $ ls -lh /usr/lib/liblapack.so.3.0.3 -rwxr-xr-x 1 root root 3.6M Mar 2 2004 /usr/lib/liblapack.so.3.0.3 -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cookedm at physics.mcmaster.ca Tue Oct 24 15:09:31 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Tue, 24 Oct 2006 15:09:31 -0400 Subject: Cross compilation In-Reply-To: <453C2374.5000208@bowdoin.edu> References: <453C2374.5000208@bowdoin.edu> Message-ID: <3826DEF0-A4ED-4288-952B-ACBA95784D7C@physics.mcmaster.ca> On Oct 22, 2006, at 22:05 , Jeremy R. Fishman wrote: > Hi, I was wondering if anyone can give me some advice on how to go > about cross-compiling NumPy. I have been searching around and > can't find any support in distutils for cross compilation. Is > there some way I can still compile Numerical Python using a mipsel- > linux compiler, on say a Cygwin host? > > The end result would be to use NumPy with an embedded Python > interpreter running on a mipsel host. In specific, The OPEN-R > system by Sony for use on Aibo robotic dogs. I need to figure out > how to get setup to use the mipsel compiler, not how to set up the > compiler. I suppose you could run the config step (using the host's compiler), then go and fix up the generated config files in the build/ directory. Hmm, looks to me if you run 'python setup.py build_src', then go poke around in build/src., you should find a numpy/ __config__.py, numpy/distutils/__config__.py (which look to be the same file, for some reason), and a numpy/core/config.h I *think* that's all you you'd have to fiddle. Then you should be able to build the modules by setting CC to your cross compiler, and doing 'python build_ext build_py' (Almost; looks like one config test is done in build_ext). Hope that helps; patches to make this cleaner are welcome. -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Wed Oct 25 13:21:27 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 25 Oct 2006 11:21:27 -0600 Subject: f2py g3 question Message-ID: <6ce0ac130610251021vffd96e5t2286a6a2249da97a@mail.gmail.com> HI, I have noticed that Pearu has been doing lots of work on f2py g3. I have heard that this will have support for derived types. What is the status of this work? Is is ready for public use? Are the derived types implemented? Thanks! Brian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tom.denniston at alum.dartmouth.org Wed Oct 25 14:53:37 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Wed, 25 Oct 2006 13:53:37 -0500 Subject: lexsort in numpy crashing on strings Message-ID: Hi, I've been using the lexsort and it is really nice. Much simpler and faster than the implementation I had before. The only problem is it seems to crash on strings. Do I just need to upgrade numpy or is it a current bug? In [2]: import numpy nu In [3]: numpy.array(['a', 'b', 'c']) Out[3]: array([a, b, c], dtype='|S1') In [4]: strArray = numpy.array(['a', 'b', 'c']) In [5]: intArray = numpy.array([1,2,3]) In [6]: numpy.__version__ Out[6]: '1.0b5' In [7]: numpy.lexsort((intArray,)) Out[7]: array([0, 1, 2]) In [8]: numpy.lexsort((intArray,intArray)) Out[8]: array([0, 1, 2]) In [9]: numpy.lexsort((strArray,intArray)) Segmentation fault (core dumped) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- _______________________________________________ SciPy-user mailing list SciPy-user at scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user From robert.kern at gmail.com Wed Oct 25 15:28:13 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 25 Oct 2006 14:28:13 -0500 Subject: lexsort in numpy crashing on strings In-Reply-To: References: Message-ID: Tom Denniston wrote: > Hi, I've been using the lexsort and it is really nice. Much simpler and > faster than the implementation I had before. The only problem is it > seems to crash on strings. > > Do I just need to upgrade numpy or is it a current bug? Upgrade numpy. I believe this bug has been fixed. http://projects.scipy.org/scipy/numpy/ticket/298 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Wed Oct 25 15:36:02 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Wed, 25 Oct 2006 22:36:02 +0300 (EEST) Subject: f2py g3 question In-Reply-To: <6ce0ac130610251021vffd96e5t2286a6a2249da97a@mail.gmail.com> References: <6ce0ac130610251021vffd96e5t2286a6a2249da97a@mail.gmail.com> Message-ID: On Wed, 25 Oct 2006, Brian Granger wrote: > HI, > > I have noticed that Pearu has been doing lots of work on f2py g3. I > have heard that this will have support for derived types. What is the > status of this work? Is is ready for public use? Are the derived > types implemented? Thanks! At the moment derived types support is implemented for types that have only scalar components. Work on the array component support is going on. If you would like to try out f2py g3, use --3g-numpy option: f2py --3g-numpy ... See numpy/f2py/lib/test_*.py files for examples that currently work with f2py g3. Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Wed Oct 25 16:46:13 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 25 Oct 2006 14:46:13 -0600 Subject: f2py g3 question In-Reply-To: References: <6ce0ac130610251021vffd96e5t2286a6a2249da97a@mail.gmail.com> Message-ID: <6ce0ac130610251346u7bf1a3deu6e8d5b88435b8d1c@mail.gmail.com> Pearu, Fantastic, I will try this out. Thanks for your work on this. Brian On 10/25/06, pearu at cens.ioc.ee wrote: > > On Wed, 25 Oct 2006, Brian Granger wrote: > > > HI, > > > > I have noticed that Pearu has been doing lots of work on f2py g3. I > > have heard that this will have support for derived types. What is the > > status of this work? Is is ready for public use? Are the derived > > types implemented? Thanks! > > At the moment derived types support is implemented for types that have > only scalar components. Work on the array component support is going > on. If you would like to try out f2py g3, use --3g-numpy option: > > f2py --3g-numpy ... > > See numpy/f2py/lib/test_*.py files for examples that currently work with > f2py g3. > > Pearu > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.h.jaffe at gmail.com Wed Oct 25 17:28:29 2006 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Wed, 25 Oct 2006 22:28:29 +0100 Subject: Still a ticket 112 "check_longfloat_repr" issue on OSX Message-ID: <453FD6FD.70709@gmail.com> OS X 10.4, PPC In [5]: import numpy as N In [6]: N.__version__ Out[6]: '1.0.1.dev3399' In [7]: print N.longfloat(0).itemsize 16 In [8]: a = N.exp(N.array([1000],dtype=N.longfloat)) In [9]: print str(a) [inf] In [10]: print str(a[0]) inf In [11]: print a.itemsize 16 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tom.denniston at alum.dartmouth.org Wed Oct 25 17:32:47 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Wed, 25 Oct 2006 16:32:47 -0500 Subject: lexsort in numpy crashing on strings In-Reply-To: References: Message-ID: Thanks Robert. Will do. On 10/25/06, Robert Kern wrote: > > Tom Denniston wrote: > > Hi, I've been using the lexsort and it is really nice. Much simpler and > > faster than the implementation I had before. The only problem is it > > seems to crash on strings. > > > > Do I just need to upgrade numpy or is it a current bug? > > Upgrade numpy. I believe this bug has been fixed. > > http://projects.scipy.org/scipy/numpy/ticket/298 > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though > it had > an underlying truth." > -- Umberto Eco > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jeremit0 at gmail.com Wed Oct 25 19:55:13 2006 From: jeremit0 at gmail.com (jeremito) Date: Wed, 25 Oct 2006 23:55:13 -0000 Subject: Matrices for testing Message-ID: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> If I understand correctly, the LAPACK package has some standard matrices included with it for testing. Does the linalg portion of numpy also have access to these matrices/arrays? Thanks, Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Wed Oct 25 20:06:31 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 25 Oct 2006 19:06:31 -0500 Subject: Matrices for testing In-Reply-To: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> Message-ID: jeremito wrote: > If I understand correctly, the LAPACK package has some standard > matrices included with it for testing. Does the linalg portion of > numpy also have access to these matrices/arrays? No. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Wed Oct 25 20:13:28 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 25 Oct 2006 19:13:28 -0500 Subject: Matrices for testing In-Reply-To: References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> Message-ID: Robert Kern wrote: > jeremito wrote: >> If I understand correctly, the LAPACK package has some standard >> matrices included with it for testing. Does the linalg portion of >> numpy also have access to these matrices/arrays? > > No. To clarify, these are routines provided with the LAPACK source distribution in the TESTING/MATGEN/ directory. However, they are not part of the LAPACK library itself, and thus we *cannot* provide wrappers for them in numpy. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From trey21 at 163.com Wed Oct 25 20:31:50 2006 From: trey21 at 163.com (=?GB2312?B?IjEx1MI0LTXI1S/Jz7qjIg==?=) Date: Thu, 26 Oct 2006 00:31:50 +0000 (UTC) Subject: =?GB2312?B?uanTpsnMudzA7dPrssm5urPJsb69tbXN0dDQ3rDg?= Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jeremit0 at gmail.com Wed Oct 25 22:43:16 2006 From: jeremit0 at gmail.com (jeremito) Date: Thu, 26 Oct 2006 02:43:16 -0000 Subject: Matrices for testing In-Reply-To: References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> Message-ID: <1161830596.758309.129670@k70g2000cwa.googlegroups.com> Robert Kern wrote: > Robert Kern wrote: > > jeremito wrote: > >> If I understand correctly, the LAPACK package has some standard > >> matrices included with it for testing. Does the linalg portion of > >> numpy also have access to these matrices/arrays? > > > > No. > > To clarify, these are routines provided with the LAPACK source distribution in > the TESTING/MATGEN/ directory. However, they are not part of the LAPACK library > itself, and thus we *cannot* provide wrappers for them in numpy. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > that is made terrible by our own mad attempt to interpret it as though it had > an underlying truth." > -- Umberto Eco > Hmmm. I guess I'll have to find another way to get some "standard/well-known" matrices to work with. Thanks for your help. Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Wed Oct 25 22:52:34 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 26 Oct 2006 11:52:34 +0900 Subject: Matrices for testing In-Reply-To: <1161830596.758309.129670@k70g2000cwa.googlegroups.com> References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> <1161830596.758309.129670@k70g2000cwa.googlegroups.com> Message-ID: On 10/26/06, jeremito wrote: > > > Hmmm. I guess I'll have to find another way to get some > "standard/well-known" matrices to work with. Thanks for your help. Well if all you want is some matrices, there's nothing stopping you from grabbing the matrices in the LAPACK distribution and using them yourself. Robert's just saying they won't be included in Numpy. There's also the matrix market, whcih has a large number of (sparse-only?) example matrices. http://math.nist.gov/MatrixMarket/index.html --bb -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ckkart at hoc.net Thu Oct 26 02:56:42 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Thu, 26 Oct 2006 06:56:42 +0000 (UTC) Subject: =?utf-8?q?distutils_data=5Ffiles_target?= Message-ID: Hi, I noticed that plain distutils and numpy.distutils differ in where they put the files specified with the keyword 'data_files' ( at least on windows XP). With distutils they go to 'PREFIX' and with numpy.distutils they go to 'PLATLIB', i.e. relative to the package installation directory. I guess this is for backward compatibility with python 2.3 which lacks a 'package_data' option. How can I install files to 'PREFIX' with numpy.disutils? Some data I really don't want to have lying around in the package dir, like e.g. documentation files, examples, etc. Regards, Christian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nwagner at iam.uni-stuttgart.de Thu Oct 26 03:19:30 2006 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Thu, 26 Oct 2006 09:19:30 +0200 Subject: Matrices for testing In-Reply-To: References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> <1161830596.758309.129670@k70g2000cwa.googlegroups.com> Message-ID: <45406182.8050005@iam.uni-stuttgart.de> Bill Baxter wrote: > > On 10/26/06, *jeremito* > wrote: > > > Hmmm. I guess I'll have to find another way to get some > "standard/well-known" matrices to work with. Thanks for your help. > > > Well if all you want is some matrices, there's nothing stopping you > from grabbing the matrices in the LAPACK distribution and using them > yourself. Robert's just saying they won't be included in Numpy. > > There's also the matrix market, whcih has a large number of > (sparse-only?) example matrices. > http://math.nist.gov/MatrixMarket/index.html > > --bb > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > You might be also interested in the Matrix Computation Toolbox which is a collection of MATLAB M-files containing functions for constructing test matrices ... http://www.ma.man.ac.uk/~higham/mctoolbox/ and http://www.mathworks.com/access/helpdesk/help/techdoc/ref/*gallery*.html BTW, you can easily import matrices given in the MatrixMarket format in scipy. See* io.mmread * mmread(source) Reads the contents of a Matrix Market file 'filename' into a matrix. Inputs: source - Matrix Market filename (extensions .mtx, .mtz.gz) or open file object. Outputs: a - sparse or full matrix Nils ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From smnkzqbda at pegasusscientific.com Thu Oct 26 03:34:06 2006 From: smnkzqbda at pegasusscientific.com (Legend) Date: Thu, 26 Oct 2006 15:34:06 +0800 Subject: smart Message-ID: <000d01c6f8d1$1e24a850$d27367da@pennyzm00dcwld> Praises am Doyle stemcell stand bay Press Media or Matters Itvcom National Postall. Opprobrium in pushing their agendas a emails adopt or saving baby an awful life or Madonna Cobain topearning dead celebsan or Jose Mercury agoap Twelve of years of after death or. Praises am Doyle stemcell stand bay Press Media or Matters Itvcom National Postall. Demon Sands Dungeons Dragons Vodcasts Host Matt Fraction Loco Megaset am Koei Ships of Dynasty Begins Videogame vet a Kathy Schoback cmp Playlogic Extend is. Players itunes music a storedvd jon cracks Apple a download protection Postthe am Tech or Report llc cio Citynews Spinall support is red hat agooracle offer Linux companys of chief. Almost facet Founded rai Applied Science of air Maritime Earth College bio Tourism art Insaurance Computing Healthcare Performing Education is Igncom Cheats Igncom in ps psp of Xbox Gamecube a. Ved am Gamers or Rotten Tomatoes continuing continued agree bound abide Agreement Advertise sql am Mysql monitored managed Pythian Remote dba de is Breaking! Alikethe profiles am Interview Gabriel is Rangethe is behind President Images saw Iiihow blood shed am. Registry item eligible in Amazon Prime items are Join today Already member of Sign or in used one List Learn is more larger in image job Starring Caine. Sellers or Offers includes Versions job Tape is Zulu Unbox a Unbox service Customers is bought also Wahlberg Carter Mini Complete Story Stoian Doukov Bullitt Steve Mcqueen Catch Thief Georgette Anys is. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Valley.gif Type: image/gif Size: 6799 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ETEEW at HOTMAIL.COM Thu Oct 26 04:47:11 2006 From: ETEEW at HOTMAIL.COM (=?iso-2022-jp?B?RVRFRVc=?=) Date: Thu, 26 Oct 2006 01:47:11 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCTDVOQT1QMnEkJEY7STgbKEI=?= Message-ID: ????????????????????? ??????????? http://o-oooo-o.com/ccc/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From lfriedri at imtek.de Thu Oct 26 04:51:57 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Thu, 26 Oct 2006 10:51:57 +0200 Subject: memory position of numpy arrays In-Reply-To: <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> Message-ID: <1161852718.6602.10.camel@localhost> Hello, thanks for your comments. Am Mittwoch, den 25.10.2006, 11:28 +0900 schrieb David Cournapeau: > Andrew Straw wrote: > > David Cournapeau wrote: > > > >> I don't know anything about your device, but a driver directly accessing > >> a memory buffer from a userland program sounds like a bug to me. > >> > > David, DMA memory (yes, I know thats an example of RAS Syndrome, > > apologies) allows hardware to fill a chunk of RAM and then hand it over > > to a userspace program. In my experience, RAM used for this purpose must > > be pre-allocated, usually in a ring-buffer type arrangement. So this is > > normal operating procedure for something like a frame grabber and not a > > bug at all. > > > What I understood from former emails was that the user is allocating a > memory buffer, and that it gives this memory buffer to the hardware. In > this sense, I don't see how it is possible to avoid kernel panic or > equivalents. If on the contrary, the driver gives you the memory buffer, > then, ok, by eg a mmap-like call, you can access directly the device > memory, but within a range fixed by the driver, which is valid if the > driver is not buggy. The API of the camera provides a function pl_exp_start_cont(hCam, buffer, size) The usage is to pass a Camera-Identifier "hCam", a pointer "buffer" and a size "size" in bytes. The camera will then start to acquire frames and put them to the buffer, which has to be size-bytes long. When the function reaches the end of the buffer, it will start at the beginning again (circular buffer mode). There is also a function available to ask for the last buffer position that is written to. The buffer has to be allocated by the caller of this API-function, so I guess it will be in *userland*. To have enough time to read a frame, the buffer has to be sufficiently big. I don't know what is sufficient, but I made it 50 frames big. With an acquisition time of lets say 10ms I have about 0.5seconds to get a frame, that was recently written to the buffer. I know, these things are not numpy and even not Python-questions, but C-questions, and I think I should go and read something about it. Does anyone have a recommendation where to find information on this memory-locking things? How is this kind of programming called? Driver-Programming? Hardware-Programming? System-Programming? Thanks Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From svetosch at gmx.net Thu Oct 26 05:09:51 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Thu, 26 Oct 2006 11:09:51 +0200 Subject: 1.0 Message-ID: <45407B5F.5010307@gmx.net> Wow! The list is so quiet despite the fact that the numpy 1.0 release is officially announced on the website, and the download is on sourceforge. Well ok, it was expected and the download counters are all at zero, but still. I want to thank everybody who made this possible very much! I'm not a numpy veteran (never used Numeric or Numarray before, or python for very long actually), but it feels great nevertheless. Since I'm not an insider, I can't give all the names that deserve to be mentioned, but it's quite obvious that one should at least mention Travis explicitly. Travis, it's great to know somebody like you is there who takes users' concerns seriously! (I should really leave it to others to praise his development skills, since due to my ignorance in that area I cannot even appreciate it fully.) Thanks, and relax a little after all the hard work! -Sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From schut at sarvision.nl Thu Oct 26 05:16:34 2006 From: schut at sarvision.nl (Vincent Schut) Date: Thu, 26 Oct 2006 11:16:34 +0200 Subject: sum over all but one axis Message-ID: <45407CF2.2050507@sarvision.nl> Hi list, probably I'm just missing something obvious, but I'm looking for a function/construcion to sum (or better, apply any numpy function) over all but one axis. Like when I have a nD array A, it will give me a 1D array with the sums of A[i, ...], but the axis that is retained in the result of course should be an argument to that function. I thought 'apply_along_axis' would do the trick, but it does not. If I apply that to a 3d array, it results in a 2d array instead of a 1d array. Of course I can easily loop over that axis, but if possible I'd like to prevent array loops... Vincent. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Thu Oct 26 05:34:15 2006 From: faltet at carabos.com (Francesc Altet) Date: Thu, 26 Oct 2006 11:34:15 +0200 Subject: 1.0 In-Reply-To: <45407B5F.5010307@gmx.net> References: <45407B5F.5010307@gmx.net> Message-ID: <1161855256.3981.2.camel@localhost.localdomain> El dj 26 de 10 del 2006 a les 11:09 +0200, en/na Sven Schreiber va escriure: > Wow! The list is so quiet despite the fact that the numpy 1.0 release is > officially announced on the website, and the download is on sourceforge. > Well ok, it was expected and the download counters are all at zero, but > still. > > I want to thank everybody who made this possible very much! I'm not a > numpy veteran (never used Numeric or Numarray before, or python for very > long actually), but it feels great nevertheless. > > Since I'm not an insider, I can't give all the names that deserve to be > mentioned, but it's quite obvious that one should at least mention > Travis explicitly. Travis, it's great to know somebody like you is there > who takes users' concerns seriously! (I should really leave it to others > to praise his development skills, since due to my ignorance in that area > I cannot even appreciate it fully.) > > Thanks, and relax a little after all the hard work! Yeah, I completely agree. Congratulations to Travis and all the NumPy team. Long life to NumPy! -- Francesc Altet | Be careful about using the following code -- Carabos Coop. V. | I've only proven that it works, www.carabos.com | I haven't tested it. -- Donald Knuth ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From schut at sarvision.nl Thu Oct 26 05:39:34 2006 From: schut at sarvision.nl (Vincent Schut) Date: Thu, 26 Oct 2006 11:39:34 +0200 Subject: sum over all but one axis In-Reply-To: <45407CF2.2050507@sarvision.nl> References: <45407CF2.2050507@sarvision.nl> Message-ID: <45408256.4070804@sarvision.nl> Vincent Schut wrote: > Hi list, > > probably I'm just missing something obvious, but I'm looking for a > function/construcion to sum (or better, apply any numpy function) over > all but one axis. Like when I have a nD array A, it will give me a 1D > array with the sums of A[i, ...], but the axis that is retained in the > result of course should be an argument to that function. > I thought 'apply_along_axis' would do the trick, but it does not. If I > apply that to a 3d array, it results in a 2d array instead of a 1d array. > > Of course I can easily loop over that axis, but if possible I'd like to > prevent array loops... > > Vincent. > Replying to self... the 'solution' I just found is this: 1: create a list 'axes' with the numbers of all axes except the one I want to retain 2: use numpy.apply_over_axes(function, array, axes).flatten() If anyone knows a better / more elegant solution or any related comments of course I'd still like to hear that. :-) VS. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.u.r.e.l.i.a.n at gmx.net Thu Oct 26 05:51:34 2006 From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert) Date: Thu, 26 Oct 2006 11:51:34 +0200 Subject: sum over all but one axis In-Reply-To: <45408256.4070804@sarvision.nl> References: <45407CF2.2050507@sarvision.nl> <45408256.4070804@sarvision.nl> Message-ID: <200610261151.34676.a.u.r.e.l.i.a.n@gmx.net> On Thursday 26 October 2006 11:39, Vincent Schut wrote: > > Of course I can easily loop over that axis, but if possible I'd like to > > prevent array loops... Depending on the problem's size, this could actually be the best solution -- if each particular sum is over enough elements, the overhead of looping is negligible. Also it would be the best solution in terms of readability. Without for-loops i would probably have come up with something like your solution, only longer and uglier. ;-) Johannes ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Thu Oct 26 06:08:05 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 26 Oct 2006 19:08:05 +0900 Subject: memory position of numpy arrays In-Reply-To: <1161852718.6602.10.camel@localhost> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> <1161852718.6602.10.camel@localhost> Message-ID: <45408905.2040502@ar.media.kyoto-u.ac.jp> Lars Friedrich wrote: > Hello, > > thanks for your comments. > > Am Mittwoch, den 25.10.2006, 11:28 +0900 schrieb David Cournapeau: > >> Andrew Straw wrote: >> >>> David Cournapeau wrote: >>> >>> >>>> I don't know anything about your device, but a driver directly accessing >>>> a memory buffer from a userland program sounds like a bug to me. >>>> >>>> >>> David, DMA memory (yes, I know thats an example of RAS Syndrome, >>> apologies) allows hardware to fill a chunk of RAM and then hand it over >>> to a userspace program. In my experience, RAM used for this purpose must >>> be pre-allocated, usually in a ring-buffer type arrangement. So this is >>> normal operating procedure for something like a frame grabber and not a >>> bug at all. >>> >>> >> What I understood from former emails was that the user is allocating a >> memory buffer, and that it gives this memory buffer to the hardware. In >> this sense, I don't see how it is possible to avoid kernel panic or >> equivalents. If on the contrary, the driver gives you the memory buffer, >> then, ok, by eg a mmap-like call, you can access directly the device >> memory, but within a range fixed by the driver, which is valid if the >> driver is not buggy. >> > > The API of the camera provides a function > pl_exp_start_cont(hCam, buffer, size) > > The usage is to pass a Camera-Identifier "hCam", a pointer "buffer" and > a size "size" in bytes. The camera will then start to acquire frames and > put them to the buffer, which has to be size-bytes long. When the > function reaches the end of the buffer, it will start at the beginning > again (circular buffer mode). There is also a function available to ask > for the last buffer position that is written to. The buffer has to be > allocated by the caller of this API-function, so I guess it will be in > *userland*. > In this case, I think the data are copied from the device to your buffer. > To have enough time to read a frame, the buffer has to be sufficiently > big. I don't know what is sufficient, but I made it 50 frames big. With > an acquisition time of lets say 10ms I have about 0.5seconds to get a > frame, that was recently written to the buffer. > You have no way to check if the ringbuffer is full or not ? In audio programming under linux, what happens normally is that you have a ring buffer locked in memory (that is cannot be paged out), and you need to check if your read/write pointers overlap to detect a buffer overrun. > I know, these things are not numpy and even not Python-questions, but > C-questions, and I think I should go and read something about it. Does > anyone have a recommendation where to find information on this > memory-locking things? How is this kind of programming called? > Driver-Programming? Hardware-Programming? System-Programming? > > Real time programming, maybe ? Without a knowledge of the API of your device, it is hard to know what the problem is for me. By the way, I found the information about locking pages into memory for windows: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/virtuallock.asp It may not work as expected, though, as it happens quite often with windows :) David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Thu Oct 26 07:57:37 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 26 Oct 2006 13:57:37 +0200 Subject: Changes to bools under Numexpr Message-ID: <4540A2B1.9080509@carabos.com> Hi all. The attached diff makes some changes to Numexpr support for booleans. The changes and their rationale are below. 1. New ``True`` and ``False`` boolean constants. This is so that 1 and 0 are always proper integer constants. It is also for completeness, but I don't envision any usage for them that couldn't be expressed without the constants. 2. The only comparisons supported with booleans are ``==`` and ``!=``, so that you can compare boolean variables. Just as NumPy supports complex order comparisons and Numexpr doesn't, so goes for bools. Being relatively new, I think there is no need to keep integer-boolean compatibility in Numexpr. What was the meaning of ``True > False`` or ``2 > True`` anyway? 3. This is also why casting booleans to normal numbers is not allowed, so ``prod()`` and ``sum()`` on booleans aren't either. What is the meaning of boolean addition anyway? To make it short, this patch is kind of a strengthening of boolean values. I expect some people to disagree with the changes, and that's why I would really like to hear your opinions on it. Regards, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: numexpr-bool2.diff Type: text/x-patch Size: 8581 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From steve at shrogers.com Thu Oct 26 08:05:33 2006 From: steve at shrogers.com (Steven H. Rogers) Date: Thu, 26 Oct 2006 06:05:33 -0600 Subject: 1.0 In-Reply-To: <1161855256.3981.2.camel@localhost.localdomain> References: <45407B5F.5010307@gmx.net> <1161855256.3981.2.camel@localhost.localdomain> Message-ID: <4540A48D.8090002@shrogers.com> Concur. This has been a tremendous effort. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From themoonseeker at gmail.com Thu Oct 26 08:50:05 2006 From: themoonseeker at gmail.com (TheMoon Seeker) Date: Thu, 26 Oct 2006 14:50:05 +0200 Subject: How to install numeric on python 1.5.2 Message-ID: <93ad97c80610260550p1b645cbft417ec9dea494abef@mail.gmail.com> Hello, I would like to install numeric for python 1.5.2 and I don't found the numeric package.. Where can I find it or how can I intall it? Greets Francis -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ted.horst at earthlink.net Thu Oct 26 08:57:11 2006 From: ted.horst at earthlink.net (Ted Horst) Date: Thu, 26 Oct 2006 07:57:11 -0500 Subject: Unit test error with numpy rc3 In-Reply-To: <20061023105027.GA6741@titus.mhy.org.uk> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> Message-ID: <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> On Mac OS X tiger (10.4) ppc, long double has increased precision but the same range as double (it really is 128 bits not 80, btw), so e**1000 is inf, so this is not really an error. I'm not sure what is the right thing to do in the test, check for overflow? Also, finfo has never worked properly for this type. Ted On Oct 23, 2006, at 05:50, Mark Hymers wrote: > On Thu, 19, Oct, 2006 at 08:29:26AM -0600, Travis Oliphant spoke > thus.. >> Actually, you shouldn't be getting an INF at all. This is what the >> test is designed to test for (so I guess it's working). The test was >> actually written wrong and was never failing because previously >> keyword >> arguments to ufuncs were ignored. >> >> Can you show us what 'a' is on your platform. > > Hi, > > I've just done a Mac OS X PPC build of the SVN trunk and am getting > this > failure too. > > nidesk046:~/scratch/upstream/scipy mark$ python > Python 2.4.1 (#2, Mar 31 2005, 00:05:10) > [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> import numpy as N >>>> N.__version__ > '1.0.dev3378' >>>> N.array([1000],dtype=N.float).dtype > dtype('float64') >>>> N.array([1000],dtype=N.longfloat).dtype > dtype('float128') >>>> N.test() > ...snip... > FAIL: Ticket #112 > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ > python2.4/site-packages/numpy/core/tests/test_regression.py", line > 220, in check_longfloat_repr > assert(str(a)[1:9] == str(a[0])[:8]) > AssertionError >>>> a = N.exp(N.array([1000],dtype=N.longfloat)) >>>> str(a) > '[inf]' > > > Any ideas about this? > > Mark > > -- > Mark Hymers > > "I once absent-mindedly ordered Three Mile Island dressing in a > restaurant > and, with great presence of mind, they brought Thousand Island > Dressing and > a bottle of chili sauce." > Terry Pratchett, alt.fan.pratchett > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642______________________________ > _________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.h.jaffe at gmail.com Thu Oct 26 10:46:50 2006 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Thu, 26 Oct 2006 15:46:50 +0100 Subject: Still a ticket 112 "check_longfloat_repr" issue on OSX In-Reply-To: <453FD6FD.70709@gmail.com> References: <453FD6FD.70709@gmail.com> Message-ID: <4540CA5A.5080607@gmail.com> Andrew Jaffe wrote: > OS X 10.4, PPC > > In [5]: import numpy as N > > In [6]: N.__version__ > Out[6]: '1.0.1.dev3399' > > In [7]: print N.longfloat(0).itemsize > 16 > > In [8]: a = N.exp(N.array([1000],dtype=N.longfloat)) > > In [9]: print str(a) > [inf] > > In [10]: print str(a[0]) > inf > > In [11]: print a.itemsize > 16 Are others still seeing this at this point, or just me? A ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From yves.frederix at gmail.com Thu Oct 26 10:50:06 2006 From: yves.frederix at gmail.com (Yves) Date: Thu, 26 Oct 2006 16:50:06 +0200 Subject: Behaviour of numpy.random.shuffle Message-ID: <20061026145006.GA15978@kotnet.org> Hi, It seems that random.shuffle does not 'shuffle' anymore when presented with an n-dimensional array. In [1]: import numpy as N In [2]: a = N.arange(5).reshape((5,1)) In [3]: a Out[3]: array([[0], [1], [2], [3], [4]]) In [4]: N.random.shuffle(a) In [5]: a Out[5]: array([[0], [1], [1], [3], [3]]) In [6]: N.__version__ Out[6]: '1.0.dev3390' After application some rows appear to be duplicated. Is this documented/intended behaviour? Cheers, YVES ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From chanley at stsci.edu Thu Oct 26 11:05:30 2006 From: chanley at stsci.edu (Christopher Hanley) Date: Thu, 26 Oct 2006 11:05:30 -0400 (EDT) Subject: Still a ticket 112 "check_longfloat_repr" issue on OSX Message-ID: <20061026110530.CLV71732@comet.stsci.edu> I just tried this on one of our G4's and see the same behavior. Chris ---- Original message ---- >Date: Thu, 26 Oct 2006 15:46:50 +0100 >From: Andrew Jaffe >Subject: Re: [Numpy-discussion] Still a ticket 112 "check_longfloat_repr" issue on OSX >To: Discussion of Numerical Python > >Andrew Jaffe wrote: >> OS X 10.4, PPC >> >> In [5]: import numpy as N >> >> In [6]: N.__version__ >> Out[6]: '1.0.1.dev3399' >> >> In [7]: print N.longfloat(0).itemsize >> 16 >> >> In [8]: a = N.exp(N.array([1000],dtype=N.longfloat)) >> >> In [9]: print str(a) >> [inf] >> >> In [10]: print str(a[0]) >> inf >> >> In [11]: print a.itemsize >> 16 > >Are others still seeing this at this point, or just me? > >A ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 26 11:20:34 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 26 Oct 2006 09:20:34 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> Message-ID: On 10/26/06, Ted Horst wrote: > > On Mac OS X tiger (10.4) ppc, long double has increased precision but > the same range as double (it really is 128 bits not 80, btw), Looks like you are right: http://www.freestandards.org/spec/ELF/ppc64/PPC-elf64abi-1.9.html#PREC Actually two doubles with some strange rules, the quad value is the sum. Hmmm, I can see why compiler support was problematic. The extended precision also has some oddities. This "Extended precision" differs from the IEEE 754 Standard in the following ways: - The software support is restricted to round-to-nearest mode. Programs that use extended precision must ensure that this rounding mode is in effect when extended-precision calculations are performed. - Does not fully support the IEEE special numbers NaN and INF. These values are encoded in the high-order double value only. The low-order value is not significant. - Does not support the IEEE status flags for overflow, underflow, and other conditions. These flag have no meaning in this format. so > e**1000 is inf, so this is not really an error. > > I'm not sure what is the right thing to do in the test, check for > overflow? Also, finfo has never worked properly for this type. See above. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Thu Oct 26 12:14:33 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 26 Oct 2006 10:14:33 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> Message-ID: <4540DEE9.6070304@ieee.org> Ted Horst wrote: > On Mac OS X tiger (10.4) ppc, long double has increased precision but > the same range as double (it really is 128 bits not 80, btw), so > e**1000 is inf, so this is not really an error. > > Thanks for the clarification. Long-double is not standard across platforms with different platforms choosing to do different things with the extra bytes. This helps explain one more platform. > I'm not sure what is the right thing to do in the test, check for > overflow? Also, finfo has never worked properly for this type. > In machar.py is the code that runs to detect all of the parameters. I think the code should be moved to C as detecting precision on a long-double takes too long. The overflow check is a good idea. The test should probably check for overflow and not try to run if it's detected. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 26 13:08:46 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 26 Oct 2006 11:08:46 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <4540DEE9.6070304@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> <4540DEE9.6070304@ieee.org> Message-ID: On 10/26/06, Travis Oliphant wrote: > > Ted Horst wrote: > > On Mac OS X tiger (10.4) ppc, long double has increased precision but > > the same range as double (it really is 128 bits not 80, btw), so > > e**1000 is inf, so this is not really an error. > > > > > > Thanks for the clarification. Long-double is not standard across > platforms with different platforms choosing to do different things with > the extra bytes. This helps explain one more platform. > > > I'm not sure what is the right thing to do in the test, check for > > overflow? Also, finfo has never worked properly for this type. > > > In machar.py is the code that runs to detect all of the parameters. I > think the code should be moved to C as detecting precision on a > long-double takes too long. > > The overflow check is a good idea. The test should probably check for > overflow and not try to run if it's detected. How to check overflow? According to the documentation the flag is not set by the hardware. And the precision is variable! Somewhere in the neighborhood of 31 decimal digits, more or less, depending. So I think it is hard to figure out what to do here. Chuck. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Thu Oct 26 13:24:25 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Thu, 26 Oct 2006 11:24:25 -0600 Subject: [ANN] NumPy 1.0 release Message-ID: <4540EF49.1040605@ieee.org> We are very pleased to announce the release of NumPy 1.0 available for download at http://www.numpy.org This release is the culmination of over 18 months of effort to allow unification of the Numeric and Numarray communities. NumPy provides the features of both packages as well as comparable speeds in the domains where both were considered fast --- often beating both packages on certain problems. If there is an area where we can speed up NumPy then we are interested in hearing about the solution. NumPy is essentially a re-write of Numeric to include the features of Numarray plus more. NumPy is a C-based extension module to Python that provides an N-dimensional array object (ndarray), a collection of fast math functions, basic linear algebra, array-producing random number generators, and basic Fourier transform capabilities. Also included with NumPy are: 1) A data-type object. The data-type of all NumPy arrays are defined by a data-type object that describes how the block of memory that makes up an element of the array is to be interpreted. Supported are all basic C-types, structures containing C-types, arrays of C-types, and structures containing structures of C-types. Data-types can also be in big or little-endian order. NumPy arrays can therefore be constructed from any regularly-sized chunk of data. A chunk of data can also be a pointer to a Python object and therefore Object arrays can be constructed (including record arrays with object members). 2) Array scalars: there is a Python scalar object (inheriting from the standard object where possible) defined for every basic data-type that an array can have. 2) A matrix object so that '*' is re-defined as matrix-multiplication and '**' as matrix-power. 3) A character array object that can replace Numarray's similarly-named object. It is basically an array of strings (or unicode) with methods matching the string and unicode methods. 4) A record array that builds on the advanced data-type support of the basic array object to allow field access using attribute look-up as well as to provide more ways to build-up a record-array. 5) A memory-map object that makes it easier to use memory-mapped areas as the memory for an array object. 6) A basic container class that uses the ndarray as a member. This often facilitates multiple-inheritance. 7) A large collection of basic functions on the array. 8) Compatibility layer for Numeric including code to help in the conversion to NumPy and full C-API support. 9) Compatibility layer for NumPy including code to help in the conversion to NumPy and full C-API support. NumPy can work with Numeric and Numarray installed and while the three array objects are different to Python, they can all share each other's data through the use of the array interface. As the developers for Numeric we can definitively say development of Numeric has ceased as has effective support. You may still find an answer to a question or two and Numeric will be available for download as long as Sourceforge is around so and code written to Numeric will still work, but there will not be "official" releases of Numeric for future versions of Python (including Python2.5). The development of NumPy has been supported by the people at STScI who created Numarray and support it. They have started to port their applications to NumPy and have indicated that support for Numarray will be phased out over the next year. You are strongly encouraged to move to NumPy. The whole point of NumPy is to unite the Numeric/Numarray development and user communities. We have done our part in releasing NumPy 1.0 and doing our best to make the transistion as easy as possible. Please support us by adopting NumPy. If you have trouble with that, please let us know why so that we can address the problems you identify. Even better, help us in fixing the problems. New users should download NumPy first unless they need an older package to work with third party code. Third-party package writers should migrate to use NumPy. Though it is not difficult, there are some things that have to be altered. Several people are available to help with that process, just ask (we will do it free for open source code and as work-for-hire for commercial code). This release would not have been possible without the work of many people. Thanks go to (if we have missed your contribution please let us know): * Travis Oliphant for the majority of the code adaptation (blame him for code problems :-) ) * Jim Hugunin, Paul Dubois, Konrad Hinsen, David Ascher, Jim Fulton and many others for Numeric on which the code is based. * Perry Greenfield, J Todd Miller, Rick White, Paul Barrett for Numarray which gave much inspiration and showed the way forward. * Paul Dubois for Masked Arrays * Pearu Peterson for f2py and numpy.distutils and help with code organization * Robert Kern for mtrand, bug fixes, help with distutils, code organization, and much more. * David Cooke for many code improvements including the auto-generated C-API and optimizations. * Alexander Belopolsky (Sasha) for Masked array bug-fixes and tests, rank-0 array improvements, scalar math help and other code additions * Francesc Altet for unicode and nested record tests and much help with rooting out nested record array bugs. * Tim Hochberg for getting the build working on MSVC, optimization improvements, and code review * Charles Harris for the sorting code originally written for Numarray and for improvements to polyfit, many bug fixes, and documentation strings. * Robert Cimrman for numpy.distutils help and the set-operations for arrays * David Huard for histogram code improvements including 2-d and d-d code * Eric Jones for sundry subroutines borrowed from scipy_base * Fernando Perez for code snippets, ideas, bugfixes, and testing. * Ed Schofield for matrix.py patches, bugfixes, testing, and docstrings. * John Hunter for code snippets (from matplotlib) * Chris Hanley for help with records.py, testing, and bug fixes. * Travis Vaught, Joe Cooper, Jeff Strunk for administration of numpy.org web site and SVN * Andrew Straw for bug-reports and help with www.scipy.org * Albert Strasheim for bug-reports, unit-testing and Valgrind runs * Stefan van der Walt for bug-reports, regression-testing, and bug-fixes. * Eric Firing for bugfixes. * Arnd Baecker for 64-bit testing * A.M. Archibald for code that decreases the number of times reshape makes a copy. More information is available at http://numpy.scipy.org and http://www.scipy.org. Bug-reports and feature requests should be submitted as tickets to the Trac pages at http://projects.scipy.org/scipy/numpy/ As an anti-SPAM measure, you must create an account in order to post tickets. Enjoy the new release, Sincerely, The NumPy Developers *Disclaimer*: The main author, Travis Oliphant, has written a 350+ page book entitled "Guide to NumPy" that documents the new system fairly thoroughly. The first two chapters of this book are available on-line for free, but the remainder must be purchased (until 2010 or a certain number of total sales has been reached). See http://www.trelgol.com for more details. There is plenty of free documentation available now for NumPy, however. Go to http://www.scipy.org for more details. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cicsxtxz at plco.shockware.com Thu Oct 26 13:41:40 2006 From: cicsxtxz at plco.shockware.com (pink border) Date: Fri, 27 Oct 2006 01:41:40 +0800 Subject: Gerald MatterSee Recovering Message-ID: <000b01c6f925$fe4d9000$08671f3c@cbc> Espaolmeet Study Guidesgift Cardshome Officedvd Setsmusic Setsprints Weeks Terrific Stephen King? Espaolmeet Study Guidesgift Cardshome Officedvd Setsmusic Setsprints Weeks Terrific Stephen King? Birthday either George Washington am or Abraham Lincoln Originally given patriotic card ribbon stars! Named Edward Burkhalter designer called decorative in inserts came sizes framing well fireplaces am walls floors am prominent Crescent Building. Government Fast is Free or Ratesstore Statuseasy Returnsall Help Desk Topics use Copyright Privacy a Policycopy llc. Consumer psychology ie must Deals magazine less savings least price Shoessave mens am. Us Millers a Antiques of Online Shopping of new Items Listed September Postcards of Great Medium for am Special or Days Review the! Known Presidents celebrated third is Monday or February or where stands today While never achieved there or number st Patricks am found or place years century with girl or holding. Samuel would last produce of todays is artware ceramics offered wide variety shapes glazes grouped together pieces or similar forms treatment lines collectors today Dickens Ware line introduced. Thousands brands question Service us update provide here comments site everyone note unable respond directly or submitted via am needed feedback appreciate form Wheres of Stufftrack recent or Returnssee heres Returns Helpforgot password am. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Spokane.gif Type: image/gif Size: 7647 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 26 13:57:18 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 26 Oct 2006 11:57:18 -0600 Subject: [ANN] NumPy 1.0 release In-Reply-To: <4540EF49.1040605@ieee.org> References: <4540EF49.1040605@ieee.org> Message-ID: On 10/26/06, Travis E. Oliphant wrote: > > > We are very pleased to announce the release of NumPy 1.0 available for > download at http://www.numpy.org Congratulations, Travis. Numpy was a needed unification and involved an enormous amount of work, most of it your own. Thanks for taking this on. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Thu Oct 26 13:26:14 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 26 Oct 2006 11:26:14 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> <4540DEE9.6070304@ieee.org> Message-ID: <4540EFB6.6080504@ieee.org> Charles R Harris wrote: > > > On 10/26/06, *Travis Oliphant* > wrote: > > Ted Horst wrote: > > On Mac OS X tiger (10.4) ppc, long double has increased > precision but > > the same range as double (it really is 128 bits not 80, btw), so > > e**1000 is inf, so this is not really an error. > > > > > > Thanks for the clarification. Long-double is not standard across > platforms with different platforms choosing to do different things > with > the extra bytes. This helps explain one more platform. > > > I'm not sure what is the right thing to do in the test, check for > > overflow? Also, finfo has never worked properly for this type. > > > In machar.py is the code that runs to detect all of the > parameters. I > think the code should be moved to C as detecting precision on a > long-double takes too long. > > The overflow check is a good idea. The test should probably check for > overflow and not try to run if it's detected. > > > How to check overflow? According to the documentation the flag is not > set by the hardware. And the precision is variable! Somewhere in the > neighborhood of 31 decimal digits, more or less, depending. So I think > it is hard to figure out what to do here. Let's drop the test. Long-double is available but is not consistent across platforms and NumPy has done nothing to try and make it so. Thus, let's just let the user beware. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cjw at sympatico.ca Thu Oct 26 14:35:06 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu, 26 Oct 2006 14:35:06 -0400 Subject: Changes to bools under Numexpr In-Reply-To: <4540A2B1.9080509@carabos.com> References: <4540A2B1.9080509@carabos.com> Message-ID: <4540FFDA.1020006@sympatico.ca> Ivan Vilata i Balaguer wrote: > Hi all. The attached diff makes some changes to Numexpr support for > booleans. The changes and their rationale are below. > > 1. New ``True`` and ``False`` boolean constants. This is so that 1 and > 0 are always proper integer constants. It is also for completeness, > but I don't envision any usage for them that couldn't be expressed > without the constants. > I'm puzzled. Python already has constants True and False of the bool type. bool is a subclass of the int type. Any instance of the bool type can be converted to the int type. >>> a=1==0 >>> type(a) >>> int(a) 0 >>> a False >>> Colin W. > 2. The only comparisons supported with booleans are ``==`` and ``!=``, > so that you can compare boolean variables. Just as NumPy supports > complex order comparisons and Numexpr doesn't, so goes for bools. > Being relatively new, I think there is no need to keep > integer-boolean compatibility in Numexpr. What was the meaning of > ``True > False`` or ``2 > True`` anyway? > > 3. This is also why casting booleans to normal numbers is not allowed, > so ``prod()`` and ``sum()`` on booleans aren't either. What is the > meaning of boolean addition anyway? > > To make it short, this patch is kind of a strengthening of boolean > values. I expect some people to disagree with the changes, and that's > why I would really like to hear your opinions on it. > > Regards, > > :: > > Ivan Vilata i Balaguer >qo< http://www.carabos.com/ > C??rabos Coop. V. V V Enjoy Data > "" > > ------------------------------------------------------------------------ > > Index: interp_body.c > =================================================================== > --- interp_body.c (revisi??n: 2299) > +++ interp_body.c (copia de trabajo) > @@ -130,21 +130,22 @@ > ci_dest = c1i); > > case OP_INVERT_BB: VEC_ARG1(b_dest = !b1); > + case OP_AND_BBB: VEC_ARG2(b_dest = (b1 && b2)); > + case OP_OR_BBB: VEC_ARG2(b_dest = (b1 || b2)); > > - case OP_AND_BBB: VEC_ARG2(b_dest = b1 && b2); > - case OP_OR_BBB: VEC_ARG2(b_dest = b1 || b2); > + case OP_EQ_BBB: VEC_ARG2(b_dest = (b1 == b2)); > + case OP_NE_BBB: VEC_ARG2(b_dest = (b1 != b2)); > > - case OP_GT_BII: VEC_ARG2(b_dest = (i1 > i2) ? 1 : 0); > - case OP_GE_BII: VEC_ARG2(b_dest = (i1 >= i2) ? 1 : 0); > - case OP_EQ_BII: VEC_ARG2(b_dest = (i1 == i2) ? 1 : 0); > - case OP_NE_BII: VEC_ARG2(b_dest = (i1 != i2) ? 1 : 0); > + case OP_GT_BII: VEC_ARG2(b_dest = (i1 > i2)); > + case OP_GE_BII: VEC_ARG2(b_dest = (i1 >= i2)); > + case OP_EQ_BII: VEC_ARG2(b_dest = (i1 == i2)); > + case OP_NE_BII: VEC_ARG2(b_dest = (i1 != i2)); > > - case OP_GT_BFF: VEC_ARG2(b_dest = (f1 > f2) ? 1 : 0); > - case OP_GE_BFF: VEC_ARG2(b_dest = (f1 >= f2) ? 1 : 0); > - case OP_EQ_BFF: VEC_ARG2(b_dest = (f1 == f2) ? 1 : 0); > - case OP_NE_BFF: VEC_ARG2(b_dest = (f1 != f2) ? 1 : 0); > + case OP_GT_BFF: VEC_ARG2(b_dest = (f1 > f2)); > + case OP_GE_BFF: VEC_ARG2(b_dest = (f1 >= f2)); > + case OP_EQ_BFF: VEC_ARG2(b_dest = (f1 == f2)); > + case OP_NE_BFF: VEC_ARG2(b_dest = (f1 != f2)); > > - case OP_CAST_IB: VEC_ARG1(i_dest = (long)b1); > case OP_ONES_LIKE_II: VEC_ARG1(i_dest = 1); > case OP_NEG_II: VEC_ARG1(i_dest = -i1); > > @@ -157,7 +158,6 @@ > > case OP_WHERE_IFII: VEC_ARG3(i_dest = f1 ? i2 : i3); > > - case OP_CAST_FB: VEC_ARG1(f_dest = (long)b1); > case OP_CAST_FI: VEC_ARG1(f_dest = (double)(i1)); > case OP_ONES_LIKE_FF: VEC_ARG1(f_dest = 1.0); > case OP_NEG_FF: VEC_ARG1(f_dest = -f1); > @@ -180,8 +180,6 @@ > case OP_FUNC_FF: VEC_ARG1(f_dest = functions_f[arg2](f1)); > case OP_FUNC_FFF: VEC_ARG2(f_dest = functions_ff[arg3](f1, f2)); > > - case OP_CAST_CB: VEC_ARG1(cr_dest = (double)b1; > - ci_dest = 0); > case OP_CAST_CI: VEC_ARG1(cr_dest = (double)(i1); > ci_dest = 0); > case OP_CAST_CF: VEC_ARG1(cr_dest = f1; > @@ -203,8 +201,8 @@ > ci_dest = (c1i*c2r - c1r*c2i) / fa; > cr_dest = fb); > > - case OP_EQ_BCC: VEC_ARG2(b_dest = (c1r == c2r && c1i == c2i) ? 1 : 0); > - case OP_NE_BCC: VEC_ARG2(b_dest = (c1r != c2r || c1i != c2i) ? 1 : 0); > + case OP_EQ_BCC: VEC_ARG2(b_dest = (c1r == c2r && c1i == c2i)); > + case OP_NE_BCC: VEC_ARG2(b_dest = (c1r != c2r || c1i != c2i)); > > case OP_WHERE_CFCC: VEC_ARG3(cr_dest = f1 ? c2r : c3r; > ci_dest = f1 ? c2i : c3i); > Index: compiler.py > =================================================================== > --- compiler.py (revisi??n: 2299) > +++ compiler.py (copia de trabajo) > @@ -217,6 +217,10 @@ > for name in c.co_names: > if name == "None": > names[name] = None > + elif name == "True": > + names[name] = True > + elif name == "False": > + names[name] = False > else: > t = types.get(name, float) > names[name] = expr.VariableNode(name, type_to_kind[t]) > Index: tests/test_numexpr.py > =================================================================== > --- tests/test_numexpr.py (revisi??n: 2299) > +++ tests/test_numexpr.py (copia de trabajo) > @@ -67,10 +67,6 @@ > x = x + 5j > assert_equal(evaluate("sum(x**2+2,axis=0)"), sum(x**2+2,axis=0)) > assert_equal(evaluate("prod(x**2+2,axis=0)"), prod(x**2+2,axis=0)) > - # Check boolean (should cast to integer) > - x = (arange(10) % 2).astype(bool) > - assert_equal(evaluate("prod(x,axis=0)"), prod(x,axis=0)) > - assert_equal(evaluate("sum(x,axis=0)"), sum(x,axis=0)) > > def check_axis(self): > y = arange(9.0).reshape(3,3) > Index: interpreter.c > =================================================================== > --- interpreter.c (revisi??n: 2299) > +++ interpreter.c (copia de trabajo) > @@ -25,6 +25,9 @@ > OP_AND_BBB, > OP_OR_BBB, > > + OP_EQ_BBB, > + OP_NE_BBB, > + > OP_GT_BII, > OP_GE_BII, > OP_EQ_BII, > @@ -35,7 +38,6 @@ > OP_EQ_BFF, > OP_NE_BFF, > > - OP_CAST_IB, > OP_COPY_II, > OP_ONES_LIKE_II, > OP_NEG_II, > @@ -47,7 +49,6 @@ > OP_MOD_III, > OP_WHERE_IFII, > > - OP_CAST_FB, > OP_CAST_FI, > OP_COPY_FF, > OP_ONES_LIKE_FF, > @@ -70,7 +71,6 @@ > OP_EQ_BCC, > OP_NE_BCC, > > - OP_CAST_CB, > OP_CAST_CI, > OP_CAST_CF, > OP_ONES_LIKE_CC, > @@ -115,6 +115,8 @@ > break; > case OP_AND_BBB: > case OP_OR_BBB: > + case OP_EQ_BBB: > + case OP_NE_BBB: > if (n == 0 || n == 1 || n == 2) return 'b'; > break; > case OP_GT_BII: > @@ -131,10 +133,6 @@ > if (n == 0) return 'b'; > if (n == 1 || n == 2) return 'f'; > break; > - case OP_CAST_IB: > - if (n == 0) return 'i'; > - if (n == 1) return 'b'; > - break; > case OP_COPY_II: > case OP_ONES_LIKE_II: > case OP_NEG_II: > @@ -152,10 +150,6 @@ > if (n == 0 || n == 2 || n == 3) return 'i'; > if (n == 1) return 'f'; > break; > - case OP_CAST_FB: > - if (n == 0) return 'f'; > - if (n == 1) return 'b'; > - break; > case OP_CAST_FI: > if (n == 0) return 'f'; > if (n == 1) return 'i'; > @@ -194,10 +188,6 @@ > if (n == 0) return 'b'; > if (n == 1 || n == 2) return 'c'; > break; > - case OP_CAST_CB: > - if (n == 0) return 'c'; > - if (n == 1) return 'b'; > - break; > case OP_CAST_CI: > if (n == 0) return 'c'; > if (n == 1) return 'i'; > @@ -1296,6 +1286,10 @@ > add_op("invert_bb", OP_INVERT_BB); > add_op("and_bbb", OP_AND_BBB); > add_op("or_bbb", OP_OR_BBB); > + > + add_op("eq_bbb", OP_EQ_BBB); > + add_op("ne_bbb", OP_NE_BBB); > + > add_op("gt_bii", OP_GT_BII); > add_op("ge_bii", OP_GE_BII); > add_op("eq_bii", OP_EQ_BII); > @@ -1306,7 +1300,6 @@ > add_op("eq_bff", OP_EQ_BFF); > add_op("ne_bff", OP_NE_BFF); > > - add_op("cast_ib", OP_CAST_IB); > add_op("ones_like_ii", OP_ONES_LIKE_II); > add_op("copy_ii", OP_COPY_II); > add_op("neg_ii", OP_NEG_II); > @@ -1318,7 +1311,6 @@ > add_op("mod_iii", OP_MOD_III); > add_op("where_ifii", OP_WHERE_IFII); > > - add_op("cast_fb", OP_CAST_FB); > add_op("cast_fi", OP_CAST_FI); > add_op("copy_ff", OP_COPY_FF); > add_op("ones_like_ff", OP_ONES_LIKE_FF); > @@ -1342,7 +1334,6 @@ > add_op("eq_bcc", OP_EQ_BCC); > add_op("ne_bcc", OP_NE_BCC); > > - add_op("cast_cb", OP_CAST_CB); > add_op("cast_ci", OP_CAST_CI); > add_op("cast_cf", OP_CAST_CF); > add_op("copy_cc", OP_COPY_CC); > Index: expressions.py > =================================================================== > --- expressions.py (revisi??n: 2299) > +++ expressions.py (copia de trabajo) > @@ -62,7 +62,13 @@ > return kind_rank[n] > > def bestConstantType(x): > - for converter in bool, int, float, complex: > + # Numeric conversion to boolean values is not tried because > + # ``bool(1) == True`` (same for 0 and False), so 0 and 1 would be > + # interpreted as booleans when ``False`` and ``True`` are already > + # supported. > + if isinstance(x, bool): > + return bool > + for converter in int, float, complex: > try: > y = converter(x) > except StandardError, err: > @@ -122,10 +128,7 @@ > return a > if isinstance(a, (bool, int, float, complex)): > a = ConstantNode(a) > - kind = a.astKind > - if kind == 'bool': > - kind = 'int' > - return FuncNode('sum', [a, axis], kind=kind) > + return FuncNode('sum', [a, axis], kind=a.astKind) > > def prod_func(a, axis=-1): > axis = encode_axis(axis) > @@ -133,10 +136,7 @@ > a = ConstantNode(a) > if isinstance(a, ConstantNode): > return a > - kind = a.astKind > - if kind == 'bool': > - kind = 'int' > - return FuncNode('prod', [a, axis], kind=kind) > + return FuncNode('prod', [a, axis], kind=a.astKind) > > @ophelper > def div_op(a, b): > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From erin.sheldon at gmail.com Thu Oct 26 14:47:36 2006 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Thu, 26 Oct 2006 14:47:36 -0400 Subject: [ANN] NumPy 1.0 release In-Reply-To: References: <4540EF49.1040605@ieee.org> Message-ID: <331116dc0610261147g2dfc7e35yfd6bf74abf66f67@mail.gmail.com> Hi all - I think I speak for many when I say that this is a huge step for those who have desired to switch to numerical python from other languages (IDL, MATLAB, etc) but have been waiting for that 1.0 release. Many thanks to everyone involved. Erin Sheldon On 10/26/06, Charles R Harris wrote: > > > On 10/26/06, Travis E. Oliphant wrote: > > > > We are very pleased to announce the release of NumPy 1.0 available for > > download at http://www.numpy.org > > Congratulations, Travis. Numpy was a needed unification and involved an > enormous amount of work, most of it your own. Thanks for taking this on. > > Chuck > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jontwang at gmail.com Thu Oct 26 18:26:47 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Thu, 26 Oct 2006 17:26:47 -0500 Subject: Defining custom types Message-ID: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> I'm trying to write a Numpy extension that will encapsulate mxDateTime as a native Numpy type. I've decided to use a type inherited from Numpy's scalar double. However, I'm running into all sorts of problems. I'm using numpy 1.0b5; I realize this is somewhat out of date. For all the examples below, assume that I've created a 1x1 array, mxArr, with my custom type. The interface used by Array_FromPyScalar does not conform with the documentation's claim that a negative return value indicates an error. The return code from setitem is not checked. Instead, the code depends on a Python error being set. I seem to be able to load values into the array, but I can't extract anything out of the array, even to print it. In gdb I've verified that loading DateTime.now() correctly puts a float representation of the date into my array. However, if I try to get the value out, I get an error: >>> mxArr[0] = DateTime.now() >>> mxArr[0] Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", line 391, in array_repr ', ', "array(") File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", line 204, in array2string separator, prefix) File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", line 160, in _array2string format = _floatFormat(data, precision, suppress_small) File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", line 281, in _floatFormat non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0))) TypeError: bad operand type for abs() I'm not sure why it's trying to call abs() on my object to print it. I have a separate PyNumberMethods attached to my object type, copied from the float scalar type, and nb_absolute is set to 0. When I break at the various functions I've registered, the last thing Numpy tries to do is cast my custom data type to an object type (which it does so successfully) via _broadcast_cast. Thanks, Jonathan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 26 19:19:11 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 26 Oct 2006 17:19:11 -0600 Subject: Defining custom types In-Reply-To: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> Message-ID: <4541426F.2070108@ee.byu.edu> Jonathan Wang wrote: > I'm trying to write a Numpy extension that will encapsulate mxDateTime > as a native Numpy type. I've decided to use a type inherited from > Numpy's scalar double. However, I'm running into all sorts of > problems. I'm using numpy 1.0b5; I realize this is somewhat out of date. > Cool. The ability to create your own data-types (and define ufuncs for them) is a feature that I'd like to see explored. But, it has not received a lot of attention and so you may find bugs along the way. We'll try to fix them quickly as they arise (and there will be bug fix releases for 1.0). But, what do you mean "inheriting" from NumPy's double for your scalar data-type. This has significant implications. To define a new data-type object (that doesn't build from the VOID data-type), you need to flesh out the PyArray_Descr * structure and this can only be done in C. Perhaps you are borrowing most entries in the structure builtin double type and then filling in a few differently like setitem and getitem? Is that accurate? > For all the examples below, assume that I've created a 1x1 array, > mxArr, with my custom type. > > The interface used by Array_FromPyScalar does not conform with the > documentation's claim that a negative return value indicates an error. You must be talking about a different function. Array_FromPyScalar is an internal function and not a C-API call. It also returns a PyObject * not an integer. So, which function are you actually referring to? > The return code from setitem is not checked. Instead, the code depends > on a Python error being set. This may be true, but how is it a problem? > > I seem to be able to load values into the array, but I can't extract > anything out of the array, even to print it. In gdb I've verified that > loading DateTime.now() correctly puts a float representation of the > date into my array. However, if I try to get the value out, I get an > error: > >>> mxArr[0] = DateTime.now() > >>> mxArr[0] > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", line > 391, in array_repr > ', ', "array(") > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > line 204, in array2string > separator, prefix) > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > line 160, in _array2string > format = _floatFormat(data, precision, suppress_small) > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > line 281, in _floatFormat > non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0))) > TypeError: bad operand type for abs() > > I'm not sure why it's trying to call abs() on my object to print it. Because that's the implication of inheriting from a double. It's just part of the code that tries to format your values into an array (notice the _floatFormat). I actually borrowed this code from numarray so I can't speak to exactly what it's doing without more study. > I have a separate PyNumberMethods attached to my object type, copied > from the float scalar type, and nb_absolute is set to 0. When I break > at the various functions I've registered, the last thing Numpy tries > to do is cast my custom data type to an object type (which it does so > successfully) via _broadcast_cast. Don't confuse the Python object associated when an element of the array is extracted and the data-type of the array. Also don't confuse the PyNumberMethods of the scalar object with the ufuncs. Defining PyNumberMethods won't usually give you the ability to calculate ufuncs. Perhaps you just want to construct an "object" array of mxDateTime's. What is the reason you want to define an mxDateTime data-type? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jontwang at gmail.com Thu Oct 26 19:37:42 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Thu, 26 Oct 2006 18:37:42 -0500 Subject: Defining custom types In-Reply-To: <4541426F.2070108@ee.byu.edu> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> Message-ID: <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> On 10/26/06, Travis Oliphant wrote: > > But, what do you mean "inheriting" from NumPy's double for your scalar > data-type. This has significant implications. To define a new > data-type object (that doesn't build from the VOID data-type), you need > to flesh out the PyArray_Descr * structure and this can only be done in > C. Perhaps you are borrowing most entries in the structure builtin > double type and then filling in a few differently like setitem and > getitem? Is that accurate? Sorry, I should have been clearer. When I talk about inheritance, I mean of the type underlying the array. For example, the built-in scalar double array has an underlying type of PyDoubleArrType_Type. My underlying type is a separate PyTypeObject. The interesting changes here are to tp_repr, tp_str, and tp_as_number. The rest of the fields are inherited from PyDoubleArrType_Type using the tp_base field. The array itself has another statically defined type object of type PyArray_Descr, which I'm creating with a PyObject_New call and filling in with many of the entries from the descriptor returned by PyArray_DescrFromType(NPY_DOUBLE), while overriding getitem and setitem to handle PyObject* of type mxDateTime as you guessed. > The interface used by Array_FromPyScalar does not conform with the > > documentation's claim that a negative return value indicates an error. > > You must be talking about a different function. Array_FromPyScalar is > an internal function and not a C-API call. It also returns a PyObject * > not an integer. So, which function are you actually referring to? > > > The return code from setitem is not checked. Instead, the code depends > > on a Python error being set. This may be true, but how is it a problem? > It's just confusing as the documentation indicates that the setitem function should return 0 for success and a negative number for failure. But within Array_FromPyScalar, we have: ret->descr->f->setitem(op, ret->data, ret); if (PyErr_Occurred()) { Py_DECREF(ret); return NULL; } else { return (PyObject *)ret; } So, someone reading the documentation could return -1 on failure without setting the Python error flag, and the function would happily continue on its way and fail to perform the proper casts. > > > I seem to be able to load values into the array, but I can't extract > > anything out of the array, even to print it. In gdb I've verified that > > loading DateTime.now() correctly puts a float representation of the > > date into my array. However, if I try to get the value out, I get an > > error: > > >>> mxArr[0] = DateTime.now() > > >>> mxArr[0] > > Traceback (most recent call last): > > File "", line 1, in ? > > File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", line > > 391, in array_repr > > ', ', "array(") > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 204, in array2string > > separator, prefix) > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 160, in _array2string > > format = _floatFormat(data, precision, suppress_small) > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 281, in _floatFormat > > non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0))) > > TypeError: bad operand type for abs() > > > > I'm not sure why it's trying to call abs() on my object to print it. > > Because that's the implication of inheriting from a double. It's just > part of the code that tries to format your values into an array (notice > the _floatFormat). I actually borrowed this code from numarray so I > can't speak to exactly what it's doing without more study. Hmm, so does Numpy ignore the tp_repr and tp_str fields in the PyTypeObject of the underlying type? I admittedly haven't had a chance to look at this code closely yet. > I have a separate PyNumberMethods attached to my object type, copied > > from the float scalar type, and nb_absolute is set to 0. When I break > > at the various functions I've registered, the last thing Numpy tries > > to do is cast my custom data type to an object type (which it does so > > successfully) via _broadcast_cast. > > Don't confuse the Python object associated when an element of the array > is extracted and the data-type of the array. Also don't confuse the > PyNumberMethods of the scalar object with the ufuncs. Defining > PyNumberMethods won't usually give you the ability to calculate ufuncs. Okay, is my understanding here correct? I am defining two type descriptors: PyArray_Descr mxNumpyType - describes the Numpy array type. PyTypeObject mxNumpyDataType - describes the data type of the contents of the array (i.e. mxNumpyType->typeobj points to this), inherits from PyDoubleArrType_Type and overrides some fields as mentioned above. And the getitem and setitem functions are designed to only give/take PyObject* of type mxDateTime. I guess it's not clear to me whether the abs() referred to by the error is an abs() ufunc or the nb_absolute pointer in the PyNumberMethods. Let me try overriding ufuncs and get back to you... Perhaps you just want to construct an "object" array of mxDateTime's. > What is the reason you want to define an mxDateTime data-type? Currently I am using an object array of mxDateTime's, but it's rather frustrating that I can't treat them as normal floats internally since that's really all they are. Jonathan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Thu Oct 26 19:49:25 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 26 Oct 2006 18:49:25 -0500 Subject: Defining custom types In-Reply-To: <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> Message-ID: Jonathan Wang wrote: > It's just confusing as the documentation indicates that the setitem > function should return 0 for success and a negative number for failure. > But within Array_FromPyScalar, we have: > > ret->descr->f->setitem(op, ret->data, ret); > > if (PyErr_Occurred()) { > Py_DECREF(ret); > return NULL; > } else { > return (PyObject *)ret; > } > > So, someone reading the documentation could return -1 on failure without > setting the Python error flag, and the function would happily continue > on its way and fail to perform the proper casts. That's a documentation vagueness, then. This is a convention established by the Python C API. If an error happens in a function that returns PyObject*, then it should return NULL to inform the caller that an error happened; other functions should return 0 for success and -1 for an error. However, the function must still set an exception object. The rest is just a convenient convention. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From djm at mindrot.org Thu Oct 26 19:50:07 2006 From: djm at mindrot.org (Damien Miller) Date: Fri, 27 Oct 2006 09:50:07 +1000 (EST) Subject: numpy-1.0 regress failure on OpenBSD Message-ID: Hi, I have just got around to updating OpenBSD's numpy port from 1.0b1 to 1.0 and am running into the following regress failure: > .........................................................................................................................................................................................................................................................Warning: overflow encountered in exp > F....................................................................................................................................................................................................................................................................... > ====================================================================== > FAIL: Ticket #112 > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/ports/math/py-numpy/w-py-numpy-1.0/fake-i386/usr/local/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", line 220, in check_longfloat_repr > assert(str(a)[1:9] == str(a[0])[:8]) > AssertionError > > ---------------------------------------------------------------------- > Ran 513 tests in 3.471s > > FAILED (failures=1) The variable 'a' seems to contain '[Inf]', so the failure appears related to the overflow warning. Any clues on how I can debug this further? thanks, Damien Miller ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 26 19:56:39 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 26 Oct 2006 17:56:39 -0600 Subject: numpy-1.0 regress failure on OpenBSD In-Reply-To: References: Message-ID: <45414B37.8050402@ee.byu.edu> Damien Miller wrote: >Hi, > >I have just got around to updating OpenBSD's numpy port from 1.0b1 to >1.0 and am running into the following regress failure: > > > >>.........................................................................................................................................................................................................................................................Warning: overflow encountered in exp >>F....................................................................................................................................................................................................................................................................... >>====================================================================== >>FAIL: Ticket #112 >>---------------------------------------------------------------------- >>Traceback (most recent call last): >> File "/usr/ports/math/py-numpy/w-py-numpy-1.0/fake-i386/usr/local/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", line 220, in check_longfloat_repr >> assert(str(a)[1:9] == str(a[0])[:8]) >>AssertionError >> >>---------------------------------------------------------------------- >>Ran 513 tests in 3.471s >> >>FAILED (failures=1) >> >> > >The variable 'a' seems to contain '[Inf]', so the failure appears related to >the overflow warning. > >Any clues on how I can debug this further? > > Unless you want to help with tracking how long double is interpreted on several platforms, then just ignore the test. (It actually wasn't being run in 1.0b1). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 26 20:15:08 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 26 Oct 2006 18:15:08 -0600 Subject: Defining custom types In-Reply-To: <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> Message-ID: <45414F8C.8090002@ee.byu.edu> > It's just confusing as the documentation indicates that the setitem > function should return 0 for success and a negative number for > failure. But within Array_FromPyScalar, we have: > > ret->descr->f->setitem(op, ret->data, ret); > > if (PyErr_Occurred()) { > Py_DECREF(ret); > return NULL; > } else { > return (PyObject *)ret; > } > I see the problem. We are assuming an error is set on failure, so both -1 should be returned and an error condition set for your own setitem function. This is typical Python behavior. I'll fix the documentation. > > > > > I seem to be able to load values into the array, but I can't extract > > anything out of the array, even to print it. In gdb I've > verified that > > loading DateTime.now() correctly puts a float representation of the > > date into my array. However, if I try to get the value out, I get an > > error: > > >>> mxArr[0] = DateTime.now() > > >>> mxArr[0] > > Traceback (most recent call last): > > File "", line 1, in ? > > File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", > line > > 391, in array_repr > > ', ', "array(") > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 204, in array2string > > separator, prefix) > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 160, in _array2string > > format = _floatFormat(data, precision, suppress_small) > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 281, in _floatFormat > > non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0))) > > TypeError: bad operand type for abs() > > > > I'm not sure why it's trying to call abs() on my object to print it. > > Because that's the implication of inheriting from a double. It's > just > part of the code that tries to format your values into an array > (notice > the _floatFormat). I actually borrowed this code from numarray so I > can't speak to exactly what it's doing without more study. > > > Hmm, so does Numpy ignore the tp_repr and tp_str fields in the > PyTypeObject of the underlying type? admittedly haven't had a chance > to look at this code closely yet. How arrays print is actually user-settable. The default printing function does indeed ignore tp_repr and tp_str of the underlying scalar objects in order to be able to set precisions. Now, we could probably fix the default printing function to actually use the tp_repr and/or tp_str fields of the corresponding scalar objects. This is worth filing a ticket about. In the mean time you can create a new array print function that checks for your data-type as the type of the array and then does something different otherwise it calls the old function. Then, register this function as the print function for arrays. > > > > I have a separate PyNumberMethods attached to my object type, copied > > from the float scalar type, and nb_absolute is set to 0. When I > break > > at the various functions I've registered, the last thing Numpy > tries > > to do is cast my custom data type to an object type (which it > does so > > successfully) via _broadcast_cast. > > Don't confuse the Python object associated when an element of the > array > is extracted and the data-type of the array. Also don't confuse the > PyNumberMethods of the scalar object with the ufuncs. Defining > PyNumberMethods won't usually give you the ability to calculate > ufuncs. > > > Okay, is my understanding here correct? I am defining two type > descriptors: > PyArray_Descr mxNumpyType - describes the Numpy array type. > PyTypeObject mxNumpyDataType - describes the data type of the contents > of the array (i.e. mxNumpyType->typeobj points to this), inherits from > PyDoubleArrType_Type and overrides some fields as mentioned above. > The nomenclature is that mxNumPyType is the data-type of the array and your PyTypeObject is the "type" of the elements of the array. So, you have the names a bit backward. So, to correspond with the way I use the words "type" and "data-type", I would name them: PyArray_Descr mxNumpyDataType PyTypeObject mxNumpyType > And the getitem and setitem functions are designed to only give/take > PyObject* of type mxDateTime. > These are in the 'f' member of the PyArray_Descr structure, so presumably you have also filled in your PyArray_Descr structure with items from PyArray_DOUBLE? > I guess it's not clear to me whether the abs() referred to by the > error is an abs() ufunc or the nb_absolute pointer in the > PyNumberMethods. Let me try overriding ufuncs and get back to you... > > Perhaps you just want to construct an "object" array of mxDateTime's. > What is the reason you want to define an mxDateTime data-type? > > > Currently I am using an object array of mxDateTime's, but it's rather > frustrating that I can't treat them as normal floats internally since > that's really all they are. Ah, I see. So, you would like to be able to say view the array of mxDateTimes as an array of "floats" (using the .view method). You are correct that this doesn't make sense when you are talking about objects, but might if mxDateTime objects are really just floats. I just wanted to make sure you were aware of the object array route. The new data-type route is less well traveled but I'm anxious to smooth the wrinkles out. Your experiences will help. Bascially we are moving from Numeric being a "builtin data-types" only to a NumPy that has "arbitrary" data-types with a few special-cased "builtins" We need more experience to clarify the issues. Your identification of problems in the default printing, for example, is one thing that will help. Keep us posted. I'd love to here how things went and what can be done to improve. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ted.horst at earthlink.net Thu Oct 26 20:55:36 2006 From: ted.horst at earthlink.net (Ted Horst) Date: Thu, 26 Oct 2006 19:55:36 -0500 Subject: Unit test error with numpy rc3 In-Reply-To: <4540EFB6.6080504@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> <4540DEE9.6070304@ieee.org> <4540EFB6.6080504@ieee.org> Message-ID: <5FCC8177-0926-43CC-BD2B-05EE57FDC6B6@earthlink.net> On Oct 26, 2006, at 12:26, Travis Oliphant wrote: > Charles R Harris wrote: >> >> >> On 10/26/06, *Travis Oliphant* > > wrote: >> >> Ted Horst wrote: >>> On Mac OS X tiger (10.4) ppc, long double has increased >> precision but >>> the same range as double (it really is 128 bits not 80, btw), so >>> e**1000 is inf, so this is not really an error. >>> >>> >> >> Thanks for the clarification. Long-double is not standard >> across >> platforms with different platforms choosing to do different >> things >> with >> the extra bytes. This helps explain one more platform. >> >>> I'm not sure what is the right thing to do in the test, check for >>> overflow? Also, finfo has never worked properly for this type. >>> >> In machar.py is the code that runs to detect all of the >> parameters. I >> think the code should be moved to C as detecting precision on a >> long-double takes too long. >> >> The overflow check is a good idea. The test should probably >> check for >> overflow and not try to run if it's detected. >> >> >> How to check overflow? According to the documentation the flag is not >> set by the hardware. And the precision is variable! Somewhere in the >> neighborhood of 31 decimal digits, more or less, depending. So I >> think >> it is hard to figure out what to do here. > > Let's drop the test. Long-double is available but is not consistent > across platforms and NumPy has done nothing to try and make it so. > Thus, let's just let the user beware. > > -Travis Yeah, that seem like the thing to do. Just for completeness: >>> N.seterr(all = 'raise') >>> fa = N.array([1e308], dtype=N.float) >>> lfa = N.array([1e308], dtype=N.longfloat) >>> fa + fa Traceback (most recent call last): File "", line 1, in ? FloatingPointError: overflow encountered in add >>> lfa + lfa Traceback (most recent call last): File "", line 1, in ? FloatingPointError: overflow encountered in add >>> N.exp(fa) Traceback (most recent call last): File "", line 1, in ? FloatingPointError: overflow encountered in exp >>> N.exp(lfa) array([inf], dtype=float128) Ted ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Thu Oct 26 21:03:31 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Thu, 26 Oct 2006 18:03:31 -0700 Subject: distutils question Message-ID: <45415AE3.8080908@jpl.nasa.gov> Hi I am trying to compile a 64 bit version of numpy with gcc. When building, numpy tries to figure out the lapack/atlas version. Up to this point, everything has been compiled with gcc -m64 and all is groovy. But, when an attempt is made to get the atlas version, the link fails because the command "gcc _configtest.o -L/u/fuego0b/myeates/lib -llapack -lcblas -latlas -o _configtest" is being run (Note the lack of -m64) This generates an error ld: fatal: file _configtest.o: wrong ELF class: ELFCLASS64 ld: fatal: File processing errors. No output written to _configtest collect2: ld returned 1 exit status ld: fatal: file _configtest.o: wrong ELF class: ELFCLASS64 ld: fatal: File processing errors. No output written to _configtest collect2: ld returned 1 exit status LinkError: LinkErro...us 1',),) Anybody know how I can force gcc -m64 when linking? I already have the environment variables CFLAGS=-m64 LDFLAGS=-64 Mathew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Thu Oct 26 21:42:52 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Thu, 26 Oct 2006 18:42:52 -0700 Subject: build errors Solaris x86 64 bit python2.5 Message-ID: <4541641C.6050004@jpl.nasa.gov> yes, I got around the problem from my previous posting "distutils question". I added ld_args[:0] = ['-m64'] to line 209 of python2.5/distutils/unixcompiler.py. Lovely, yes I know. I now get an error "numpy/core/src/multiarraymodule.c:7230: error: `NPY_ALLOW_THREADS' undeclared". This is after several billion warning messages of the form numpy/core/src/multiarraymodule.c:5010: warning: dereferencing type-punned pointer will break strict-aliasing rules Any help, suggestions gratefully accepted. Mathew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 26 23:34:01 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 26 Oct 2006 21:34:01 -0600 Subject: build errors Solaris x86 64 bit python2.5 In-Reply-To: <4541641C.6050004@jpl.nasa.gov> References: <4541641C.6050004@jpl.nasa.gov> Message-ID: On 10/26/06, Mathew Yeates wrote: > > yes, I got around the problem from my previous posting "distutils > question". I added ld_args[:0] = ['-m64'] to line 209 of > python2.5/distutils/unixcompiler.py. Lovely, yes I know. > > I now get an error "numpy/core/src/multiarraymodule.c:7230: error: > `NPY_ALLOW_THREADS' undeclared". This is after several billion warning > messages of the form > numpy/core/src/multiarraymodule.c:5010: warning: dereferencing > type-punned pointer will break strict-aliasing rules GCC? Needs the -no-strict-aliasing flag. Everybody hates the default except the compiler writers because you can't cast pointers between different sized types, something the linux kernel and numpy do a lot. Things can fail badly if you don't set the flag and ignore the warnings. I don't know about the other warning, maybe some syntax error causing the declaration to be missed. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From george.sakkis at gmail.com Thu Oct 26 23:58:28 2006 From: george.sakkis at gmail.com (George Sakkis) Date: Fri, 27 Oct 2006 03:58:28 +0000 (UTC) Subject: numpy - scipy version hell References: <200610191314.54482.jkgruet@sandia.gov> Message-ID: Robert Kern gmail.com> writes: > It looks like you linked against a FORTRAN LAPACK, but didn't manage to link the > FORTRAN runtime library libg2c. Can you give us the output of your build? > I just installed Numpy, ATLAS and LAPACK on Centos a few hours ago and I got the exact same error. You're right, libg2c is never linked. Here's a sample line from the linking: gcc -pthread -shared build/temp.linux-i686-2.4/numpy/linalg/lapack_litemodule.o -L/usr/local/lib/atlas/ -llapack -lptf77blas -lptcblas -latlas -o build/lib.linux-i686-2.4/numpy/linalg/lapack_lite.so I've been looking into numpy's distutils for the last hour or so but didn't track down the problem yet; something seems to be broken with Redhat's setup... George ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jk985 at tom.com Fri Oct 27 00:24:19 2006 From: jk985 at tom.com (=?GB2312?B?IjEx1MI0LTXI1S+xsb6pIg==?=) Date: Fri, 27 Oct 2006 04:24:19 +0000 (UTC) Subject: =?GB2312?B?RVhDRUy6zVBQVNTaxvPStbncwO3Kx7XEuN+8ttOm08M=?= Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Fri Oct 27 01:04:57 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 27 Oct 2006 00:04:57 -0500 Subject: numpy - scipy version hell In-Reply-To: References: <200610191314.54482.jkgruet@sandia.gov> Message-ID: George Sakkis wrote: > Robert Kern gmail.com> writes: > >> It looks like you linked against a FORTRAN LAPACK, but didn't manage to link > the >> FORTRAN runtime library libg2c. Can you give us the output of your build? >> > > I just installed Numpy, ATLAS and LAPACK on Centos a few hours ago and I got the > exact same error. You're right, libg2c is never linked. Here's a sample line > from the linking: > > gcc -pthread -shared build/temp.linux-i686-2.4/numpy/linalg/lapack_litemodule.o > -L/usr/local/lib/atlas/ -llapack -lptf77blas -lptcblas -latlas -o > build/lib.linux-i686-2.4/numpy/linalg/lapack_lite.so > > I've been looking into numpy's distutils for the last hour or so but didn't > track down the problem yet; something seems to be broken with Redhat's setup... Did you do anything to configure the libraries for ATLAS? Like editing site.cfg? If so, you will need to add -lg2c after -latlas. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From lfriedri at imtek.de Fri Oct 27 01:55:06 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Fri, 27 Oct 2006 07:55:06 +0200 Subject: memory position of numpy arrays In-Reply-To: <45408905.2040502@ar.media.kyoto-u.ac.jp> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> <1161852718.6602.10.camel@localhost> <45408905.2040502@ar.media.kyoto-u.ac.jp> Message-ID: <1161928506.6608.4.camel@localhost> Am Donnerstag, den 26.10.2006, 19:08 +0900 schrieb David Cournapeau: > By the way, I found the information about locking pages into memory for > windows: > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/virtuallock.asp > Thanks for the link, I now use a virtuallock in my Dll. I don't get the "paging-error"-bluescreens anymore, but now I get other ones ("DRIVER_IRQL_NOT_LESS_OR_EQUAL"), but I think this is another issue, and I am working on it... If anyone is using python / numpy / ctypes for hardware control (say, Cameras with grabber-cards or fire-wire / DCAM; National Instruments acquisition cards using NIDAQmx, ...) I am interested in discussion! Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gael.varoquaux at normalesup.org Fri Oct 27 02:46:13 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 27 Oct 2006 08:46:13 +0200 Subject: memory position of numpy arrays In-Reply-To: <1161928506.6608.4.camel@localhost> References: <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> <1161852718.6602.10.camel@localhost> <45408905.2040502@ar.media.kyoto-u.ac.jp> <1161928506.6608.4.camel@localhost> Message-ID: <20061027064613.GF17064@clipper.ens.fr> On Fri, Oct 27, 2006 at 07:55:06AM +0200, Lars Friedrich wrote: > If anyone is using python / numpy / ctypes for hardware control (say, > Cameras with grabber-cards or fire-wire / DCAM; National Instruments > acquisition cards using NIDAQmx, ...) I am interested in discussion! Worked great for me ! My approach was to write a small wrapper C (actually C++, with "extern C" linking) library that exposed only what I needed of the camera interface, in a "python-friendly" way, and to wrap it with ctypes. I controlled a "Pixis" princeton instruments camera this way. As I said, it worked surprisingly well. I can send the code as an example if you wish. Ga?l ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Fri Oct 27 03:21:18 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Fri, 27 Oct 2006 09:21:18 +0200 Subject: Changes to bools under Numexpr In-Reply-To: <4540FFDA.1020006@sympatico.ca> References: <4540A2B1.9080509@carabos.com> <4540FFDA.1020006@sympatico.ca> Message-ID: <4541B36E.7080403@carabos.com> En/na Colin J. Williams ha escrit: > Ivan Vilata i Balaguer wrote: >> Hi all. The attached diff makes some changes to Numexpr support for >> booleans. The changes and their rationale are below. >> >> 1. New ``True`` and ``False`` boolean constants. This is so that 1 and >> 0 are always proper integer constants. It is also for completeness, >> but I don't envision any usage for them that couldn't be expressed >> without the constants. >> > I'm puzzled. > Python already has constants True and False of the bool type. bool is a > subclass of the int type. > Any instance of the bool type can be converted to the int type. > >>> a=1==0 > >>> type(a) > > >>> int(a) > 0 > >>> a > False > >>> > > Colin W. > >> 2. The only comparisons supported with booleans are ``==`` and ``!=``, >> so that you can compare boolean variables. Just as NumPy supports >> complex order comparisons and Numexpr doesn't, so goes for bools. >> Being relatively new, I think there is no need to keep >> integer-boolean compatibility in Numexpr. What was the meaning of >> ``True > False`` or ``2 > True`` anyway? Well, the ``True`` and ``False`` constants where not previously supported in Numexpr because they had to be defined somewhere. Now they are. Regarding the Python int and bool types and their relationships, it is a very elegant solution introduced in Python 2.3 since previous versions didn't have a proper boolean type, so the 0 and 1 ints where used for that. What I'm proposing here is, since Numexpr has a recent story and most probably there isn't much code affected by the change, why not define the boolean type as a purely logical one and leave its numeric compatibility issues out? By the way, it simplifies Numexpr's virtual machine (less casting opcodes). I admit again this looks a little baffling, of course, but I don't think it would mean many noticeable changes to the user. Regards, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Fri Oct 27 03:36:59 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 27 Oct 2006 01:36:59 -0600 Subject: build errors Solaris x86 64 bit python2.5 In-Reply-To: References: <4541641C.6050004@jpl.nasa.gov> Message-ID: On 10/26/06, Charles R Harris wrote: > > > > On 10/26/06, Mathew Yeates wrote: > > > > yes, I got around the problem from my previous posting "distutils > > question". I added ld_args[:0] = ['-m64'] to line 209 of > > python2.5/distutils/unixcompiler.py. Lovely, yes I know. > > > > I now get an error "numpy/core/src/multiarraymodule.c:7230: error: > > `NPY_ALLOW_THREADS' undeclared". This is after several billion warning > > messages of the form > > numpy/core/src/multiarraymodule.c:5010: warning: dereferencing > > type-punned pointer will break strict-aliasing rules > > > GCC? Needs the -no-strict-aliasing flag. Everybody hates the default > except the compiler writers because you can't cast pointers between > different sized types, something the linux kernel and numpy do a lot. Things > can fail badly if you don't set the flag and ignore the warnings. > Make that -fno-strict-aliasing. The whole command line on x86 looks like: gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fPIC Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ivilata at carabos.com Fri Oct 27 03:43:32 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Fri, 27 Oct 2006 09:43:32 +0200 Subject: [ANN] NumPy 1.0 release In-Reply-To: <4540EF49.1040605@ieee.org> References: <4540EF49.1040605@ieee.org> Message-ID: <4541B8A4.3000201@carabos.com> En/na Travis E. Oliphant ha escrit:: > We are very pleased to announce the release of NumPy 1.0 available for > download at http://www.numpy.org > > This release is the culmination of over 18 months of effort to allow > unification of the Numeric and Numarray communities. [...] Wow, let me say this must be for so many people in this group like having a child at last! Congratulations and lots of thanks to all developers (and very especially to Travis) and contributors for the impressive designing, coding, testing and, last but not least, community driving of the last months. Let's hope this goes on like this for a looong time! Cheers, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david at ar.media.kyoto-u.ac.jp Fri Oct 27 05:53:28 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 27 Oct 2006 18:53:28 +0900 Subject: pyaudio, a module to make noise from numpy arrays Message-ID: <4541D718.6040308@ar.media.kyoto-u.ac.jp> Hi, I announce the first release of pyaudio, a module to make noise from numpy arrays (read, write and play audio files with numpy arrays). * WHAT FOR ?: The Goal is to give to a numpy/scipy environmenet some basic audio IO facilities (ala sound, wavread, wavwrite of matlab). With pyaudio, you should be able to read and write most common audio files from and to numpy arrays. The underlying IO operations are done using libsndfile from Erik Castro Lopo (http://www.mega-nerd.com/libsndfile/), so he is the one who did the hard work. As libsndfile has support for a vast number of audio files (including wav, aiff, but also htk, ircam, and flac, an open source lossless codec), pyaudio enables the import from and export to a fairly large number of file formats. There is also a really crude player, which uses tempfile to play audio, and which only works for linux-alsa anyway. I intend to add better support at least for linux, and for other platforms if this does not involve too much hassle. So basically, if you are lucky enough to use a recent linux system, pyaudio already gives you the equivalent of wavread, wavwrite and sound. * DOWNLOAD: http://www.ar.media.kyoto-u.ac.jp/members/david/pyaudio.tar.gz * INSTALLATION INSTRUCTIONS: Just untar the package and drop it into scipy/Lib/sandbox, and add the two following lines to scipy/Lib/sandbox/setup.py: # Package to make some noise using numpy config.add_subpackage('pyaudio') (if libsndfile.so is not in /usr/lib, a fortiori if you are a windows user, you should also change set the right location for libsndfile in pyaudio/pysndfile.py, at the line _snd.cdll.LoadLibrary('/usr/lib/libsndfile.so') ) * EXAMPLE USAGE == Reading example == # Reading from '/home/david/blop.flac' from scipy.sandbox.pyaudio import sndfile a = sndfile('/home/david/blop.flac') print a tmp = a.read_frames_float(1024) --> Prints: File : /home/david/blop.flac Sample rate : 44100 Channels : 2 Frames : 9979776 Format : 0x00170002 Sections : 1 Seekable : True Duration : 00:03:46.298 And put into tmp the 1024 first frames (a frame is the equivalent of a sample, but taking into account the number of channels: so 1024 frames gives you 2048 samples here). == Writing example == # Writing to a wavfile: from scipy.sandbox.pyaudio import sndfile import numpy as N noise = N.random.randn((44100)) a = sndfile('/home/david/blop.flac', sfm['SFM_WRITE'], sf_format['SF_FORMAT_WAV'] | sf_format['SF_FORMAT_PCM16'], 1, 44100) a.write_frames(noise, 44100) a.close() -> should gives you a lossless compressed white noise ! This is really a first release, not really tested, not much documentation, I can just say it works for me. I haven't found a good way to emulate enumerations, which libsndfile uses a lot, so I am using dictionaries generated from the library C header to get a relation enum label <=> value. If someone has a better idea, I am open to suggestions ! Cheers, David From lfriedri at imtek.de Fri Oct 27 07:04:00 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Fri, 27 Oct 2006 13:04:00 +0200 Subject: memory position of numpy arrays In-Reply-To: <20061027064613.GF17064@clipper.ens.fr> References: <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> <1161852718.6602.10.camel@localhost> <45408905.2040502@ar.media.kyoto-u.ac.jp> <1161928506.6608.4.camel@localhost> <20061027064613.GF17064@clipper.ens.fr> Message-ID: <1161947040.6608.9.camel@localhost> Hello Gael (sorry, I just don't get the dots...), Am Freitag, den 27.10.2006, 08:46 +0200 schrieb Gael Varoquaux: > Worked great for me ! My approach was to write a small wrapper C > (actually C++, with "extern C" linking) library that exposed only what I > needed of the camera interface, in a "python-friendly" way, and to wrap > it with ctypes. I controlled a "Pixis" princeton instruments camera this > way. As I said, it worked surprisingly well. I can send the code as an > example if you wish. Yes, that would be really nice! I think, I am doing the same thing. In the attachment, you find my dll-code, which I am still working on --so it is not ready to use!--, if you have any comments I will be happy. Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de -------------- next part -------------- A non-text attachment was scrubbed... Name: pvcamDLL.cpp Type: text/x-c++src Size: 4061 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cjw at sympatico.ca Fri Oct 27 09:33:35 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri, 27 Oct 2006 09:33:35 -0400 Subject: Changes to bools under Numexpr In-Reply-To: <4541B36E.7080403@carabos.com> References: <4540A2B1.9080509@carabos.com> <4540FFDA.1020006@sympatico.ca> <4541B36E.7080403@carabos.com> Message-ID: <45420AAF.1080605@sympatico.ca> Ivan Vilata i Balaguer wrote: > En/na Colin J. Williams ha escrit: > > >> Ivan Vilata i Balaguer wrote: >> >>> Hi all. The attached diff makes some changes to Numexpr support for >>> booleans. The changes and their rationale are below. >>> >>> 1. New ``True`` and ``False`` boolean constants. This is so that 1 and >>> 0 are always proper integer constants. It is also for completeness, >>> but I don't envision any usage for them that couldn't be expressed >>> without the constants. >>> >>> >> I'm puzzled. >> Python already has constants True and False of the bool type. bool is a >> subclass of the int type. >> Any instance of the bool type can be converted to the int type. >> >>> a=1==0 >> >>> type(a) >> >> >>> int(a) >> 0 >> >>> a >> False >> >>> >> >> Colin W. >> >> >>> 2. The only comparisons supported with booleans are ``==`` and ``!=``, >>> so that you can compare boolean variables. Just as NumPy supports >>> complex order comparisons and Numexpr doesn't, so goes for bools. >>> Being relatively new, I think there is no need to keep >>> integer-boolean compatibility in Numexpr. What was the meaning of >>> ``True > False`` or ``2 > True`` anyway? >>> > > Well, the ``True`` and ``False`` constants where not previously > supported in Numexpr because they had to be defined somewhere. Now they > are. > > Regarding the Python int and bool types and their relationships, it is a > very elegant solution introduced in Python 2.3 since previous versions > didn't have a proper boolean type, so the 0 and 1 ints where used for > that. What I'm proposing here is, since Numexpr has a recent story and > most probably there isn't much code affected by the change, why not > define the boolean type as a purely logical one and leave its numeric > compatibility issues out? By the way, it simplifies Numexpr's virtual > machine (less casting opcodes). > > I admit again this looks a little baffling, of course, but I don't think > it would mean many noticeable changes to the user. > > Regards, > > Ivan, I'm afraid I'm still baffled. Are you saying that your proposal is necessary to preserve compatibility with Python versions before 2.3? Otherwise, it appears to introduce clutter with no clear benefit. I don't find Numexpr in NumPy, are you referring to a scipy module? Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jeremit0 at gmail.com Fri Oct 27 09:43:54 2006 From: jeremit0 at gmail.com (jeremito) Date: Fri, 27 Oct 2006 13:43:54 -0000 Subject: Matrices for testing In-Reply-To: <45406182.8050005@iam.uni-stuttgart.de> References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> <1161830596.758309.129670@k70g2000cwa.googlegroups.com> <45406182.8050005@iam.uni-stuttgart.de> Message-ID: <1161956634.143048.4230@e3g2000cwe.googlegroups.com> > Well if all you want is some matrices, there's nothing stopping you > from grabbing the matrices in the LAPACK distribution and using them > yourself. Robert's just saying they won't be included in Numpy. > There's also the matrix market, whcih has a large number of > (sparse-only?) example matrices. > http://math.nist.gov/MatrixMarket/index.html > --bb _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > You might be also interested in the Matrix Computation Toolbox which is > a collection of MATLAB M-files containing functions for constructing > test matrices ... > http://www.ma.man.ac.uk/~higham/mctoolbox/ > > and > http://www.mathworks.com/access/helpdesk/help/techdoc/ref/*gallery*.html > > BTW, you can easily import matrices given in the MatrixMarket format in > scipy. See* io.mmread * > > mmread(source) > Reads the contents of a Matrix Market file 'filename' into a matrix. > > Inputs: > > source - Matrix Market filename (extensions .mtx, .mtz.gz) > or open file object. > > Outputs: > > a - sparse or full matrix > > Nils Thanks Bill and Nils. After my response, I had discovered the Matrix Market and realized it would be easy for me create some of the matrices myself. However having a way to read in the files already is really helpful. Thanks for pointing that out. Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From xtdcplp at pianometz.com Fri Oct 27 09:58:35 2006 From: xtdcplp at pianometz.com (Helsing was) Date: Fri, 27 Oct 2006 15:58:35 +0200 Subject: Routines moves Message-ID: <000d01c6f9cf$fea88000$80991d53@domjwd5vafgfgp> Walmart States Drug or Plan is Kfrom cbs News is worlds biggest retailer is said Thursday extending its generic another states added were am. Walmart States Drug or Plan is Kfrom cbs News is worlds biggest retailer is said Thursday extending its generic another states added were am. Help or Pivoting Small is pivotsleft or foot pivot pivot point Move heel upward Take pivots or backwards Repeat right foot degree Left half in way back Routines of. Explore jr Catalog am Gift Guides of Match Parts Phone Calling in Rent an This Weeks Specials see all the. Backwards ball that backward four There want work a Remember long defense short number could Increase range row miss in continue distances maximum range. Unveil colorful Tetris Russia Kfound via kottkeorg or Video find bbc or creator Alexey Pajitnov covers rivalries between Atari buying country doesnt believe property am much less property Ultimately great very important! Via kottkeorg Video find bbc creator Alexey Pajitnov covers rivalries or between Atari buying country doesnt of believe property much less property Ultimately great a very important gamemore. Default Gothic Dreams Green Fields am Cowboy Memories Orange Flowers Pink Rose Purple red! Generation schools will in teach boys girls rule under Title ix gives school districts a latitude create allgirl of allboy job Board is. Useful Search Kthe Offical takes or look using verbmore Seven reports generates timely bulletins presented characters current affairs say system produce tailored persons or. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: step.gif Type: image/gif Size: 7101 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jontwang at gmail.com Fri Oct 27 10:35:10 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Fri, 27 Oct 2006 09:35:10 -0500 Subject: Defining custom types In-Reply-To: <45414F8C.8090002@ee.byu.edu> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> Message-ID: <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> On 10/26/06, Travis Oliphant wrote: > > > Okay, is my understanding here correct? I am defining two type > > descriptors: > > PyArray_Descr mxNumpyType - describes the Numpy array type. > > PyTypeObject mxNumpyDataType - describes the data type of the contents > > of the array (i.e. mxNumpyType->typeobj points to this), inherits from > > PyDoubleArrType_Type and overrides some fields as mentioned above. > > > The nomenclature is that mxNumPyType is the data-type of the array and > your PyTypeObject is the "type" of the elements of the array. So, you > have the names a bit backward. > > So, to correspond with the way I use the words "type" and "data-type", I > would name them: > > PyArray_Descr mxNumpyDataType > PyTypeObject mxNumpyType Okay, I will use this convention going forwards. > And the getitem and setitem functions are designed to only give/take > > PyObject* of type mxDateTime. > > > These are in the 'f' member of the PyArray_Descr structure, so > presumably you have also filled in your PyArray_Descr structure with > items from PyArray_DOUBLE? That's correct. I have all members of the 'f' member identical to that from PyArray_DOUBLE, except: mxNumpyType->f->dotfunc = NULL; mxNumpyType->f->getitem = date_getitem; mxNumpyType->f->setitem = date_setitem; mxNumpyType->f->cast[PyArray_DOUBLE] = (PyArray_VectorUnaryFunc*) dateToDouble; mxNumpyType->f->cast[PyArray_OBJECT] = (PyArray_VectorUnaryFunc*) dateToObject; All other cast functions are NULL. If I redefine the string function, I encounter another, perhaps more serious problem leading to a segfault. I've defined my string function to be extremely simple: >>> def printer(arr): ... return str(arr[0]) Now, if I try to print an element of the array: >>> mxArr[0] I get to this stack trace: #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at scalartypes.inc.src :68 #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, base=0x814e7a8) at arrayobject.c:1419 #2 0x007d259f in array_subscript_nice (self=0x814e7a8, op=0x804eb8c) at arrayobject.c:1985 #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at Objects/abstract.c:94 (Note: for some reason gdb claims that arrayobject.c:1985 is array_subscript_nice, but looking at my source this line is actually in array_item_nice. *boggle*) But scalar_value returns NULL for all non-native types. So, destptr in PyArray_Scalar is set to NULL, and the call the copyswap segfaults. Perhaps scalar_value should be checking the scalarkind field of PyArray_Descr, or using the elsize and alignment fields to figure out the pointer to return if scalarkind isn't set? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cjw at sympatico.ca Fri Oct 27 11:55:29 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri, 27 Oct 2006 11:55:29 -0400 Subject: Python Enhancement Proposal (PEP) - Current status? Message-ID: <45422BF1.2090201@sympatico.ca> A July posting sets out the intent: http://scipy.org/BaseArray Version 3 of the draft: http://numpy.scipy.org/array_interface.shtml There is a description, from a C Structure perspective: http://svn.scipy.org/svn/PEP/PEP_basearray.txt What is the current status of the plan to develop a PEP? Is there a more recent version of the PEP, from a Python user's perspective? Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 12:02:18 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 10:02:18 -0600 Subject: Python Enhancement Proposal (PEP) - Current status? In-Reply-To: <45422BF1.2090201@sympatico.ca> References: <45422BF1.2090201@sympatico.ca> Message-ID: <45422D8A.8060805@ieee.org> Colin J. Williams wrote: > A July posting sets out the intent: > http://scipy.org/BaseArray > Version 3 of the draft: > http://numpy.scipy.org/array_interface.shtml > There is a description, from a C Structure perspective: > http://svn.scipy.org/svn/PEP/PEP_basearray.txt > > What is the current status of the plan to develop a PEP? > > Is there a more recent version of the PEP, from a Python user's perspective? > > I'm working on two PEP's after discussing this with Guido at SciPy 2006. 1) A data-type object PEP 2) An extension of the buffer protocol to include the array interface. These are being done instead of the basearray PEP for now. I'm hoping to have a draft of these ready for discussion in a few weeks. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 12:05:10 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 10:05:10 -0600 Subject: Defining custom types In-Reply-To: <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> Message-ID: <45422E36.3010502@ieee.org> Jonathan Wang wrote: > On 10/26/06, *Travis Oliphant* > wrote: > > > Okay, is my understanding here correct? I am defining two type > > descriptors: > > PyArray_Descr mxNumpyType - describes the Numpy array type. > > PyTypeObject mxNumpyDataType - describes the data type of the > contents > > of the array (i.e. mxNumpyType->typeobj points to this), > inherits from > > PyDoubleArrType_Type and overrides some fields as mentioned above. > > > The nomenclature is that mxNumPyType is the data-type of the array and > your PyTypeObject is the "type" of the elements of the array. > So, you > have the names a bit backward. > > So, to correspond with the way I use the words "type" and > "data-type", I > would name them: > > PyArray_Descr mxNumpyDataType > PyTypeObject mxNumpyType > > > Okay, I will use this convention going forwards. > > > And the getitem and setitem functions are designed to only > give/take > > PyObject* of type mxDateTime. > > > These are in the 'f' member of the PyArray_Descr structure, so > presumably you have also filled in your PyArray_Descr structure with > items from PyArray_DOUBLE? > > > That's correct. I have all members of the 'f' member identical to that > from PyArray_DOUBLE, except: > > mxNumpyType->f->dotfunc = NULL; > mxNumpyType->f->getitem = date_getitem; > mxNumpyType->f->setitem = date_setitem; > mxNumpyType->f->cast[PyArray_DOUBLE] = (PyArray_VectorUnaryFunc*) > dateToDouble; > mxNumpyType->f->cast[PyArray_OBJECT] = (PyArray_VectorUnaryFunc*) > dateToObject; > > All other cast functions are NULL. > > If I redefine the string function, I encounter another, perhaps more > serious problem leading to a segfault. I've defined my string function > to be extremely simple: > >>> def printer(arr): > ... return str(arr[0]) > > Now, if I try to print an element of the array: > >>> mxArr[0] > > I get to this stack trace: > #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at > scalartypes.inc.src:68 > #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, > base=0x814e7a8) at arrayobject.c:1419 > #2 0x007d259f in array_subscript_nice (self=0x814e7a8, op=0x804eb8c) > at arrayobject.c:1985 > #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at > Objects/abstract.c:94 > > (Note: for some reason gdb claims that arrayobject.c:1985 is > array_subscript_nice, but looking at my source this line is actually > in array_item_nice. *boggle*) > > But scalar_value returns NULL for all non-native types. So, destptr in > PyArray_Scalar is set to NULL, and the call the copyswap segfaults. > > Perhaps scalar_value should be checking the scalarkind field of > PyArray_Descr, or using the elsize and alignment fields to figure out > the pointer to return if scalarkind isn't set? Hmmm... It looks like the modifications to scalar_value did not take into account user-defined types. I've added a correction so that user-defined types will use setitem to set the scalar value into the array. Presumably your setitem function can handle setting the array with scalars of your new type? I've checked the changes into SVN. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cjw at sympatico.ca Fri Oct 27 12:09:57 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri, 27 Oct 2006 12:09:57 -0400 Subject: Python Enhancement Proposal (PEP) - Current status? In-Reply-To: <45422D8A.8060805@ieee.org> References: <45422BF1.2090201@sympatico.ca> <45422D8A.8060805@ieee.org> Message-ID: <45422F55.7040901@sympatico.ca> Travis Oliphant wrote: > Colin J. Williams wrote: > >> A July posting sets out the intent: >> http://scipy.org/BaseArray >> Version 3 of the draft: >> http://numpy.scipy.org/array_interface.shtml >> There is a description, from a C Structure perspective: >> http://svn.scipy.org/svn/PEP/PEP_basearray.txt >> >> What is the current status of the plan to develop a PEP? >> >> Is there a more recent version of the PEP, from a Python user's perspective? >> >> >> > I'm working on two PEP's after discussing this with Guido at SciPy 2006. > > 1) A data-type object PEP > 2) An extension of the buffer protocol to include the array interface. > > These are being done instead of the basearray PEP for now. I'm hoping > to have a draft of these ready for discussion in a few weeks. > > > -Travis > > Travis, Many thanks, I'll watch out. In an earlier draft, I believe it was proposed that the type specifier be a single character. I hope that thought will be given to the use of something similar to dtype as the mnemonic codes are much more user-friendly than the letter codes. Thanks also for all the slogging you have done over the last many months. Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From joris at ster.kuleuven.be Fri Oct 27 11:58:54 2006 From: joris at ster.kuleuven.be (Joris De Ridder) Date: Fri, 27 Oct 2006 17:58:54 +0200 Subject: astype() behaviour Message-ID: <200610271758.54379.joris@ster.kuleuven.be> Hi, Is the following behaviour of astype() intentional in NumPy 1.0? >>> x = array([1,2,3]) >>> x.astype(None) array([ 1., 2., 3.]) That is, the int32 is converted to float64. J. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From joris at ster.kuleuven.be Fri Oct 27 12:13:21 2006 From: joris at ster.kuleuven.be (Joris De Ridder) Date: Fri, 27 Oct 2006 18:13:21 +0200 Subject: dtype() examples that used to work Message-ID: <200610271813.21500.joris@ster.kuleuven.be> Hi, The following did once work in NumPy: >>> dtype([int16]) >>> dtype([[int16]]) >>> dtype([uint,int32]) >>> dtype(['f8','S10']) but now they all generate a "TypeError: data type not understood". Why? I could not find the answer in help(dtype), the RecordArrays tutorial on scipy.org, the NumPy release notes, nor in the NumPy book. J. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jeremit0 at gmail.com Fri Oct 27 10:44:46 2006 From: jeremit0 at gmail.com (jeremito) Date: Fri, 27 Oct 2006 14:44:46 -0000 Subject: Sorting eigenvalues and vectors together Message-ID: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> I am using a = numpy.linalg.eig(A) to get the eigenvalues and eigenvectors. I am interested only in the largest eigenvalue so I would like to sort the first element of a. But if I do that, I won't know what is the associated eigenvector. Is there a function that will sort the values and vectors together or do I need to write it myself. Thanks, Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Fri Oct 27 13:01:38 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 27 Oct 2006 19:01:38 +0200 Subject: astype() behaviour In-Reply-To: <200610271758.54379.joris@ster.kuleuven.be> References: <200610271758.54379.joris@ster.kuleuven.be> Message-ID: <200610271901.39809.faltet@carabos.com> A Divendres 27 Octubre 2006 17:58, Joris De Ridder va escriure: > Hi, > > Is the following behaviour of astype() intentional in NumPy 1.0? > > >>> x = array([1,2,3]) > >>> x.astype(None) > > array([ 1., 2., 3.]) > > That is, the int32 is converted to float64. Yes, I think the behaviour is intended. This is because 'float64' is the default type in NumPy from some months ago (before the default was 'int_') HTH, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 13:02:00 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 11:02:00 -0600 Subject: Python Enhancement Proposal (PEP) - Current status? In-Reply-To: <45423A03.7010401@noaa.gov> References: <45422BF1.2090201@sympatico.ca> <45422D8A.8060805@ieee.org> <45423A03.7010401@noaa.gov> Message-ID: <45423B88.60401@ieee.org> Christopher Barker wrote: > Travis Oliphant wrote: > >> I'm working on two PEP's after discussing this with Guido at SciPy 2006. >> >> 1) A data-type object PEP >> 2) An extension of the buffer protocol to include the array interface. >> >> These are being done instead of the basearray PEP for now. >> > > Those are a great starts, and I think it is a good idea to break these > out. Thanks for working on this. > > However, I still think it is a VERY good idea to have an n-d array in > the standard lib -- I hope that idea won't get forgotten. > I think any nd-array in the standard library will just be an "Object" nd-array, though. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Fri Oct 27 13:06:13 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 27 Oct 2006 19:06:13 +0200 Subject: dtype() examples that used to work In-Reply-To: <200610271813.21500.joris@ster.kuleuven.be> References: <200610271813.21500.joris@ster.kuleuven.be> Message-ID: <200610271906.14099.faltet@carabos.com> A Divendres 27 Octubre 2006 18:13, Joris De Ridder va escriure: > Hi, > > The following did once work in NumPy: > >>> dtype([int16]) > >>> dtype([[int16]]) > >>> dtype([uint,int32]) > >>> dtype(['f8','S10']) > > but now they all generate a "TypeError: data type not understood". Why? Should be intended as well. If you try to set a dtype from a list, it has to follow the format of a description as specified in: http://numpy.scipy.org/array_interface.shtml for example: In [67]: dtype([('f1', int16)]) Out[67]: dtype([('f1', '0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Fri Oct 27 12:55:31 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Fri, 27 Oct 2006 09:55:31 -0700 Subject: Python Enhancement Proposal (PEP) - Current status? In-Reply-To: <45422D8A.8060805@ieee.org> References: <45422BF1.2090201@sympatico.ca> <45422D8A.8060805@ieee.org> Message-ID: <45423A03.7010401@noaa.gov> Travis Oliphant wrote: > I'm working on two PEP's after discussing this with Guido at SciPy 2006. > > 1) A data-type object PEP > 2) An extension of the buffer protocol to include the array interface. > > These are being done instead of the basearray PEP for now. Those are a great starts, and I think it is a good idea to break these out. Thanks for working on this. However, I still think it is a VERY good idea to have an n-d array in the standard lib -- I hope that idea won't get forgotten. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 27 13:19:40 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 27 Oct 2006 13:19:40 -0400 Subject: Sorting eigenvalues and vectors together In-Reply-To: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> Message-ID: On 27/10/06, jeremito wrote: > I am using a = numpy.linalg.eig(A) to get the eigenvalues and > eigenvectors. I am interested only in the largest eigenvalue so I > would like to sort the first element of a. But if I do that, I won't > know what is the associated eigenvector. Is there a function that will > sort the values and vectors together or do I need to write it myself. I think that they are sorted, but argsort() will do what you want (or argmax(), even). A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Fri Oct 27 13:33:52 2006 From: David.L.Goldsmith at noaa.gov (David L. Goldsmith) Date: Fri, 27 Oct 2006 10:33:52 -0700 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <4541D718.6040308@ar.media.kyoto-u.ac.jp> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> Message-ID: <45424300.60805@noaa.gov> I'm sure some others _might_ regard this as frivolous, so let me just say: "Way Cool"! Thanks! DG David Cournapeau wrote: > Hi, > I announce the first release of pyaudio, a module to make noise from > numpy arrays (read, write and play audio files with numpy arrays). > > * WHAT FOR ?: > > The Goal is to give to a numpy/scipy environmenet some basic audio IO > facilities (ala sound, wavread, wavwrite of matlab). > > With pyaudio, you should be able to read and write most common audio > files from and to numpy arrays. The underlying IO operations are done > using libsndfile from Erik Castro Lopo > (http://www.mega-nerd.com/libsndfile/), so he is the one who did the > hard work. As libsndfile has support for a vast number of audio files > (including wav, aiff, but also htk, ircam, and flac, an open source > lossless codec), pyaudio enables the import from and export to a fairly > large number of file formats. > There is also a really crude player, which uses tempfile to play > audio, and which only works for linux-alsa anyway. I intend to add > better support at least for linux, and for other platforms if this does > not involve too much hassle. > > So basically, if you are lucky enough to use a recent linux system, > pyaudio already gives you the equivalent of wavread, wavwrite and sound. > > * DOWNLOAD: > > http://www.ar.media.kyoto-u.ac.jp/members/david/pyaudio.tar.gz > > * INSTALLATION INSTRUCTIONS: > > Just untar the package and drop it into scipy/Lib/sandbox, and add > the two following lines to scipy/Lib/sandbox/setup.py: > > # Package to make some noise using numpy > config.add_subpackage('pyaudio') > > (if libsndfile.so is not in /usr/lib, a fortiori if you are a windows > user, you should also change set the right location for libsndfile in > pyaudio/pysndfile.py, at the line > _snd.cdll.LoadLibrary('/usr/lib/libsndfile.so') ) > > * EXAMPLE USAGE > > == Reading example == > > # Reading from '/home/david/blop.flac' > from scipy.sandbox.pyaudio import sndfile > > a = sndfile('/home/david/blop.flac') > print a > > tmp = a.read_frames_float(1024) > > --> Prints: > > File : /home/david/blop.flac > Sample rate : 44100 > Channels : 2 > Frames : 9979776 > Format : 0x00170002 > Sections : 1 > Seekable : True > Duration : 00:03:46.298 > > And put into tmp the 1024 first frames (a frame is the equivalent of > a sample, but taking into account the number of channels: so 1024 frames > gives you 2048 samples here). > > == Writing example == > # Writing to a wavfile: > from scipy.sandbox.pyaudio import sndfile > import numpy as N > > noise = N.random.randn((44100)) > a = sndfile('/home/david/blop.flac', sfm['SFM_WRITE'], > sf_format['SF_FORMAT_WAV'] | sf_format['SF_FORMAT_PCM16'], > 1, 44100) > > a.write_frames(noise, 44100) > a.close() > > -> should gives you a lossless compressed white noise ! > > This is really a first release, not really tested, not much > documentation, I can just say it works for me. I haven't found a good > way to emulate enumerations, which libsndfile uses a lot, so I am using > dictionaries generated from the library C header to get a relation enum > label <=> value. If someone has a better idea, I am open to suggestions ! > > Cheers, > > David > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Fri Oct 27 13:23:46 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Fri, 27 Oct 2006 10:23:46 -0700 Subject: Sorting eigenvalues and vectors together In-Reply-To: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> Message-ID: On 10/27/06, jeremito wrote: > I am using a = numpy.linalg.eig(A) to get the eigenvalues and > eigenvectors. I am interested only in the largest eigenvalue so I > would like to sort the first element of a. But if I do that, I won't > know what is the associated eigenvector. Is there a function that will > sort the values and vectors together or do I need to write it myself. You can get the ranking of the eigenvalues with idx = a[0].argsort().argsort() or idx = (-a[0]).argsort().argsort(). And then use that to sort both the eigenvalues and eigenvectors. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From djm at mindrot.org Fri Oct 27 13:37:10 2006 From: djm at mindrot.org (Damien Miller) Date: Sat, 28 Oct 2006 03:37:10 +1000 (EST) Subject: numpy-1.0 regress failure on OpenBSD In-Reply-To: <45414B37.8050402@ee.byu.edu> References: <45414B37.8050402@ee.byu.edu> Message-ID: On Thu, 26 Oct 2006, Travis Oliphant wrote: > Unless you want to help with tracking how long double is interpreted on > several platforms, then just ignore the test. (It actually wasn't being > run in 1.0b1). I'm happy to help - do you have a testcase that I can run on the various platforms that OpenBSD supports? -d ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 13:01:05 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 11:01:05 -0600 Subject: dtype() examples that used to work In-Reply-To: <200610271813.21500.joris@ster.kuleuven.be> References: <200610271813.21500.joris@ster.kuleuven.be> Message-ID: <45423B51.3080006@ieee.org> Joris De Ridder wrote: > Hi, > > The following did once work in NumPy: > > >>>> dtype([int16]) >>>> dtype([[int16]]) >>>> dtype([uint,int32]) >>>> dtype(['f8','S10']) >>>> > > but now they all generate a "TypeError: data type not understood". Why? > I could not find the answer in help(dtype), the RecordArrays tutorial on > scipy.org, the NumPy release notes, nor in the NumPy book. > It's in the release notes. Look in the section from 1.0b2 to 1.0b3. It's highlighted with a Note: *NOTE: * When creating data-type objects from lists, the list must be in array descriptor format. The "list of data-type objects" approach is no longer supported due to its ambiguity with the very common array descriptor approach. You can still get this behavior using a comma-separated string or changing your list to an array_descriptor: [('', x) for x in oldlist] -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Fri Oct 27 14:17:16 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 27 Oct 2006 12:17:16 -0600 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <45424300.60805@noaa.gov> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> <45424300.60805@noaa.gov> Message-ID: On 10/27/06, David L. Goldsmith wrote: > I'm sure some others _might_ regard this as frivolous, so let me just > say: "Way Cool"! Thanks! +1, and not frivolous at all. It's /really/ neat to be able to pull in data from standard image formats (say jpeg) into arrays to quickly do numerics on images. Adding similar capabilities to audio signals is, IMHO, a great contribution. So I heartily join the congratulation. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Fri Oct 27 13:22:29 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 27 Oct 2006 12:22:29 -0500 Subject: Sorting eigenvalues and vectors together In-Reply-To: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> Message-ID: jeremito wrote: > I am using a = numpy.linalg.eig(A) to get the eigenvalues and > eigenvectors. I am interested only in the largest eigenvalue so I > would like to sort the first element of a. But if I do that, I won't > know what is the associated eigenvector. Is there a function that will > sort the values and vectors together or do I need to write it myself. Use argsort() on the eigenvalues and fancy indexing on the eigenvectors. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 14:33:03 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 12:33:03 -0600 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <4541D718.6040308@ar.media.kyoto-u.ac.jp> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> Message-ID: <454250DF.3060402@ieee.org> David Cournapeau wrote: > Hi, > I announce the first release of pyaudio, a module to make noise from > numpy arrays (read, write and play audio files with numpy arrays). > Very nice. Thank you. I'd like to see exactly this kind of thing for video files too. We can get a lot of mileage out of ctypes. By the way. I noticed your setup.py file is missing a .todict() to convert the configuration object into a dictionary that setup(** ) can handle. if __name__ == "__main__": from numpy.distutils.core import setup setup(**configuration(top_path='').todict()) You might also check to see if libsndfile can be imported during the setup phase and then warn the user before installing that they need it. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Fri Oct 27 14:57:14 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Fri, 27 Oct 2006 21:57:14 +0300 (EEST) Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <454250DF.3060402@ieee.org> References: <454250DF.3060402@ieee.org> Message-ID: On Fri, 27 Oct 2006, Travis Oliphant wrote: > David Cournapeau wrote: > > Hi, > > I announce the first release of pyaudio, a module to make noise from > > numpy arrays (read, write and play audio files with numpy arrays). > > > > Very nice. Thank you. I'd like to see exactly this kind of thing for > video files too. We can get a lot of mileage out of ctypes. > > By the way. I noticed your setup.py file is missing a .todict() to > convert the configuration object into a dictionary that setup(** ) can > handle. > > if __name__ == "__main__": > from numpy.distutils.core import setup > setup(**configuration(top_path='').todict()) Actually, one should use here setup(configuration=configuration) Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Fri Oct 27 15:11:11 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Fri, 27 Oct 2006 12:11:11 -0700 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <454250DF.3060402@ieee.org> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> <454250DF.3060402@ieee.org> Message-ID: <454259CF.9090608@noaa.gov> > David Cournapeau wrote: >> Hi, >> I announce the first release of pyaudio, a module to make noise from >> numpy arrays (read, write and play audio files with numpy arrays). Does this have anything to do with this pyaudio? http://people.csail.mit.edu/hubert/pyaudio/ -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jontwang at gmail.com Fri Oct 27 15:37:01 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Fri, 27 Oct 2006 14:37:01 -0500 Subject: Defining custom types In-Reply-To: <45422E36.3010502@ieee.org> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> <45422E36.3010502@ieee.org> Message-ID: <4f7636bf0610271237p57a7b83bkb3b6cf00a738a833@mail.gmail.com> On 10/27/06, Travis Oliphant wrote: > > > If I redefine the string function, I encounter another, perhaps more > > serious problem leading to a segfault. I've defined my string function > > to be extremely simple: > > >>> def printer(arr): > > ... return str(arr[0]) > > > > Now, if I try to print an element of the array: > > >>> mxArr[0] > > > > I get to this stack trace: > > #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at > > scalartypes.inc.src:68 > > #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, > > base=0x814e7a8) at arrayobject.c:1419 > > #2 0x007d259f in array_subscript_nice (self=0x814e7a8, op=0x804eb8c) > > at arrayobject.c:1985 > > #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at > > Objects/abstract.c:94 > > > > (Note: for some reason gdb claims that arrayobject.c:1985 is > > array_subscript_nice, but looking at my source this line is actually > > in array_item_nice. *boggle*) > > > > But scalar_value returns NULL for all non-native types. So, destptr in > > PyArray_Scalar is set to NULL, and the call the copyswap segfaults. > > > > Perhaps scalar_value should be checking the scalarkind field of > > PyArray_Descr, or using the elsize and alignment fields to figure out > > the pointer to return if scalarkind isn't set? > > Hmmm... It looks like the modifications to scalar_value did not take > into account user-defined types. I've added a correction so that > user-defined types will use setitem to set the scalar value into the > array. Presumably your setitem function can handle setting the array > with scalars of your new type? > > I've checked the changes into SVN. Do there also need to be changes in scalartypes.inc.src to use getitem if a user-defined type does not inherit from a Numpy scalar? i.e. at scalartypes.inc.src:114 we should return some pointer calculated from the PyArray_Descr's elsize and alignment field to get the destination for the "custom scalar" type to be copied. As it stands, if the user-defined type does not inherit from a Numpy scalar, lots of things continue to break. Furthermore it seems like the scalar conversions prefer the builtin types, but it seems to me that the user-defined type should be preferred. i.e. if I try to get an element from my mxDateTime array, I get a float back: >>> mxArr[0] = DateTime.now() >>> mxArr[0][0] 732610.60691268521 But what I really want is the mxDateTime, which, oddly enough, is what happens if I use tolist(): >>> mxArr.tolist()[0] [] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Fri Oct 27 16:02:56 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 14:02:56 -0600 Subject: Defining custom types In-Reply-To: <4f7636bf0610271237p57a7b83bkb3b6cf00a738a833@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> <45422E36.3010502@ieee.org> <4f7636bf0610271237p57a7b83bkb3b6cf00a738a833@mail.gmail.com> Message-ID: <454265F0.7000805@ieee.org> Jonathan Wang wrote: > On 10/27/06, *Travis Oliphant* > wrote: > > > If I redefine the string function, I encounter another, perhaps more > > serious problem leading to a segfault. I've defined my string > function > > to be extremely simple: > > >>> def printer(arr): > > ... return str(arr[0]) > > > > Now, if I try to print an element of the array: > > >>> mxArr[0] > > > > I get to this stack trace: > > #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at > > scalartypes.inc.src:68 > > #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, > > base=0x814e7a8) at arrayobject.c:1419 > > #2 0x007d259f in array_subscript_nice (self=0x814e7a8, > op=0x804eb8c) > > at arrayobject.c:1985 > > #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at > > Objects/abstract.c:94 > > > > (Note: for some reason gdb claims that arrayobject.c:1985 is > > array_subscript_nice, but looking at my source this line is > actually > > in array_item_nice. *boggle*) > > > > But scalar_value returns NULL for all non-native types. So, > destptr in > > PyArray_Scalar is set to NULL, and the call the copyswap segfaults. > > > > Perhaps scalar_value should be checking the scalarkind field of > > PyArray_Descr, or using the elsize and alignment fields to > figure out > > the pointer to return if scalarkind isn't set? > > Hmmm... It looks like the modifications to scalar_value did not take > into account user-defined types. I've added a correction so that > user-defined types will use setitem to set the scalar value into the > array. Presumably your setitem function can handle setting the array > with scalars of your new type? > > I've checked the changes into SVN. > > > Do there also need to be changes in scalartypes.inc.src to use getitem > if a user-defined type does not inherit from a Numpy scalar? This needs to be clarified. I don't think it's possible to do it without inheriting from a numpy scalar at this point (the void numpy scalar can be inherited from and is pretty generic). I know I was not considering that case when I wrote the code. > i.e. at scalartypes.inc.src:114 we should return some pointer > calculated from the PyArray_Descr's elsize and alignment field to get > the destination for the "custom scalar" type to be copied. I think this is a good idea. I doubt it's enough to fix all places that don't inherit from numpy scalars, but it's a start. It seems like we need to figure out where the beginning of the data is for the type which is assumed to be defined on alignment boundaries after a PyObject_HEAD (right)? This could actually be used for everything and all the switch and if statements eliminated. I think the alignment field is the only thing needed, though. I don't see how I would use the elsize field? > > As it stands, if the user-defined type does not inherit from a Numpy > scalar, lots of things continue to break. Not surprising, I did not make sure and support this. > Furthermore it seems like the scalar conversions prefer the builtin > types, but it seems to me that the user-defined type should be preferred. I'm not sure what this means. > > > i.e. if I try to get an element from my mxDateTime array, I get a > float back: > >>> mxArr[0] = DateTime.now() > >>> mxArr[0][0] > 732610.60691268521 Why can you index mxArr[0]? What is mxArr[0]? If it's a scalar, then why can you index it? What is type(mxArr[0])? > > But what I really want is the mxDateTime, which, oddly enough, is what > happens if I use tolist(): > >>> mxArr.tolist()[0] > [] That's not surprising because tolist just calls getitem on each element in the array to construct the list. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jeremit0 at gmail.com Fri Oct 27 16:23:38 2006 From: jeremit0 at gmail.com (jeremito) Date: Fri, 27 Oct 2006 20:23:38 -0000 Subject: Sorting eigenvalues and vectors together In-Reply-To: References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> Message-ID: <1161980618.503699.300800@e3g2000cwe.googlegroups.com> argsort() will do the trick. Thanks once again. Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Fri Oct 27 17:04:13 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Fri, 27 Oct 2006 23:04:13 +0200 Subject: Sorting eigenvalues and vectors together In-Reply-To: <1161980618.503699.300800@e3g2000cwe.googlegroups.com> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> <1161980618.503699.300800@e3g2000cwe.googlegroups.com> Message-ID: <4542744D.6040801@gmx.net> jeremito schrieb: > argsort() will do the trick. Thanks once again. > Jeremy > I was a bit confused by your question, maybe you can clarify what you did in the end. IIRC, if the eigenvalues returned by numpy are real numbers (due to the type of the underlying matrix and algorithm), then they are automatically returned ascending (again, IIRC). That would mean you don't have to do anything. But if they're complex, it's not obvious how to sort them. Probably by modulus. So what exactly was your problem? ciao, Sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jontwang at gmail.com Fri Oct 27 17:08:08 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Fri, 27 Oct 2006 16:08:08 -0500 Subject: Defining custom types In-Reply-To: <454265F0.7000805@ieee.org> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> <45422E36.3010502@ieee.org> <4f7636bf0610271237p57a7b83bkb3b6cf00a738a833@mail.gmail.com> <454265F0.7000805@ieee.org> Message-ID: <4f7636bf0610271408p7adcedc7h547cd33434456a2c@mail.gmail.com> On 10/27/06, Travis Oliphant wrote: > > Jonathan Wang wrote: > > On 10/27/06, *Travis Oliphant* > > wrote: > > > > > If I redefine the string function, I encounter another, perhaps > more > > > serious problem leading to a segfault. I've defined my string > > function > > > to be extremely simple: > > > >>> def printer(arr): > > > ... return str(arr[0]) > > > > > > Now, if I try to print an element of the array: > > > >>> mxArr[0] > > > > > > I get to this stack trace: > > > #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at > > > scalartypes.inc.src:68 > > > #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, > > > base=0x814e7a8) at arrayobject.c:1419 > > > #2 0x007d259f in array_subscript_nice (self=0x814e7a8, > > op=0x804eb8c) > > > at arrayobject.c:1985 > > > #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at > > > Objects/abstract.c:94 > > > > > > (Note: for some reason gdb claims that arrayobject.c:1985 is > > > array_subscript_nice, but looking at my source this line is > > actually > > > in array_item_nice. *boggle*) > > > > > > But scalar_value returns NULL for all non-native types. So, > > destptr in > > > PyArray_Scalar is set to NULL, and the call the copyswap > segfaults. > > > > > > Perhaps scalar_value should be checking the scalarkind field of > > > PyArray_Descr, or using the elsize and alignment fields to > > figure out > > > the pointer to return if scalarkind isn't set? > > > > Hmmm... It looks like the modifications to scalar_value did not take > > into account user-defined types. I've added a correction so that > > user-defined types will use setitem to set the scalar value into the > > array. Presumably your setitem function can handle setting the > array > > with scalars of your new type? > > > > I've checked the changes into SVN. > > > > > > Do there also need to be changes in scalartypes.inc.src to use getitem > > if a user-defined type does not inherit from a Numpy scalar? > This needs to be clarified. I don't think it's possible to do it > without inheriting from a numpy scalar at this point (the void numpy > scalar can be inherited from and is pretty generic). I know I was not > considering that case when I wrote the code. > > i.e. at scalartypes.inc.src:114 we should return some pointer > > calculated from the PyArray_Descr's elsize and alignment field to get > > the destination for the "custom scalar" type to be copied. > I think this is a good idea. I doubt it's enough to fix all places that > don't inherit from numpy scalars, but it's a start. > > It seems like we need to figure out where the beginning of the data is > for the type which is assumed to be defined on alignment boundaries > after a PyObject_HEAD (right)? This could actually be used for > everything and all the switch and if statements eliminated. > > I think the alignment field is the only thing needed, though. I don't > see how I would use the elsize field? Hmm, yeah, I guess alignment would be sufficient. Worst case, you could delegate to setitem, right? It would be useful to support arbitrary types. Suppose, for example, that I wanted to make an array of structs. In keeping with the date/time example, I might want to store a long and a double, the long for days in the Gregorian calendar and the double for seconds from midnight on that day. > Furthermore it seems like the scalar conversions prefer the builtin > > types, but it seems to me that the user-defined type should be > preferred. > I'm not sure what this means. > > > > > > i.e. if I try to get an element from my mxDateTime array, I get a > > float back: > > >>> mxArr[0] = DateTime.now() > > >>> mxArr[0][0] > > 732610.60691268521 > Why can you index mxArr[0]? What is mxArr[0]? If it's a scalar, then > why can you index it? What is type(mxArr[0])? Ah, I am mistaken here - I am correctly getting my mxNumpyDateTime type back: mxArr is a 1x1 matrix: >>> mxArr = numpy.empty((1,1), dtype = libMxNumpy.type) >>> mxArr[0] = DateTime.now() >>> type(mxArr) >>> type(mxArr[0]) >>> type(mxArr[0][0]) >>> mxArr.shape (1, 1) > But what I really want is the mxDateTime, which, oddly enough, is what > > happens if I use tolist(): > > >>> mxArr.tolist()[0] > > [] > > That's not surprising because tolist just calls getitem on each element > in the array to construct the list. I guess this is a degenerate case, since I have getitem returning a mxDateTime while the actual type of the elements in the array is mxNumpyDateTime (i.e. mxNumpyType). Would the correct behavior, then, be for getitem to return a mxNumpyDateTime and register the object cast function to return a mxDateTime? If I try to do math on the array, it seems like the operation is performed via object pointers (mxDateTime - mxDateTime returns a DateTimeDelta object, and mxNumpyDateTime is a float): >>> mxArr = numpy.empty((1,1), dtype = libMxNumpy.type) >>> mxArr[0][0] = DateTime.now() >>> mxArr2 = numpy.empty((1,1), dtype = libMxNumpy.type) >>> mxArr2[0][0] = DateTime.DateTimeFrom('2006-01-01') >>> type(mxArr[0][0]) >>> type(mxArr2[0][0]) >>> sub = mxArr - mxArr2 >>> type(sub[0][0]) I'm guessing I need to register ufunc loops for all the basic math on my types? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Fri Oct 27 18:11:15 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 27 Oct 2006 17:11:15 -0500 Subject: Sorting eigenvalues and vectors together In-Reply-To: <4542744D.6040801@gmx.net> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> <1161980618.503699.300800@e3g2000cwe.googlegroups.com> <4542744D.6040801@gmx.net> Message-ID: Sven Schreiber wrote: > IIRC, if the eigenvalues returned by numpy are real numbers (due to the > type of the underlying matrix and algorithm), then they are > automatically returned ascending (again, IIRC). This should be considered an implementation detail that may change in the future, so don't rely on it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Fri Oct 27 18:37:19 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sat, 28 Oct 2006 00:37:19 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <45379D7E.30609@ee.byu.edu> Message-ID: <000301c6fa18$76fed890$0a83a8c0@ratbert> Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant > Sent: Thursday, October 19, 2006 5:45 PM > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] adding an attribute to an nd-array > > Stefan van der Walt wrote: > > >On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > >>On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: > >>>A quick question on extending numpy arrays: is it possible to easily > >>>add an attribute to an ndarray? > >>It might be easier to create a subclass: pleasehave a look here: > >>http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_s > ubclasses.py > >>That's a tiny example of subclassing ndarrays, with some extra > attributes. > >>(BTW, I'm not sure that's the most obvious place where to look: if it > turns > >>out to be useful, I'll put it on the scipy wiki) > > > >Thanks very much, Pierre. > > > >If I understand correctly, the following should work: > > > >import numpy as N > > > >class InfoArray(N.ndarray): > > def __new__(info_arr_cls,arr,info={}): > > info_arr_cls.info = info > > return N.array(arr).view(info_arr_cls) > > > >When does __array_finalize__ get called, and is it always necessary to > >specify it? > > Actually something as simple as > > class InfoArray(N.ndarray): > pass > > will allow you to add attributes to InfoArray. I would like to create an "info" array like this to attach some metadata, mostly sampling frequency, to some of my arrays. The first problem I'm running into is that this new InfoArray type needs something like the array function so that I can create it from an existing "normal" array. It would also be useful if metadata could survive through type conversions like this: y = array(x, 'f4') One frequently wants to do this where you start with a 8-bit or 16-bit samples of a speech signal and you then want to convert to a floating point type for further processing. I browsed through the NumPy book and looked at the __new__ and __array_finalize__ stuff in Pierre GM's new MaskedArray, but I'm still thoroughly confused about how to create a simple array+metadata subclass that Just Works(TM). Does anybody have some code that they would be willing to share? Thanks. Regards, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Fri Oct 27 20:59:36 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sat, 28 Oct 2006 02:59:36 +0200 Subject: NumPy on Intel C/MKL Message-ID: <000401c6fa2c$57041c30$0a83a8c0@ratbert> Hello all I managed to get NumPy to compile with the free for non-commercial use Linux versions of the Intel C compiler, version 9.1 and Intel MKL 8.1. Instructions at the bottom of this page: http://www.scipy.org/Installing_SciPy/Linux I ran the NumPy test suite, and it turned up a few test failures in test_ufunclike -- it seems the Intel C/MKL combo comes up with different answers for the sign of NaN. Compiling with -Wall turned up a few warnings, but nothing too serious. I submitted a ticket for the ones that seem easily fixable and might warrant a quick look: http://projects.scipy.org/scipy/numpy/ticket/366 I recently started building some of my other C code on Windows with the Intel compiler, and the speed increases in my application are quite dramatic. The Intel code is about 4 times faster than the code produced by Visual Studio .NET 2003. I suspect the speed increase is due to some loop vectorization. When compiling NumPy on Linux, some interesting messages pop up: numpy/core/src/arraytypes.inc.src(658) : (col. 9) remark: LOOP WAS VECTORIZED. numpy.distutils doesn't seem to support the Intel compiler on Windows, but if you have other C code lying around, you can easily use SCons to compile it with the Intel compiler (Intel offers 30-day evaluation versions of most of their products). Have fun! Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mforbes at phys.washington.edu Fri Oct 27 22:18:52 2006 From: mforbes at phys.washington.edu (Michael McNeil Forbes) Date: Fri, 27 Oct 2006 19:18:52 -0700 Subject: recarray.__setattr__ bug? Message-ID: Is the following the desired behaviour for setting recarray attributes?? This seems to clash with the semantics for arrays. ?>>> from numpy import * ?>>> a = array([1,2,3]) ?>>> b = a.view([('x',int),('y',int),('z',int)]) ?>>> r = b.view(recarray) ?>>> b['x'] = 0 ?>>> r.y = 0 ?>>> a ?array([0, 2, 3]) ?>>> r.x ?array([0]) ?>>> r.y ?0 Setting r.y creates a local variable in r.__dict__ which is then accessed rather than the array element referred to by fielddict. I am not sure how to fix this: trying to set an attribute that does not exist in an ndarray raises an error, so I figured that changing the second line of recarray.__setattr__ in numpy/core/records.py to ?? ?def __setattr__(self, attr, val): ? ? ? ? try: ? ? ? ? ? ? return sb.ndarray.__setattr__(self, attr, val) would work, but this sets the variable in r.__dict__ (and I am not exactly sure where this is defined, so I can't comment further).? Is there a simple solution like this or does one need to explicitly check '__dict__'. One could reverse the order and call 'setfield' if the value is in the 'fielddict' first and then revert to the base setter, but this would potentially hide other members like 'shape'.? (Making this change does not break any tests however.) A test case for this would be: ?? ?def check_recarray_setattr1(self): """Make sure __setattr__ sets array fields.""" a = array([1,2]) r = rec.fromarrays(a,names=['x','y']) b = a.view(r.dtype).view(recarray) b.x = 0 assert 0 == a[0] ? ? def check_recarray_setattr2(self): """Make sure shape attribute is not hidden.""" a = array([[1,2],[3,4]]) r = rec.fromarrays(a,names=['x','shape']) b = a.view(r.dtype).view(recarray) assert 3 == b[1,0][0] b.shape = (1,2) assert 3 == b[0,1][0] Michael. -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Sat Oct 28 05:14:14 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sat, 28 Oct 2006 05:14:14 -0400 Subject: adding an attribute to an nd-array In-Reply-To: <000301c6fa18$76fed890$0a83a8c0@ratbert> References: <000301c6fa18$76fed890$0a83a8c0@ratbert> Message-ID: <200610280514.14604.pgmdevlist@gmail.com> On Friday 27 October 2006 18:37, Albert Strasheim wrote: > Hello all ... > I would like to create an "info" array like this to attach some metadata, > mostly sampling frequency, to some of my arrays. > ... > I browsed through the NumPy book and looked at the __new__ and > __array_finalize__ stuff in Pierre GM's new MaskedArray, but I'm still > thoroughly confused about how to create a simple array+metadata subclass > that Just Works(TM). > > Does anybody have some code that they would be willing to share? I'm not sure about the JustWorks(R)(TM), but I just created a page on the Wiki describing a simple example of InfoArray class http://www.scipy.org/Subclasses That should do what you want, but let me know if you have some more specific requests. I guess I should go more into the details of subclassing on that page, but I'm quite far from being a specialist... ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Sat Oct 28 06:19:06 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sat, 28 Oct 2006 06:19:06 -0400 Subject: recarray.__setattr__ bug? In-Reply-To: References: Message-ID: <200610280619.06841.pgmdevlist@gmail.com> On Friday 27 October 2006 22:18, Michael McNeil Forbes wrote: > Is the following the desired behaviour for setting recarray attributes?? I ran into the same problem recently... What about modifying __setattr__ to the following ? def __setattr__(self, attr, val): fielddict = sb.ndarray.__getattribute__(self,'dtype').fields if attr in fielddict.keys(): return sb.ndarray.__setitem__(self, attr, val) try: return object.__setattr__(self, attr, val) except AttributeError: # Must be a fieldname raise AttributeError, "record array has no attribute %s" % attr The order is changed, as in Michael's suggestion, but that doesn't set any variable in the __dict__, nor does it mask anything. Instead of modifying the an attribute, we modify the field directly... ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jk985 at tom.com Sat Oct 28 13:56:54 2006 From: jk985 at tom.com (=?GB2312?B?IjEx1MI0LTXI1S+xsb6pIg==?=) Date: Sat, 28 Oct 2006 17:56:54 +0000 (UTC) Subject: =?GB2312?B?RVhDRUy6zVBQVNTaxvPStbncwO3Kx7XEuN+8ttOm08M=?= Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Sat Oct 28 14:28:35 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 28 Oct 2006 12:28:35 -0600 Subject: recarray.__setattr__ bug? In-Reply-To: References: Message-ID: <4543A153.4030505@ieee.org> Michael McNeil Forbes wrote: > Is the following the desired behaviour for setting recarray attributes? > This seems to clash with the semantics for arrays. > > >>> from numpy import * > >>> a = array([1,2,3]) > >>> b = a.view([('x',int),('y',int),('z',int)]) > >>> r = b.view(recarray) > >>> b['x'] = 0 > >>> r.y = 0 > >>> a > array([0, 2, 3]) > >>> r.x > array([0]) > >>> r.y > 0 > > Setting r.y creates a local variable in r.__dict__ which is then > accessed rather than the array element referred to by fielddict. > Hmm.... I know that the code was changed at some point a few months ago specifically to this behavior because of some concerns Perry, Chris (people at STScI) had. Originally, field names came first, but we changed it so they could set known attributes of a record array even if they were also field names. This may be an unintentional side-effect. So, let's not just change things again and create problems for them. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From vdeelocbad at planetprint.com Sat Oct 28 15:03:37 2006 From: vdeelocbad at planetprint.com (life.We an) Date: Sat, 28 Oct 2006 21:03:37 +0200 Subject: Seal Campaign aquatic life.We Message-ID: <000b01c6fac3$c61d6620$851336c2@mama> Hints avoid common would like help of distribute of fliers club meetings events feel download in current flier. Bottom Balticon Wyndham downtown Baltimore in main hallway across entrance in first Here some advice which provides or useful hints avoid common would. Hints avoid common would like help of distribute of fliers club meetings events feel download in current flier. One or more writers in how to am improve own writing Past panels included writing about the part Trek express a point view of in story or trials being in writer planets of stars am galaxies that. Door or telling where pick up or wi ll starting October check page now am may obtain forms of clicking now closed refer is. More writers how to improve own writing Past panels included writing about in. Atthedoor bring take in place location posted should of arrive prior close start possible or There door telling am where pick up wi in ll starting October check page now may obtain. Useful hints avoid common would like help distribute fliers club meetings events feel download current flier pdf format make am copies Thanks much. Location posted or should arrive prior close start is possible There door telling where pick up wi ll starting October check page now is may obtain forms clicking. May obtain forms clicking now closed refer Hotel or Hunt Valley inn well overflow or hotelsthe has begun am accepting or person Farpoint February same Leave is visit table in bottom am. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: with.gif Type: image/gif Size: 8199 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Sat Oct 28 15:30:22 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sat, 28 Oct 2006 15:30:22 -0400 Subject: recarray.__setattr__ bug? In-Reply-To: <4543A153.4030505@ieee.org> References: <4543A153.4030505@ieee.org> Message-ID: <200610281530.22929.pgmdevlist@gmail.com> On Saturday 28 October 2006 14:28, Travis Oliphant wrote: > > Hmm.... I know that the code was changed at some point a few months ago > specifically to this behavior because of some concerns Perry, Chris > (people at STScI) had. Originally, field names came first, but we > changed it so they could set known attributes of a record array even if > they were also field names. So that I could have a field named 'shape', and modifying r.shape would change the shape of the array, not the content of the field ? That makes sense, but isn't it a bit dangerous ? Shouldn't we have a list of reserved keywords (dtype, shape...) that would raise an exception if used as field names ? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Sat Oct 28 15:36:23 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sat, 28 Oct 2006 15:36:23 -0400 Subject: A recarray of arrays Message-ID: <200610281536.23189.pgmdevlist@gmail.com> Folks, What is the easiest way to define a recarray of arrays ? For example, I'd need something like that: Given three arrays >>> x = N.arange(5) >>> y = x+1 >>> z = N.sqrt(x) and a list of names: >>> n = ['x','y','z'] Define a 3-record array with two fields: the first one being a ndarray (in x, y or), the second a string (in n). I'm a bit at loss with the definition of the corresponding dtype... ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From service at visa.com Sat Oct 28 17:10:27 2006 From: service at visa.com (service at visa.com) Date: Sat, 28 Oct 2006 17:10:27 -0400 Subject: VISA Update Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Sat Oct 28 20:08:53 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 28 Oct 2006 18:08:53 -0600 Subject: A recarray of arrays In-Reply-To: <200610281536.23189.pgmdevlist@gmail.com> References: <200610281536.23189.pgmdevlist@gmail.com> Message-ID: <4543F115.4030004@ieee.org> Pierre GM wrote: > Folks, > What is the easiest way to define a recarray of arrays ? > For example, I'd need something like that: > Given three arrays > >>>> x = N.arange(5) >>>> y = x+1 >>>> z = N.sqrt(x) >>>> > and a list of names: > >>>> n = ['x','y','z'] >>>> > Define a 3-record array with two fields: the first one being a ndarray (in x, > y or), the second a string (in n). > > I'm a bit at loss with the definition of the corresponding dtype... > I'm a little unclear one what you want. Here is a data-type whose first field is a name and whose second field is a 5-element array: dt = N.dtype([('name', 'S10'), ('data', float, (5,))]) Then: a = array([('x',x),('y',y),('z',z)],dt) gives array([('x', [0.0, 1.0, 2.0, 3.0, 4.0]), ('y', [1.0, 2.0, 3.0, 4.0, 5.0]), ('z', [0.0, 1.0, 1.4142135623730951, 1.7320508075688772, 2.0])], dtype=[('name', '|S10'), ('data', ' <4543A153.4030505@ieee.org> Message-ID: In article <4543A153.4030505 at ieee.org>, Travis Oliphant wrote: > Hmm.... I know that the code was changed at some point a few months ago > specifically to this behavior because of some concerns Perry, Chris > (people at STScI) had. Originally, field names came first, but we > changed it so they could set known attributes of a record array even if > they were also field names. > > This may be an unintentional side-effect. So, let's not just change > things again and create problems for them. Were any test cases generated for this? Changing the code did not break anything: if this is important, a test should probably be added. Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mforbes at phys.washington.edu Sat Oct 28 22:45:27 2006 From: mforbes at phys.washington.edu (Michael McNeil Forbes) Date: Sat, 28 Oct 2006 19:45:27 -0700 Subject: recarray.__setattr__ bug? References: <4543A153.4030505@ieee.org> <200610281530.22929.pgmdevlist@gmail.com> Message-ID: In article <200610281530.22929.pgmdevlist at gmail.com>, Pierre GM wrote: > So that I could have a field named 'shape', and modifying > r.shape would change the shape of the array, not the content of the field ? > > That makes sense, but isn't it a bit dangerous ? Shouldn't we have a list of > reserved keywords (dtype, shape...) that would raise an exception if used as > field names ? Since the field names can be used with regular arrays without any consequences, I think it would be bad to raise an exception with recarrays where there was no problem with regular arrays, and regular arrays should allow field names such as 'shape'. A note should be added to the recarray documentation, however, pointing out this pitfall. Perhaps a warning could be raised when a recarray is constructed with poor fieldname choices, but I would not raise an exception. In any case, I think is is dangerous to allow the fieldnames to hide standard array members. This would break the array interface for such recarrays causing many subtle problems for any algorithm that assumed the recarray to behave as a standard array: This code could be deeply buried. Instead, the user's code would behave 'strangely', but it is also the user's code that assigned bad fieldnames, and a warning would alert of this. Another concern that I found with the recarray interface is that the current implementation exhibits quite poor performance, but I am not sure how to fix this yet. Is there any documentation about the general design decisions behind recarrays? I think they could be very useful for simplifying code, but could use several improvements. Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Sun Oct 29 01:00:38 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sun, 29 Oct 2006 14:00:38 +0900 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <454259CF.9090608@noaa.gov> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> <454250DF.3060402@ieee.org> <454259CF.9090608@noaa.gov> Message-ID: <45443576.9030908@ar.media.kyoto-u.ac.jp> Christopher Barker wrote: > > Does this have anything to do with this pyaudio? > > http://people.csail.mit.edu/hubert/pyaudio/ > > -Chris > Not at all. I should have looked for pyaudio as a name on google :) Before coding my small package, I looked at other python bindings for audio, but either they were not cross platform, or not uptodate, or depended on too much external code. I didn't find this one, though. I think using something like portaudio just to be able to play or record data is a bit overkill: those libraries are supposed to be used for audio applications, with all the constraints: real time capabilities, mixing capabilities, being able to list soundcards, etc... My scope is really much simpler: I just want to be able to import audio files as numpy arrays, process them, and to listen to the result; it really just intend to be an equivalent of wavread, wavwrite and sound/soundsc of matlab. libsndfile is the defacto standard on linux (almost all audio applications on linux with IO needs use it), is really high quality, and is available on linux, windows and mac OSX. cheers, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Sun Oct 29 01:09:27 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sun, 29 Oct 2006 01:09:27 -0400 Subject: recarray.__setattr__ bug? In-Reply-To: References: <200610281530.22929.pgmdevlist@gmail.com> Message-ID: <200610290109.27473.pgmdevlist@gmail.com> On Saturday 28 October 2006 22:45, Michael McNeil Forbes wrote: > Since the field names can be used with regular arrays without any > consequences, I think it would be bad to raise an exception with > recarrays where there was no problem with regular arrays, and regular > arrays should allow field names such as 'shape'. I agree to a certain extent: as switching from regular arrays to recarrays is straightforward, any field can potentially can be accessed as an attribute and this can brings some serious surprises. If an exception is too restrictive, a warning should at least be raised (just as the new warnings about dividing by zeros) when an array is created with field names in a exclude list. As a starter, the exclude list could be most (all ?) of the basic attributes/methods of a regular ndarray, along with 'fields', 'mask'... > Another concern that I found with the recarray interface is that the > current implementation exhibits quite poor performance, but I am not > sure how to fix this yet. Well, there's a lot of __getattribute__ access: that tends to slow things down. But wouldn't checking whether the fields are in the 'exclude' list degrade the performance even more ? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Sun Oct 29 02:40:32 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sun, 29 Oct 2006 01:40:32 -0600 Subject: recarray.__setattr__ bug? In-Reply-To: References: <4543A153.4030505@ieee.org> Message-ID: <45445AF0.9010503@ieee.org> Michael McNeil Forbes wrote: > In article <4543A153.4030505 at ieee.org>, > Travis Oliphant wrote: > >> Hmm.... I know that the code was changed at some point a few months ago >> specifically to this behavior because of some concerns Perry, Chris >> (people at STScI) had. Originally, field names came first, but we >> changed it so they could set known attributes of a record array even if >> they were also field names. >> >> This may be an unintentional side-effect. So, let's not just change >> things again and create problems for them. >> > > Were any test cases generated for this? Changing the code did not break > anything: if this is important, a test should probably be added. > You are right. In fact there are lots of tests that need to be added. Stefan as been doing a great job of regression testing. I've committed some fixes to this particular issue that allows setattr to be used to set field names. Now 1) The standard setattr is tried first 2) If it fails, then the field-dictionary is checked to see if there is a field with that name. a) If not, then the error created by 1 is raised b) if the name is a field, then an attempt is made to set the field with that name 3) If the standard setattr succeeds then either i) a built-in attribute was correctly set or ii) a new attribute was created and attached to the dictionary. a) If the attribute is not in the fields, then return b) Otherwise, see if this name was in the .__dict__ before the call (if it's not then we just added it so delete it). i) If the deletion succeeds then set set the field ii If the deletion fails, then return the result of standard setattr setting. I've added a test for some of these behaviors. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Sun Oct 29 04:15:49 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Sun, 29 Oct 2006 10:15:49 +0100 Subject: Changes to bools under Numexpr In-Reply-To: <45420AAF.1080605@sympatico.ca> References: <4540A2B1.9080509@carabos.com> <4540FFDA.1020006@sympatico.ca> <4541B36E.7080403@carabos.com> <45420AAF.1080605@sympatico.ca> Message-ID: <45447145.1010100@carabos.com> En/na Colin J. Williams ha escrit:: > I'm afraid I'm still baffled. Are you saying that your proposal is > necessary > to preserve compatibility with Python versions before 2.3? Otherwise, it > appears > to introduce clutter with no clear benefit. > > I don't find Numexpr in NumPy, are you referring to a scipy module? From http://www.scipy.org/SciPyPackages/NumExpr: The scipy.sandbox.numexpr package supplies routines for the fast evaluation of array expressions elementwise by using a vector-based virtual machine. It's comparable to scipy.weave.blitz (in Weave), but doesn't require a separate compile step of C or C++ code. If your SciPy package doesn't include it, you can get a checkout from the repository:: $ svn co http://svn.scipy.org/svn/scipy/trunk/Lib/sandbox/numexpr Numexpr is not usable under Python <= 2.4 since it uses decorators (and maybe for other reasons), so my proposal clearly hasn't the intention on Python 2.3 compatibility. Instead, it is a simplification of Numexpr's type system by stricter boolean types. I suggest that you try Numexpr. It is a great piece of software. Cheers, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gerard.vermeulen at grenoble.cnrs.fr Sun Oct 29 04:49:26 2006 From: gerard.vermeulen at grenoble.cnrs.fr (Gerard Vermeulen) Date: Sun, 29 Oct 2006 10:49:26 +0100 Subject: ANN: PyQwt3D-0.1.2 Message-ID: <20061029104926.6388c04e.gerard.vermeulen@grenoble.cnrs.fr> What is PyQwt3D? - it is a set of Python bindings for the QwtPlot3D C++ class library which extends the Qt framework with widgets for 3D data visualization. PyQwt3D inherits the snappy feel from QwtPlot3D. The examples at http://pyqwt.sourceforge.net/pyqwt3d-examples.html show how easy it is to make a 3D plot and how to save a 3D plot to an image or an (E)PS/PDF file. - it requires and extends PyQt, a set of Python bindings for Qt. - it supports the use of PyQt, Qt, Qwt, the Numerical Python extensions (any combination of NumPy, numarray, and numarray) and optionally SciPy in a GUI Python application or in an interactive Python session. - it runs on POSIX, Mac OS X and Windows platforms (practically any platform supported by Qt and Python). New features: - support for Python-2.5 (requires PyQt and SIP snapshots or the future PyQt-3.17, PyQt-4.1 and SIP-4.5) - support for Qt-4.2.x and -4.1.x - support for NumPy-1.0, see http://numpy.scipy.org - PyQwt3D.OpenGL gives access to a small subset of the OpenGL API - EnrichmentDemo.py example demonstrates PyQwt3D.OpenGL The home page of PyQwt3D is http://pyqwt.sourceforge.net. PyQwt3D-0.1.2 requires: 1. Python-2.5.x,-2.4.x, or -2.3. 2. PyQt-4.0.x, or -3.16.x (or the future PyQt-4.1 or PyQt-3.17) 3. SIP-4.4.x (or the future SIP-4.5) 4. Qt-4.2.x, Qt-4.1.x, Qt-3.3.x, or -3.2.x. 5. QwtPlot3D-0.2.6. Have fun -- Gerard Vermeulen ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From hivktdnzyc at openheart.com Sun Oct 29 06:37:27 2006 From: hivktdnzyc at openheart.com (to manage) Date: Sun, 29 Oct 2006 20:37:27 +0900 Subject: series Message-ID: <001101c6fb4e$9ca66810$ceb30d3b@yourkbvfzpdt4n> Corporate is welfare Adaware secures business dangerous identity is is important at school home or office now can. Enough to manage worry about without added stress or protecting private company Well am relieve war against spy is ware requires is heavy defense protect corporate welfare am Adaware is secures. Corporate is welfare Adaware secures business dangerous identity is is important at school home or office now can. Freeskins free Support Education Lists the Press us Header our a? About without am added stress protecting in private company Well relieve war against is spy ware of requires heavy defense in protect corporate welfare a Adaware secures business dangerous is identity. Foundation Incread Quick Center Forums or What News aim new is Blog Software in Reuters oct Spoofing am bug Found am ie or Cnet News Footer links Useprivacy. Enough to manage worry about without added stress protecting private company Well relieve war against spy ware requires heavy defense a protect. In finding right here see theproduct Chart or Project eco Clean up malicious threats a growing series standalone toolsread is More ser a Awards takes? Our Mission yourself is family are top is priority provides you in with safety security privacy giving real peace. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Mission..gif Type: image/gif Size: 8552 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cjw at sympatico.ca Sun Oct 29 06:55:20 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Sun, 29 Oct 2006 06:55:20 -0500 Subject: Changes to bools under Numexpr In-Reply-To: <45447145.1010100@carabos.com> References: <4540A2B1.9080509@carabos.com> <4540FFDA.1020006@sympatico.ca> <4541B36E.7080403@carabos.com> <45420AAF.1080605@sympatico.ca> <45447145.1010100@carabos.com> Message-ID: <454496A8.1090002@sympatico.ca> Ivan Vilata i Balaguer wrote: > En/na Colin J. Williams ha escrit:: > > >> I'm afraid I'm still baffled. Are you saying that your proposal is >> necessary >> to preserve compatibility with Python versions before 2.3? Otherwise, it >> appears >> to introduce clutter with no clear benefit. >> >> I don't find Numexpr in NumPy, are you referring to a scipy module? >> > > From http://www.scipy.org/SciPyPackages/NumExpr: > > The scipy.sandbox.numexpr package supplies routines for the fast > evaluation of array expressions elementwise by using a vector-based > virtual machine. It's comparable to scipy.weave.blitz (in Weave), > but doesn't require a separate compile step of C or C++ code. > > If your SciPy package doesn't include it, you can get a checkout from > the repository:: > > $ svn co http://svn.scipy.org/svn/scipy/trunk/Lib/sandbox/numexpr > > Numexpr is not usable under Python <= 2.4 since it uses decorators (and > maybe for other reasons), so my proposal clearly hasn't the intention on > Python 2.3 compatibility. Instead, it is a simplification of Numexpr's > type system by stricter boolean types. > > I suggest that you try Numexpr. It is a great piece of software. Cheers, > > :: > > Ivan Vilata i Balaguer >qo< http://www.carabos.com/ > C??rabos Coop. V. V V Enjoy Data > > Ivan, Many thanks for this. It's a long time since I looked at SciPy, though I will look at Numexpr. Currently, I'm focusing on Numpy. Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Sun Oct 29 10:52:02 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sun, 29 Oct 2006 10:52:02 -0500 Subject: recarray.__setattr__ bug? In-Reply-To: <45445AF0.9010503@ieee.org> References: <45445AF0.9010503@ieee.org> Message-ID: <200610291052.03425.pgmdevlist@gmail.com> On Sunday 29 October 2006 02:40, Travis Oliphant wrote: > I've committed some fixes to this particular issue that allows setattr > to be used to set field names. I just started playing with the new recarray: that works great ! Travis, Thanks a lot ! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tom.denniston at alum.dartmouth.org Sun Oct 29 17:53:37 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Sun, 29 Oct 2006 16:53:37 -0600 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. Message-ID: I recently upgraded to numpy 1.0 from 1.0b5. I noticed that numpy.argmaxbehavior is very strange on object arrays. See below: (Pdb) numpy.__version__ '1.0' (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object)) 0 (Pdb) numpy.argmax(numpy.array([2, 3], dtype=int)) 1 (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object), axis=0) 0 I would expect the argmax to behave the same on the dtype=int and dtype=object examples but it doesn't. Am I missing some subtelty or is this just a bug? 1.0 is the most recent version, right? --Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cjw at sympatico.ca Sun Oct 29 18:22:13 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Sun, 29 Oct 2006 18:22:13 -0500 Subject: matrixdef.py Message-ID: <454537A5.1030202@sympatico.ca> Line 71 has: data.view(subtype) This appears to involve a call to __array_finalize__. Is this an unconditional call? If not, what are the conditions? Why not use __init__ to handle these things, since it is always called after the __new__? Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sun Oct 29 19:05:37 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 29 Oct 2006 17:05:37 -0700 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: On 10/29/06, Tom Denniston wrote: > > I recently upgraded to numpy 1.0 from 1.0b5. I noticed that numpy.argmaxbehavior is very strange on object arrays. See below: > > (Pdb) numpy.__version__ > '1.0' > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object)) > 0 > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=int)) > 1 > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object), axis=0) > 0 > > > I would expect the argmax to behave the same on the dtype=int and > dtype=object examples but it doesn't. Am I missing some subtelty or is this > just a bug? 1.0 is the most recent version, right? > Suppose In [22]: array([1,[2,3]], dtype=object) Out[22]: array([1, [2, 3]], dtype=object) How would you compare the elements? In [27]: 2 < [0,0] Out[27]: True In [28]: [0,0] > 2 Out[28]: True Compares memory locations? In [28]: [2] < [0,0] Out[28]: False Lexical ordering? I don't know how python interprets these things. That said, I suspect your example should behave better, but it might give strange results sometimes anyway. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tom.denniston at alum.dartmouth.org Sun Oct 29 20:10:05 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Sun, 29 Oct 2006 19:10:05 -0600 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: Oh. My mistake. I thought I had an array of 2 objects which were ints. I actually had an array of one list of 2 ints. It works properly if I construct the array properly. In [14]: a = numpy.empty((2), dtype=object) In [15]: a[0:]=[2,3] In [16]: a Out[16]: array([2, 3], dtype=object) In [17]: a.shape Out[17]: (2,) In [18]: a[0] Out[18]: 2 In [19]: a[1] Out[19]: 3 In [20]: numpy.argmax(a) Out[20]: 1 On 10/29/06, Charles R Harris wrote: > > > > On 10/29/06, Tom Denniston wrote: > > > > I recently upgraded to numpy 1.0 from 1.0b5. I noticed that > > numpy.argmax behavior is very strange on object arrays. See below: > > > > (Pdb) numpy.__version__ > > '1.0' > > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object)) > > 0 > > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=int)) > > 1 > > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object), axis=0) > > 0 > > > > > > I would expect the argmax to behave the same on the dtype=int and > > dtype=object examples but it doesn't. Am I missing some subtelty or is this > > just a bug? 1.0 is the most recent version, right? > > > > Suppose > > In [22]: array([1,[2,3]], dtype=object) > Out[22]: array([1, [2, 3]], dtype=object) > > How would you compare the elements? > > In [27]: 2 < [0,0] > Out[27]: True > > In [28]: [0,0] > 2 > Out[28]: True > > Compares memory locations? > > In [28]: [2] < [0,0] > Out[28]: False > > Lexical ordering? > > > I don't know how python interprets these things. That said, I suspect your > example should behave better, but it might give strange results sometimes > anyway. > > Chuck > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Sun Oct 29 21:17:34 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 29 Oct 2006 19:17:34 -0700 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: On 10/29/06, Tom Denniston wrote: > > Oh. My mistake. I thought I had an array of 2 objects which were ints. I > actually had an array of one list of 2 ints. It works properly if I > construct the array properly. > I think there is actually a bug here: In [61]: sort(array([3,2], dtype=object)) Out[61]: array([2, 3], dtype=object) In [62]: argmax(array([2,3], dtype=object)) Out[62]: 0 See, the sort works fine. I suspect argmax is using the wrong comparison function. I was just pointing out that sometimes it is hard to know what is going on with objects. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david at ar.media.kyoto-u.ac.jp Mon Oct 30 03:26:36 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 30 Oct 2006 17:26:36 +0900 Subject: A few questions about numpy.distutils Message-ID: <4545B73C.8040005@ar.media.kyoto-u.ac.jp> Hi there, I would like to improve a bit the packaging of pyaudio, and for that, I need to do the following when configuring my package: - to be able to retrieve a header file (sndfile.h). I need the full pathname, because I need to read its content - to be able to retrieve a library file. I need the full pathname, including the library prefix and suffix/extension, because I need to load it with ctypes. I've shamelessly copied and pasted code from numpy.distutils.system_info, and I manage to retrieve informations for my library, and to retrieve the full pathname for sndfile.h, but I don't know how to get the full shared library file. For now, my sndfile_info looks like that: class sndfile_info(system_info): #variables to override section = 'sndfile' notfounderror = SndfileNotFoundError libname = 'sndfile' header = 'sndfile.h' def __init__(self): system_info.__init__(self) def calc_info(self): """ Compute the informations of the library """ # Look for the shared library sndfile_libs = self.get_libs('sndfile_libs', self.libname) lib_dirs = self.get_lib_dirs() for i in lib_dirs: tmp = self.check_libs(i, sndfile_libs) if tmp is not None: info = tmp break else: return # Look for the header file include_dirs = self.get_include_dirs() inc_dir = None for d in include_dirs: p = self.combine_paths(d,self.header) if p: inc_dir = os.path.dirname(p[0]) headername = os.path.abspath(p[0]) break if inc_dir is not None: dict_append(info, include_dirs=[inc_dir], headername = headername) self.set_info(**info) return Any help to get the full library name would be appreciated (I cannot use find_library from ctypes.util, because I have no guarantee that it will return the same one than distutils), Thanks, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From yves.frederix at gmail.com Mon Oct 30 05:47:57 2006 From: yves.frederix at gmail.com (Yves) Date: Mon, 30 Oct 2006 11:47:57 +0100 Subject: How to check for nan in an array? Message-ID: <20061030104757.GA6967@kotnet.org> Hi, Is there a way of checking for nan's in an array, as comparing to N.nan doesn't seem to work. In [1]: import numpy as N In [2]: a = N.asarray(0.)/0 Warning: invalid value encountered in divide In [3]: a Out[3]: nan In [4]: a==N.nan Out[4]: False In [5]: b = a.copy() In [6]: a==b Out[6]: False In [7]: b Out[7]: nan In [8]: N.__version__ Out[8]: '1.0.dev3390' Many thanks, YVES ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From yves.frederix at gmail.com Mon Oct 30 05:53:45 2006 From: yves.frederix at gmail.com (Yves) Date: Mon, 30 Oct 2006 11:53:45 +0100 Subject: How to check for nan in an array? In-Reply-To: <20061030104757.GA6967@kotnet.org> References: <20061030104757.GA6967@kotnet.org> Message-ID: <20061030105345.GA7117@kotnet.org> Forget it. N.isnan() does the job. YVES On Mon, Oct 30, 2006 at 11:47:57AM +0100, Yves wrote: > Hi, > > Is there a way of checking for nan's in an array, as comparing to N.nan > doesn't seem to work. > > In [1]: import numpy as N > > In [2]: a = N.asarray(0.)/0 > Warning: invalid value encountered in divide > > In [3]: a > Out[3]: nan > > In [4]: a==N.nan > Out[4]: False > > In [5]: b = a.copy() > > In [6]: a==b > Out[6]: False > > In [7]: b > Out[7]: nan > > In [8]: N.__version__ > Out[8]: '1.0.dev3390' > > > Many thanks, > YVES > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From eddkrne at pc119.net Mon Oct 30 06:16:25 2006 From: eddkrne at pc119.net (GMT Hours) Date: Mon, 30 Oct 2006 12:16:25 +0100 Subject: porting Message-ID: <001001c6fc14$d8598a10$bc0a53d9@SS> Timestamp stack trace etc so too in However am happen before am fixed in are correctly? Its displayed n v notably par am Consoles im getting or two know help much figuring problem thought let know Quote Error am exception returned failure a nsresult is location frame data getvisible latter appears am switching after something havent yet is figured! Pri msg in Quote asqueella is Russia Moscow Nice thinks in that logged though worldquot is pm wrote am flaws uploaded update which allows these together fri updatexml or pointing Mass Installer am note Sorry that is fd dt am wow sweet thanks do or think accesskey dropdown its displayed n v notably a par Consoles im getting two know is help much figuring problem or thought let am know Quote. Faq dont like a default layout in simply right click a select Customize have am your way please keyboard shortcut displaying wont happen Either use of keyconfig or in Developer adding shortcut id of get more detailed timestamp stack trace etc so too However am happen am before fixed are of correctly am look tell me where agree colored messages Just mrtechs modify them in. Trace etc so too However happen is before am fixed are correctly look tell in me or where agree colored a messages! Selected eg selective copying deleted reopen Complete easier porting single patches dialog is visible least a out a Included Winstripe themes appearance of available four additional languages German flickering in gone filtering loading Categories applied agree in. Single patches dialog visible least out Included Winstripe in themes appearance available four additional in languages German flickering gone filtering loading Categories applied a agree a refer such a updown preview handler occasional funny up down Patch didnt official Pinstripe rotates a mode features Collapse is recent or one Quick access a Copy is clipboard of Lang related a Evaluating expression twice produces am results remains am changing. Do think accesskey dropdown a its is displayed n v notably par am. Slovak Czech locales thanks Infoman or Leszek is teo or yczkowski Rastislav Huboan Michal Gebauer Claiming or Flock button open sidebar respected currently Trunk Portuguese French Teboga Menet from is blocked domains is slip when opening possible a several consoles is rare of break Xpcom were Content Ported Lithuanian locale Edgaras Januauskas ignore until closed a. Needed am about replacing am original see a Testing in Version a Revisions or Jumps correct number am an external of editor requires form is issue introduced! Logo Copyright Rights a Reserved Privacy Policy log Register my Posts Watched Topics Login Rules or Forums Knowledge Base submit news. Default layout is simply is right click select Customize have your way please keyboard shortcut displaying wont happen Either use keyconfig or or Developer adding a shortcut id get more of. Changed accesskeys is Context of gt Evaluate problems hopefully hardly any or never Initial is Last edited sat times or total? Huboan Michal Gebauer Claiming a Flock button is open sidebar respected currently Trunk Portuguese in French a Teboga Menet from blocked domains slip. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: though..gif Type: image/gif Size: 7941 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From winkliesrp at oyakeuro.com Mon Oct 30 07:19:23 2006 From: winkliesrp at oyakeuro.com (reserved. are) Date: Mon, 30 Oct 2006 13:19:23 +0100 Subject: our Message-ID: <000801c6fc1d$a2195c60$17bdb651@b00df3904d6014> Jewel Quest Sandscript Classic Sudoku Express Bejeweled or Tumblebugs see Classicsgo a retro am Atari Hasbro Signin Scrabble Monopoly Yahtzee Centipede is Favorite. Hotties at Bellagios upscale is nightclub Book or entry a Hiro Late is night action is this exotic club the hip Maritime Hotel Opium Smoke line in South a Beachs ultimate garden a Tanned a beauties flock Halloween Take turn around track Mario a Andrettis racing of school Make tracks or across Mojave our Hummer Glide over gators wet wild Everglades airboat ride sideways in. Travelsite Welcome Gamescom games plus community gaming forums friends gamescom llc a Aolgames gdc America or ecc Vmware virtual in machine of pc Network Vmtn Forumsuser Groupsvmtn Store in Nonstore am Orders in Cart Pricing Stor a ffb Faqs. Music News Finance Estate Research Learn Shopping is Business Television Travelsite Welcome of Gamescom games plus community gaming forums friends gamescom llc Aolgames gdc America ecc Vmware a virtual machine pc Network a Vmtn Forumsuser Groupsvmtn Store Nonstore Orders Cart Pricing of Stor ffb in Faqs am Find Reseller in Company or Account Product Licensing Sign Register Logout of Help advanced Homeintro Interfaces Homeserver of. Wild Everglades airboat ride sideways in in Wine Country Have minute speedboat Manhattan Scope! Mexico Antigua Aruba Atlanta Bahamas Bahama Island Barbados a Bermuda Boston Cabo Lucaslos Cabos Cancun Cayman in Islands Grand Chicago Cozumel Daytona Beach Disney of World Orlando Dominican rep Puerto a Plata Punta Cana Santo of Domingo Fort Lauderdale of big Kauai Maui Oahu is Honolulu Houston Ixtapa Jamaica in all! Diner is Dash is Restaurant Rescue Defender of Gold Miner Fish Tycoon Panda in Craze Board two Kind Solitaire Mahjong Escape of Ancient China is Mahjomino Aloha Tripeaks am Casino Poker pop Spanish Poker Wsop or Texas Holdem Limit Blackjack Three Download Luxor Scuba of Jewel. Yahtzee Centipede in Favorite Gamesin Expedition mission recover is lost artifacts Titanic is trial check out popular Vegasfish Video. Company Account of Product Licensing Sign in Register or Logout Help advanced Homeintro Interfaces Homeserver am Test Homedata Center Desktop Paperstech nical Success Forumsyour Customer Alliance Consulting vac Signinfind Usabout Homenews Investor Email Page of Print Appliances or Steps Compare Browser Resources Support in Legal or Requests in Usage. Mediocre next or levelnew own private some matching fun Hidden Titanic Mystery Files Prime Suspects am Rumble Cube am Deluxe Tropix Plantasia zen Scrabble Blast Travels Texttwist of Flip Words Arcade Diner! Key in West a via eyw Lake am Tahoe is London Mazatlan Myrtle Orleans City Paris Phoenix Playa del Carmen Rico Vallarta Reno. Many things look when am player use it can be real hassle scanning through loads in small ones that fit needs am come want Buyer Guide has made. Angeles san Francisco Nightlife a Adventure Travel Special Events Nightclubs Stuff Magazine Home is get There Stay am Party. Maya in Rome Italy of Antonio Diego am Scottsdale Arizona Seattle st a Croix Usvi st of Lucia Castries Vieux. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: scanning.gif Type: image/gif Size: 7791 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From joris at ster.kuleuven.be Mon Oct 30 07:53:32 2006 From: joris at ster.kuleuven.be (Joris De Ridder) Date: Mon, 30 Oct 2006 13:53:32 +0100 Subject: astype() behaviour In-Reply-To: <200610271901.39809.faltet@carabos.com> References: <200610271758.54379.joris@ster.kuleuven.be> <200610271901.39809.faltet@carabos.com> Message-ID: <200610301353.32926.joris@ster.kuleuven.be> On Friday 27 October 2006 19:01, Francesc Altet wrote: [FA]: A Divendres 27 Octubre 2006 17:58, Joris De Ridder va escriure: [FA]: > Hi, [FA]: > [FA]: > Is the following behaviour of astype() intentional in NumPy 1.0? [FA]: > [FA]: > >>> x = array([1,2,3]) [FA]: > >>> x.astype(None) [FA]: > [FA]: > array([ 1., 2., 3.]) [FA]: > [FA]: > That is, the int32 is converted to float64. [FA]: [FA]: Yes, I think the behaviour is intended. This is because 'float64' is the [FA]: default type in NumPy from some months ago (before the default was 'int_') OK, updated the astype() example in the www.scipy.org/Numpy_Example_List . J. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From joris at ster.kuleuven.be Mon Oct 30 08:58:39 2006 From: joris at ster.kuleuven.be (Joris De Ridder) Date: Mon, 30 Oct 2006 14:58:39 +0100 Subject: dtype() examples that used to work In-Reply-To: <200610271906.14099.faltet@carabos.com> References: <200610271813.21500.joris@ster.kuleuven.be> <200610271906.14099.faltet@carabos.com> Message-ID: <200610301458.39353.joris@ster.kuleuven.be> On Friday 27 October 2006 19:06, Francesc Altet wrote: [FA]: for example: [FA]: [FA]: In [67]: dtype([('f1', int16)]) [FA]: Out[67]: dtype([('f1', ' Hi, I have a script that crashes, but only if it runs over 9~10 hours, with the following backtrace from gdb. The script uses PyMC, and repeatedly calls (> 1000000) likelihood functions written in fortran and wrapped with f2py. Numpy: 1.0.dev3327 Python: 2.4.3 Does this backtrace give enough info to track the problem or do the gurus need more ? Thanks, David *** glibc detected *** free(): invalid pointer: 0x00002aaaac1257e0 *** Program received signal SIGABRT, Aborted. [Switching to Thread 46912504440528 (LWP 25269)] 0x00002aaaab09011d in raise () from /lib/libc.so.6 (gdb) backtrace #0 0x00002aaaab09011d in raise () from /lib/libc.so.6 #1 0x00002aaaab09184e in abort () from /lib/libc.so.6 #2 0x00002aaaab0c4e41 in __fsetlocking () from /lib/libc.so.6 #3 0x00002aaaab0ca90e in malloc_usable_size () from /lib/libc.so.6 #4 0x00002aaaab0cac56 in free () from /lib/libc.so.6 #5 0x00002aaaabff7770 in PyArray_FromArray (arr=0x1569500, newtype=0x2aaaac1257e0, flags=0) at arrayobject.c:7804 #6 0x00002aaaabfece56 in PyArray_FromAny (op=0x1569500, newtype=0x0, min_depth=0, max_depth=0, flags=0, context=0x0) at arrayobject.c:8257 #7 0x00002aaaabff40b1 in PyArray_MultiIterNew (n=2) at arrayobject.c:10253 #8 0x00002aaaabff44bc in _broadcast_cast (out=0x62b5, in=0x6, castfunc=0x2aaaabfbf5a0 , iswap=-1, oswap=6) at arrayobject.c:7445 #9 0x00002aaaabffe301 in PyArray_CastToType (mp=0x156dca0, at=, fortran_=0) at arrayobject.c:7344 #10 0x00002aaaabffe785 in PyArray_FromScalar (scalar=0x1573b30, outcode=0x2aaaac1257e0) at scalartypes.inc.src:219 #11 0x00002aaaabfecff5 in PyArray_FromAny (op=0x1573b30, newtype=0x2aaaac1257e0, min_depth=0, max_depth=, flags=0, context=0x0) at arrayobject.c:8260 #12 0x00002aaab6038b7b in array_from_pyobj (type_num=11, dims=0x7fffff8f6200, rank=1, intent=, obj=0x1573b30) at build/src.linux-x86_64-2.4/fortranobject.c:653 #13 0x00002aaab6034aa9 in f2py_rout_flib_beta ( capi_self=, capi_args=, capi_keywds=, f2py_func=0x2aaab603e830 ) at build/src.linux-x86_64-2.4/PyMC/flibmodule.c:2601 #14 0x0000000000414490 in PyObject_Call () #15 0x0000000000475de5 in PyEval_EvalFrame () #16 0x00000000004bdf69 in PyDescr_NewGetSet () #17 0x00000000004143eb in PyIter_Next () #18 0x000000000046ba53 in _PyUnicodeUCS4_IsNumeric () #19 0x0000000000477ab1 in PyEval_EvalFrame () #20 0x00000000004783ff in PyEval_EvalCodeEx () #21 0x000000000047699b in PyEval_EvalFrame () #22 0x0000000000476ab6 in PyEval_EvalFrame () #23 0x0000000000476ab6 in PyEval_EvalFrame () #24 0x00000000004783ff in PyEval_EvalCodeEx () #25 0x000000000047699b in PyEval_EvalFrame () #26 0x00000000004783ff in PyEval_EvalCodeEx () #27 0x000000000047699b in PyEval_EvalFrame () #28 0x00000000004783ff in PyEval_EvalCodeEx () #29 0x000000000047699b in PyEval_EvalFrame () #30 0x00000000004783ff in PyEval_EvalCodeEx () #31 0x0000000000478512 in PyEval_EvalCode () #32 0x000000000049c222 in PyRun_FileExFlags () #33 0x000000000049c4ae in PyRun_SimpleFileExFlags () #34 0x0000000000410a80 in Py_Main () #35 0x00002aaaab07d49b in __libc_start_main () from /lib/libc.so.6 #36 0x000000000040ffba in _start () -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tom.denniston at alum.dartmouth.org Mon Oct 30 11:32:20 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Mon, 30 Oct 2006 10:32:20 -0600 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: Yes I definately agree about the latter! On 10/29/06, Charles R Harris wrote: > > > > On 10/29/06, Tom Denniston wrote: > > > > Oh. My mistake. I thought I had an array of 2 objects which were ints. > > I actually had an array of one list of 2 ints. It works properly if I > > construct the array properly. > > > > I think there is actually a bug here: > > In [61]: sort(array([3,2], dtype=object)) > Out[61]: array([2, 3], dtype=object) > > In [62]: argmax(array([2,3], dtype=object)) > Out[62]: 0 > > See, the sort works fine. I suspect argmax is using the wrong comparison > function. I was just pointing out that sometimes it is hard to know what is > going on with objects. > > > Chuck > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From faltet at carabos.com Mon Oct 30 11:49:03 2006 From: faltet at carabos.com (Francesc Altet) Date: Mon, 30 Oct 2006 17:49:03 +0100 Subject: dtype() examples that used to work In-Reply-To: <200610301458.39353.joris@ster.kuleuven.be> References: <200610271813.21500.joris@ster.kuleuven.be> <200610271906.14099.faltet@carabos.com> <200610301458.39353.joris@ster.kuleuven.be> Message-ID: <1162226943.3965.7.camel@localhost.localdomain> El dl 30 de 10 del 2006 a les 14:58 +0100, en/na Joris De Ridder va escriure: > IMO, record arrays seem powerful, but also intimidating at a first glance. Agreed, specially if you start to nest datatypes. > I think many didactical examples will help getting them into common use. > I made some effort to get updated examples in the Numpy Example List: > > www.scipy.org/Numpy_Example_List#dtype > www.scipy.org/Numpy_Example_List#array > > Could people who already have some experience with it, have a look at > them and give me their opinion? Looks good. I've taken the freedom to add some examples of nested types and recarrays. -- Francesc Altet | Be careful about using the following code -- Carabos Coop. V. | I've only proven that it works, www.carabos.com | I haven't tested it. -- Donald Knuth ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jmakem01 at qub.ac.uk Mon Oct 30 12:14:31 2006 From: jmakem01 at qub.ac.uk (Jonathan Makem) Date: Mon, 30 Oct 2006 17:14:31 -0000 Subject: Abaqus Python Message-ID: Hi, I work with Abaqus 6.6.1 finite element software. To access the results from a simulation, the Abaqus scripting interface is used in Python by importing OdbAccess modules. These modules can only be accessed using the version of Python that is installed with Abaqus. However, I cannot install numpy for this version of Python. And I can't use Python 2.4 or 2.5 because then I can't use the odbAcces modules. Is it possible to use numpy with the Abaqus version of Python? If so, how? Regards, Jonny. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Mon Oct 30 14:03:48 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 30 Oct 2006 12:03:48 -0700 Subject: Conversion about getting array interface into Python Message-ID: <45464C94.4080900@ieee.org> If anybody has a desire to see the array interface into Python, please help by voicing an opinion on python-dev in the discussion about adding data-type objects to Python. There are a few prominent people who don't get why applications would need to share data-type information about memory areas. I need help giving reasons why. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 14:18:52 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 12:18:52 -0700 Subject: glibc invalid pointer error In-Reply-To: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> Message-ID: On 10/30/06, David Huard wrote: > Hi, > I have a script that crashes, but only if it runs over 9~10 hours, with the > following backtrace from gdb. The script uses PyMC, and repeatedly calls (> > 1000000) likelihood functions written in fortran and wrapped with f2py. > Numpy: 1.0.dev3327 > Python: 2.4.3 This sounds awfully reminiscent of the bug I recently mentioned: http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 We left a fresh run over the weekend, but my office mate is currently out of the office and his terminal is locked, so I don't know what the result is. I'll report shortly: we followed Travis' instructions and ran with a fresh SVN build which includes the extra warnings he added to the dealloc routines. You may want to try the same advice, perhaps with information from both of us the gurus may zero in on the problem, if indeed it is the same. Note that I'm not positive it's the same problem, and our backtraces aren't quite the same. But the rest of the scenario is similar: low-level memory crash from glibc, very long run is needed to fire the bug, potentially millions of calls to both numpy and to f2py-wrapped in-house libraries. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From xqmlkttvz at pason.com Mon Oct 30 14:28:25 2006 From: xqmlkttvz at pason.com (programs) Date: Mon, 30 Oct 2006 14:28:25 -0500 Subject: Subaru Discount Message-ID: <000c01c6fc59$91bdc5e0$a12cffcf@homem9afjgt740> Mark is overrated Joel is Stretch Welcome why Many taken in pine. Simply is exactly what minimum skill practice in every is far a should camp am from water sources pitch tent how build in fire if yo first place Rather foremost attitude ethic respecting caring doing part. Spread Have question Want work a for Membership Benefits Overview am Principles Traveling Trainers Team Bios Program or Details Events Requests East West Subaru Discount State is Advocate a Partner Front Country Tools Teaching Awareness Workshops of Trainer Courses Master Educator Course Schedule Locate in Resources Roster Submission Corporate or Nonprofit Education Outfitter Guide of Service Retailer Parks Agencies Staff Contact us History Media Room. Online Today Learn about or Peak and other innovative is Trace programs get the training you is need find out where can it over partners helping to spread Have am question Want work for Membership a Benefits is Overview am Principles Traveling Trainers Team is Bios or Program Details Events Requests or East West Subaru Discount State Advocate Partner Front Country Tools of Teaching or Awareness Workshops is. Enjoy am are critical risk am Also risk continued access wildlands as. Board of Directors Archived lnt Site or Leaving your or mark am is of overrated Joel is Stretch Welcome why of Many taken pine cone or of rock veered off trail dodge mud puddles in gotten in too close wildlife tossed or an apple of core into woods While these in actions may seem or harmless at time until we reduce our impact quality outdoor! Resources Roster is Submission a Corporate Nonprofit Education of Outfitter Guide or Service a Retailer am Parks Agencies Staff Contact us History Media Room Board of Directors in Archived lnt Site of Leaving in your mark is overrated in Joel Stretch Welcome why Many taken of pine a cone or am rock veered. Rock veered of off trail dodge of mud puddles gotten too close! Time is until we reduce our impact quality in outdoor enjoy are critical risk is Also risk continued access wildlands as land management sometimes is take is action protect they manage Unless catches up of with behavior in all outdoors unchanged is possible by presence Solution a Center am Ethics believes. Durable Surfaces a Dispose Waste Properly Minimize am Campfire Respect be Visitors Click of here am detailed am Seven Give back? Can it over is partners helping to spread Have question Want a work for a Membership is Benefits Overview Principles Traveling am Trainers Team is. Submission Corporate Nonprofit Education Outfitter Guide a Service Retailer Parks in Agencies in. Far should camp from water sources pitch a tent how build fire if yo in first place Rather a. That impacts or widespread causes complex simple change am through a research one or person of time set rules nor simply or exactly am what. Courses Master Educator Course Schedule Locate Resources Roster Submission Corporate Nonprofit. Need find out where can of it over partners helping or to or spread Have am question Want work for! Quality am outdoor enjoy are critical risk of Also risk am continued access wildlands am as. From water sources pitch tent how build fire if yo first place Rather foremost attitude is ethic. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: protect.gif Type: image/gif Size: 7946 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Mon Oct 30 14:57:43 2006 From: david.huard at gmail.com (David Huard) Date: Mon, 30 Oct 2006 14:57:43 -0500 Subject: glibc invalid pointer error In-Reply-To: References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> Message-ID: <91cf711d0610301157o19f2ba70o214ec1d08adfc406@mail.gmail.com> Ok, I'll update numpy and give it another try tonight. Regards, David 2006/10/30, Fernando Perez : > > On 10/30/06, David Huard wrote: > > Hi, > > I have a script that crashes, but only if it runs over 9~10 hours, with > the > > following backtrace from gdb. The script uses PyMC, and repeatedly calls > (> > > 1000000) likelihood functions written in fortran and wrapped with f2py. > > Numpy: 1.0.dev3327 > > Python: 2.4.3 > > This sounds awfully reminiscent of the bug I recently mentioned: > > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > > We left a fresh run over the weekend, but my office mate is currently > out of the office and his terminal is locked, so I don't know what the > result is. I'll report shortly: we followed Travis' instructions and > ran with a fresh SVN build which includes the extra warnings he added > to the dealloc routines. You may want to try the same advice, perhaps > with information from both of us the gurus may zero in on the problem, > if indeed it is the same. > > Note that I'm not positive it's the same problem, and our backtraces > aren't quite the same. But the rest of the scenario is similar: > low-level memory crash from glibc, very long run is needed to fire the > bug, potentially millions of calls to both numpy and to f2py-wrapped > in-house libraries. > > Cheers, > > f > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez.net at gmail.com Mon Oct 30 15:08:00 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 13:08:00 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: <453D4AD1.5060508@ieee.org> References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/23/06, Travis Oliphant wrote: > I've placed them in SVN (r3384): > > arraydescr_dealloc needs to do something like. > > if (self->fields == Py_None) { > print something > incref(self) > return; > } Here is some more info. We left a long-running job over the weekend with the prints you suggested. Oddly, something happened at the OS level which killed our SSH connection to that machine, but the above numpy dealloc() warning never printed (we logged this). What did happen is that the refcount you suggested we print: sys.getrefcount(numpy.dtype('float')) eventually seems to have wrapped around and gone negative. I'm attaching the log file with those print statements, the key point is that this happens eventually: PSVD Iteration 19 Ref count 1989827662 bar 444 PSVD Iteration 0 Ref count 2021353399 PSVD Iteration 1 Ref count 2143386207 PSVD Iteration 2 Ref count -2001245193 PSVD Iteration 3 Ref count -1915816437 PSVD Iteration 4 Ref count -1902698473 That refcount is for dtype('float') as indicated above. Is it not a problem that this particular refcount goes negative? Eventually it may continue increasing and hit a zero, point at which I imagine that the bad dealloc will occur. Are refcounts stored in signed 32-bit ints? Why? I'd have naively expected them to be stored in unsigned longs to avoid wraparound problems, but maybe I'm completely missing the real problem here. We've started another run to see if we can get the actual crash to happen, will report. Cheers, f -------------- next part -------------- A non-text attachment was scrubbed... Name: numpy_report Type: application/octet-stream Size: 3307 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Mon Oct 30 16:08:23 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 30 Oct 2006 18:08:23 -0300 Subject: [Numpy-discussion] Conversion about getting array interface into Python In-Reply-To: <45464C94.4080900@ieee.org> References: <45464C94.4080900@ieee.org> Message-ID: On 10/30/06, Travis Oliphant wrote: > > If anybody has a desire to see the array interface into Python, please > help by voicing an opinion on python-dev in the discussion about adding > data-type objects to Python. There are a few prominent people who > don't get why applications would need to share data-type information > about memory areas. I need help giving reasons why. > > -Travis Python-Dev is sometimes a hard place to get into. I remember posting some proposals or even bugs (considered bugs for me and others) and getting rather crude responses. Travis said "a few prominent people...". I consider Travis a *very* prominent people in Python world. His **terrific** contribution to NumPy reveals a really smart, always-looking-ahead way of doing things. However, I've seen before strong and disparate opposition to his proposals in Python-Dev. Perhaps the reason for this is simple: few Python core developers are involved in scientific computing and do not have a clear idea of what it is needed for this. I really believe that NumPy/Scipy community should try to raise his voice on Python-Dev. Many NumPy/Scipy users/developers really want to run high-performance Python code. Python is being used in supercomputers, some applications taking advantage of Python (SPaSM) have even won the Gordon Bell Performance Prize. An 25 Tflop/s application involving Python programing language is really a good example of what can be achieved with Python and compiled code interaction. In short, I fully support Travis in his initiative to standardize access to low level binary data, and encourage others like me who really want this to post to Python-Dev. From my part, I will try to post my reasons in connection with my (small) experience developing MPI for Python. Regars, -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From dalcinl at gmail.com Mon Oct 30 16:15:28 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 30 Oct 2006 18:15:28 -0300 Subject: Abaqus Python In-Reply-To: References: Message-ID: On 10/30/06, Jonathan Makem wrote: > > Hi, > > I work with Abaqus 6.6.1 finite element software. To access the results > from a simulation, the Abaqus scripting interface is used in Python by > importing OdbAccess modules. These modules can only be accessed using the > version of Python that is installed with Abaqus. However, I cannot install > numpy for this version of Python. And I can't use Python 2.4 or 2.5 because > then I can't use the odbAcces modules. Is it possible to use numpy with the > Abaqus version of Python? If so, how? > Jonny. Could you specify which version of Python is used with ABAQUS? can you launch the ABAQUS-provided Python interpreter and do: >>> import sys >>> print sys.version Perhaps the 'odbAcces' module is a module provided by ABAQUS? Which platform are you using? Linux? Windows? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Mon Oct 30 16:54:10 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 30 Oct 2006 14:54:10 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Fernando Perez wrote: > > On 10/23/06, Travis Oliphant wrote: > > > I've placed them in SVN (r3384): > > > > arraydescr_dealloc needs to do something like. > > > > if (self->fields == Py_None) { > > print something > > incref(self) > > return; > > } > > Here is some more info. We left a long-running job over the weekend > with the prints you suggested. Oddly, something happened at the OS > level which killed our SSH connection to that machine, but the above > numpy dealloc() warning never printed (we logged this). > > What did happen is that the refcount you suggested we print: > > sys.getrefcount(numpy.dtype('float')) > > eventually seems to have wrapped around and gone negative. I'm > attaching the log file with those print statements, the key point is > that this happens eventually: > > PSVD Iteration 19 > Ref count 1989827662 > bar 444 > PSVD Iteration 0 > Ref count 2021353399 > PSVD Iteration 1 > Ref count 2143386207 > PSVD Iteration 2 > Ref count -2001245193 > PSVD Iteration 3 > Ref count -1915816437 > PSVD Iteration 4 > Ref count -1902698473 > > That refcount is for dtype('float') as indicated above. Is it not a > problem that this particular refcount goes negative? Eventually it > may continue increasing and hit a zero, point at which I imagine that > the bad dealloc will occur. > > Are refcounts stored in signed 32-bit ints? Why? I'd have naively > expected them to be stored in unsigned longs to avoid wraparound > problems, but maybe I'm completely missing the real problem here. I suspect the real problem is that the refcount keeps going up. Even if it was unsigned it would eventually wrap to zero and with a bit of luck get garbage collected. So probably something isn't decrementing the refcount. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez.net at gmail.com Mon Oct 30 17:01:51 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 15:01:51 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Charles R Harris wrote: > I suspect the real problem is that the refcount keeps going up. Even if it > was unsigned it would eventually wrap to zero and with a bit of luck get > garbage collected. So probably something isn't decrementing the refcount. Oops, my bad: I meant *unsigned long long*, so that the refcount is a 64-bit object. By the time it wraps around, you'll have run out of memory long ago. Having 32 bit ref counters can potentially mean you run out of the counter before you run out of RAM on a system with sufficient memory. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Mon Oct 30 17:13:37 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 30 Oct 2006 15:13:37 -0700 Subject: glibc invalid pointer error In-Reply-To: References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> Message-ID: <45467911.2060207@ee.byu.edu> Fernando Perez wrote: >On 10/30/06, David Huard wrote: > > >>Hi, >>I have a script that crashes, but only if it runs over 9~10 hours, with the >>following backtrace from gdb. The script uses PyMC, and repeatedly calls (> >>1000000) likelihood functions written in fortran and wrapped with f2py. >>Numpy: 1.0.dev3327 >>Python: 2.4.3 >> >> > >This sounds awfully reminiscent of the bug I recently mentioned: > >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > > It actually looks very much like it. I think the problem may be in f2py or in one of the C-API calls where-in there is a reference-count problem with the built-in data-type objects. NumPy won't try to free those anymore which will solve the immediate problem, but there is still a reference-count problem somewhere. The reference to the data-type objects is consumed by constructors that take PyArray_Descr * arguments. So, you often need to INCREF before passing to those constructors. It looks like this INCREF is forgotten in some extension module (perhaps in f2py or PyMC). It's possible it's in NumPy itself, though I've re-checked the code lots of times looking for that specific problem. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 17:31:51 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 15:31:51 -0700 Subject: glibc invalid pointer error In-Reply-To: <45467911.2060207@ee.byu.edu> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> <45467911.2060207@ee.byu.edu> Message-ID: On 10/30/06, Travis Oliphant wrote: > Fernando Perez wrote: > >This sounds awfully reminiscent of the bug I recently mentioned: > > > >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > > > > > > It actually looks very much like it. I think the problem may be in f2py > or in one of the C-API calls where-in there is a reference-count problem > with the built-in data-type objects. > > NumPy won't try to free those anymore which will solve the immediate > problem, but there is still a reference-count problem somewhere. > > The reference to the data-type objects is consumed by constructors that > take PyArray_Descr * arguments. So, you often need to INCREF before > passing to those constructors. It looks like this INCREF is forgotten > in some extension module (perhaps in f2py or PyMC). It's possible it's > in NumPy itself, though I've re-checked the code lots of times looking > for that specific problem. As a data point, our code has almost no manual memory management in C, but lots and lots of f2py-generated wrappers, as well as a lot of weave.inline-generated code. We do have hand-written C extensions, but most of them operate on externally allocated arrays. The one little snippet where we manually manage memory is a copy of numpy's innerproduct() which I simplified and tuned for our purposes; it just does: ret = (PyArrayObject *)PyArray_SimpleNew(nd,dimensions, ap1->descr->type_num); if (ret == NULL) goto fail; [ do computational loop to fill in ret array, no memory management here ] return (PyObject *)ret; fail: Py_XDECREF(ret); return NULL; That's the full extent of our manual memory management, and I don't see any problem with it, but maybe there is: I copied this from numpy months ago and haven't really looked again. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Mon Oct 30 17:36:32 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 30 Oct 2006 15:36:32 -0700 Subject: glibc invalid pointer error In-Reply-To: References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> Message-ID: <45467E70.60700@ee.byu.edu> Fernando Perez wrote: >On 10/30/06, David Huard wrote: > > >>Hi, >>I have a script that crashes, but only if it runs over 9~10 hours, with the >>following backtrace from gdb. The script uses PyMC, and repeatedly calls (> >>1000000) likelihood functions written in fortran and wrapped with f2py. >>Numpy: 1.0.dev3327 >>Python: 2.4.3 >> >> > >This sounds awfully reminiscent of the bug I recently mentioned: > >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > >We left a fresh run over the weekend, but my office mate is currently >out of the office and his terminal is locked, so I don't know what the >result is. I'll report shortly: we followed Travis' instructions and >ran with a fresh SVN build which includes the extra warnings he added >to the dealloc routines. You may want to try the same advice, perhaps >with information from both of us the gurus may zero in on the problem, >if indeed it is the same. > I talked about the reference counting issue. One problem is not incrementing the reference count when it needs to be. The other problem could occur if the reference-count was not decremented when it needed to be and the reference count wrapped from MAX_LONG to 0. This could also create the problem and would be expected for "long-running" processes. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 17:41:22 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 15:41:22 -0700 Subject: glibc invalid pointer error In-Reply-To: <45467E70.60700@ee.byu.edu> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> <45467E70.60700@ee.byu.edu> Message-ID: On 10/30/06, Travis Oliphant wrote: > Fernando Perez wrote: > > >On 10/30/06, David Huard wrote: > > > > > >>Hi, > >>I have a script that crashes, but only if it runs over 9~10 hours, with the > >>following backtrace from gdb. The script uses PyMC, and repeatedly calls (> > >>1000000) likelihood functions written in fortran and wrapped with f2py. > >>Numpy: 1.0.dev3327 > >>Python: 2.4.3 > >> > >> > > > >This sounds awfully reminiscent of the bug I recently mentioned: > > > >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > > > >We left a fresh run over the weekend, but my office mate is currently > >out of the office and his terminal is locked, so I don't know what the > >result is. I'll report shortly: we followed Travis' instructions and > >ran with a fresh SVN build which includes the extra warnings he added > >to the dealloc routines. You may want to try the same advice, perhaps > >with information from both of us the gurus may zero in on the problem, > >if indeed it is the same. > > > I talked about the reference counting issue. One problem is not > incrementing the reference count when it needs to be. The other problem > could occur if the reference-count was not decremented when it needed to > be and the reference count wrapped from MAX_LONG to 0. This could also > create the problem and would be expected for "long-running" processes. I just posted the log from that run in the other thread. I'm not sure if that helps you any though. I'm running the code again to see if we see your new warning fire, and will report back. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Mon Oct 30 17:49:38 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Mon, 30 Oct 2006 17:49:38 -0500 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 30/10/06, Fernando Perez wrote: > On 10/30/06, Charles R Harris wrote: > > > I suspect the real problem is that the refcount keeps going up. Even if it > > was unsigned it would eventually wrap to zero and with a bit of luck get > > garbage collected. So probably something isn't decrementing the refcount. > > Oops, my bad: I meant *unsigned long long*, so that the refcount is a > 64-bit object. By the time it wraps around, you'll have run out of > memory long ago. Having 32 bit ref counters can potentially mean you > run out of the counter before you run out of RAM on a system with > sufficient memory. Yes, this is a feature(?) of python as it currently stands (I checked 2.5) - reference counts are 32-bit signed integers, so if you have an object that has enough references, python will be exceedingly unhappy: http://mail.python.org/pipermail/python-dev/2002-September/028679.html It is of course possible that you actually have that many references to some object, but it seems to me you'd notice twenty-four gigabytes of pointers floating around... A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From matthew.brett at gmail.com Mon Oct 30 17:52:26 2006 From: matthew.brett at gmail.com (Matthew Brett) Date: Mon, 30 Oct 2006 22:52:26 +0000 Subject: dtype.hasobject value Message-ID: <1e2af89e0610301452r7d79aad7pa47b64ae906ebe31@mail.gmail.com> Hi, I notice that the value for: zeros((1,), dtype=object).dtype.hasobject is now 63, whereas previously it had been 1. Is this intended? Thanks, Matthew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Mon Oct 30 18:02:34 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 30 Oct 2006 16:02:34 -0700 Subject: glibc invalid pointer error In-Reply-To: <91cf711d0610301157o19f2ba70o214ec1d08adfc406@mail.gmail.com> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> <91cf711d0610301157o19f2ba70o214ec1d08adfc406@mail.gmail.com> Message-ID: <4546848A.60008@ee.byu.edu> David Huard wrote: > Ok, > I'll update numpy and give it another try tonight. > I just fixed some reference-count problems in f2py today. These were of the variety that there was a missing decref that would cause the reference count of certain often-used data-types to increase without bound and eventually wrap (to 0) in long-running processes using f2py. I suspect this is the fundamental problem in both cases. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Mon Oct 30 18:05:49 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 30 Oct 2006 20:05:49 -0300 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Fernando Perez wrote: > On 10/30/06, Charles R Harris wrote: > > > I suspect the real problem is that the refcount keeps going up. Even if it > > was unsigned it would eventually wrap to zero and with a bit of luck get > > garbage collected. So probably something isn't decrementing the refcount. > > Oops, my bad: I meant *unsigned long long*, so that the refcount is a > 64-bit object. By the time it wraps around, you'll have run out of > memory long ago. Having 32 bit ref counters can potentially mean you > run out of the counter before you run out of RAM on a system with > sufficient memory. > > Cheers, FYI, this is what is defined in Include/object.h /* PyObject_HEAD defines the initial segment of every PyObject. */ #define PyObject_HEAD \ _PyObject_HEAD_EXTRA \ Py_ssize_t ob_refcnt; \ struct _typeobject *ob_type; #define Py_INCREF(op) ( \ _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \ (op)->ob_refcnt++) #define Py_DECREF(op) \ if (_Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA \ --(op)->ob_refcnt != 0) \ _Py_CHECK_REFCNT(op) \ else \ _Py_Dealloc((PyObject *)(op)) And '_Py_CHECK_REFCNT' macro will finally call Py_FatalError 'ob_refcnt' is a Py_ssize_t integer, so I think you will not be able to overflow it, unless in case of C code with refcounting bugs. Am I right? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Mon Oct 30 18:10:09 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 30 Oct 2006 20:10:09 -0300 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Lisandro Dalcin wrote: > FYI, this is what is defined in Include/object.h > I forgoy to say in Python-2.5 -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Mon Oct 30 18:48:12 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Mon, 30 Oct 2006 15:48:12 -0800 Subject: Conversion about getting array interface into Python In-Reply-To: References: <45464C94.4080900@ieee.org> Message-ID: <45468F3C.8020407@noaa.gov> Lisandro Dalcin wrote: > Perhaps the reason for this is simple: > few Python core developers are involved in scientific computing and do > not have a clear idea of what it is needed for this. Perhaps true, but..... I imagine one of the issues that the python-dev crowd has is that they don't want to add features solely for the scientific computing crowd. Indeed, they don't need to. The features are there in numpy, everyone who uses numpy has access to them. The reason I want to have these features in the standard library is that there are a lot of good uses for them outside the scientific computing community, and, in particular, for interaction between scientific computing and more general purpose packages (GUI toolkits, Image processing, web frameworks, who knows?) Specifically, this is useful stuff for exchanging data with PIL, wxPython (two specific uses I have), and Matplotlib could really use it with all of its back-ends (TK, GTK, QT, etc....). In addition, I don't think it's on the table at the moment, but an n-d array has a lot of use well outside of number crunching. I've introduced a handful of users to numpy just to get access to a nice n-d object array type for completely no-numeric uses. Off to go post on python-dev..... -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 19:23:42 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 17:23:42 -0700 Subject: glibc invalid pointer error In-Reply-To: <4546848A.60008@ee.byu.edu> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> <91cf711d0610301157o19f2ba70o214ec1d08adfc406@mail.gmail.com> <4546848A.60008@ee.byu.edu> Message-ID: On 10/30/06, Travis Oliphant wrote: > David Huard wrote: > > > Ok, > > I'll update numpy and give it another try tonight. > > > > I just fixed some reference-count problems in f2py today. These were of > the variety that there was a missing decref that would cause the > reference count of certain often-used data-types to increase without > bound and eventually wrap (to 0) in long-running processes using f2py. > > I suspect this is the fundamental problem in both cases. Many thanks, Travis. We're rebuilding numpy and all of our f2py-generated wrappers, and will start a new run. I'll report on the results as well. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 19:28:39 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 17:28:39 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Lisandro Dalcin wrote: > FYI, this is what is defined in Include/object.h > > /* PyObject_HEAD defines the initial segment of every PyObject. */ > #define PyObject_HEAD \ > _PyObject_HEAD_EXTRA \ > Py_ssize_t ob_refcnt; \ > struct _typeobject *ob_type; > > #define Py_INCREF(op) ( \ > _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \ > (op)->ob_refcnt++) > > #define Py_DECREF(op) \ > if (_Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA \ > --(op)->ob_refcnt != 0) \ > _Py_CHECK_REFCNT(op) \ > else \ > _Py_Dealloc((PyObject *)(op)) > > And '_Py_CHECK_REFCNT' macro will finally call Py_FatalError > > 'ob_refcnt' is a Py_ssize_t integer, so I think you will not be able > to overflow it, unless in case of C code with refcounting bugs. Am I > right? I think you are, and fortunately this indicates that they /did/ change to a longer data type for refcounting in newer pythons. The box where we have this problem is running 2.3 though, and obviously a runaway refcount in f2py can still die even if it's a longer data type. However, Travis mentioned he just fixed precisely a bug like that in f2py, so I'm optimistic, and I'm currently making a new test. Thanks for the info, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cjw at sympatico.ca Mon Oct 30 20:01:27 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Mon, 30 Oct 2006 20:01:27 -0500 Subject: Conversion about getting array interface into Python In-Reply-To: <45464C94.4080900@ieee.org> References: <45464C94.4080900@ieee.org> Message-ID: <4546A067.6000405@sympatico.ca> Travis Oliphant wrote: > If anybody has a desire to see the array interface into Python, please > help by voicing an opinion on python-dev in the discussion about adding > data-type objects to Python. There are a few prominent people who > don't get why applications would need to share data-type information > about memory areas. I need help giving reasons why. > > -Travis > It seems to make sense to me. Perhaps a pointer to some document on just what is being proposed would help. Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From strawman at astraw.com Mon Oct 30 20:40:41 2006 From: strawman at astraw.com (Andrew Straw) Date: Mon, 30 Oct 2006 17:40:41 -0800 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: <4546A999.4000003@astraw.com> Fernando Perez wrote: > Here is some more info. We left a long-running job over the weekend > with the prints you suggested. Oddly, something happened at the OS > level which killed our SSH connection to that machine, but the above > numpy dealloc() warning never printed (we logged this). As an aside, I always use GNU screen when starting long-running jobs just in case something like this happens. screen lets you reconnect to a session from any login to a machine. Just a point of information... -Andrew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 30 23:47:25 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 30 Oct 2006 21:47:25 -0700 Subject: dtype.hasobject value In-Reply-To: <1e2af89e0610301452r7d79aad7pa47b64ae906ebe31@mail.gmail.com> References: <1e2af89e0610301452r7d79aad7pa47b64ae906ebe31@mail.gmail.com> Message-ID: <4546D55D.30402@ieee.org> Matthew Brett wrote: > Hi, > > I notice that the value for: > > zeros((1,), dtype=object).dtype.hasobject > > is now 63, whereas previously it had been 1. Is this intended? > > Yes. We are using hasobject as an 8-bit flag now and separating out some of the concepts that make object arrays object arrays. This will be in 1.0.1 which will come out soon. It should be binary compatible with 1.0. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 31 00:19:45 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 30 Oct 2006 22:19:45 -0700 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: <4546DCF1.3080105@ieee.org> Tom Denniston wrote: > I recently upgraded to numpy 1.0 from 1.0b5. I noticed that > numpy.argmax behavior is very strange on object arrays. See below: > > (Pdb) numpy.__version__ > '1.0' > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object)) > 0 > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=int)) > 1 > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object), axis=0) > 0 > > > I would expect the argmax to behave the same on the dtype=int and > dtype=object examples but it doesn't. Am I missing some subtelty or > is this just a bug? 1.0 is the most recent version, right? This is a bug. I've fixed in in SVN. Thanks for the test. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 31 00:22:32 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 30 Oct 2006 22:22:32 -0700 Subject: matrixdef.py In-Reply-To: <454537A5.1030202@sympatico.ca> References: <454537A5.1030202@sympatico.ca> Message-ID: <4546DD98.5020307@ieee.org> Colin J. Williams wrote: > Line 71 has: data.view(subtype) > > This appears to involve a call to __array_finalize__. Is this an > unconditional call? > Yes. Every-time an array is constructed the __array_finalize__ method is called unless the __array_finalize__ attribute is None > If not, what are the conditions? > > Why not use __init__ to handle these things, since it is always called > after the __new__? > Because __init__ must have the same calling signature as __new__ and we have no way of knowing what it is. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gnata at obs.univ-lyon1.fr Tue Oct 31 06:04:41 2006 From: gnata at obs.univ-lyon1.fr (Xavier Gnata) Date: Tue, 31 Oct 2006 12:04:41 +0100 Subject: A way to compute a 2d histogram In-Reply-To: <91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> <453F57C6.2020101@obs.univ-lyon1.fr> <91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com> Message-ID: <45472DC9.2090802@obs.univ-lyon1.fr> Hi, Your histograms functions look fine for me :) As it is a quite usual operation on an array, I would suggest to put it in numpy as numpy.histogram. IMHO, there is no point to create an numpy.stats only for histograms (or do you have plans to move other stats related function to numpy.stats?) Xavier. > Nicolas, thanks for the bug report, I fooled around with argument > passing and should have checked every case. > > You'll find the histogram function that deals with weights on the > numpy trac ticket 189, > I'm waiting for some hints as to where the histogram function should > reside (numpy.histogram, numpy.stats.histogram, ...) before submitting > a patch . > > Salut, > David > > > 2006/10/25, Nicolas Champavert >: > > Hi, > > it would be great if you could add the weight option in the 1D > histogram too. > > Nicolas > > David Huard a ?crit : > > Xavier, > > Here is the patch against svn. Please report any bug. I haven't had > > the time to test it extensively, something that should be done > before > > commiting the patch to the repo. I'd appreciate your feedback. > > > > David > > > > 2006/10/24, David Huard < david.huard at gmail.com > > > >>: > > > > Hi Xavier, > > > > You could tweak histogram2d to do what you want, or you > could give > > me a couple of days and I'll do it and let you know. If you want > > to help, you could write a test using your particular > application > > and data. > > > > David > > > > > > 2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr > > > >>: > > > > Hi, > > > > I have a set of 3 1D large arrays. > > The first 2 one stand for the coordinates of particules and > > the last one > > for their masses. > > I would like to be able to plot this data ie to compute > a 2D > > histogram > > summing the masses in each bin. > > I cannot find a way to do that without any loop on the > indices > > resulting > > too a very slow function. > > > > I'm looking for an elegant way to do that with numpy (or > > scipy??) function. > > > > For instance, scipy.histogram2d cannot do the job because it > > only counts > > the number of samples in each bin. > > There is no way to deal with weights. > > > > Xavier. > > > > > > -- > > ############################################ > > Xavier Gnata > > CRAL - Observatoire de Lyon > > 9, avenue Charles Andr? > > 69561 Saint Genis Laval cedex > > Phone: +33 4 78 86 85 28 > > Fax: +33 4 78 86 83 86 > > E-mail: gnata at obs.univ-lyon1.fr > > > > ############################################ > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web > > services, security? > > Get stuff done quickly with pre-integrated technology to > make > > your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on > > Apache Geronimo > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > > > > > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > > > > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- ############################################ Xavier Gnata CRAL - Observatoire de Lyon 9, avenue Charles Andr? 69561 Saint Genis Laval cedex Phone: +33 4 78 86 85 28 Fax: +33 4 78 86 83 86 E-mail: gnata at obs.univ-lyon1.fr ############################################ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 31 07:57:37 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 31 Oct 2006 05:57:37 -0700 Subject: Need more comments from scientific community on python-dev Message-ID: <45474841.9030802@ieee.org> I'm recruiting more comments on python-dev regarding my two proposals for improving Python's native ability to share ndarray-like information. There is a dearth of scientific-computing and number-crunching-aware people on python-dev. The result is that I sound like a lone voice arguing for something that nobody cares about. When, I don't think that is true. Please, please. If you want Python to grow support for the array interface (or something like it), then please speak up on python-dev. Even something as simple as I really see the need for a way to exchange data-format information between two objects sharing the buffer protocol can be helpful. You can post through the gmane newsgroup interface: gmane.comp.python.devel Find any of the posts on the PEP's I've introduced. Thanks for your help. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Tue Oct 31 08:37:31 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 31 Oct 2006 22:37:31 +0900 Subject: pyaudio 0.3, with docs ! Message-ID: <4547519B.4040201@ar.media.kyoto-u.ac.jp> Hi, I improved pyaudio last WE using indications given by various people on the list or privately, and as I finally got the motivation to set-up something which looks like a webpage, there is a doc with examples which show how to use it. The API to open files for writing is much saner, and the setup.py should be smart enough to grab all informations necessary to the wrapper, including the location of the shared libsndfile: download: http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pyaudio/#installation doc + examples: http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pyaudio/ I would appreciate to hear reports on platforms which are not linux (windows, mac os X) to see if my the setup.py works there, Cheers, David From aisaac at american.edu Tue Oct 31 11:14:00 2006 From: aisaac at american.edu (Alan Isaac) Date: Tue, 31 Oct 2006 11:14:00 -0500 Subject: [Numpy-discussion] Need more comments from scientific community on python-dev In-Reply-To: <45474841.9030802@ieee.org> References: <45474841.9030802@ieee.org> Message-ID: On Tue, 31 Oct 2006, Travis Oliphant wrote: > Please, please. If you want Python to grow support for > the array interface (or something like it), then please > speak up on python-dev. The easiest access to this discussion for me was http://news.gmane.org/group/gmane.comp.python.devel/ I cannot add to this discussion, but I REALLY hope others will help Travis out here. (A few have.) He is fielding a lot of questions, some of which look to me to be from individuals who are ready to have fairly strong opinions without really understanding the "why" of his proposals. The good news is, there seems to be (on my naive reading) some sympathy for what Travis is trying to do. I think more motivating examples would prove helpful in swinging things. Cheers, Alan Isaac From oliphant at ee.byu.edu Tue Oct 31 12:04:48 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 10:04:48 -0700 Subject: [Numpy-discussion] Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> Message-ID: <45478230.2050709@ee.byu.edu> Fernando Perez wrote: >On 10/31/06, Rich Shepard wrote: > > >>On Tue, 31 Oct 2006, Alan Isaac wrote: >> >> >> >>>The easiest access to this discussion for me was >>>http://news.gmane.org/group/gmane.comp.python.devel/ I cannot add to this >>>discussion, but I REALLY hope others will help Travis out here. (A few >>>have.) He is fielding a lot of questions, some of which look to me to be >>>from individuals who are ready to have fairly strong opinions without >>>really understanding the "why" of his proposals. >>> >>> >> All this is sufficiently far from my areas of expertise that I cannot >>contribute anything useful. Otherwise, I'd be happy to lend support. >> >> > >I actually worry about the same: I really would like to help, but >after reading the whole discussion, I realized that the low-level >details being asked and discussed are something I don't really know >enough to say anything. And I don't want to sound simply saying 'Hey, >Travis is great, listen to him!' to python-dev, since that (asides >from looking silly) can be somewhat counter-productive. > > >How does that sound, Travis? Is that something you think might help >you, esp. since so many of us are feeling woefully underqualified to >lend a useful hand in the actual discussion on python-dev? > > That would be great. I think a couple of things would also be useful. 1) Some way to indicate to python-dev that I'm actually speaking for more than just myself. So, while I agree that just supporting my PEP (which probably in reality needs work) without understanding it is counter-productive, a voice that says. "We really do need this kind of functionality" is at least one more voice. 2) Examples of sharing memory between two objects. PIL is the classic example and has some merit, but because the internal memory layout of the PIL is 'pointer-to-pointers' instead of 'big-chunk-of-memory' it's not a 1-1 match to NumPy and the array interface only can comunicate information about the "mode." But, I can see other examples. PyMedia, PyGame, PyVideo? CVXOPT, PyVoxel. All of these seem to define their own objects which are basically just interpretations of chunks of memory. At one time, we might have said "these should all be sub-classes of the ndarray". Now, we are thinking more along the lines of "these should all expose an array interface". The array interface is still more bulky then it needs to be (it has to go through the attribute-lookup process which can be slow). It would be much better if the extended buffer protocol were available as a function-pointer on the type object of the type. If you have an application where you've ever wanted NumPy in the core. See if the extended buffer protocol serves your purposes and if you agree, voice your approval for the PEP. In my mind, the data-format PEP does not need to go through if there really is a better way to pass data-format information through the buffer protocol. But, the extended buffer protocol we *do* need. -Travis >Regards, > >f >_______________________________________________ >SciPy-user mailing list >SciPy-user at scipy.org >http://projects.scipy.org/mailman/listinfo/scipy-user > > From jdhunter at ace.bsd.uchicago.edu Tue Oct 31 12:13:24 2006 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Tue, 31 Oct 2006 11:13:24 -0600 Subject: [Numpy-discussion] Need more comments from scientific community on python-dev In-Reply-To: <45478230.2050709@ee.byu.edu> (Travis Oliphant's message of "Tue, 31 Oct 2006 10:04:48 -0700") References: <45474841.9030802@ieee.org> <45478230.2050709@ee.byu.edu> Message-ID: <87r6wo5ryj.fsf@peds-pc311.bsd.uchicago.edu> >>>>> "Travis" == Travis Oliphant writes: Travis> All of these seem to define their own objects which are Travis> basically just interpretations of chunks of memory. At Travis> one time, we might have said "these should all be Travis> sub-classes of the ndarray". Now, we are thinking more What about blitting pixel buffers from mpl or chaco agg into various GUI windows, GTK, Tk, WX, etc.... This seems like a ready made case for the array interface. I could pipe in with an example like this if it would help. JDH From oliphant at ee.byu.edu Tue Oct 31 12:25:20 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 10:25:20 -0700 Subject: dtype.hasobject value In-Reply-To: <4546D55D.30402@ieee.org> References: <1e2af89e0610301452r7d79aad7pa47b64ae906ebe31@mail.gmail.com> <4546D55D.30402@ieee.org> Message-ID: <45478700.10207@ee.byu.edu> Travis Oliphant wrote: >Matthew Brett wrote: > > >>Hi, >> >>I notice that the value for: >> >>zeros((1,), dtype=object).dtype.hasobject >> >>is now 63, whereas previously it had been 1. Is this intended? >> >> >> >> > >Yes. We are using hasobject as an 8-bit flag now and separating out >some of the concepts that make object arrays object arrays. > >This will be in 1.0.1 which will come out soon. It should be binary >compatible with 1.0. > > Let me follow up with this. I've changed back dtype.hasobject so that it returns a Python boolean object. Thus comparisons to 1 should work fine. Under the covers it is (dtype->hasobject & NPY_ITEM_HASOBJECT) == NPY_ITEM_HASOBJECT) There is a new attribute (flags) that contains all the flags in the hasobject variable. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Tue Oct 31 12:46:22 2006 From: David.L.Goldsmith at noaa.gov (David L Goldsmith) Date: Tue, 31 Oct 2006 09:46:22 -0800 Subject: Need more comments from scientific community on python-dev In-Reply-To: <45474841.9030802@ieee.org> References: <45474841.9030802@ieee.org> Message-ID: <45478BEE.5040008@noaa.gov> Perhaps you could please elaborate on what you regard as "the issue(s)" and "what's at stake"? I'm still comparatively new (been using python and numpy circa 8-9 months), but so far I haven't encountered any problems I'd attribute to "Python's ... [in]ability to share ndarray-like information". (The most vexing problems I've had have all been with matplotlib, its dual-mode API, and its rather clumsy and haphazard documentation.) I'd like to be supportive, but not blindly-so. DG Travis Oliphant wrote: > I'm recruiting more comments on python-dev regarding my two proposals > for improving Python's native ability to share ndarray-like information. > > There is a dearth of scientific-computing and number-crunching-aware > people on python-dev. The result is that I sound like a lone voice > arguing for something that nobody cares about. When, I don't think that > is true. Please, please. If you want Python to grow support for the > array interface (or something like it), then please speak up on > python-dev. > > Even something as simple as I really see the need for a way to exchange > data-format information between two objects sharing the buffer protocol > can be helpful. > > You can post through the gmane newsgroup interface: > > gmane.comp.python.devel > > Find any of the posts on the PEP's I've introduced. Thanks for your help. > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- HMRD/ORR/NOS/NOAA ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 31 12:55:51 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 31 Oct 2006 10:55:51 -0700 Subject: [SciPy-user] Need more comments from scientific community on python-dev In-Reply-To: <87r6wo5ryj.fsf@peds-pc311.bsd.uchicago.edu> References: <45474841.9030802@ieee.org> <45478230.2050709@ee.byu.edu> <87r6wo5ryj.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: On 10/31/06, John Hunter wrote: > > >>>>> "Travis" == Travis Oliphant writes: > Travis> All of these seem to define their own objects which are > Travis> basically just interpretations of chunks of memory. At > Travis> one time, we might have said "these should all be > Travis> sub-classes of the ndarray". Now, we are thinking more > > What about blitting pixel buffers from mpl or chaco agg into various > GUI windows, GTK, Tk, WX, etc.... This seems like a ready made case > for the array interface. I could pipe in with an example like this if > it would help. I would also like to see the socket object take a more general type than python strings. I suspect there are a lot of such things in python -- interfaces to services and such -- that would benefit from using a more general interface to memory blocks. But I am not a python developer and don't really know how to make that case. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez.net at gmail.com Tue Oct 31 13:18:03 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 31 Oct 2006 11:18:03 -0700 Subject: [Numpy-discussion] Need more comments from scientific community on python-dev In-Reply-To: <45478230.2050709@ee.byu.edu> References: <45474841.9030802@ieee.org> <45478230.2050709@ee.byu.edu> Message-ID: On 10/31/06, Travis Oliphant wrote: > Fernando Perez wrote: > >I actually worry about the same: I really would like to help, but > >after reading the whole discussion, I realized that the low-level > >details being asked and discussed are something I don't really know > >enough to say anything. And I don't want to sound simply saying 'Hey, > >Travis is great, listen to him!' to python-dev, since that (asides > >from looking silly) can be somewhat counter-productive. > > > > > >How does that sound, Travis? Is that something you think might help > >you, esp. since so many of us are feeling woefully underqualified to > >lend a useful hand in the actual discussion on python-dev? > > > > > > That would be great. I think a couple of things would also be useful. OK, my experience so far has been that there's a certain 'activation barrier' with Wikis, but once pages are there, people for some reason feel more comfortable filling in. So even though it's mostly a place holder, I went ahead and made this: http://www.scipy.org/ArrayInterfacePEP Hopefully as the discussion evolves, this page can be filled in with all the necessary info in one place, and it will become in a few days a solid, organized repository of all the key points in this discussion. This will be a LOT easier to refer to in the python-dev battle than randomly scattered bits of emails in the discussion thread. I'll try to spend more time on it over the next few days to fill in, but I'm pretty busy with other things as well, so hopefully others can pitch in as well. Cheers, f ps - one more thing. This guy: http://blog.vrplumber.com/ has been rewriting the OpenGL bindings using ctypes, and I've seen posts from him about numpy (in his blog). He might be able to contribute something... From aisaac at american.edu Tue Oct 31 13:44:02 2006 From: aisaac at american.edu (Alan Isaac) Date: Tue, 31 Oct 2006 13:44:02 -0500 Subject: installing numpy on Tiger Message-ID: If other users will not profit from this question, please consider it OT and ignore it. But perhaps it will prove useful to some potential numpy users. I am a long time Windows user who has been happily using the Python and numpy installers. At work I need to use a Mac for a while. I installed Python 2.5 from python.org painlessly, but it looks like I'll have to compile numpy from source. Do I understand that right? I have used interpreted languages for pretty much all my computing needs, so I am hoping someone patient will point me to or lead me through the steps I need to take to successfully install numpy on this platform. Thank you, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Tue Oct 31 13:54:28 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 31 Oct 2006 11:54:28 -0700 Subject: installing numpy on Tiger In-Reply-To: References: Message-ID: On 10/31/06, Alan Isaac wrote: > If other users will not profit from this question, > please consider it OT and ignore it. But perhaps > it will prove useful to some potential numpy users. > > I am a long time Windows user who has been happily > using the Python and numpy installers. At work I > need to use a Mac for a while. I installed Python 2.5 > from python.org painlessly, but it looks like > I'll have to compile numpy from source. > Do I understand that right? > > I have used interpreted languages for pretty much > all my computing needs, so I am hoping someone patient > will point me to or lead me through the steps I need > to take to successfully install numpy on this platform. This is probably a good starting point: http://www.scipy.org/Installing_SciPy/Mac_OS_X There have been numerous threads on this issue recently, so you may also want to do a bit of searching of the mailing list archives. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From lists.steve at arachnedesign.net Tue Oct 31 13:58:40 2006 From: lists.steve at arachnedesign.net (Steve Lianoglou) Date: Tue, 31 Oct 2006 13:58:40 -0500 Subject: installing numpy on Tiger In-Reply-To: References: Message-ID: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> Hi Alan, > I installed Python 2.5 > from python.org painlessly, but it looks like > I'll have to compile numpy from source. > Do I understand that right? I believe you'll have to if you want to use Python 2.5. > ... so I am hoping someone patient > will point me to or lead me through the steps I need > to take to successfully install numpy on this platform. I think the instructions on the scipy wiki are pretty thorough: http://scipy.org/Installing_SciPy/Mac_OS_X One thing to note is that I think I've seen on this list that people have been having problems compiling and using num/scipy on Python 2.5 (on any platform (?)) ... I haven't tried it myself, but that's the general feeling I've gotten from some of the emails on that subject. I could be wrong though. Another thing to consider would be to install the "Superpack" which is linked to in the mac section from the scipy downloads page: http://www.scipy.org/Download I also remember seeing on the ML that there were some problems w/ that superpack it not including some config file for ipython or something, but I'm not sure. For what it's worth, I'm using MacPorts (http://macports.org) to manage my python (and a other) installations, and have been compiling scipy/numpy/matplotlib/ipython from svn checkouts every now and again ... this has been working well for me. The version of python you get from macports is 2.4.3 Hope that helps. -steve ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From erin.sheldon at gmail.com Tue Oct 31 14:38:35 2006 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Tue, 31 Oct 2006 14:38:35 -0500 Subject: installing numpy on Tiger In-Reply-To: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> References: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> Message-ID: <331116dc0610311138m2eb27c23g4404a14a7084ad45@mail.gmail.com> Hi Alan - I have not had luck with the binary distros. There is always something that doesn't work, so I will be interested in the results of your efforts. The biggest problem with compiling things yourself is going to be dealing with LAPACK and fortran issues, and the backends in matplotlib. FWIW, the most stable solution I have found that doesn't involve dealing with dependencies yourself is darwinports (now macports). It currently has 1.0b5 numpy and scipy 0.5.1, which depend on python 2.4. Matplotlib worked with the wxPython backend but not others. While not the latest versions it will get you started, and I would guess 1.0 will be available soon (anyone?). I had absolutely zero problems using this once I realized that I had to use wxPython for the plotting backend. The snag is that gfortran is needed, which requires compiling gcc 4.0 which took 6 hours on my powerbook, so you will need some patience. Erin On 10/31/06, Steve Lianoglou wrote: > Hi Alan, > > > > I installed Python 2.5 > > from python.org painlessly, but it looks like > > I'll have to compile numpy from source. > > Do I understand that right? > > I believe you'll have to if you want to use Python 2.5. > > > ... so I am hoping someone patient > > will point me to or lead me through the steps I need > > to take to successfully install numpy on this platform. > > I think the instructions on the scipy wiki are pretty thorough: > http://scipy.org/Installing_SciPy/Mac_OS_X > > One thing to note is that I think I've seen on this list that people > have been having problems compiling and using num/scipy on Python 2.5 > (on any platform (?)) ... I haven't tried it myself, but that's the > general feeling I've gotten from some of the emails on that subject. > I could be wrong though. > > Another thing to consider would be to install the "Superpack" which > is linked to in the mac section from the scipy downloads page: > http://www.scipy.org/Download > > I also remember seeing on the ML that there were some problems w/ > that superpack it not including some config file for ipython or > something, but I'm not sure. > > For what it's worth, I'm using MacPorts (http://macports.org) to > manage my python (and a other) installations, and have been compiling > scipy/numpy/matplotlib/ipython from svn checkouts every now and > again ... this has been working well for me. The version of python > you get from macports is 2.4.3 > > Hope that helps. > -steve > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ndarray at mac.com Tue Oct 31 15:10:01 2006 From: ndarray at mac.com (Sasha) Date: Tue, 31 Oct 2006 16:10:01 -0400 Subject: Need more comments from scientific community on python-dev In-Reply-To: <45474841.9030802@ieee.org> References: <45474841.9030802@ieee.org> Message-ID: On 10/31/06, Travis Oliphant wrote: > > I'm recruiting more comments on python-dev regarding my two proposals > for improving Python's native ability to share ndarray-like information. > I would love to help, but I feel that I will be on the other side of the disagreement. (That's why I reply here rather than on python-dev first.) I've suggested that numpy could use ctypes way to describe binary data a long time ago: http://sourceforge.net/mailarchive/message.php?msg_id=15395017 I agree that it would not work out of the box, but I don't see any major obstacles to adding the missing features. I think we should attempt to implement a ctypes approach and compare the result to the dtype object approach. If we discover any unimplementable features along the way, this will be a strong argument to abandon ctypes as a standard. The last time I looked at the issue, the only feature that ctypes were lacking for me was the itemsize information (endianness is probably lacking as well, but I did not need it.) I am not sure, however that this information belongs to the type description object rather than carried separately. Note that it is trivial to add an itemsize attribute to ctypes because each primitive ctype has a fixed itemsize and Structure can easily compute its itemsize. Doubling the number of ctypes to allow for different endianness is a more troubling proposition. In any case, I think we should seriously discuss the relative merits of the two approaches on this list and present our view to python-dev when we reach some consensus. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Tue Oct 31 15:23:27 2006 From: aisaac at american.edu (Alan G Isaac) Date: Tue, 31 Oct 2006 15:23:27 -0500 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> Message-ID: On Tue, 31 Oct 2006, Sasha apparently wrote: > I think we should attempt to implement a ctypes approach You are probably aware that Travis has addressed this at some length on comp.python.devel (for example http://article.gmane.org/gmane.comp.python.devel/84661 http://article.gmane.org/gmane.comp.python.devel/84670 ) but just in case ... Cheers, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 31 15:29:41 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 13:29:41 -0700 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> Message-ID: <4547B235.6050904@ee.byu.edu> Sasha wrote: >On 10/31/06, Travis Oliphant wrote: > > >>I'm recruiting more comments on python-dev regarding my two proposals >>for improving Python's native ability to share ndarray-like information. >> >> >> > >I would love to help, but I feel that I will be on the other side of >the disagreement. (That's why I reply here rather than on python-dev >first.) > > Please read my posts about the Python type-object verses normal Python Object situation. That really is the crux of the matter. Ctypes uses a Python type object for every data-format. NumPy uses an instance of a data-type object for every data-format. What advantage do we gain by making every instance of a data-type object *also* a Python type object? We get a lot of head-ache. Have you seen what ctypes had to do? It had to define a new Dictionary object so it could attach it to the tp_dict parameter because you can't just inherit from the PyTypeObject and add the fields you want to the structure. This is my argument. I for one am not going to put any effort in that direction. People are free to do it, if they want, of course. But, it's no small change. I would, however, put effort into "undertstanding ctypes objects" as data-type objects. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 31 15:30:39 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 13:30:39 -0700 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> Message-ID: <4547B26F.80103@ee.byu.edu> Sasha wrote: >On 10/31/06, Travis Oliphant wrote: > > >>I'm recruiting more comments on python-dev regarding my two proposals >>for improving Python's native ability to share ndarray-like information. >> >> >> > >I would love to help, but I feel that I will be on the other side of >the disagreement. (That's why I reply here rather than on python-dev >first.) > > There's actually two issues here as well. 1) The extended buffer protocol 2) How to exchange data-format information through it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Beating_Back_the_Shorters_nathaniel at gmail.com Tue Oct 31 16:30:43 2006 From: Beating_Back_the_Shorters_nathaniel at gmail.com (Crystal Xiong) Date: Tue, 31 Oct 2006 15:30:43 -0600 Subject: Haley Padgett Message-ID: <160519020459.XAA13898Beating_Back_the_Shorters_nathaniel@gmail.com> A non-text attachment was scrubbed... Name: not available Type: multipart/alternative Size: 20 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ndarray at mac.com Tue Oct 31 16:36:58 2006 From: ndarray at mac.com (Sasha) Date: Tue, 31 Oct 2006 17:36:58 -0400 Subject: Need more comments from scientific community on python-dev In-Reply-To: <4547B235.6050904@ee.byu.edu> References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu> Message-ID: On 10/31/06, Travis Oliphant wrote: > Please read my posts about the Python type-object verses normal Python > Object situation. That really is the crux of the matter. > I read the whole python-dev thread before replying. I may be a little biased because I never liked somewhat cryptic letter codes in Numeric and the change of codes from Numeric to numpy did not contribute to my sympathy particularly when the convert script changed all unrelated instances of 'b' in my code to something else. I am also not a big fan of record arrays. I believe that numeric data should be stored in "inverted tables," where columns of homogeneous data are stored contiguously. With this disclaimer, I will address a few issues below. > Ctypes uses a Python type object for every data-format. > NumPy uses an instance of a data-type object for every data-format. > Yes, but AFAIK this is a recent innovation. Numarray used type objects and Numeric simply used letter codes. > What advantage do we gain by making every instance of a data-type object > *also* a Python type object? I think the main advantage is that you can have instances: >>> c_int(42) c_int(42) Of course, numpy has scalars for that, but ctypes also has fixed-length arrays, that are somewhat different from ndarrays: >>> a10 = c_int*10 >>> a10() <__main__.c_int_Array_10 object at 0x2a95816958> > We get a lot of head-ache. Have you seen > what ctypes had to do? It had to define a new Dictionary object so it > could attach it to the tp_dict parameter because you can't just inherit > from the PyTypeObject and add the fields you want to the structure. > This is my argument. > But, isn't this someone else's head-ache? Someone has already gone through all these contortions, why not reuse the effort? Others on python-dev described some specific needs of ctypes that your datatype object does not address. Your point seems to be that numpy does not share these needs and could use a much simpler approach. > I for one am not going to put any effort in that direction. People are > free to do it, if they want, of course. But, it's no small change. Can you present some use cases that illustrate the advantage of your datatype approach best? I will try to implement them with ctypes. > I would, however, put effort into "undertstanding ctypes objects" as > data-type objects. Yes, this is the key. I think we should better understand ctypes limitations before proposing an alternative. At the end of the day, it is better to have buffer protocol that describes the data using ctypes types than to have no standard type information at all. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 31 17:34:01 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 15:34:01 -0700 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu> Message-ID: <4547CF59.4020905@ee.byu.edu> Sasha wrote: >On 10/31/06, Travis Oliphant wrote: > > >Yes, this is the key. I think we should better understand ctypes >limitations before proposing an alternative. > I already understand it's practical limitations --- type objects as data-type instances is too bulky and too restrictive. You have to "be a Python type object" in order to be a data-type object. I wish others would actually understand the difference. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 31 17:48:19 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 15:48:19 -0700 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu> Message-ID: <4547D2B3.5000808@ee.byu.edu> Sasha wrote: >On 10/31/06, Travis Oliphant wrote: > > > >>Please read my posts about the Python type-object verses normal Python >>Object situation. That really is the crux of the matter. >> >> >> >I read the whole python-dev thread before replying. I may be a little >biased because I never liked somewhat cryptic letter codes in Numeric >and the change of codes from Numeric to numpy did not contribute to my >sympathy particularly when the convert script changed all unrelated >instances of 'b' in my code to something else. > The letter codes are not data-type objects. The letter codes are only there for historical reasons (and they are in the struct and array modules too so blame Python...) >I am also not a big >fan of record arrays. I believe that numeric data should be stored in >"inverted tables," where columns of homogeneous data are stored >contiguously. > Sure. I understand this. This was my argument too when Numarray was first proposed. But, how do you memory-map a record where the data is actually stored differently? Numeric's answer is "you don't" but that is un-acceptable. This is why record-arrays were created and the whole pandora's box of data-types was opened. > > >>Ctypes uses a Python type object for every data-format. >>NumPy uses an instance of a data-type object for every data-format. >> >> >> >Yes, but AFAIK this is a recent innovation. Numarray used type >objects and Numeric simply used letter codes. > > > Recent if 1 year is recent, and only if you worry precisely about when they were Python objects. Numeric always used a PyArray_Descr * structure (which was a PyObject_HEAD away from being a Python object) to describe data. The letter codes were just simple ways to "represent" those underlying structures (which could have been exposed to Python as "data-types" from the very beginning. >>What advantage do we gain by making every instance of a data-type object >>*also* a Python type object? >> >> > >I think the main advantage is that you can have instances: > > >>>>c_int(42) >>>> >>>> >c_int(42) > >Of course, numpy has scalars for that, but ctypes also has >fixed-length arrays, that are somewhat different from ndarrays: > > Sure, but you don't "need" an instance for every memory-layout description. If you want one, then great, ctypes gives it to you. But "requiring" data-type to be encoded in a type object is over-kill. >> We get a lot of head-ache. Have you seen >>what ctypes had to do? It had to define a new Dictionary object so it >>could attach it to the tp_dict parameter because you can't just inherit >>from the PyTypeObject and add the fields you want to the structure. >>This is my argument. >> >> >> > >But, isn't this someone else's head-ache? Someone has already gone >through all these contortions, why not reuse the effort? > No, it's not that simple. We have a headache whenever we want to do something like I just did and separate out the concepts of what makes a Python Object a Python object. Now, we don't just modify a simple C-structure (PyArray_Descr *), we have to modify a "meta-type" or a altered dictionary and get that change put in to ctypes. Inheriting from Python type objects is harder. People who have apparently never tried seem to think it's not but it is. >Others on >python-dev described some specific needs of ctypes that your datatype >object does not address. Your point seems to be that numpy does not >share these needs and could use a much simpler approach. > > No, that's not the point. The data-type object could easily be extended to accomodate those needs. The point is using data-types as instances of a regular Python object or data-types as instances of a "type" object. >>I for one am not going to put any effort in that direction. People are >>free to do it, if they want, of course. But, it's no small change. >> >> > >Can you present some use cases that illustrate the advantage of your >datatype approach best? > > >I will try to implement them with ctypes. > > This is missing the point. I have no doubt ctypes "could" be used on the Python side to describe a data-type. But, we need the PyArray_Descr * structure in NumPy. What are you going to replace the PyArray_Descr * structure with? That is the point. >> I would, however, put effort into "undertstanding ctypes objects" as >>data-type objects. >> >> > >Yes, this is the key. I think we should better understand ctypes >limitations before proposing an alternative. > Please understand what I meant. I meant putting effort into getting PyArray_DescrConverter to allow ctypes inputs and convert to the appropriate PyArray_Descr * structure. I already understand ctypes objects. I want the dtype() command to also understand them. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From josh.p.marshall at gmail.com Tue Oct 31 17:51:16 2006 From: josh.p.marshall at gmail.com (Josh Marshall) Date: Wed, 1 Nov 2006 09:51:16 +1100 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: Message-ID: On 11/31/06, Fernando Perez wrote: > Fernando Perez wrote: > ps - one more thing. This guy: > > http://blog.vrplumber.com/ > > has been rewriting the OpenGL bindings using ctypes, and I've seen > posts from him about numpy (in his blog). He might be able to > contribute something... I've been working on the OSX port and extensions for OpenGL-ctypes. (now released as PyOpenGL-3.00a, please test!). What Mike has done is define setuptools plugins to interface to different kinds of array data. The default type is ctypes arrays, or numpy arrays if numpy is installed. The data types handled are ctypes sized arrays, ctypes pointers, strings (read-only), and Python lists. (and old Numeric/ numarray via a non-default build.) Summaries of development and usage, resp, are found: http://pyopengl.sourceforge.net/ctypes/development.html http://pyopengl.sourceforge.net/ctypes/using.html Notes on array handling from the above: > Perhaps the most complex mechanisms in OpenGL-ctypes are those > which implement the array-based operations which allow for using > low-level blocks of formatted data to communicate with the OpenGL > implementation. OpenGL-ctypes preferred basic array implementation > is the (new) numpy reimplementation of the original Numeric Python. > > The array handling functionality provided within OpenGL-ctypes is > localised to the OpenGL.arrays sub-package. Within the package, > there are two major classes, one (the FormatHandler) which > implements an interface to a way of storing data in Python, and > another (the ArrayDatatype) which models an OpenGL array format. > The ArrayDatatype classes use FormatHandlers to manipulate array- > compatible objects for use in the system. and on FormatHandlers, for each of the datatypes I mentioned earlier: > Each format handler is responsible for implementing an API that > ArrayDatatypes can use to work with the Python data-format. Data- > formats can support a subset of the API, they only need to support > those aspects of the data-format which make sense. Now, I haven't spent much time looking at these parts of OpenGL- ctypes, as they have just worked for me. I would think that it would be trivial to write a FormatHandler which uses the ndarray interface and data type description to use any object implenting it as an input for OpenGL. This would include things such as PIL images. Mike, can you give us your opinion on how a standardised data type descriptor would be helpful for PyOpenGL? The PEP and some information about it can be found here: http://www.scipy.org/ArrayInterfacePEP Cheers, Josh ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Tue Oct 31 19:26:22 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Wed, 1 Nov 2006 09:26:22 +0900 Subject: Need more comments from scientific community on python-dev In-Reply-To: <45474841.9030802@ieee.org> References: <45474841.9030802@ieee.org> Message-ID: One thing I see with PIL and Matplotlib and PyOpenGL is that they're having to add specific support for Numpy and numeric and numarray and PIL and ctypes etc. I'm also in the camp that doesn't really understand this stuff well enough to really argue in favor of it, but would the proposed extensions make it possible for all these different packages to just have to support "buffer protocol" and magically be able to work with numpy or numeric without having to introduce explicit dependencies and support code for all those other packages? If I've got the facts straight then I'd be happy to go and say that on comp.lang.python. --bb On 10/31/06, Travis Oliphant wrote: > > > I'm recruiting more comments on python-dev regarding my two proposals > for improving Python's native ability to share ndarray-like information. > > There is a dearth of scientific-computing and number-crunching-aware > people on python-dev. The result is that I sound like a lone voice > arguing for something that nobody cares about. When, I don't think that > is true. Please, please. If you want Python to grow support for the > array interface (or something like it), then please speak up on > python-dev. > > Even something as simple as I really see the need for a way to exchange > data-format information between two objects sharing the buffer protocol > can be helpful. > > You can post through the gmane newsgroup interface: > > gmane.comp.python.devel > > Find any of the posts on the PEP's I've introduced. Thanks for your help. > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From alexander.belopolsky at gmail.com Tue Oct 31 18:51:57 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 31 Oct 2006 18:51:57 -0500 Subject: Need more comments from scientific community on python-dev In-Reply-To: <4547D2B3.5000808@ee.byu.edu> References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu> <4547D2B3.5000808@ee.byu.edu> Message-ID: On 10/31/06, Travis Oliphant wrote: ... > Please understand what I meant. I meant putting effort into getting > PyArray_DescrConverter to allow ctypes inputs and convert to the > appropriate PyArray_Descr * structure. I already understand ctypes > objects. I want the dtype() command to also understand them. > I think I am starting to understand. Forgive me for being slow. Is it correct that you don't mind writing c_int * 10 instead of dtype((' References: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> <331116dc0610311138m2eb27c23g4404a14a7084ad45@mail.gmail.com> Message-ID: <6ce0ac130610311646m79374541l57a5e1753af56e81@mail.gmail.com> Nice binaries for gfortran can be found here: hpc.sourceforge.net/ On 10/31/06, Erin Sheldon wrote: > Hi Alan - > > I have not had luck with the binary distros. There is always > something that doesn't work, so I will be interested in > the results of your efforts. > > The biggest problem with compiling things yourself is going > to be dealing with LAPACK and fortran issues, and the > backends in matplotlib. > > FWIW, the most stable solution I have found that doesn't involve > dealing with dependencies yourself is darwinports (now > macports). It currently has 1.0b5 numpy and scipy 0.5.1, which > depend on python 2.4. Matplotlib worked with the wxPython > backend but not others. While not the latest versions it will get > you started, and I would guess 1.0 will be available soon (anyone?). > I had absolutely zero problems using this once I realized that I > had to use wxPython for the plotting backend. The snag > is that gfortran is needed, which requires compiling gcc 4.0 which > took 6 hours on my powerbook, so you will need some patience. > > Erin > > On 10/31/06, Steve Lianoglou wrote: > > Hi Alan, > > > > > > > I installed Python 2.5 > > > from python.org painlessly, but it looks like > > > I'll have to compile numpy from source. > > > Do I understand that right? > > > > I believe you'll have to if you want to use Python 2.5. > > > > > ... so I am hoping someone patient > > > will point me to or lead me through the steps I need > > > to take to successfully install numpy on this platform. > > > > I think the instructions on the scipy wiki are pretty thorough: > > http://scipy.org/Installing_SciPy/Mac_OS_X > > > > One thing to note is that I think I've seen on this list that people > > have been having problems compiling and using num/scipy on Python 2.5 > > (on any platform (?)) ... I haven't tried it myself, but that's the > > general feeling I've gotten from some of the emails on that subject. > > I could be wrong though. > > > > Another thing to consider would be to install the "Superpack" which > > is linked to in the mac section from the scipy downloads page: > > http://www.scipy.org/Download > > > > I also remember seeing on the ML that there were some problems w/ > > that superpack it not including some config file for ipython or > > something, but I'm not sure. > > > > For what it's worth, I'm using MacPorts (http://macports.org) to > > manage my python (and a other) installations, and have been compiling > > scipy/numpy/matplotlib/ipython from svn checkouts every now and > > again ... this has been working well for me. The version of python > > you get from macports is 2.4.3 > > > > Hope that helps. > > -steve > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, security? > > Get stuff done quickly with pre-integrated technology to make your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From iwncfbs at pcibuf.com Tue Oct 31 21:19:13 2006 From: iwncfbs at pcibuf.com (always remind) Date: Wed, 1 Nov 2006 10:19:13 +0800 Subject: RatesStore StatusEasy ReturnsAll Message-ID: <000a01c6fd5c$1fcee870$b6b0a8dc@cyserver> Might got is some! All abilities asking me is questions about many different things. Bar During California Ladies State! Rulings tax Tournament vol? Gambling Live Coverage of Schedule Results Database of. Additional blind after winning preceding pot am. Display of among in them Noble Powells in Syndetic Solutions or Ebsco Thomson of. Washington in Tavis Smiley of inspire am mentors achieving success. Action seven five days Come am View Article? >From players all a. Solutions Ebsco Thomson Gale Statement photocopy material? Put you too in Thats need of ask anyones permission go? Card Player Magazine am What if Wanna Leave. Pay not getting. Analyst Issues for Gift of Blogs a. Around he ready a reader Irene s. Analyst is Issues for Gift Blogs is it is of. Ccc Rosewood Danvers ma provides licenses variety. Might got is some! Rookies Holdem Humor a Legal Lowball Money. Inspire mentors achieving success Clean. Soonbampn Annex Oreilly Computing Gamespc Video Statefind. Support Press Contact Terms Copyright is copy rights reserved. Pooh Little Einstein Princess favorites! Makes fair a table remain protect them. Rulings tax Tournament vol? Librarians in serious buyers holds. Just do a anytime? Humor Legal Lowball Money Management Omaha Math Psychology! All abilities asking me is questions about many different things. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Us.gif Type: image/gif Size: 8792 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From george.sakkis at gmail.com Tue Oct 31 18:38:43 2006 From: george.sakkis at gmail.com (George Sakkis) Date: Tue, 31 Oct 2006 23:38:43 -0000 Subject: Reading records from file and sorting Message-ID: <1162337923.747767.323100@b28g2000cwb.googlegroups.com> Is there a more elegant and/or faster way to read some records from a file and then sort them by different fields ? What I have now is too specific and error-prone in general: import numpy as N records = N.fromfile(a_file, dtype=N.dtype('i2,i4')) records_by_f0 = records.take(records.getfield('i2').argsort()) records_by_f1 = records.take(records.getfield('i4',2).argsort()) If there's a better way, I'd like to see it; bonus points for in-place sorting. Thanks, George ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From erin.sheldon at gmail.com Tue Oct 31 23:02:51 2006 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Tue, 31 Oct 2006 23:02:51 -0500 Subject: installing numpy on Tiger In-Reply-To: <6ce0ac130610311646m79374541l57a5e1753af56e81@mail.gmail.com> References: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> <331116dc0610311138m2eb27c23g4404a14a7084ad45@mail.gmail.com> <6ce0ac130610311646m79374541l57a5e1753af56e81@mail.gmail.com> Message-ID: <331116dc0610312002x38212d63o1d495745186607eb@mail.gmail.com> On 10/31/06, Brian Granger wrote: > Nice binaries for gfortran can be found here: > > hpc.sourceforge.net/ Do you know how to make macports recognize this compiler if it is outside the macports tree? I had a version of gfortran (4.2.0) already on my system, but it insisted on compiling its own version (4.1.1) Erin ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Sun Oct 1 07:49:31 2006 From: faltet at carabos.com (Francesc Altet) Date: Sun, 01 Oct 2006 13:49:31 +0200 Subject: [Numpy-discussion] Non-writeable default for numpy.ndarray In-Reply-To: <451D9DBF.1030303@ee.byu.edu> References: <200609291723.26896.faltet@carabos.com> <451D45D9.6050005@ieee.org> <200609291844.07928.faltet@carabos.com> <451D9DBF.1030303@ee.byu.edu> Message-ID: <1159703372.3950.29.camel@localhost.localdomain> El dv 29 de 09 del 2006 a les 16:27 -0600, en/na Travis Oliphant va escriure: > Francesc Altet wrote: > > >I see. Thanks for the explanation. > > > > > You deserve the thanks for the great testing of less-traveled corners of > NumPy. It's exactly the kind of thing needed to get NumPy ready for > release. You are welcome, but actually the merit is more of the pytables test suite than mine :-) Travis, I've expressed publicly many times my gratitude to you, but I'm going to do so once more(so, be ready :-): I was absolutely thrilled that each and everyone of my bug reports were either solved or declared features in a matter of 1 or 2 days as maximum, and this is just *amazing* in a complex software like NumPy. I don't need to say it again (but will do): You have made of NumPy *the* reference of python array packages, and better that this, you have made this only in one year and a half! Definitely, I have come to the conclusion that you are not human, but rather a MONSTER! :-) Also thanks goes to Charles Harris, for implementing very fine-tuned sorting algorithms in both numpy and numarray packages. His work is very useful for us. Finally, and although not directly related with numpy, I'd like to express my gratitude and acknowledgement to David Cooke (and Tim Hochberg) for it's work on numexpr: it's a perfect complement to numpy in that it allows to accelerate even more many of the basic operations with arrays. All of you, and the rest of the numpy community are a first-class team, and this is why it makes me so happy to help you! -- >0,0< Francesc Altet http://www.carabos.com/ V V C?rabos Coop. V. Enjoy Data "-" From gael.varoquaux at normalesup.org Sun Oct 1 15:42:43 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 1 Oct 2006 21:42:43 +0200 Subject: [Numpy-discussion] Adding an assert to ctypeslib.py Message-ID: <20061001194242.GC17427@clipper.ens.fr> Hi, I just lost a bit of time because I had an old version of ctypes (the one that comes with ubuntu dapper). It doesn't work with ctypeslib.py. I think we should add a warning just after the import. Cheers, Ga?l -------------- next part -------------- 22d21 < assert ctypes.__version__ >= '0.9.9.6' From kferrio at cox.net Sun Oct 1 22:32:15 2006 From: kferrio at cox.net (Kyle Ferrio) Date: Sun, 01 Oct 2006 19:32:15 -0700 Subject: [Numpy-discussion] Problem building from svn under cygwin Message-ID: <45207A2F.6020409@cox.net> You're correct. I built python with cygwin. Thanks for the clue. Robert Kern wrote: Coatimundi wrote: > I have the latest cygwin environment and am using gcc 3.4.4. to build numpy. > > Configure works: > > python setup.py configure --compiler=mingw32 > > Build fails: > > python setup.py build --compiler=mingw32 What Python are you using? It looks like you built Python 2.5 using cygwin (that is, the UNIX emulation layer). You cannot build your extensions with mingw32 in that case. Don't use --compiler at all and you will be fine. If you are using the regular Python binaries, then you should be using --compiler=mingw32, but you're getting the wrong Python headers in that case. -- Robert Kern From olivetti at itc.it Mon Oct 2 04:44:51 2006 From: olivetti at itc.it (Emanuele Olivetti) Date: Mon, 02 Oct 2006 10:44:51 +0200 Subject: [Numpy-discussion] bincount on multidimensional array (missing 'axis' parameter) Message-ID: <4520D183.4030003@itc.it> Hi, I have to perform bincount on each column of a matrix so I was wondering why there is no 'axis=?' parameter in bincount function. Currently I'm looping through columns using a python loop but I'd appreciate to have the loop embedded in the bincount function (and possibily using a C loop, since I'm doing this stuff thousands times in a case of many columns). Is there a good alternative solution to extending bincount? Suggestions? Thanks in advance, Emanuele From perhamyhavv at intel.net Mon Oct 2 13:21:46 2006 From: perhamyhavv at intel.net (Jeno Kingsley) Date: Mon, 2 Oct 2006 18:21:46 +0100 Subject: [Numpy-discussion] Hi Message-ID: <01c6e63e$dadd39d0$7801a8c0@zuccoli> Hi, ClAxLIS VlAxGRA AMBxlEN VALxlUM Economize over 50% http://www.defunjeriondsafun.com _____ farms. The only really puzzling thing about the 3D map was what looked enthusiastically. My medication was beginning to override the pain and that I could see-except in the close vicinity of the device. I moved -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mailer-Daemon at lists.sourceforge.net Mon Oct 2 12:52:49 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Mon, 02 Oct 2006 09:52:49 -0700 Subject: [Numpy-discussion] Warning: message 1GUFPK-0001wW-5k delayed 10 hours Message-ID: This message was created automatically by mail delivery software. A message that you sent has not yet been delivered to one or more of its recipients after more than 10 hours on the queue on externalmx-1.sourceforge.net. The message identifier is: 1GUFPK-0001wW-5k The date of the message is: Mon, 02 Oct 2006 09:32:13 -0200 The subject of the message is: doctor, at the revolt is located treating the planet he gold; The address to which the message has not yet been delivered is: numpy-discussion at lists.sourceforge.net Delay reason: SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is No action is required on your part. Delivery attempts will continue for some time, and this warning may be repeated at intervals if the message remains undelivered. Eventually the mail delivery software will give up, and when that happens, the message will be returned to you. From rowen at cesmail.net Mon Oct 2 14:08:11 2006 From: rowen at cesmail.net (Russell E. Owen) Date: Mon, 02 Oct 2006 11:08:11 -0700 Subject: [Numpy-discussion] numarray and/or Numeric with python 2.5? Message-ID: Has anyone tried to build numarray or Numeric with python 2.5? Since the type of array index was changed, I'm wondering if it works (and if so, if there are any gotchas to worry about). (And yes, I do plan to switch to numpy, but meanwhile I'm still using numarray and Numeric.) Regards, -- Russell From jmiller at stsci.edu Mon Oct 2 15:02:32 2006 From: jmiller at stsci.edu (Todd Miller) Date: Mon, 02 Oct 2006 15:02:32 -0400 Subject: [Numpy-discussion] numarray and/or Numeric with python 2.5? In-Reply-To: References: Message-ID: <45216248.60905@stsci.edu> Russell E. Owen wrote: > Has anyone tried to build numarray or Numeric with python 2.5? Since the > numarray builds and runs fine for Python-2.5, but only as 32-bit. > type of array index was changed, I'm wondering if it works (and if so, > NOTE: numarray does not exploit the new ssize_t index in Python-2.5 so numarray is not 64-bit enabled. Regards, Todd From lars.bittrich at googlemail.com Mon Oct 2 03:26:56 2006 From: lars.bittrich at googlemail.com (Lars Bittrich) Date: Mon, 2 Oct 2006 09:26:56 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) Message-ID: <200610020926.56965.lars.bittrich@googlemail.com> Hi all, recently I tried to compile numpy/scipy with Python 2.5 and Intel C++ compiler 9.1. At first everything was fine, but the scipy test produced a few errors. The reason was a little difference: numpy svn(1.0.dev3233) with Intel C compiler (Python 2.5): ----------------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:x[i] Out[4]:1.0 numpy svn(1.0.dev3233) with GCC 3.3 (Python 2.3): -------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:print x[i] Out[4]:array([1]) The Intel version gives me a scalar whereas the gcc version an array. Maybe Python 2.5 is causing this problem but my first guess was the compiler. A typical error message from scipy.test(10): ====================================================================== ERROR: check_chebyc (scipy.special.tests.test_basic.test_chebyc) ---------------------------------------------------------------------- Traceback (most recent call last): File "[...]/lib/python2.5/site-packages/scipy/special/tests/test_basic.py", line 667, in check_chebyc C1 = chebyc(1) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 461, in chebyc p = orthopoly1d(x,w,hn,kn,wfunc=lambda x: 1.0/sqrt(1-x*x/4.0),limits=(-2,2),monic=monic) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 78, in __init__ equiv_weights = [weights[k] / wfunc(roots[k]) for k in range(len(roots))] TypeError: object of type 'numpy.complex128' has no len() Does anyone know how to resolve the problem? Best regards, Lars From oliphant at ee.byu.edu Mon Oct 2 17:53:08 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 02 Oct 2006 15:53:08 -0600 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) In-Reply-To: <200610020926.56965.lars.bittrich@googlemail.com> References: <200610020926.56965.lars.bittrich@googlemail.com> Message-ID: <45218A44.3070101@ee.byu.edu> Lars Bittrich wrote: >Hi all, > >recently I tried to compile numpy/scipy with Python 2.5 and Intel C++ compiler >9.1. At first everything was fine, but the scipy test produced a few errors. >The reason was a little difference: > >numpy svn(1.0.dev3233) with Intel C compiler (Python 2.5): >----------------------------------------------------------- >In [1]:from numpy import ones, zeros, integer > >In [2]: > >In [2]:x = ones(1) > >In [3]:i = zeros(1, integer) > >In [4]:x[i] >Out[4]:1.0 > >numpy svn(1.0.dev3233) with GCC 3.3 (Python 2.3): >-------------------------------------------------- >In [1]:from numpy import ones, zeros, integer > >In [2]: > >In [2]:x = ones(1) > >In [3]:i = zeros(1, integer) > >In [4]:print x[i] >Out[4]:array([1]) > >The Intel version gives me a scalar whereas the gcc version an array. Maybe >Python 2.5 is causing this problem but my first guess was the compiler. > > > This is a Python 2.5 issue (the new __index__ method) was incorrectly implemented and allowing a 1-d array to be interpreted as an index. This should be fixed in SVN. -Travis From lars.bittrich at googlemail.com Sun Oct 1 08:35:06 2006 From: lars.bittrich at googlemail.com (Lars Bittrich) Date: Sun, 1 Oct 2006 14:35:06 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) Message-ID: <200610011435.06426.lars.bittrich@googlemail.com> Hi all, recently I tried to compile numpy/scipy with Python 2.5 and Intel C++ compiler 9.1. At first everything was fine, but the scipy test produced a few errors. The reason was a little difference: numpy svn(1.0.dev3233) with Intel C compiler (Python 2.5): ----------------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:x[i] Out[4]:1.0 numpy svn(1.0.dev3233) with GCC 3.3 (Python 2.3): -------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:print x[i] Out[4]:array([1]) The Intel version gives me a scalar whereas the gcc version an array. Maybe Python 2.5 is causing this problem but my first guess was the compiler. A typical error message from scipy.test(10): ====================================================================== ERROR: check_chebyc (scipy.special.tests.test_basic.test_chebyc) ---------------------------------------------------------------------- Traceback (most recent call last): File "[...]/lib/python2.5/site-packages/scipy/special/tests/test_basic.py", line 667, in check_chebyc C1 = chebyc(1) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 461, in chebyc p = orthopoly1d(x,w,hn,kn,wfunc=lambda x: 1.0/sqrt(1-x*x/4.0),limits=(-2,2),monic=monic) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 78, in __init__ equiv_weights = [weights[k] / wfunc(roots[k]) for k in range(len(roots))] TypeError: object of type 'numpy.complex128' has no len() Does anyone know how to resolve the problem? Best regards, Lars From lars.bittrich at googlemail.com Sun Oct 1 08:10:34 2006 From: lars.bittrich at googlemail.com (Lars Bittrich) Date: Sun, 1 Oct 2006 14:10:34 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) Message-ID: <200610011410.34552.lars.bittrich@googlemail.com> Hi all, recently I tried to compile numpy/scipy with Python 2.5 and Intel C++ compiler 9.1. At first everything was fine, but the scipy test produced a few errors. The reason was a little difference: numpy svn(1.0.dev3233) with Intel C compiler (Python 2.5): ----------------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:x[i] Out[4]:1.0 numpy svn(1.0.dev3233) with GCC 3.3 (Python 2.3): -------------------------------------------------- In [1]:from numpy import ones, zeros, integer In [2]: In [2]:x = ones(1) In [3]:i = zeros(1, integer) In [4]:print x[i] Out[4]:array([1]) The Intel version gives me a scalar whereas the gcc version an array. Maybe Python 2.5 is causing this problem but my first guess was the compiler. A typical error message from scipy.test(10): ====================================================================== ERROR: check_chebyc (scipy.special.tests.test_basic.test_chebyc) ---------------------------------------------------------------------- Traceback (most recent call last): File "[...]/lib/python2.5/site-packages/scipy/special/tests/test_basic.py", line 667, in check_chebyc C1 = chebyc(1) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 461, in chebyc p = orthopoly1d(x,w,hn,kn,wfunc=lambda x: 1.0/sqrt(1-x*x/4.0),limits=(-2,2),monic=monic) File "[...]/lib/python2.5/site-packages/scipy/special/orthogonal.py", line 78, in __init__ equiv_weights = [weights[k] / wfunc(roots[k]) for k in range(len(roots))] TypeError: object of type 'numpy.complex128' has no len() Does anyone know how to resolve the problem? Best regards, Lars From 13640887 at sun.ac.za Mon Oct 2 18:10:33 2006 From: 13640887 at sun.ac.za (Albert Strasheim) Date: Tue, 3 Oct 2006 00:10:33 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that Message-ID: Hello all I recently started looking at David Cournapeau's PyEM package, specifically his implementation of the K-Means algorithm. He implemented part of this algorithm with in pure Python version and also provided a Pyrex alternative that is significantly faster (about 10 times with the data I tested with). I tried to figure out why this is so. The job of this part of the algorithm is pretty simple: from a bunch of cluster means (the codebook) find the nearest cluster mean for each data point. The first attempt at implementing this algorithm might use two for loops, one over the data points and one over the cluster means, computing a Euclidean distance at each step. Slightly better is to use one loop and some broadcasting. By randomly trying some code, I arrived at the following one-liner: N.sum((data[...,N.newaxis,...]-code)**2, 2).argmin(axis=1) where data = N.random.randn(2000, 39) nclusters = 64 code = data[0:nclusters,:] This code was still slow, so I mangled it a bit so that I could profile it using cProfile under Python 2.5 (use profile if you have an older version of Python): def kmean(): data = N.random.randn(2000, 39) nclusters = 64 code = data[0:nclusters,:] for i in xrange(10): def a(): return data[...,N.newaxis,...] z = a() def b(): return z-code z = b() def c(): return z**2 z = c() def d(): return N.sum(z, 2) z = d() def e(): return z.argmin(axis=1) z = e() if __name__ == '__main__': import cProfile cProfile.run("kmean()") Profiler output: In [570]: %run -t -e kmean2.py 84 function calls in 8.567 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 8.567 8.567 :1() 10 0.000 0.000 0.835 0.084 fromnumeric.py:375(sum) 10 0.000 0.000 0.000 0.000 kmean2.py:10(a) 10 6.837 0.684 6.837 0.684 kmean2.py:12(b) 10 0.623 0.062 0.623 0.062 kmean2.py:14(c) 10 0.000 0.000 0.835 0.084 kmean2.py:16(d) 10 0.080 0.008 0.080 0.008 kmean2.py:18(e) 1 0.180 0.180 8.567 8.567 kmean2.py:5(kmean2) 10 0.000 0.000 0.000 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 1 0.012 0.012 0.012 0.012 {method 'randn' of 'mtrand.RandomState' objects} 10 0.835 0.083 0.835 0.083 {method 'sum' of 'numpy.ndarray' objects} It seems the b part of the computation is taking most of the time. I tried to simulate this separately: In [571]: x1 = N.random.randn(2000,39) In [572]: y1 = N.random.randn(64,39) In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms per loop In [575]: z1.shape Out[575]: (2000, 64, 39) As far as I can figure, this operation is doing 2000*64*39 subtractions. Doing this straight up yields the following: In [576]: x2 = N.random.randn(2000,64,39) In [577]: y2 = N.random.randn(2000,64,39) In [578]: %timeit z2 = x2-y2 10 loops, best of 3: 108 ms per loop Does anybody have any ideas on why this is so much faster? Hopefully I didn't mess up somewhere... Thanks! Regards, Albert P.S. I'm used NumPy 1.0rc1 with Python 2.5 on Windows for these tests. I get similar results with 1.0.dev3239 with Python 2.4.3 on Linux. P.P.S. SourceForge and GMail *really* don't like each other right now. From oliphant at ee.byu.edu Mon Oct 2 19:03:50 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 02 Oct 2006 17:03:50 -0600 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: References: Message-ID: <45219AD6.5020203@ee.byu.edu> Albert Strasheim wrote: >In [571]: x1 = N.random.randn(2000,39) > >In [572]: y1 = N.random.randn(64,39) > >In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms >per loop > >In [575]: z1.shape >Out[575]: (2000, 64, 39) > >As far as I can figure, this operation is doing 2000*64*39 subtractions. >Doing this straight up yields the following: > >In [576]: x2 = N.random.randn(2000,64,39) > >In [577]: y2 = N.random.randn(2000,64,39) > >In [578]: %timeit z2 = x2-y2 >10 loops, best of 3: 108 ms per loop > >Does anybody have any ideas on why this is so much faster? Hopefully I >didn't mess up somewhere... > > I suspect I know why, although the difference seems rather large. There is code optimization that is being taken advantage of in the second case. If you have contiguous arrays (no broadcasting needed), then 1 C-loop is used for the subtraction (your second case). In the first case you are using broadcasting to generate the larger array. This requires more complicated looping constructs under the covers which causes your overhead. Bascially, you will have 64*39 1-d loops of 2000 elements each in the first example with a bit of calculation over-head to reset the pointers before each loop. In the ufunc code, compare the ONE_UFUNCLOOP case with the NOBUFFER_UFUNCLOOP case. If you want to be sure what is running un-comment the fprintf statements so you can tell. I'm surprised the overhead of adjusting pointers is so high, but then again you are probably getting a lot of cache misses in the first case so there is more to it than that, the loops may run more slowly too. -Travis From tim.hochberg at ieee.org Mon Oct 2 19:49:52 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 02 Oct 2006 16:49:52 -0700 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45219AD6.5020203@ee.byu.edu> References: <45219AD6.5020203@ee.byu.edu> Message-ID: <4521A5A0.8040902@ieee.org> Travis Oliphant wrote: > Albert Strasheim wrote: > > >> In [571]: x1 = N.random.randn(2000,39) >> >> In [572]: y1 = N.random.randn(64,39) >> >> In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms >> per loop >> >> In [575]: z1.shape >> Out[575]: (2000, 64, 39) >> >> As far as I can figure, this operation is doing 2000*64*39 subtractions. >> Doing this straight up yields the following: >> >> In [576]: x2 = N.random.randn(2000,64,39) >> >> In [577]: y2 = N.random.randn(2000,64,39) >> >> In [578]: %timeit z2 = x2-y2 >> 10 loops, best of 3: 108 ms per loop >> >> Does anybody have any ideas on why this is so much faster? Hopefully I >> didn't mess up somewhere... >> >> >> > > I suspect I know why, although the difference seems rather large. There > is code optimization that is being taken advantage of in the second > case. If you have contiguous arrays (no broadcasting needed), then 1 > C-loop is used for the subtraction (your second case). > > In the first case you are using broadcasting to generate the larger > array. This requires more complicated looping constructs under the > covers which causes your overhead. Bascially, you will have 64*39 1-d > loops of 2000 elements each in the first example with a bit of > calculation over-head to reset the pointers before each loop. > > In the ufunc code, compare the ONE_UFUNCLOOP case with the > NOBUFFER_UFUNCLOOP case. If you want to be sure what is running > un-comment the fprintf statements so you can tell. > > I'm surprised the overhead of adjusting pointers is so high, but then > again you are probably getting a lot of cache misses in the first case > so there is more to it than that, the loops may run more slowly too. > > I suspect that Travis is basically right about why your simple subtraction runs much faster than your test case. However, that doesn't mean you can't do better than at present. When dealing with large, multidimensional arrays my experience has been that unrolling all of the for loops is frequently counterproductive. I chalk this up to two factors: first that you tend to end up generating large temporary arrays and this in turn leads to cache misses. Second, you loose flexibility in how you perform the calculation, which in turns limits other possible explanations. I just spent a while playing with this, and assuming I've correctly translated your original intent I've come up with two alternative, looping versions that run, respectively 2 and 3 times faster. I've a feeling that kmean3, the fastest one, still has a little more room to be sped up, but I'm out of time now. Code is below -tim import numpy as N data = N.random.randn(2000, 39) def kmean0(data): nclusters = 64 code = data[0:nclusters,:] return N.sum((data[...,N.newaxis,...]-code)**2, 2).argmin(axis=1) def kmean1(data): nclusters = 64 code = data[0:nclusters,:] z = data[:,N.newaxis,:] z = z-code z = z**2 z = N.sum(z, 2) return z.argmin(axis=1) def kmean2(data): nclusters = 64 naxes = data.shape[-1] code = data[0:nclusters,:] data = data[:, N.newaxis] allz = N.zeros([len(data)]) for i, x in enumerate(data): z = (x - code) z **= 2 allz[i] = z.sum(-1).argmin(0) return allz def kmean3(data): nclusters = 64 naxes = data.shape[-1] code = data[0:nclusters] totals = N.zeros([nclusters, len(data)], float) transdata = data.transpose().copy() for cluster, tot in zip(code, totals): for di, ci in zip(transdata, cluster): delta = di - ci delta **=2 tot += delta return totals.argmin(axis=0) if __name__ == '__main__': assert N.alltrue(kmean0(data) == kmean1(data)) assert N.alltrue(kmean0(data) == kmean2(data)) assert N.alltrue(kmean0(data) == kmean3(data)) from timeit import Timer print Timer('kmean0(data)', 'from scratch import kmean0, data').timeit(3) print Timer('kmean1(data)', 'from scratch import kmean1, data').timeit(3) print Timer('kmean2(data)', 'from scratch import kmean2, data').timeit(3) print Timer('kmean3(data)', 'from scratch import kmean3, data').timeit(3) From tim.hochberg at ieee.org Mon Oct 2 20:17:46 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 02 Oct 2006 17:17:46 -0700 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521A5A0.8040902@ieee.org> References: <45219AD6.5020203@ee.byu.edu> <4521A5A0.8040902@ieee.org> Message-ID: <4521AC2A.5050101@ieee.org> Tim Hochberg wrote: > Travis Oliphant wrote: > >> Albert Strasheim wrote: >> >> >> >>> In [571]: x1 = N.random.randn(2000,39) >>> >>> In [572]: y1 = N.random.randn(64,39) >>> >>> In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms >>> per loop >>> >>> In [575]: z1.shape >>> Out[575]: (2000, 64, 39) >>> >>> As far as I can figure, this operation is doing 2000*64*39 subtractions. >>> Doing this straight up yields the following: >>> >>> In [576]: x2 = N.random.randn(2000,64,39) >>> >>> In [577]: y2 = N.random.randn(2000,64,39) >>> >>> In [578]: %timeit z2 = x2-y2 >>> 10 loops, best of 3: 108 ms per loop >>> >>> Does anybody have any ideas on why this is so much faster? Hopefully I >>> didn't mess up somewhere... >>> >>> >>> [SNIP] > > > I just spent a while playing with this, and assuming I've correctly > translated your original intent I've come up with two alternative, > looping versions that run, respectively 2 and 3 times faster. I've a > feeling that kmean3, the fastest one, still has a little more room to be > sped up, but I'm out of time now. Code is below > > -tim > > > One more iterations, this time a little algorithmic improvement, and were up to 4x as fast as the original code. Here we take advantage of the fact that the term di**2 is constant across the axis we are minimizing on, computation of ci**2 can be hoisted out of the loop and the -2*di*ci term can be rejiggered to di*ci by appropriate rescaling. This reduces the computation in the inner loop from a subtract and a multiply to just a multiply. def kmean4(data): nclusters = 64 naxes = data.shape[-1] code = data[:nclusters] transdata = data.transpose().copy() totals = N.empty([nclusters, len(data)], float) code2 = (code**2).sum(-1) code2 *= -0.5 totals[:] = code2[:, N.newaxis] for cluster, tot in zip(code, totals): for di, ci in zip(transdata, cluster): tot += di*ci return totals.argmax(axis=0) [SNIP CODE] From oliphant at ee.byu.edu Mon Oct 2 20:32:26 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 02 Oct 2006 18:32:26 -0600 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45219AD6.5020203@ee.byu.edu> References: <45219AD6.5020203@ee.byu.edu> Message-ID: <4521AF9A.50701@ee.byu.edu> Travis Oliphant wrote: > >I suspect I know why, although the difference seems rather large. > [snip] >I'm surprised the overhead of adjusting pointers is so high, but then >again you are probably getting a lot of cache misses in the first case >so there is more to it than that, the loops may run more slowly too. > > I'm personally bothered that this example runs so much more slowly. I don't think it should. Perhaps it is unavoidable because of the memory-layout issues. It is just hard to believe that the overhead for calling into the loop and adjusting the pointers is so much higher. But, that isn't the problem, here. Notice the following: x3 = N.random.rand(39,2000) x4 = N.random.rand(39,64,1) %timeit z3 = x3[:,None,:] - x4 10 loops, best of 3: 76.4 ms per loop Hmm... It looks like cache misses are a lot more important than making sure the inner loop is taken over the largest number of variables (that's the current way ufuncs decide which axis ought to be used as the 1-d loop). Perhaps those inner 1-d loops could be optimized (using prefetch or something) to reduce the number of cache misses on the inner computation, and the concept of looping over the largest dimension (instead of the last dimension) should be re-considered. Ideas, -Travis From 13640887 at sun.ac.za Mon Oct 2 20:42:24 2006 From: 13640887 at sun.ac.za (Albert Strasheim) Date: Tue, 3 Oct 2006 02:42:24 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521AF9A.50701@ee.byu.edu> Message-ID: Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant > Sent: 03 October 2006 02:32 > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] Vectorizing code, for loops, and all that > > Travis Oliphant wrote: > > > > >I suspect I know why, although the difference seems rather large. > > > [snip] > > >I'm surprised the overhead of adjusting pointers is so high, but then > >again you are probably getting a lot of cache misses in the first case > >so there is more to it than that, the loops may run more slowly too. > > > > > > I'm personally bothered that this example runs so much more slowly. I > don't think it should. Perhaps it is unavoidable because of the > memory-layout issues. It is just hard to believe that the overhead for > calling into the loop and adjusting the pointers is so much higher. Firstly, thanks to Tim... I'll try his functions tomorrow. Meanwhile, I can confirm that the NOBUFFER_UFUNCLOOP case in PyUFunc_GenericFunction is getting exercised in the slower case. Here's some info on what's happening, courtesy of Rational Quantify: case NOBUFFER_UFUNCLOOP: while (loop->index < loop->size) { for (i=0; inargs; i++) loop->bufptr[i] = loop->iters[i]->dataptr; [1] loop->function((char **)loop->bufptr, &(loop->bufcnt), loop->steps, loop->funcdata); [2] UFUNC_CHECK_ERROR(loop); for (i=0; inargs; i++) { PyArray_ITER_NEXT(loop->iters[i]); [3] } loop->index++; } break; [1] 12.97% of function time [2] 8.65% of functiont ime [3] 62.14% of function time If statistics from elsewhere in the code would be helpful, let me know, and I'll see if I can convince Quantify to cough it up. > Thanks for all code and suggestions. Cheers, Albert From oliphant at ee.byu.edu Mon Oct 2 21:50:35 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 02 Oct 2006 19:50:35 -0600 Subject: [Numpy-discussion] ***[Possible UCE]*** Re: Vectorizing code, for loops, and all that In-Reply-To: References: Message-ID: <4521C1EB.30205@ee.byu.edu> Albert Strasheim wrote: >Hello all > > > >>-----Original Message----- >>From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- >>discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant >>Sent: 03 October 2006 02:32 >>To: Discussion of Numerical Python >>Subject: Re: [Numpy-discussion] Vectorizing code, for loops, and all that >> >>Travis Oliphant wrote: >> >> >> >>>I suspect I know why, although the difference seems rather large. >>> >>> >>> >>[snip] >> >> >> >>>I'm surprised the overhead of adjusting pointers is so high, but then >>>again you are probably getting a lot of cache misses in the first case >>>so there is more to it than that, the loops may run more slowly too. >>> >>> >>> >>> >>I'm personally bothered that this example runs so much more slowly. I >>don't think it should. Perhaps it is unavoidable because of the >>memory-layout issues. It is just hard to believe that the overhead for >>calling into the loop and adjusting the pointers is so much higher. >> >> > >Firstly, thanks to Tim... I'll try his functions tomorrow. > >Meanwhile, I can confirm that the NOBUFFER_UFUNCLOOP case in >PyUFunc_GenericFunction is getting exercised in the slower case. Here's some >info on what's happening, courtesy of Rational Quantify: > >case NOBUFFER_UFUNCLOOP: >while (loop->index < loop->size) { > for (i=0; inargs; i++) > loop->bufptr[i] = loop->iters[i]->dataptr; [1] > > loop->function((char **)loop->bufptr, &(loop->bufcnt), > loop->steps, loop->funcdata); [2] > UFUNC_CHECK_ERROR(loop); > > for (i=0; inargs; i++) { > PyArray_ITER_NEXT(loop->iters[i]); [3] > } > loop->index++; >} >break; > >[1] 12.97% of function time >[2] 8.65% of functiont ime >[3] 62.14% of function time > >If statistics from elsewhere in the code would be helpful, let me know, and >I'll see if I can convince Quantify to cough it up. > > > Please run the same test but using x1 = N.random.rand(39,2000) x2 = N.random.rand(39,64,1) z1 = x1[:,N.newaxis,:] - x2 Thanks, -Travis From Mailer-Daemon at lists.sourceforge.net Tue Oct 3 02:06:57 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Mon, 02 Oct 2006 23:06:57 -0700 Subject: [Numpy-discussion] Warning: message 1GUFPK-0001wW-5k delayed 24 hours Message-ID: This message was created automatically by mail delivery software. A message that you sent has not yet been delivered to one or more of its recipients after more than 24 hours on the queue on externalmx-1.sourceforge.net. The message identifier is: 1GUFPK-0001wW-5k The date of the message is: Mon, 02 Oct 2006 09:32:13 -0200 The subject of the message is: doctor, at the revolt is located treating the planet he gold; The address to which the message has not yet been delivered is: numpy-discussion at lists.sourceforge.net Delay reason: SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is No action is required on your part. Delivery attempts will continue for some time, and this warning may be repeated at intervals if the message remains undelivered. Eventually the mail delivery software will give up, and when that happens, the message will be returned to you. From david at ar.media.kyoto-u.ac.jp Tue Oct 3 03:33:01 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 03 Oct 2006 16:33:01 +0900 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: References: Message-ID: <4522122D.9070407@ar.media.kyoto-u.ac.jp> Albert Strasheim wrote: > Hello all > > I recently started looking at David Cournapeau's PyEM package, specifically > his implementation of the K-Means algorithm. He implemented part of this > algorithm with in pure Python version and also provided a Pyrex alternative > that is significantly faster (about 10 times with the data I tested with). I > tried to figure out why this is so. > > For what it worths, PyEM has been quietly updated in the past weeks: most responses I got initially were people reporting errors almost always related to change in numpy API changes, so I decided to keep quiet for a while, waiting for an API freeze on numpy side (last versions use vq from scipy.cluster, for example, the plotting have been much improved, a fast Ctype version for diagonal gaussian kernel enables to run EM on hundred of thousand frames of several dozens dimensions with several dozens of gaussian in the mixture in reasonable times) > The job of this part of the algorithm is pretty simple: from a bunch of > cluster means (the codebook) find the nearest cluster mean for each data > point. The first attempt at implementing this algorithm might use two for > loops, one over the data points and one over the cluster means, computing a > Euclidean distance at each step. Slightly better is to use one loop and some > broadcasting. > > By randomly trying some code, I arrived at the following one-liner: > > N.sum((data[...,N.newaxis,...]-code)**2, 2).argmin(axis=1) > > where > > data = N.random.randn(2000, 39) > nclusters = 64 > code = data[0:nclusters,:] > > This code was still slow, so I mangled it a bit so that I could profile it > using cProfile under Python 2.5 (use profile if you have an older version of > Python): > > def kmean(): > data = N.random.randn(2000, 39) > nclusters = 64 > code = data[0:nclusters,:] > for i in xrange(10): > def a(): return data[...,N.newaxis,...] > z = a() > def b(): return z-code > z = b() > def c(): return z**2 > z = c() > def d(): return N.sum(z, 2) > z = d() > def e(): return z.argmin(axis=1) > z = e() > > if __name__ == '__main__': > import cProfile > cProfile.run("kmean()") > > Profiler output: > I got exactly the same kind of weird behaviour in other places of PyEM (apparently simple substractions taking a lot of time compared to other parts although it should theoratically be negligeable: eg X - mu where X is (n, d) and mu (d, 1) takeing almost as much time as exp( X**2) !). It would be great to at least know the origin of this non-intuitive result, David From 13640887 at sun.ac.za Tue Oct 3 05:12:36 2006 From: 13640887 at sun.ac.za (Albert Strasheim) Date: Tue, 3 Oct 2006 11:12:36 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521C1EB.30205@ee.byu.edu> Message-ID: Hello all > -----Original Message----- > From: Travis Oliphant [mailto:oliphant at ee.byu.edu] > Sent: 03 October 2006 03:51 > To: Discussion of Numerical Python; Albert Strasheim > Subject: Re: ***[Possible UCE]*** Re: [Numpy-discussion] Vectorizing > code, for loops, and all that > > > >Meanwhile, I can confirm that the NOBUFFER_UFUNCLOOP case in > >PyUFunc_GenericFunction is getting exercised in the slower case. > >Here's > some > >info on what's happening, courtesy of Rational Quantify: > > > >case NOBUFFER_UFUNCLOOP: > >while (loop->index < loop->size) { > > for (i=0; inargs; i++) > > loop->bufptr[i] = loop->iters[i]->dataptr; [1] > > > > loop->function((char **)loop->bufptr, &(loop->bufcnt), > > loop->steps, loop->funcdata); [2] > > UFUNC_CHECK_ERROR(loop); > > > > for (i=0; inargs; i++) { > > PyArray_ITER_NEXT(loop->iters[i]); [3] > > } > > loop->index++; > >} > >break; > > > >[1] 12.97% of function time > >[2] 8.65% of functiont ime > >[3] 62.14% of function time > > > >If statistics from elsewhere in the code would be helpful, let me > >know, > and > >I'll see if I can convince Quantify to cough it up. > > > Please run the same test but using > > x1 = N.random.rand(39,2000) > x2 = N.random.rand(39,64,1) > > z1 = x1[:,N.newaxis,:] - x2 Very similar results to what I had previously: [1] 10.88% [2] 7.25% [3] 68.25% Cheers, Albert From david at ar.media.kyoto-u.ac.jp Tue Oct 3 06:55:31 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 03 Oct 2006 19:55:31 +0900 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) Message-ID: <452241A3.3000702@ar.media.kyoto-u.ac.jp> Hi, I was wondering if there was any way to speed up the following code: y = N.zeros((n, K)) for i in range(K): y[:, i] = gauss_den(data, mu[i, :], va[i, :]) Where K is of order 1e1, n of order 1e5. Normally, gauss_den is a quite expensive function, but the profiler tells me that the indexing y[:,i] takes almost as much time as the gauss_den computation (which computes n exp !). To see if the profiler is "right", i replaces with the (non valid) following function: y = N.zeros((n, K)) for i in range(K): yt = gauss_den(data, mu[i, :], va[i, :]) return y Where more than 99% of the code is spent inside gauss_den. I guess the problem is coming from the fact that y being C order, y[:, i] needs accessing data in a non 'linear' way. Is there a way to speed this up ? I did something like this: y = N.zeros((K, n)) for i in range(K): y[i] = gauss_den(data, mu[i, :], va[i, :]) return y.T which works, but I don't like it very much. Isn't there any other way ? David From oliphant.travis at ieee.org Tue Oct 3 10:59:18 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 03 Oct 2006 08:59:18 -0600 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: References: Message-ID: <45227AC6.4010801@ieee.org> Albert Strasheim wrote: >>> [1] 12.97% of function time >>> [2] 8.65% of functiont ime >>> [3] 62.14% of function time >>> >>> If statistics from elsewhere in the code would be helpful, let me >>> know, >>> >> and >> >>> I'll see if I can convince Quantify to cough it up. >>> >>> >> Please run the same test but using >> >> x1 = N.random.rand(39,2000) >> x2 = N.random.rand(39,64,1) >> >> z1 = x1[:,N.newaxis,:] - x2 >> > > Very similar results to what I had previously: > > [1] 10.88% > [2] 7.25% > [3] 68.25% > > Thanks, I've got some ideas about how to speed this up by eliminating some of the unnecessary calculations going on outside of the function loop, but there will still be some speed issues depending on how the array is traversed once you get above a certain size. I'm not sure there anyway around that, ultimately, due to memory access being slow on most hardware. If anyone has any ideas, I'd love to hear them. I won't be able to get to implementing my ideas until at least Friday (also when rc2 will be released). -Travis From oliphant.travis at ieee.org Tue Oct 3 11:04:33 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 03 Oct 2006 09:04:33 -0600 Subject: [Numpy-discussion] RC2 to be released Friday Message-ID: <45227C01.6080109@ieee.org> I'm going to have to put off release of rc2 for Friday. I'm just too busy right now. That might help us get some speed-ups into the NOBUFFER_UFUNCLOOP code as well. My speed-up ideas are: 1) Only keep track of 1 set of coordinates instead of self->nargs sets (1 for each iterator). 2) Keep track of the dataptr for each iterator in the bufptr array (so a copy isn't necessary) 3) Not increment the index on each iterator separately. All of these changes will be made directly in the NOBUFFER_UFUNCLOOP code. More generally, it would be nice to take these ideas and push them into other areas of the code --- perhaps through the multi-iterator that is already present. Probably, this will have to wait until 1.0.1 though. -Travis From kortmann at ideaworks.com Tue Oct 3 12:03:25 2006 From: kortmann at ideaworks.com (kortmann at ideaworks.com) Date: Tue, 3 Oct 2006 09:03:25 -0700 (PDT) Subject: [Numpy-discussion] ValueError: object too deep for desired array In-Reply-To: References: Message-ID: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\Documents and Settings\kenny>cd c:\lameness C:\lameness>c:\Python24\python.exe templatewindow.py a = [[ 1.00013175e+00 2.63483019e-05 1.00006740e+00 5.22246363e-05 1.00008735e+00 -2.77694969e-07 -1.30756273e-07 1.03202730e-06] [ 2.63483019e-05 6.95644927e-05 -7.15426839e-07 1.99534228e-05 5.29400631e-05 -3.07638369e-09 -5.52729618e-06 -7.61431767e-06] [ 1.00006740e+00 -7.15426839e-07 1.00011917e+00 2.50407751e-05 1.00006219e+00 -2.77757947e-07 -1.30856101e-07 1.23058301e-07] [ 5.22246363e-05 1.99534228e-05 2.50407751e-05 8.21505582e-05 5.26966037e-05 -6.13563429e-09 -2.76420755e-06 -3.80791858e-06] [ 1.00008735e+00 5.29400631e-05 1.00006219e+00 5.26966037e-05 1.00020132e+00 5.01389982e-05 3.45959412e-05 3.17129503e-05] [ -2.77694969e-07 -3.07638369e-09 -2.77757947e-07 -6.13563429e-09 5.01389982e-05 3.45959412e-05 3.17129503e-05 3.27035490e-05] [ -1.30756273e-07 -5.52729618e-06 -1.30856101e-07 -2.76420755e-06 3.45959412e-05 3.17129503e-05 3.27035490e-05 3.59732704e-05] [ 1.03202730e-06 -7.61431767e-06 1.23058301e-07 -3.80791858e-06 3.17129503e-05 3.27035490e-05 3.59732704e-05 4.12184645e-05]] F = [[ -1.44231014e+03] [ -7.54006917e-02] [ -1.44227222e+03] [ -7.49199956e-02] [ -1.44242446e+03] [ -4.24684780e-02] [ -2.49566072e-02] [ -2.16978637e-02]] Traceback (most recent call last): File "C:\lameness\PlotPanel.py", line 476, in OnNo self.parent.peaktime() File "C:\lameness\PlotPanel.py", line 64, in peaktime self._replot() File "C:\lameness\PlotPanel.py", line 159, in _replot self.drawpeak() File "C:\lameness\PlotPanel.py", line 233, in drawpeak COE[:,j1-1] = linalg.solve(A,F) ValueError: object too deep for desired array Anyone know what this error could be coming from? Is it because the floating point numbers are too large, memory wise? -Kenny From tim.hochberg at ieee.org Tue Oct 3 12:12:37 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 09:12:37 -0700 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45227AC6.4010801@ieee.org> References: <45227AC6.4010801@ieee.org> Message-ID: <45228BF5.1070604@ieee.org> Travis Oliphant wrote: > Albert Strasheim wrote: > >>>> [1] 12.97% of function time >>>> [2] 8.65% of functiont ime >>>> [3] 62.14% of function time >>>> >>>> If statistics from elsewhere in the code would be helpful, let me >>>> know, >>>> >>>> >>> and >>> >>> >>>> I'll see if I can convince Quantify to cough it up. >>>> >>>> >>>> >>> Please run the same test but using >>> >>> x1 = N.random.rand(39,2000) >>> x2 = N.random.rand(39,64,1) >>> >>> z1 = x1[:,N.newaxis,:] - x2 >>> >>> >> Very similar results to what I had previously: >> >> [1] 10.88% >> [2] 7.25% >> [3] 68.25% >> >> >> > Thanks, > > I've got some ideas about how to speed this up by eliminating some of > the unnecessary calculations going on outside of the function loop, but > there will still be some speed issues depending on how the array is > traversed once you get above a certain size. I'm not sure there anyway > around that, ultimately, due to memory access being slow on most hardware. > > If anyone has any ideas, I'd love to hear them. I won't be able to > get to implementing my ideas until at least Friday (also when rc2 will > be released). > > I had an idea regarding which axis to operate on first. Rather than operate on strictly the longest axis or strictly the innermost axis, a hybrid approach could be used. We would operate on the longest axis that would not result in the inner loop overflowing the cache. The idea is minimize the loop overhead as we do now by choosing the largest axis, while at the same time attempting to maintain cache friendliness. Let's suppose that it's safe to stuff 2000 array elements into the cache[1], and lets consider two arrays where the lengths of the axes[2], from innermost to outermost are [10,100,1000] and [100,10,1000]. Under this scheme, we would loop over the length 100 axis in both cases. Looping over the length-1000 axis pull 1e6 elements into the cache. If the innermost axis alone exceeded our cache imposed limit, we would of course just loop over that axis first. -tim [1] This would really be specified in bytes and thus the number of allowable elements would vary by data type. It might also need to be tuned based on architecture. [2] I believe that there's some consolidation of adjacent axes that goes on where possible, let's assume that's already happened at this point. From oliphant at ee.byu.edu Tue Oct 3 12:14:32 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 03 Oct 2006 10:14:32 -0600 Subject: [Numpy-discussion] ValueError: object too deep for desired array In-Reply-To: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> References: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> Message-ID: <45228C68.5030302@ee.byu.edu> kortmann at ideaworks.com wrote: >Microsoft Windows XP [Version 5.1.2600] >(C) Copyright 1985-2001 Microsoft Corp. > >C:\Documents and Settings\kenny>cd c:\lameness > >C:\lameness>c:\Python24\python.exe templatewindow.py >a = [[ 1.00013175e+00 2.63483019e-05 1.00006740e+00 5.22246363e-05 > 1.00008735e+00 -2.77694969e-07 -1.30756273e-07 1.03202730e-06] > [ 2.63483019e-05 6.95644927e-05 -7.15426839e-07 1.99534228e-05 > 5.29400631e-05 -3.07638369e-09 -5.52729618e-06 -7.61431767e-06] > [ 1.00006740e+00 -7.15426839e-07 1.00011917e+00 2.50407751e-05 > 1.00006219e+00 -2.77757947e-07 -1.30856101e-07 1.23058301e-07] > [ 5.22246363e-05 1.99534228e-05 2.50407751e-05 8.21505582e-05 > 5.26966037e-05 -6.13563429e-09 -2.76420755e-06 -3.80791858e-06] > [ 1.00008735e+00 5.29400631e-05 1.00006219e+00 5.26966037e-05 > 1.00020132e+00 5.01389982e-05 3.45959412e-05 3.17129503e-05] > [ -2.77694969e-07 -3.07638369e-09 -2.77757947e-07 -6.13563429e-09 > 5.01389982e-05 3.45959412e-05 3.17129503e-05 3.27035490e-05] > [ -1.30756273e-07 -5.52729618e-06 -1.30856101e-07 -2.76420755e-06 > 3.45959412e-05 3.17129503e-05 3.27035490e-05 3.59732704e-05] > [ 1.03202730e-06 -7.61431767e-06 1.23058301e-07 -3.80791858e-06 > 3.17129503e-05 3.27035490e-05 3.59732704e-05 4.12184645e-05]] >F = [[ -1.44231014e+03] > [ -7.54006917e-02] > [ -1.44227222e+03] > [ -7.49199956e-02] > [ -1.44242446e+03] > [ -4.24684780e-02] > [ -2.49566072e-02] > [ -2.16978637e-02]] >Traceback (most recent call last): > File "C:\lameness\PlotPanel.py", line 476, in OnNo > self.parent.peaktime() > File "C:\lameness\PlotPanel.py", line 64, in peaktime > self._replot() > File "C:\lameness\PlotPanel.py", line 159, in _replot > self.drawpeak() > File "C:\lameness\PlotPanel.py", line 233, in drawpeak > COE[:,j1-1] = linalg.solve(A,F) >ValueError: object too deep for desired array > >Anyone know what this error could be coming from? Is it because the >floating point numbers are too large, memory wise? > > No, it's because you are trying to fit a 2-d array into a 1-d position. linalg.solve(A,F) produces a 2-d array in this instance (because F is a 2-d array). COE[:,j1-1:] =linalg.solve(A,F) should work or squeezing the result of the solution to a 1-d array would also work. -Travis From tim.hochberg at ieee.org Tue Oct 3 12:16:02 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 09:16:02 -0700 Subject: [Numpy-discussion] ValueError: object too deep for desired array In-Reply-To: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> References: <3267.12.216.231.149.1159891405.squirrel@webmail.ideaworks.com> Message-ID: <45228CC2.1010101@ieee.org> kortmann at ideaworks.com wrote: > Microsoft Windows XP [Version 5.1.2600] > (C) Copyright 1985-2001 Microsoft Corp. > > C:\Documents and Settings\kenny>cd c:\lameness > > C:\lameness>c:\Python24\python.exe templatewindow.py > a = [[ 1.00013175e+00 2.63483019e-05 1.00006740e+00 5.22246363e-05 > 1.00008735e+00 -2.77694969e-07 -1.30756273e-07 1.03202730e-06] > [ 2.63483019e-05 6.95644927e-05 -7.15426839e-07 1.99534228e-05 > 5.29400631e-05 -3.07638369e-09 -5.52729618e-06 -7.61431767e-06] > [ 1.00006740e+00 -7.15426839e-07 1.00011917e+00 2.50407751e-05 > 1.00006219e+00 -2.77757947e-07 -1.30856101e-07 1.23058301e-07] > [ 5.22246363e-05 1.99534228e-05 2.50407751e-05 8.21505582e-05 > 5.26966037e-05 -6.13563429e-09 -2.76420755e-06 -3.80791858e-06] > [ 1.00008735e+00 5.29400631e-05 1.00006219e+00 5.26966037e-05 > 1.00020132e+00 5.01389982e-05 3.45959412e-05 3.17129503e-05] > [ -2.77694969e-07 -3.07638369e-09 -2.77757947e-07 -6.13563429e-09 > 5.01389982e-05 3.45959412e-05 3.17129503e-05 3.27035490e-05] > [ -1.30756273e-07 -5.52729618e-06 -1.30856101e-07 -2.76420755e-06 > 3.45959412e-05 3.17129503e-05 3.27035490e-05 3.59732704e-05] > [ 1.03202730e-06 -7.61431767e-06 1.23058301e-07 -3.80791858e-06 > 3.17129503e-05 3.27035490e-05 3.59732704e-05 4.12184645e-05]] > F = [[ -1.44231014e+03] > [ -7.54006917e-02] > [ -1.44227222e+03] > [ -7.49199956e-02] > [ -1.44242446e+03] > [ -4.24684780e-02] > [ -2.49566072e-02] > [ -2.16978637e-02]] > Traceback (most recent call last): > File "C:\lameness\PlotPanel.py", line 476, in OnNo > self.parent.peaktime() > File "C:\lameness\PlotPanel.py", line 64, in peaktime > self._replot() > File "C:\lameness\PlotPanel.py", line 159, in _replot > self.drawpeak() > File "C:\lameness\PlotPanel.py", line 233, in drawpeak > COE[:,j1-1] = linalg.solve(A,F) > ValueError: object too deep for desired array > > Anyone know what this error could be coming from? Is it because the > floating point numbers are too large, memory wise?\ > No, this error has to do with the number of dimensions being wrong somewhere. Try printing out the shapes of COE[:,j-1] and linalg.solve(A,F); you'll probably see that they don't match. -tim From tim.hochberg at ieee.org Tue Oct 3 12:24:10 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 09:24:10 -0700 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: <452241A3.3000702@ar.media.kyoto-u.ac.jp> References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> Message-ID: <45228EAA.9040301@ieee.org> David Cournapeau wrote: > Hi, > > I was wondering if there was any way to speed up the following code: > > y = N.zeros((n, K)) > for i in range(K): > y[:, i] = gauss_den(data, mu[i, :], va[i, :]) > > Where K is of order 1e1, n of order 1e5. Normally, gauss_den is a quite > expensive function, but the profiler tells me that the indexing y[:,i] > takes almost as much time as the gauss_den computation (which computes n > exp !). To see if the profiler is "right", i replaces with the (non > valid) following function: > > y = N.zeros((n, K)) > for i in range(K): > yt = gauss_den(data, mu[i, :], va[i, :]) > return y > > Where more than 99% of the code is spent inside gauss_den. > > I guess the problem is coming from the fact that y being C order, y[:, > i] needs accessing data in a non 'linear' way. Is there a way to speed > this up ? I did something like this: > > y = N.zeros((K, n)) > for i in range(K): > y[i] = gauss_den(data, mu[i, :], va[i, :]) > return y.T > > which works, but I don't like it very much. Why not? > Isn't there any other way That depends on the details of gauss_den. A general note: for this kind of microoptimization puzzle, it's much easier to help if you can post a self contained example, preferably something fairly simple that still illustrates the speed issue, that we can experiment with. -tim From faltet at carabos.com Tue Oct 3 12:55:31 2006 From: faltet at carabos.com (Francesc Altet) Date: Tue, 3 Oct 2006 18:55:31 +0200 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? Message-ID: <200610031855.32056.faltet@carabos.com> Hi, I thought that numpy.isscalar was a good way of distinguising a numpy scalar from a python scalar, but it seems not: >>> numpy.isscalar(numpy.string_('3')) True >>> numpy.isscalar('3') True Is there an easy (and fast, if possible) way to check whether an object is a numpy scalar or a python one? Thanks, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From kortmann at ideaworks.com Tue Oct 3 13:02:51 2006 From: kortmann at ideaworks.com (Kenny Ortmann) Date: Tue, 3 Oct 2006 12:02:51 -0500 Subject: [Numpy-discussion] ValueError: object too deep for References: Message-ID: <00fd01c6e70d$c33af590$0700a8c0@yairlap> Yeah thanks guys, I printed them out the size and realized what it was doing then used a for loop to put it into the 1D array, but thanks Travis the way you said makes it easier than the for loop. -Kenny From haase at msg.ucsf.edu Tue Oct 3 13:24:28 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Tue, 3 Oct 2006 10:24:28 -0700 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? In-Reply-To: <200610031855.32056.faltet@carabos.com> References: <200610031855.32056.faltet@carabos.com> Message-ID: <200610031024.28188.haase@msg.ucsf.edu> Hi, a noticed the underscore in "numpy.string_" ... I thought the underscores were removed in favour of handling the "from numpy import *" separately via the __all__ variable. Is this done only for *some* members of numpy ? (sorry for hijacking your thread ...) -Sebastian Haase On Tuesday 03 October 2006 09:55, Francesc Altet wrote: > Hi, > > I thought that numpy.isscalar was a good way of distinguising a numpy > scalar > > from a python scalar, but it seems not: > >>> numpy.isscalar(numpy.string_('3')) > > True > > >>> numpy.isscalar('3') > > True > > Is there an easy (and fast, if possible) way to check whether an object is > a numpy scalar or a python one? > > Thanks, From tim.hochberg at ieee.org Tue Oct 3 13:27:13 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 10:27:13 -0700 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? In-Reply-To: <200610031855.32056.faltet@carabos.com> References: <200610031855.32056.faltet@carabos.com> Message-ID: <45229D71.3010800@ieee.org> Francesc Altet wrote: > Hi, > > I thought that numpy.isscalar was a good way of distinguising a numpy scalar > from a python scalar, but it seems not: > > >>>> numpy.isscalar(numpy.string_('3')) >>>> > True > >>>> numpy.isscalar('3') >>>> > True > > Is there an easy (and fast, if possible) way to check whether an object is a > numpy scalar or a python one? > > It looks like isinstance(x, numpy.generic) works, but I didn't test it extensively. -tim From rowen at cesmail.net Tue Oct 3 13:31:26 2006 From: rowen at cesmail.net (Russell E. Owen) Date: Tue, 03 Oct 2006 10:31:26 -0700 Subject: [Numpy-discussion] numarray and/or Numeric with python 2.5? References: <45216248.60905@stsci.edu> Message-ID: In article <45216248.60905 at stsci.edu>, Todd Miller wrote: > Russell E. Owen wrote: > > Has anyone tried to build numarray or Numeric with python 2.5? Since the > > > numarray builds and runs fine for Python-2.5, but only as 32-bit. > > type of array index was changed, I'm wondering if it works (and if so, > > > NOTE: numarray does not exploit the new ssize_t index in Python-2.5 so > numarray is not 64-bit enabled. Thank you very much. That's great. It means my current code is compatible with Python 2.5 (assuming Numeric works the same way, which seems likely). I am really looking forward to switching to numpy, but now is just a bit too early. -- Russell From oliphant at ee.byu.edu Tue Oct 3 13:47:05 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 03 Oct 2006 11:47:05 -0600 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? In-Reply-To: <200610031024.28188.haase@msg.ucsf.edu> References: <200610031855.32056.faltet@carabos.com> <200610031024.28188.haase@msg.ucsf.edu> Message-ID: <4522A219.504@ee.byu.edu> Sebastian Haase wrote: >Hi, >a noticed the underscore in "numpy.string_" ... >I thought the underscores were removed in favour of handling the >"from numpy import *" separately via the __all__ variable. >Is this done only for *some* members of numpy ? > > Yeah. The underscores are still on the conflicting type names. -Travis From oliphant at ee.byu.edu Tue Oct 3 13:47:55 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 03 Oct 2006 11:47:55 -0600 Subject: [Numpy-discussion] Fastest way of distinguish a numpy scalar of a python scalar? In-Reply-To: <45229D71.3010800@ieee.org> References: <200610031855.32056.faltet@carabos.com> <45229D71.3010800@ieee.org> Message-ID: <4522A24B.5010205@ee.byu.edu> Tim Hochberg wrote: >Francesc Altet wrote: > > >>Hi, >> >>I thought that numpy.isscalar was a good way of distinguising a numpy scalar >>from a python scalar, but it seems not: >> >> >> >> >>>>>numpy.isscalar(numpy.string_('3')) >>>>> >>>>> >>>>> >>True >> >> >> >>>>>numpy.isscalar('3') >>>>> >>>>> >>>>> >>True >> >>Is there an easy (and fast, if possible) way to check whether an object is a >>numpy scalar or a python one? >> >> >> >> >It looks like isinstance(x, numpy.generic) works, but I didn't test it >extensively. > > > That should definitely work. All the array scalars are in a hierarchy inheriting from numpy.generic. There are also sub-levels (numpy.integer, numpy.inexact, etc...) -Travis From kortmann at ideaworks.com Tue Oct 3 16:01:48 2006 From: kortmann at ideaworks.com (Kenny Ortmann) Date: Tue, 3 Oct 2006 15:01:48 -0500 Subject: [Numpy-discussion] complex numbers References: Message-ID: <005f01c6e726$c3317650$0700a8c0@yairlap> excuse my laziness for not looking this up, I googled it but could not find a solution. matlab has a isreal(array) where if the array is full of real numbers the value returned is 1. I'm translating matlab code and ran across if ~isreal(array) array = abs(array) Is there a way to check to see if a number is real or complex? and if so is there a way to extract the(a + ib) because the absolute value of a complex number is like the pythagorean therom on a and b? thanks for your help, Kenny From David.L.Goldsmith at noaa.gov Tue Oct 3 16:22:38 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Tue, 03 Oct 2006 13:22:38 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <005f01c6e726$c3317650$0700a8c0@yairlap> References: <005f01c6e726$c3317650$0700a8c0@yairlap> Message-ID: <4522C68E.4030101@noaa.gov> numpy.isreal(a) is a "top level" function (i.e., not a class member function) in numpy; here's its help doc: Help on function isreal in module numpy.lib.type_check: isreal(x) Return a boolean array where elements are True if that element is real (has zero imaginary part) For scalars, return a boolean. As for checking for pure imaginaries (i.e., real part == 0), one could use: x.imag == -j*x or numpy.isreal(i*x) or simply (x.real == 0.0) and (x.imag != 0.0) DG Kenny Ortmann wrote: > excuse my laziness for not looking this up, I googled it but could not find > a solution. > matlab has a > isreal(array) > where if the array is full of real numbers the value returned is 1. > I'm translating matlab code and ran across > > if ~isreal(array) > array = abs(array) > > Is there a way to check to see if a number is real or complex? and if so is > there a way to extract the(a + ib) because the absolute value of a complex > number is like the pythagorean therom on a and b? > > thanks for your help, > Kenny > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From David.L.Goldsmith at noaa.gov Tue Oct 3 16:33:19 2006 From: David.L.Goldsmith at noaa.gov (David L Goldsmith) Date: Tue, 03 Oct 2006 13:33:19 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <005f01c6e726$c3317650$0700a8c0@yairlap> References: <005f01c6e726$c3317650$0700a8c0@yairlap> Message-ID: <4522C90F.9050206@noaa.gov> PS: The Python built in function (i.e., you don't even need numpy for this) abs(x) is "vectorized" (i.e., accepts a (nested) sequence, incl. numpy array, argument) and overloaded to give the modulus (i.e., Pythagorean "length") of a complex number when such is its argument. DG Kenny Ortmann wrote: > excuse my laziness for not looking this up, I googled it but could not find > a solution. > matlab has a > isreal(array) > where if the array is full of real numbers the value returned is 1. > I'm translating matlab code and ran across > > if ~isreal(array) > array = abs(array) > > Is there a way to check to see if a number is real or complex? and if so is > there a way to extract the(a + ib) because the absolute value of a complex > number is like the pythagorean therom on a and b? > > thanks for your help, > Kenny > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- HMRD/ORR/NOS/NOAA From tim.hochberg at ieee.org Tue Oct 3 16:34:54 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 13:34:54 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <005f01c6e726$c3317650$0700a8c0@yairlap> References: <005f01c6e726$c3317650$0700a8c0@yairlap> Message-ID: <4522C96E.2010401@ieee.org> Kenny Ortmann wrote: > excuse my laziness for not looking this up, I googled it but could not find > a solution. > matlab has a > isreal(array) > where if the array is full of real numbers the value returned is 1. > I'm translating matlab code and ran across > > if ~isreal(array) > array = abs(array) > > Is there a way to check to see if a number is real or complex? There may be a better way, but:: alltrue(isreal(x)) Would work. As would: not sometrue(x.imag) In the above test you are already negating the test, so you could just drop the not. > and if so is > there a way to extract the(a + ib) because the absolute value of a complex > number is like the pythagorean therom on a and b? > I'm not entirely sure what you are looking for here, but x.imag and x.real will give you the real and imaginary parts. abs(x) will return the magnitude of x whether x is real or complex. x.conj() will return complex conjugate. I'm somewhat suspicious of that matlab code. The code given is discontinuous as you cross the negative real axis. Does the result subsequently get squared or something? I'm guessing that either the matlab code is doing extra work, or there are some hidden assumptions (all values are in the positive real half-plane). Or some such. In either case, you'd probably be OK just skipping the check for realness and always taking the absolute of array. I can't say for sure without more context though. -tim From tim.hochberg at ieee.org Tue Oct 3 16:38:15 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 13:38:15 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <4522C90F.9050206@noaa.gov> References: <005f01c6e726$c3317650$0700a8c0@yairlap> <4522C90F.9050206@noaa.gov> Message-ID: <4522CA37.9060108@ieee.org> David L Goldsmith wrote: > PS: The Python built in function (i.e., you don't even need numpy for > this) abs(x) is "vectorized" (i.e., accepts a (nested) sequence, incl. > numpy array, argument) and overloaded to give the modulus (i.e., > Pythagorean "length") of a complex number when such is its argument. > This isn't quite right. The built in abs function looks for the special method __abs__. So, abs(x) is equivalent to x.__abs__(). Arrays supply an appropriate __abs__ method, lists do not. For example: >>> l = [1,2,3,4] >>> abs(l) Traceback (most recent call last): File "", line 1, in ? TypeError: bad operand type for abs() >>> a = numpy.arange(5) >>> abs(a) array([0, 1, 2, 3, 4]) -tim > DG > > Kenny Ortmann wrote: > >> excuse my laziness for not looking this up, I googled it but could not find >> a solution. >> matlab has a >> isreal(array) >> where if the array is full of real numbers the value returned is 1. >> I'm translating matlab code and ran across >> >> if ~isreal(array) >> array = abs(array) >> >> Is there a way to check to see if a number is real or complex? and if so is >> there a way to extract the(a + ib) because the absolute value of a complex >> number is like the pythagorean therom on a and b? >> >> thanks for your help, >> Kenny >> >> >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share your >> opinions on IT & business topics through brief surveys -- and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> > > > From David.L.Goldsmith at noaa.gov Tue Oct 3 17:21:55 2006 From: David.L.Goldsmith at noaa.gov (David L Goldsmith) Date: Tue, 03 Oct 2006 14:21:55 -0700 Subject: [Numpy-discussion] complex numbers In-Reply-To: <4522CA37.9060108@ieee.org> References: <005f01c6e726$c3317650$0700a8c0@yairlap> <4522C90F.9050206@noaa.gov> <4522CA37.9060108@ieee.org> Message-ID: <4522D473.8020506@noaa.gov> Thanks Tim, DG Tim Hochberg wrote: > David L Goldsmith wrote: > >> PS: The Python built in function (i.e., you don't even need numpy for >> this) abs(x) is "vectorized" (i.e., accepts a (nested) sequence, incl. >> numpy array, argument) and overloaded to give the modulus (i.e., >> Pythagorean "length") of a complex number when such is its argument. >> >> > This isn't quite right. The built in abs function looks for the special > method __abs__. So, abs(x) is equivalent to x.__abs__(). Arrays supply > an appropriate __abs__ method, lists do not. For example: > > >>> l = [1,2,3,4] > >>> abs(l) > Traceback (most recent call last): > File "", line 1, in ? > TypeError: bad operand type for abs() > >>> a = numpy.arange(5) > >>> abs(a) > array([0, 1, 2, 3, 4]) > > > -tim > >> DG >> >> Kenny Ortmann wrote: >> >> >>> excuse my laziness for not looking this up, I googled it but could not find >>> a solution. >>> matlab has a >>> isreal(array) >>> where if the array is full of real numbers the value returned is 1. >>> I'm translating matlab code and ran across >>> >>> if ~isreal(array) >>> array = abs(array) >>> >>> Is there a way to check to see if a number is real or complex? and if so is >>> there a way to extract the(a + ib) because the absolute value of a complex >>> number is like the pythagorean therom on a and b? >>> >>> thanks for your help, >>> Kenny >>> >>> >>> >>> ------------------------------------------------------------------------- >>> Take Surveys. Earn Cash. Influence the Future of IT >>> Join SourceForge.net's Techsay panel and you'll get the chance to share your >>> opinions on IT & business topics through brief surveys -- and earn cash >>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >>> _______________________________________________ >>> Numpy-discussion mailing list >>> Numpy-discussion at lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >>> >>> >>> >> >> > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- HMRD/ORR/NOS/NOAA From lfqfbhj at sofresh.com Tue Oct 3 17:34:39 2006 From: lfqfbhj at sofresh.com (Alerts. Terms) Date: Tue, 03 Oct 2006 21:34:39 -0000 Subject: [Numpy-discussion] News Maps more Message-ID: <000e01c389f6$2576ab00$15f9347d@expvcianpc> VideoNew News Maps more Maps more raquo Sign in Web fewer your on can try Answers for expert different keywords. general fewer your on can fewer your on raquo Advanced Search Members: Alerts Create new group About raquo Advanced Search Members: users: Join Alerts Create groups Your search did not match any Make sure words About Searched all groups Your search did not match any any Make sure words are spelled Try different Search Members: latest the latest messages emailed search. Get the latest Try different -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: in.gif Type: image/gif Size: 9782 bytes Desc: not available URL: From kortmann at ideaworks.com Tue Oct 3 17:49:29 2006 From: kortmann at ideaworks.com (Kenny Ortmann) Date: Tue, 3 Oct 2006 16:49:29 -0500 Subject: [Numpy-discussion] Complex numbers response References: Message-ID: <007d01c6e735$cebe2a90$0700a8c0@yairlap> > There may be a better way, but:: > > alltrue(isreal(x)) > > Would work. As would: > > not sometrue(x.imag) > > In the above test you are already negating the test, so you could just > drop the not. >> and if so is >> there a way to extract the(a + ib) because the absolute value of a >> complex >> number is like the pythagorean therom on a and b? what i was looking for in the above is that when I looked up the abs(complexnum) i saw that the abs() of a complex num is (a^2 + b^2)^.5 so i was asking for a way to do this because i was assuming that i was going to have to parse through the array in a for loop and if i found a complex number then get the a and b to do the pythagorean theorom but it seems i will not have to do this, as for the matlab code see below > I'm not entirely sure what you are looking for here, but x.imag and > x.real will give you the real and imaginary parts. abs(x) will return > the magnitude of x whether x is real or complex. x.conj() will return > complex conjugate. > > I'm somewhat suspicious of that matlab code. The code given is > discontinuous as you cross the negative real axis. Does the result > subsequently get squared or something? I'm guessing that either the > matlab code is doing extra work, or there are some hidden assumptions > (all values are in the positive real half-plane). Or some such. In > either case, you'd probably be OK just skipping the check for realness > and always taking the absolute of array. I can't say for sure without > more context though. > > -tim this is taken out of a program i am converting from matlab to python, so I can not explain all of it, it would take forever, BUT COEmix, the input, is equal to a different arrays 0 and 2 rows added together. So with that being said, it can not be empty, and must be actual numbers. The reason I am "scared" to remove the isreal statement that i asked about is because the array before hand has values that are taken from square roots. I do not know if these could ever end up being complex and I am meeting with the head of this project tomorrow to discuss it, but you asked so i thought i would share. function [pospeakind,negpeakind]=peakdetect(COEmix) if ~nargin COEmix=input('enter COEmix vector or return for empty outputs\n'); if isempty(COEmix) pospeakind=[]; negpeakind=[]; return end end sizsig=size(COEmix); while isempty(COEmix)|~isnumeric(COEmix)|~all(all(isfinite(COEmix)))... |length(sizsig)>2|min(sizsig)~=1 COEmix=input(['COEmix is empty, nonnumeric, nonfinite, or nonvector:\nenter '... 'finite vector or return for empty outputs\n']); if isempty(COEmix) pospeakind=[]; negpeakind=[]; return end sizsig=size(COEmix); end if ~isreal(COEmix) COEmix=abs(COEmix); end if ~any(COEmix-COEmix(1)) pospeakind=[]; negpeakind=[]; disp('constant COEmix graph suppressed') return end From tim.hochberg at ieee.org Tue Oct 3 18:59:57 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Tue, 03 Oct 2006 15:59:57 -0700 Subject: [Numpy-discussion] Complex numbers response In-Reply-To: <007d01c6e735$cebe2a90$0700a8c0@yairlap> References: <007d01c6e735$cebe2a90$0700a8c0@yairlap> Message-ID: <4522EB6D.5010604@ieee.org> Kenny Ortmann wrote: >> There may be a better way, but:: >> >> alltrue(isreal(x)) >> >> Would work. As would: >> >> not sometrue(x.imag) >> >> In the above test you are already negating the test, so you could just >> drop the not. >> >>> and if so is >>> there a way to extract the(a + ib) because the absolute value of a >>> complex >>> number is like the pythagorean therom on a and b? >>> > > what i was looking for in the above is that when I looked up the > abs(complexnum) i saw that the abs() of a complex num is (a^2 + b^2)^.5 so i > was asking for a way to do this because i was assuming that i was going to > have to parse through the array in a for loop and if i found a complex > number then get the a and b to do the pythagorean theorom but it seems i > will not have to do this, as for the matlab code see below > OK. Yeah, you shouldn't have to do element by element calculations here. >> I'm not entirely sure what you are looking for here, but x.imag and >> x.real will give you the real and imaginary parts. abs(x) will return >> the magnitude of x whether x is real or complex. x.conj() will return >> complex conjugate. >> >> I'm somewhat suspicious of that matlab code. The code given is >> discontinuous as you cross the negative real axis. Does the result >> subsequently get squared or something? I'm guessing that either the >> matlab code is doing extra work, or there are some hidden assumptions >> (all values are in the positive real half-plane). Or some such. In >> either case, you'd probably be OK just skipping the check for realness >> and always taking the absolute of array. I can't say for sure without >> more context though. >> >> -tim >> > > this is taken out of a program i am converting from matlab to python, so I > can not explain all of it, it would take forever, BUT > COEmix, the input, is equal to a different arrays 0 and 2 rows added > together. So with that being said, it can not be empty, and must be actual > numbers. The reason I am "scared" to remove the isreal statement that i > asked about is because the array before hand has values that are taken from > square roots. I do not know if these could ever end up being complex and I > am meeting with the head of this project tomorrow to discuss it, but you > asked so i thought i would share. > Just to beat a deceased equine a bit more: what I'm concerned about here is that the code as it stands will result in a positive number for (-x+jy), not matter how small y is, while resulting in a negative number for -x. That discontinuity seems unlikely to be what is desired. Actually, it's worse than that, if there are any complex numbers in the array, all the numbers will be converted to positive real numbers. So, for example: [-1, 1, -2, 2, 5] => [-1, 1, -2, 2, 5] while: [-1, 1, -2, 2, 3+4j] => [1, 1, 2, 2, 5] Now it's quite possible that the way COEmix is generated constrains it's value in some way. It's possible that it's constrained to the positive half plane, or perhaps more likely that it's ideally a positive real number that may stray slightly off the real axis, so to clean it up the absolute value is being taken. Since, I can't think of a case where you actually want the discontinuity described above, I'm guessing that the code would be just fine as COEmix = abs(COEmix) That is, unconditionally take the absolute value. However, that *is* just a guess. The answer lies in how COEmix is originally generated and perhaps in how it is used as well. Regards, -tim [... CODE...] From peridot.faceted at gmail.com Tue Oct 3 02:06:42 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Tue, 3 Oct 2006 02:06:42 -0400 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521AF9A.50701@ee.byu.edu> References: <45219AD6.5020203@ee.byu.edu> <4521AF9A.50701@ee.byu.edu> Message-ID: On 02/10/06, Travis Oliphant wrote: > Perhaps those inner 1-d loops could be optimized (using prefetch or > something) to reduce the number of cache misses on the inner > computation, and the concept of looping over the largest dimension > (instead of the last dimension) should be re-considered. Cache control seems to be the main factor deciding the speed of many algorithms. Prefectching could make a huge difference, particularly on NUMA machines (like a dual opteron). I think GCC has a moderately portable way to request it (though it may be only in beta versions as yet). More generally, all the tricks that ATLAS uses to accelerate BLAS routines would (in principle) be applicable here. The implementation would be extremely difficult, though, even if all the basic loops could be expressed in a few primitives. A. M. Archibald From lars.bittrich at googlemail.com Tue Oct 3 05:53:22 2006 From: lars.bittrich at googlemail.com (Lars Bittrich) Date: Tue, 3 Oct 2006 11:53:22 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) In-Reply-To: <45218A44.3070101@ee.byu.edu> References: <200610020926.56965.lars.bittrich@googlemail.com> <45218A44.3070101@ee.byu.edu> Message-ID: <200610031153.22422.lars.bittrich@googlemail.com> On Monday 02 October 2006 23:53, Travis Oliphant wrote: > This is a Python 2.5 issue (the new __index__ method) was incorrectly > implemented and allowing a 1-d array to be interpreted as an index. > > This should be fixed in SVN. Ok, thank you. I tried that but the version of python-svn is 2.6 and there are much more complications like: File "[...]/lib/python2.6/site-packages/numpy/core/numeric.py", line 284 as = a.shape ^ SyntaxError: invalid syntax File "[...]/lib/python2.6/site-packages/numpy/f2py/crackfortran.py", line 1647 as=b['args'] ^ SyntaxError: invalid syntax I should try Python 2.4 for more stable results. P.S. Sorry for spaming you. I had problems with my mail address settings. I did not expect the mails to arrive after a few days. I thought that they were rejected. From gael.varoquaux at normalesup.org Tue Oct 3 22:11:51 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Wed, 4 Oct 2006 04:11:51 +0200 Subject: [Numpy-discussion] return type diffences of indexed arrays with Intel C++ compiler (Python 2.5) In-Reply-To: <200610031153.22422.lars.bittrich@googlemail.com> References: <200610020926.56965.lars.bittrich@googlemail.com> <45218A44.3070101@ee.byu.edu> <200610031153.22422.lars.bittrich@googlemail.com> Message-ID: <20061004021151.GA12486@clipper.ens.fr> On Tue, Oct 03, 2006 at 11:53:22AM +0200, Lars Bittrich wrote: > On Monday 02 October 2006 23:53, Travis Oliphant wrote: > > This is a Python 2.5 issue (the new __index__ method) was incorrectly > > implemented and allowing a 1-d array to be interpreted as an index. > > This should be fixed in SVN. > Ok, thank you. I tried that but the version of python-svn is 2.6 No, Travis was talking of the numpy svn, not the python svn. Gael From robert.kern at gmail.com Tue Oct 3 23:01:16 2006 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 03 Oct 2006 22:01:16 -0500 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python Message-ID: Has anyone implemented an easier or more efficient way to broadcast arrays to a common shape at the Python level? I was hoping that the broadcast iterator would actually provide the broadcasted arrays, but it does not. I've attached my best pure-Python effort. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -------------- next part -------------- A non-text attachment was scrubbed... Name: broadcast_array.py Type: application/x-python Size: 1280 bytes Desc: not available URL: From peridot.faceted at gmail.com Tue Oct 3 12:38:55 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Tue, 3 Oct 2006 12:38:55 -0400 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45228BF5.1070604@ieee.org> References: <45227AC6.4010801@ieee.org> <45228BF5.1070604@ieee.org> Message-ID: On 03/10/06, Tim Hochberg wrote: > I had an idea regarding which axis to operate on first. Rather than > operate on strictly the longest axis or strictly the innermost axis, a > hybrid approach could be used. We would operate on the longest axis that > would not result in the inner loop overflowing the cache. The idea is > minimize the loop overhead as we do now by choosing the largest axis, > while at the same time attempting to maintain cache friendliness. If elements are smaller than cache lines (usually at least eight bytes, I think), we might end up pulling many times as many bytes into the cache as we actually need if we don't loop along axes with small strides first. Can BLAS be used for some of these operations? A. M. Archibald A. M. Archibald From fullung at gmail.com Mon Oct 2 18:00:10 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 3 Oct 2006 00:00:10 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that Message-ID: <5eec5f300610021500g6034c106h11f4c32d77e642ac@mail.gmail.com> Hello all I recently started looking at David Cournapeau's PyEM package, specifically his implementation of the K-Means algorithm. He implemented part of this algorithm with in pure Python version and also provided a Pyrex alternative that is significantly faster (about 10 times with the data I tested with). I tried to figure out why this is so. The job of this part of the algorithm is pretty simple: from a bunch of cluster means (the codebook) find the nearest cluster mean for each data point. The first attempt at implementing this algorithm might use two for loops, one over the data points and one over the cluster means, computing a Euclidean distance at each step. Slightly better is to use one loop and some broadcasting. By randomly trying some code, I arrived at the following one-liner: N.sum((data[...,N.newaxis,...]-code)**2, 2).argmin(axis=1) where data = N.random.randn(2000, 39) nclusters = 64 code = data[0:nclusters,:] This code was still slow, so I mangled it a bit so that I could profile it using cProfile under Python 2.5 (use profile if you have an older version of Python): def kmean(): data = N.random.randn(2000, 39) nclusters = 64 code = data[0:nclusters,:] for i in xrange(10): def a(): return data[...,N.newaxis,...] z = a() def b(): return z-code z = b() def c(): return z**2 z = c() def d(): return N.sum(z, 2) z = d() def e(): return z.argmin(axis=1) z = e() if __name__ == '__main__': import cProfile cProfile.run("kmean()") Profiler output: In [570]: %run -t -e kmean2.py 84 function calls in 8.567 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 8.567 8.567 :1() 10 0.000 0.000 0.835 0.084 fromnumeric.py:375(sum) 10 0.000 0.000 0.000 0.000 kmean2.py:10(a) 10 6.837 0.684 6.837 0.684 kmean2.py:12(b) 10 0.623 0.062 0.623 0.062 kmean2.py:14(c) 10 0.000 0.000 0.835 0.084 kmean2.py:16(d) 10 0.080 0.008 0.080 0.008 kmean2.py:18(e) 1 0.180 0.180 8.567 8.567 kmean2.py:5(kmean2) 10 0.000 0.000 0.000 0.000 {isinstance} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 1 0.012 0.012 0.012 0.012 {method 'randn' of 'mtrand.RandomState' objects} 10 0.835 0.083 0.835 0.083 {method 'sum' of 'numpy.ndarray' objects} It seems the b part of the computation is taking most of the time. I tried to simulate this separately: In [571]: x1 = N.random.randn(2000,39) In [572]: y1 = N.random.randn(64,39) In [574]: %timeit z1=x1[...,N.newaxis,...]-y1 10 loops, best of 3: 703 ms per loop In [575]: z1.shape Out[575]: (2000, 64, 39) As far as I can figure, this operation is doing 2000*64*39 subtractions. Doing this straight up yields the following: In [576]: x2 = N.random.randn(2000,64,39) In [577]: y2 = N.random.randn(2000,64,39) In [578]: %timeit z2 = x2-y2 10 loops, best of 3: 108 ms per loop Does anybody have any ideas on why this is so much faster? Hopefully I didn't mess up somewhere... Thanks! Regards, Albert P.S. I'm used NumPy 1.0rc1 with Python 2.5 on Windows for these tests. I get similar results with 1.0.dev3239 with Python 2.4.3 on Linux. From peridot.faceted at gmail.com Wed Oct 4 01:20:44 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 4 Oct 2006 01:20:44 -0400 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python In-Reply-To: References: Message-ID: On 03/10/06, Robert Kern wrote: > Has anyone implemented an easier or more efficient way to broadcast arrays to a > common shape at the Python level? I was hoping that the broadcast iterator would > actually provide the broadcasted arrays, but it does not. How about vectorize(lambda *args: args)? Almost works, only it sometimes segfaults with more than two arguments... but that's clearly a numpy bug. A. M. Archibald From wbaxter at gmail.com Tue Oct 3 05:15:25 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Tue, 3 Oct 2006 18:15:25 +0900 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4522122D.9070407@ar.media.kyoto-u.ac.jp> References: <4522122D.9070407@ar.media.kyoto-u.ac.jp> Message-ID: Er, is this the PyEM in question? It doesn't have much of a web presence... http://www.ar.media.kyoto-u.ac.jp/members/david/ --bb On 10/3/06, David Cournapeau wrote: > Albert Strasheim wrote: > > Hello all > > > > I recently started looking at David Cournapeau's PyEM package, specifically > > his implementation of the K-Means algorithm. He implemented part of this > > algorithm with in pure Python version and also provided a Pyrex alternative > > that is significantly faster (about 10 times with the data I tested with). I > > tried to figure out why this is so. From fullung at gmail.com Tue Oct 3 05:12:02 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 3 Oct 2006 11:12:02 +0200 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <4521C1EB.30205@ee.byu.edu> Message-ID: Hello all > -----Original Message----- > From: Travis Oliphant [mailto:oliphant at ee.byu.edu] > Sent: 03 October 2006 03:51 > To: Discussion of Numerical Python; Albert Strasheim > Subject: Re: ***[Possible UCE]*** Re: [Numpy-discussion] Vectorizing code, > for loops, and all that > > > >Meanwhile, I can confirm that the NOBUFFER_UFUNCLOOP case in > >PyUFunc_GenericFunction is getting exercised in the slower case. Here's > some > >info on what's happening, courtesy of Rational Quantify: > > > >case NOBUFFER_UFUNCLOOP: > >while (loop->index < loop->size) { > > for (i=0; inargs; i++) > > loop->bufptr[i] = loop->iters[i]->dataptr; [1] > > > > loop->function((char **)loop->bufptr, &(loop->bufcnt), > > loop->steps, loop->funcdata); [2] > > UFUNC_CHECK_ERROR(loop); > > > > for (i=0; inargs; i++) { > > PyArray_ITER_NEXT(loop->iters[i]); [3] > > } > > loop->index++; > >} > >break; > > > >[1] 12.97% of function time > >[2] 8.65% of functiont ime > >[3] 62.14% of function time > > > >If statistics from elsewhere in the code would be helpful, let me know, > and > >I'll see if I can convince Quantify to cough it up. > > > Please run the same test but using > > x1 = N.random.rand(39,2000) > x2 = N.random.rand(39,64,1) > > z1 = x1[:,N.newaxis,:] - x2 Very similar results to what I had previously: [1] 10.88% [2] 7.25% [3] 68.25% Cheers, Albert From robert.kern at gmail.com Wed Oct 4 01:33:05 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 04 Oct 2006 00:33:05 -0500 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python In-Reply-To: References: Message-ID: A. M. Archibald wrote: > On 03/10/06, Robert Kern wrote: >> Has anyone implemented an easier or more efficient way to broadcast arrays to a >> common shape at the Python level? I was hoping that the broadcast iterator would >> actually provide the broadcasted arrays, but it does not. > > How about vectorize(lambda *args: args)? Almost works, only it > sometimes segfaults with more than two arguments... but that's clearly > a numpy bug. Yeah, a segfault would be problematic. Otherwise, it works and is in fact faster than what I wrote. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From peridot.faceted at gmail.com Wed Oct 4 01:37:55 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 4 Oct 2006 01:37:55 -0400 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python In-Reply-To: References: Message-ID: On 04/10/06, Robert Kern wrote: > Yeah, a segfault would be problematic. Otherwise, it works and is in fact faster > than what I wrote. It's a bit tricky to trigger but I think it was fixed in 1.0rc1 (in changeset 3125, in fact: http://projects.scipy.org/scipy/numpy/changeset/3125 ) Would it be useful for me to contribute the tiny script I wrote to trigger it as a regression test? A. M. Archibald -------------- next part -------------- A non-text attachment was scrubbed... Name: bug.py Type: text/x-python Size: 248 bytes Desc: not available URL: From Mailer-Daemon at lists.sourceforge.net Wed Oct 4 01:57:41 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Tue, 03 Oct 2006 22:57:41 -0700 Subject: [Numpy-discussion] Warning: message 1GUFPK-0001wW-5k delayed 48 hours Message-ID: This message was created automatically by mail delivery software. A message that you sent has not yet been delivered to one or more of its recipients after more than 48 hours on the queue on externalmx-1.sourceforge.net. The message identifier is: 1GUFPK-0001wW-5k The date of the message is: Mon, 02 Oct 2006 09:32:13 -0200 The subject of the message is: doctor, at the revolt is located treating the planet he gold; The address to which the message has not yet been delivered is: numpy-discussion at lists.sourceforge.net Delay reason: SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is No action is required on your part. Delivery attempts will continue for some time, and this warning may be repeated at intervals if the message remains undelivered. Eventually the mail delivery software will give up, and when that happens, the message will be returned to you. From david at ar.media.kyoto-u.ac.jp Wed Oct 4 06:58:25 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 04 Oct 2006 19:58:25 +0900 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: <45228EAA.9040301@ieee.org> References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> <45228EAA.9040301@ieee.org> Message-ID: <452393D1.2020308@ar.media.kyoto-u.ac.jp> Tim Hochberg wrote: >> >> >> I guess the problem is coming from the fact that y being C order, y[:, >> i] needs accessing data in a non 'linear' way. Is there a way to speed >> this up ? I did something like this: >> >> y = N.zeros((K, n)) >> for i in range(K): >> y[i] = gauss_den(data, mu[i, :], va[i, :]) >> return y.T >> >> which works, but I don't like it very much. > Why not? > Mainly because using those transpose do not really reflect the intention, and this does not seem natural. >> Isn't there any other way > That depends on the details of gauss_den. > > A general note: for this kind of microoptimization puzzle, it's much > easier to help if you can post a self contained example, preferably > something fairly simple that still illustrates the speed issue, that we > can experiment with. > Here we are (the difference may not seem that much between the two multiple_ga, but in reality, _diag_gauss_den is an internal function which is done in C, and is much faster... By writing this example, I've just realized that the function _diag_gauss_den may be slow for exactly the same reasons): #! /usr/bin/env python # Last Change: Wed Oct 04 07:00 PM 2006 J import numpy as N from numpy.random import randn def _diag_gauss_den(x, mu, va): """ This function is the actual implementation of gaussian pdf in scalar case. It assumes all args are conformant, so it should not be used directly Call gauss_den instead""" # Diagonal matrix case d = mu.size inva = 1/va[0] fac = (2*N.pi) ** (-d/2.0) * N.sqrt(inva) y = (x[:,0] - mu[0]) ** 2 * inva * -0.5 for i in range(1, d): inva = 1/va[i] fac *= N.sqrt(inva) y += (x[:,i] - mu[i]) ** 2 * inva * -0.5 y = fac * N.exp(y) def multiple_gauss_den1(data, mu, va): """Helper function to generate several Gaussian pdf (different parameters) from the same data: unoptimized version""" K = mu.shape[0] n = data.shape[0] d = data.shape[1] y = N.zeros((n, K)) for i in range(K): y[:, i] = _diag_gauss_den(data, mu[i, :], va[i, :]) return y def multiple_gauss_den2(data, mu, va): """Helper function to generate several Gaussian pdf (different parameters) from the same data: optimized version""" K = mu.shape[0] n = data.shape[0] d = data.shape[1] y = N.zeros((K, n)) for i in range(K): y[i] = _diag_gauss_den(data, mu[i, :], va[i, :]) return y.T def bench(): #=========================================== # GMM of 30 comp, 15 dimension, 1e4 frames #=========================================== d = 15 k = 30 nframes = 1e4 niter = 10 mode = 'diag' mu = randn(k, d) va = randn(k, d) ** 2 X = randn(nframes, d) print "=============================================================" print "(%d dim, %d components) GMM with %d iterations, for %d frames" \ % (d, k, niter, nframes) for i in range(niter): y1 = multiple_gauss_den1(X, mu, va) for i in range(niter): y2 = multiple_gauss_den2(X, mu, va) se = N.sum(y1-y2) print se if __name__ == '__main__': import hotshot, hotshot.stats profile_file = 'foo.prof' prof = hotshot.Profile(profile_file, lineevents=1) prof.runcall(bench) p = hotshot.stats.load(profile_file) print p.sort_stats('cumulative').print_stats(20) prof.close() I am a bit puzzled by all those C vs F storage, though. In Matlab, where the storage was always F as far as I know, I have never encountered such differences (eg between y(:, i) and y(:, i)); I don't know if this is because I am doing it badly, or because matlab is much more clever than numpy at handling those cases, or if that is the price to pay for the added flexibility of numpy... David From david at ar.media.kyoto-u.ac.jp Wed Oct 4 07:13:22 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 04 Oct 2006 20:13:22 +0900 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: <452393D1.2020308@ar.media.kyoto-u.ac.jp> References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> <45228EAA.9040301@ieee.org> <452393D1.2020308@ar.media.kyoto-u.ac.jp> Message-ID: <45239752.2030401@ar.media.kyoto-u.ac.jp> David Cournapeau wrote: > Here we are (the difference may not seem that much between the two > multiple_ga, but in reality, _diag_gauss_den is an internal function > which is done in C, and is much faster... By writing this example, I've > just realized that the function _diag_gauss_den may be slow for exactly > the same reasons): > > I checked my assumption about the _diag_gauss_den function being slow because of the same problem, and this is indeed the case: If I replace X = randn(nframes, d) by X = randn(d, nframes); X = X.T, the function is mode than twice as fast ! This seems way to much of a difference to me.... David From ivilata at carabos.com Wed Oct 4 07:23:52 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Wed, 04 Oct 2006 13:23:52 +0200 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays Message-ID: <452399C8.7050805@carabos.com> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or so, but I have come across some alignment or striding problems which can be seen with the following code:: import numpy import numexpr array_length = 10 array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] array = numpy.empty((array_length,), dtype=array_descr) for i in xrange(array_length): array['c1'][i] = i array['c2'][i] = 0xaaaa print numexpr.evaluate('c1', {'c1': array['c1']}) print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 (unmodified) this gives the following result:: [ 0 109226 -1431699456 2 240298 -1431699456 4 371370 8 633514] [0 1 2 3 4 5 6 7 8 9] The test works right when ``evaluate()`` is used with 'c2' instead of 'c1', and also when 'c2' also measures 32 bits and fields are aligned. Maybe the ``memsteps`` value is not getting used somewhere. Any ideas on this? :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: From bsouthey at gmail.com Wed Oct 4 09:26:53 2006 From: bsouthey at gmail.com (Bruce Southey) Date: Wed, 4 Oct 2006 08:26:53 -0500 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <5eec5f300610021500g6034c106h11f4c32d77e642ac@mail.gmail.com> References: <5eec5f300610021500g6034c106h11f4c32d77e642ac@mail.gmail.com> Message-ID: Hi, On 10/2/06, Albert Strasheim wrote: > I recently started looking at David Cournapeau's PyEM package, > specifically his implementation of the K-Means algorithm. He > implemented part of this algorithm with in pure Python version and > also provided a Pyrex alternative that is significantly faster (about > 10 times with the data I tested with). I tried to figure out why this > is so. Just of of curosity, have you seen the implementation at BioPython (http://biopython.org)? Michiel de Hoon and other wrote 'The C Clustering Library', see the following document for more details (linked from documentation page - http://biopython.org/wiki/Documentation): http://biopython.org/DIST/docs/cluster/cluster.pdf Regards Bruce From bsouthey at gmail.com Wed Oct 4 11:36:09 2006 From: bsouthey at gmail.com (Bruce Southey) Date: Wed, 4 Oct 2006 10:36:09 -0500 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: <452393D1.2020308@ar.media.kyoto-u.ac.jp> References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> <45228EAA.9040301@ieee.org> <452393D1.2020308@ar.media.kyoto-u.ac.jp> Message-ID: Hi, I think what you are after is the multivariate normal distribution. Assuming I have it correctly, it is clearer to see (and probably more accurate to compute) in the log form as: -(N/2)*log(2*PI) - 0.5*log(determinant of V) - 0.5*(transpose of (x-mu))*inverse(V)*(x-mu) where N is the number of observations, PI is math constant, V is the known variance co-variance matrix, x is vector of values, mu is the known mean. If so, then you can vectorize the density calculation. Of course if V has a simple structure (it is unclear from your code on this) and can be factored out, then the only 'hard' part to compute is the product of the vector transpose with the vector. If mu and va are scalars for each calculation, then you can factor them out for so the only multidimensional calculations are squares and sums of each frame. Regards Bruce On 10/4/06, David Cournapeau wrote: > Tim Hochberg wrote: > >> > >> > >> I guess the problem is coming from the fact that y being C order, y[:, > >> i] needs accessing data in a non 'linear' way. Is there a way to speed > >> this up ? I did something like this: > >> > >> y = N.zeros((K, n)) > >> for i in range(K): > >> y[i] = gauss_den(data, mu[i, :], va[i, :]) > >> return y.T > >> > >> which works, but I don't like it very much. > > Why not? > > > Mainly because using those transpose do not really reflect the > intention, and this does not seem natural. > >> Isn't there any other way > > That depends on the details of gauss_den. > > > > A general note: for this kind of microoptimization puzzle, it's much > > easier to help if you can post a self contained example, preferably > > something fairly simple that still illustrates the speed issue, that we > > can experiment with. > > > Here we are (the difference may not seem that much between the two > multiple_ga, but in reality, _diag_gauss_den is an internal function > which is done in C, and is much faster... By writing this example, I've > just realized that the function _diag_gauss_den may be slow for exactly > the same reasons): > > > #! /usr/bin/env python > # Last Change: Wed Oct 04 07:00 PM 2006 J > > import numpy as N > from numpy.random import randn > > def _diag_gauss_den(x, mu, va): > """ This function is the actual implementation > of gaussian pdf in scalar case. It assumes all args > are conformant, so it should not be used directly > > Call gauss_den instead""" > # Diagonal matrix case > d = mu.size > inva = 1/va[0] > fac = (2*N.pi) ** (-d/2.0) * N.sqrt(inva) > y = (x[:,0] - mu[0]) ** 2 * inva * -0.5 > for i in range(1, d): > inva = 1/va[i] > fac *= N.sqrt(inva) > y += (x[:,i] - mu[i]) ** 2 * inva * -0.5 > y = fac * N.exp(y) > > def multiple_gauss_den1(data, mu, va): > """Helper function to generate several Gaussian > pdf (different parameters) from the same data: unoptimized version""" > K = mu.shape[0] > n = data.shape[0] > d = data.shape[1] > > y = N.zeros((n, K)) > for i in range(K): > y[:, i] = _diag_gauss_den(data, mu[i, :], va[i, :]) > > return y > > def multiple_gauss_den2(data, mu, va): > """Helper function to generate several Gaussian > pdf (different parameters) from the same data: optimized version""" > K = mu.shape[0] > n = data.shape[0] > d = data.shape[1] > > y = N.zeros((K, n)) > for i in range(K): > y[i] = _diag_gauss_den(data, mu[i, :], va[i, :]) > > return y.T > > def bench(): > #=========================================== > # GMM of 30 comp, 15 dimension, 1e4 frames > #=========================================== > d = 15 > k = 30 > nframes = 1e4 > niter = 10 > mode = 'diag' > > mu = randn(k, d) > va = randn(k, d) ** 2 > X = randn(nframes, d) > > print "=============================================================" > print "(%d dim, %d components) GMM with %d iterations, for %d frames" \ > % (d, k, niter, nframes) > > for i in range(niter): > y1 = multiple_gauss_den1(X, mu, va) > > for i in range(niter): > y2 = multiple_gauss_den2(X, mu, va) > > se = N.sum(y1-y2) > > print se > > if __name__ == '__main__': > import hotshot, hotshot.stats > profile_file = 'foo.prof' > prof = hotshot.Profile(profile_file, lineevents=1) > prof.runcall(bench) > p = hotshot.stats.load(profile_file) > print p.sort_stats('cumulative').print_stats(20) > prof.close() > > I am a bit puzzled by all those C vs F storage, though. In Matlab, where > the storage was always F as far as I know, I have never encountered such > differences (eg between y(:, i) and y(:, i)); I don't know if this is > because I am doing it badly, or because matlab is much more clever than > numpy at handling those cases, or if that is the price to pay for the > added flexibility of numpy... > > David > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From tim.hochberg at ieee.org Wed Oct 4 12:32:24 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 04 Oct 2006 09:32:24 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <452399C8.7050805@carabos.com> References: <452399C8.7050805@carabos.com> Message-ID: <4523E218.1030204@ieee.org> Ivan Vilata i Balaguer wrote: > It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > so, but I have come across some alignment or striding problems which can > be seen with the following code:: > > import numpy > import numexpr > > array_length = 10 > array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] > > array = numpy.empty((array_length,), dtype=array_descr) > for i in xrange(array_length): > array['c1'][i] = i > array['c2'][i] = 0xaaaa > > print numexpr.evaluate('c1', {'c1': array['c1']}) > print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) > > Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 > (unmodified) this gives the following result:: > > [ 0 109226 -1431699456 2 240298 -1431699456 > 4 371370 8 633514] > [0 1 2 3 4 5 6 7 8 9] > > The test works right when ``evaluate()`` is used with 'c2' instead of > 'c1', and also when 'c2' also measures 32 bits and fields are aligned. > Maybe the ``memsteps`` value is not getting used somewhere. Any ideas > on this? > I suspect that there are some assumptions that the element separation is an integral multiple of the element size. I certainly didn't have record arrays in mind when I was working on the striding stuff, so it wouldn't surprise me. This should be fixed: preferably to do the right thing and at a minimum to cleanly raise an exception rather than spitting out garbage. I don't know that I'll have time to mess with it soon though. -tim From torgil.svensson at gmail.com Wed Oct 4 12:40:01 2006 From: torgil.svensson at gmail.com (Torgil Svensson) Date: Wed, 4 Oct 2006 18:40:01 +0200 Subject: [Numpy-discussion] Experience with Visit? In-Reply-To: References: Message-ID: Sourceforge mailing list doesn't like me. Here's another try. I've managed to use visit through it's cli and giving it data through pyvtk so that this code if __name__=='__main__': from numpy import * from numpy.random import randn n=10000 t=linspace(0,1,n)+randn(n)*0.01 a=t+t*(1+randn(n)*0.1) phi=randn(n)*2*pi*0.05 scatter4( a*cos(2.0*2*pi*t+phi) , a*sin(1.5*2*pi*t+phi) , t ,1.0/sqrt(1+t**2) ) gives this result: http://www.torgil.org/visit/scatter4.png You can grab the source using git: (see http://git.or.cz/ for details) git clone http://torgil.org/pyvisit.git I'm using the pty module in python to communicate with the visit cli which unfortunately is only available on Unix. If you do any updates, like an usefule API or windows support, please share (that's why I put it in git due to it's excellent distributed development capabilities). This was more of a proof of concept for me. > On 9/15/06, Rob Hetland wrote: > > > > I have looked at it, and I even know somebody who is using python to > > put in a data reader for FVCOM (a finite volume ocean model). He has > > a prototype working. I have no experience with the python bindings > > personally. > > > > I looked into compiling it from source (for an intel mac), but it was > > daunting so I gave up. > > > > I also contacted them to try to put in lower level python bindings, > > so that we could create 3D graphics from the command line, but they > > didn't seem too interested in that. > > > > Otherwise, it is a very nice workable app. > > > > -Rob > > > > > > On Sep 15, 2006, at 11:23 AM, O'Keefe, Michael wrote: > > > > > I haven't tried VisIT before but thanks for the link. I also > > > downloaded and am checking it out. > > > > > > Along this same line of discussion, has anyone tried OOF2 which is > > > an FEA package that also has some strong python connections? > > > > > > http://www.ctcms.nist.gov/oof/oof2.html > > > > > > I'm working on a Windows machine and it the current code-base > > > doesn't seem to support Windows out of the box (if at all). Looks > > > like you can put it together for *nix or Mac OS X, though... > > > > > > --Michael > > > > > >> -----Original Message----- > > >> From: numpy-discussion-bounces at lists.sourceforge.net > > >> [mailto:numpy-discussion-bounces at lists.sourceforge.net] On > > >> Behalf Of Robert Cimrman > > >> Sent: Friday, September 15, 2006 8:29 > > >> To: Discussion of Numerical Python > > >> Subject: Re: [Numpy-discussion] Experience with Visit? > > >> > > >> Travis Oliphant wrote: > > >>> Has anybody had any experience with the 3-D visualization software > > >>> VISIT? It has Python bindings and seems to be pretty > > >> sophisticated. > > >>> I'm wondering why I haven't heard more about it. > > >>> > > >>> http://www.llnl.gov/visit/ > > >> > > >> No reaction up to now, so... > > >> > > >> I have just tried the 'getting started' part and was quite impressed, > > >> thanks for posting the link! Up to now I have used ParaView > > >> and was very > > >> satisfied, but the Python bindings of VisIt are a great lure. > > >> > > >> r. > > >> > > >> -------------------------------------------------------------- > > >> ----------- > > >> Using Tomcat but need to do more? Need to support web > > >> services, security? > > >> Get stuff done quickly with pre-integrated technology to make > > >> your job easier > > >> Download IBM WebSphere Application Server v.1.0.1 based on > > >> Apache Geronimo > > >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& > > >> dat=121642 > > >> _______________________________________________ > > >> Numpy-discussion mailing list > > >> Numpy-discussion at lists.sourceforge.net > > >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > >> > > > > > > ---------------------------------------------------------------------- > > > --- > > > Using Tomcat but need to do more? Need to support web services, > > > security? > > > Get stuff done quickly with pre-integrated technology to make your > > > job easier > > > Download IBM WebSphere Application Server v.1.0.1 based on Apache > > > Geronimo > > > http://sel.as-us.falkag.net/sel? > > > cmd=lnk&kid=120709&bid=263057&dat=121642 > > > _______________________________________________ > > > Numpy-discussion mailing list > > > Numpy-discussion at lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ---- > > Rob Hetland, Associate Professor > > Dept. of Oceanography, Texas A&M University > > http://pong.tamu.edu/~rob > > phone: 979-458-0096, fax: 979-845-6331 > > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, security? > > Get stuff done quickly with pre-integrated technology to make your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > From haase at msg.ucsf.edu Wed Oct 4 12:47:52 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Wed, 4 Oct 2006 09:47:52 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523E218.1030204@ieee.org> References: <452399C8.7050805@carabos.com> <4523E218.1030204@ieee.org> Message-ID: <200610040947.52533.haase@msg.ucsf.edu> Quick question hopefully somewhat related to this: Does numexpr fully support float32 arrays ? -Sebastian On Wednesday 04 October 2006 09:32, Tim Hochberg wrote: > Ivan Vilata i Balaguer wrote: > > It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > > so, but I have come across some alignment or striding problems which can > > be seen with the following code:: > > > > import numpy > > import numexpr > > > > array_length = 10 > > array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] > > > > array = numpy.empty((array_length,), dtype=array_descr) > > for i in xrange(array_length): > > array['c1'][i] = i > > array['c2'][i] = 0xaaaa > > > > print numexpr.evaluate('c1', {'c1': array['c1']}) > > print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) > > > > Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 > > (unmodified) this gives the following result:: > > > > [ 0 109226 -1431699456 2 240298 -1431699456 > > 4 371370 8 633514] > > [0 1 2 3 4 5 6 7 8 9] > > > > The test works right when ``evaluate()`` is used with 'c2' instead of > > 'c1', and also when 'c2' also measures 32 bits and fields are aligned. > > Maybe the ``memsteps`` value is not getting used somewhere. Any ideas > > on this? > > I suspect that there are some assumptions that the element separation > is an integral multiple of the element size. I certainly didn't have > record arrays in mind when I was working on the striding stuff, so it > wouldn't surprise me. This should be fixed: preferably to do the right > thing and at a minimum to cleanly raise an exception rather than > spitting out garbage. I don't know that I'll have time to mess with it > soon though. > > -tim > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your opinions on IT & business topics through brief surveys -- and earn > cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Wed Oct 4 13:13:10 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 04 Oct 2006 10:13:10 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <200610040947.52533.haase@msg.ucsf.edu> References: <452399C8.7050805@carabos.com> <4523E218.1030204@ieee.org> <200610040947.52533.haase@msg.ucsf.edu> Message-ID: <4523EBA6.8070107@ieee.org> Sebastian Haase wrote: > Quick question hopefully somewhat related to this: > Does numexpr fully support float32 arrays ? > I don't recall. At one point there was a tentative plan to support float32 by casting them a block at a time to float64, operating on them and them casting them back. That's to limit the number of bytecodes that we need to support and keep the switch statement at a manageable size. However, it doesn't look like that ever got implemented, so the answer is probably no. -tim > -Sebastian > > > On Wednesday 04 October 2006 09:32, Tim Hochberg wrote: > >> Ivan Vilata i Balaguer wrote: >> >>> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >>> so, but I have come across some alignment or striding problems which can >>> be seen with the following code:: >>> >>> import numpy >>> import numexpr >>> >>> array_length = 10 >>> array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] >>> >>> array = numpy.empty((array_length,), dtype=array_descr) >>> for i in xrange(array_length): >>> array['c1'][i] = i >>> array['c2'][i] = 0xaaaa >>> >>> print numexpr.evaluate('c1', {'c1': array['c1']}) >>> print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) >>> >>> Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 >>> (unmodified) this gives the following result:: >>> >>> [ 0 109226 -1431699456 2 240298 -1431699456 >>> 4 371370 8 633514] >>> [0 1 2 3 4 5 6 7 8 9] >>> >>> The test works right when ``evaluate()`` is used with 'c2' instead of >>> 'c1', and also when 'c2' also measures 32 bits and fields are aligned. >>> Maybe the ``memsteps`` value is not getting used somewhere. Any ideas >>> on this? >>> >> I suspect that there are some assumptions that the element separation >> is an integral multiple of the element size. I certainly didn't have >> record arrays in mind when I was working on the striding stuff, so it >> wouldn't surprise me. This should be fixed: preferably to do the right >> thing and at a minimum to cleanly raise an exception rather than >> spitting out garbage. I don't know that I'll have time to mess with it >> soon though. >> >> -tim >> >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share >> your opinions on IT & business topics through brief surveys -- and earn >> cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > From tim.hochberg at ieee.org Wed Oct 4 13:19:08 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 04 Oct 2006 10:19:08 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <452399C8.7050805@carabos.com> References: <452399C8.7050805@carabos.com> Message-ID: <4523ED0C.4070306@ieee.org> Ivan Vilata i Balaguer wrote: > It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > so, but I have come across some alignment or striding problems which can > be seen with the following code:: I looked at this just a little bit and clearly this bit from interp_body cannot work in the presence of recor arrays: //.... intp sf1 = sb1 / sizeof(double); \ //... #define f1 ((double *)x1)[j*sf1] There are clearly some assumptions that sb1 is evenly divisible by sizeof(double). Blech!. This is likely my fault, and I expect it won't be too horrible to fix, but I don't know that I'll have time immediately. -tim From pfdubois at gmail.com Wed Oct 4 13:19:53 2006 From: pfdubois at gmail.com (Paul Dubois) Date: Wed, 4 Oct 2006 10:19:53 -0700 Subject: [Numpy-discussion] Experience with Visit? In-Reply-To: References: Message-ID: Sorry I didn't post earlier. VisIt is used extensively at LLNL for our 3D apps. I will try to get someone who uses it to comment. On 04 Oct 2006 09:42:20 -0700, Torgil Svensson wrote: > > Sourceforge mailing list doesn't like me. Here's another try. > > I've managed to use visit through it's cli and giving it data through > pyvtk so that this code > > if __name__=='__main__': > from numpy import * > from numpy.random import randn > n=10000 > t=linspace(0,1,n)+randn(n)*0.01 > a=t+t*(1+randn(n)*0.1) > phi=randn(n)*2*pi*0.05 > scatter4( a*cos(2.0*2*pi*t+phi) , a*sin(1.5*2*pi*t+phi) , t > ,1.0/sqrt(1+t**2) ) > > gives this result: > http://www.torgil.org/visit/scatter4.png > > You can grab the source using git: (see http://git.or.cz/ for details) > git clone http://torgil.org/pyvisit.git > > I'm using the pty module in python to communicate with the visit cli > which unfortunately is only available on Unix. If you do any updates, > like an usefule API or windows support, please share (that's why I put > it in git due to it's excellent distributed development capabilities). > This was more of a proof of concept for me. > > > On 9/15/06, Rob Hetland wrote: > > > > > > I have looked at it, and I even know somebody who is using python to > > > put in a data reader for FVCOM (a finite volume ocean model). He has > > > a prototype working. I have no experience with the python bindings > > > personally. > > > > > > I looked into compiling it from source (for an intel mac), but it was > > > daunting so I gave up. > > > > > > I also contacted them to try to put in lower level python bindings, > > > so that we could create 3D graphics from the command line, but they > > > didn't seem too interested in that. > > > > > > Otherwise, it is a very nice workable app. > > > > > > -Rob > > > > > > > > > On Sep 15, 2006, at 11:23 AM, O'Keefe, Michael wrote: > > > > > > > I haven't tried VisIT before but thanks for the link. I also > > > > downloaded and am checking it out. > > > > > > > > Along this same line of discussion, has anyone tried OOF2 which is > > > > an FEA package that also has some strong python connections? > > > > > > > > http://www.ctcms.nist.gov/oof/oof2.html > > > > > > > > I'm working on a Windows machine and it the current code-base > > > > doesn't seem to support Windows out of the box (if at all). Looks > > > > like you can put it together for *nix or Mac OS X, though... > > > > > > > > --Michael > > > > > > > >> -----Original Message----- > > > >> From: numpy-discussion-bounces at lists.sourceforge.net > > > >> [mailto:numpy-discussion-bounces at lists.sourceforge.net] On > > > >> Behalf Of Robert Cimrman > > > >> Sent: Friday, September 15, 2006 8:29 > > > >> To: Discussion of Numerical Python > > > >> Subject: Re: [Numpy-discussion] Experience with Visit? > > > >> > > > >> Travis Oliphant wrote: > > > >>> Has anybody had any experience with the 3-D visualization software > > > >>> VISIT? It has Python bindings and seems to be pretty > > > >> sophisticated. > > > >>> I'm wondering why I haven't heard more about it. > > > >>> > > > >>> http://www.llnl.gov/visit/ > > > >> > > > >> No reaction up to now, so... > > > >> > > > >> I have just tried the 'getting started' part and was quite > impressed, > > > >> thanks for posting the link! Up to now I have used ParaView > > > >> and was very > > > >> satisfied, but the Python bindings of VisIt are a great lure. > > > >> > > > >> r. > > > >> > > > >> -------------------------------------------------------------- > > > >> ----------- > > > >> Using Tomcat but need to do more? Need to support web > > > >> services, security? > > > >> Get stuff done quickly with pre-integrated technology to make > > > >> your job easier > > > >> Download IBM WebSphere Application Server v.1.0.1 based on > > > >> Apache Geronimo > > > >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& > > > >> dat=121642 > > > >> _______________________________________________ > > > >> Numpy-discussion mailing list > > > >> Numpy-discussion at lists.sourceforge.net > > > >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > >> > > > > > > > > > ---------------------------------------------------------------------- > > > > --- > > > > Using Tomcat but need to do more? Need to support web services, > > > > security? > > > > Get stuff done quickly with pre-integrated technology to make your > > > > job easier > > > > Download IBM WebSphere Application Server v.1.0.1 based on Apache > > > > Geronimo > > > > http://sel.as-us.falkag.net/sel? > > > > cmd=lnk&kid=120709&bid=263057&dat=121642 > > > > _______________________________________________ > > > > Numpy-discussion mailing list > > > > Numpy-discussion at lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > ---- > > > Rob Hetland, Associate Professor > > > Dept. of Oceanography, Texas A&M University > > > http://pong.tamu.edu/~rob > > > phone: 979-458-0096, fax: 979-845-6331 > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > Using Tomcat but need to do more? Need to support web services, > security? > > > Get stuff done quickly with pre-integrated technology to make your job > easier > > > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > > _______________________________________________ > > > Numpy-discussion mailing list > > > Numpy-discussion at lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From haase at msg.ucsf.edu Wed Oct 4 13:23:25 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Wed, 4 Oct 2006 10:23:25 -0700 Subject: [Numpy-discussion] =?iso-8859-1?q?Problems_with_Numexpr_and_disco?= =?iso-8859-1?q?ntiguous=09arrays?= In-Reply-To: <4523EBA6.8070107@ieee.org> References: <452399C8.7050805@carabos.com> <200610040947.52533.haase@msg.ucsf.edu> <4523EBA6.8070107@ieee.org> Message-ID: <200610041023.25343.haase@msg.ucsf.edu> On Wednesday 04 October 2006 10:13, Tim Hochberg wrote: > Sebastian Haase wrote: > > Quick question hopefully somewhat related to this: > > Does numexpr fully support float32 arrays ? > > I don't recall. At one point there was a tentative plan to support > float32 by casting them a block at a time to float64, operating on them > and them casting them back. That's to limit the number of bytecodes that > we need to support and keep the switch statement at a manageable size. > However, it doesn't look like that ever got implemented, so the answer > is probably no. > > -tim Does that mean its considered "impratical" to ever add native float32 support ? Is the switch-statement you mention written by hand or is that automatically generated ? -Sebastian > > > -Sebastian > > > > On Wednesday 04 October 2006 09:32, Tim Hochberg wrote: > >> Ivan Vilata i Balaguer wrote: > >>> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > >>> so, but I have come across some alignment or striding problems which > >>> can be seen with the following code:: > >>> > >>> import numpy > >>> import numexpr > >>> > >>> array_length = 10 > >>> array_descr = [('c1', numpy.int32), ('c2', numpy.uint16)] > >>> > >>> array = numpy.empty((array_length,), dtype=array_descr) > >>> for i in xrange(array_length): > >>> array['c1'][i] = i > >>> array['c2'][i] = 0xaaaa > >>> > >>> print numexpr.evaluate('c1', {'c1': array['c1']}) > >>> print numexpr.evaluate('c1', {'c1': array['c1'].copy()}) > >>> > >>> Im my computer, Pentium IV with NumPy 1.0rc1 and Numexpr r2239 > >>> (unmodified) this gives the following result:: > >>> > >>> [ 0 109226 -1431699456 2 240298 > >>> -1431699456 4 371370 8 633514] > >>> [0 1 2 3 4 5 6 7 8 9] > >>> > >>> The test works right when ``evaluate()`` is used with 'c2' instead of > >>> 'c1', and also when 'c2' also measures 32 bits and fields are aligned. > >>> Maybe the ``memsteps`` value is not getting used somewhere. Any ideas > >>> on this? > >> > >> I suspect that there are some assumptions that the element separation > >> is an integral multiple of the element size. I certainly didn't have > >> record arrays in mind when I was working on the striding stuff, so it > >> wouldn't surprise me. This should be fixed: preferably to do the right > >> thing and at a minimum to cleanly raise an exception rather than > >> spitting out garbage. I don't know that I'll have time to mess with it > >> soon though. > >> > >> -tim > >> > >> > >> ------------------------------------------------------------------------ > >>- Take Surveys. Earn Cash. Influence the Future of IT > >> Join SourceForge.net's Techsay panel and you'll get the chance to share > >> your opinions on IT & business topics through brief surveys -- and earn > >> cash > >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE > >>V _______________________________________________ > >> Numpy-discussion mailing list > >> Numpy-discussion at lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share > > your opinions on IT & business topics through brief surveys -- and earn > > cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your opinions on IT & business topics through brief surveys -- and earn > cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cookedm at physics.mcmaster.ca Wed Oct 4 13:27:36 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Wed, 4 Oct 2006 13:27:36 -0400 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523ED0C.4070306@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> Message-ID: <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> On Wed, 04 Oct 2006 10:19:08 -0700 Tim Hochberg wrote: > Ivan Vilata i Balaguer wrote: > > It seemed that discontiguous arrays worked OK in Numexpr since r1977 or > > so, but I have come across some alignment or striding problems which can > > be seen with the following code:: > I looked at this just a little bit and clearly this bit from interp_body > cannot work in the presence of recor arrays: > > //.... > intp sf1 = sb1 / sizeof(double); \ > //... > #define f1 ((double *)x1)[j*sf1] > > > There are clearly some assumptions that sb1 is evenly divisible by > sizeof(double). Blech!. This is likely my fault, and I expect it won't > be too horrible to fix, but I don't know that I'll have time immediately. My thinking is that this should be handled by a copy, so that the opcodes always work on contiguous data. The copy can be another opcode. One advantage of operating on contiguous data is that it's easier to use the processor's vector instructions, if applicable. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From tim.hochberg at ieee.org Wed Oct 4 13:42:33 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 04 Oct 2006 10:42:33 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> Message-ID: <4523F289.901@ieee.org> David M. Cooke wrote: > On Wed, 04 Oct 2006 10:19:08 -0700 > Tim Hochberg wrote: > > >> Ivan Vilata i Balaguer wrote: >> >>> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >>> so, but I have come across some alignment or striding problems which can >>> be seen with the following code:: >>> >> I looked at this just a little bit and clearly this bit from interp_body >> cannot work in the presence of recor arrays: >> >> //.... >> intp sf1 = sb1 / sizeof(double); \ >> //... >> #define f1 ((double *)x1)[j*sf1] >> >> >> There are clearly some assumptions that sb1 is evenly divisible by >> sizeof(double). Blech!. This is likely my fault, and I expect it won't >> be too horrible to fix, but I don't know that I'll have time immediately. >> > > My thinking is that this should be handled by a copy, so that the opcodes > always work on contiguous data. The copy can be another opcode. One advantage > of operating on contiguous data is that it's easier to use the processor's > vector instructions, if applicable. > That would be easy to do. Right now the opcodes should work correctly on data that is spaced in multiples of the itemsize on the last axis. Other arrays are copied (no opcode required, it's embedded at the top of interp_body lines 64-80). The record array case apparently slips through the cracks when we're checking whether an array is suitable to be used correctly (interpreter.c 1086-1103). It would certainly not be any harder to only allow contiguous arrays than to correctly deal with record arrays. Only question I have is whether the extra copy will overwhelm the savings of that operating on contiguous data gives. The thing to do is probably try it and see what happens. -tim From cookedm at physics.mcmaster.ca Wed Oct 4 13:44:53 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Wed, 4 Oct 2006 13:44:53 -0400 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <200610041023.25343.haase@msg.ucsf.edu> References: <452399C8.7050805@carabos.com> <200610040947.52533.haase@msg.ucsf.edu> <4523EBA6.8070107@ieee.org> <200610041023.25343.haase@msg.ucsf.edu> Message-ID: <20061004134453.754f5bb6@arbutus.physics.mcmaster.ca> On Wed, 4 Oct 2006 10:23:25 -0700 Sebastian Haase wrote: > On Wednesday 04 October 2006 10:13, Tim Hochberg wrote: > > Sebastian Haase wrote: > > > Quick question hopefully somewhat related to this: > > > Does numexpr fully support float32 arrays ? > > > > I don't recall. At one point there was a tentative plan to support > > float32 by casting them a block at a time to float64, operating on them > > and them casting them back. That's to limit the number of bytecodes that > > we need to support and keep the switch statement at a manageable size. > > However, it doesn't look like that ever got implemented, so the answer > > is probably no. > > > > -tim > > Does that mean its considered "impratical" to ever add native float32 > support ? Is the switch-statement you mention written by hand or is that > automatically generated ? > -Sebastian > Currently by hand. I've got a rewrite lying around that generates the C code for it using a description in Python, but I haven't finished it yet. It should make it much easier to add different types, along with different methods of calculating and switching (switch vs. gcc's label pointers, for instance). Probably, if float32 is added, there will be two (internal) implementations: one that uses float64 mainly, and coerces float32 to float64, and one that does the reverse (this would be invisible to the user, of course). The same would handle int32 and int64. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From oliphant at ee.byu.edu Wed Oct 4 14:21:43 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 04 Oct 2006 12:21:43 -0600 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523F289.901@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> Message-ID: <4523FBB7.1060500@ee.byu.edu> Tim Hochberg wrote: >David M. Cooke wrote: > > >>On Wed, 04 Oct 2006 10:19:08 -0700 >>Tim Hochberg wrote: >> >> >> >> >>>Ivan Vilata i Balaguer wrote: >>> >>> >>> >>>>It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >>>>so, but I have come across some alignment or striding problems which can >>>>be seen with the following code:: >>>> >>>> >>>> >>>I looked at this just a little bit and clearly this bit from interp_body >>>cannot work in the presence of recor arrays: >>> >>>//.... >>> intp sf1 = sb1 / sizeof(double); \ >>>//... >>> #define f1 ((double *)x1)[j*sf1] >>> >>> >>>There are clearly some assumptions that sb1 is evenly divisible by >>>sizeof(double). Blech!. This is likely my fault, and I expect it won't >>>be too horrible to fix, but I don't know that I'll have time immediately. >>> >>> >>> >>My thinking is that this should be handled by a copy, so that the opcodes >>always work on contiguous data. The copy can be another opcode. One advantage >>of operating on contiguous data is that it's easier to use the processor's >>vector instructions, if applicable. >> >> >> > >That would be easy to do. Right now the opcodes should work correctly on >data that is spaced in multiples of the itemsize on the last axis. Other >arrays are copied (no opcode required, it's embedded at the top of >interp_body lines 64-80). The record array case apparently slips through >the cracks when we're checking whether an array is suitable to be used >correctly (interpreter.c 1086-1103). It would certainly not be any >harder to only allow contiguous arrays than to correctly deal with >record arrays. Only question I have is whether the extra copy will >overwhelm the savings of that operating on contiguous data gives. > With record arrays you have to worry about alignment issues. The most complicated part of the ufunc code is to handle that. The usual approach is to copy (and possibly byte-swap at least the axis you are working on) over to a buffer (the copyswapn functions will do that using a pretty fast approach for each data-type). This is ultimately how the ufuncs work (though the buffer-size is fixed so the data is copied and operated on in chunks). -Travis From haase at msg.ucsf.edu Wed Oct 4 19:28:27 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Wed, 4 Oct 2006 16:28:27 -0700 Subject: [Numpy-discussion] OSX 10.4 vs 10.3 - long double dynlib problem Message-ID: <200610041628.27926.haase@msg.ucsf.edu> Hi, I just compiled CVS numpy on OSX 10.4 (tiger). Runs fine. I was surprised when I tried to run the same mode on 10.3 (panther?) and got a importError (dyld:...) related to missing acoshl (long double) functions: ImportError: Failure linking new module: : dyld: /Library/Frameworks/Python.framework/Versions/2.4/Resources/Python.app/Contents/MacOS/Python Undefined symbols: /Volumes/haase/PrMacNPPC/numpy/core/umath.so undefined reference to _acoshl$LDBL128 expected to be defined in /usr/lib/libmx.A.dylib /Volumes/haase/PrMacNPPC/numpy/core/umath.so undefined reference to _acosl$LDBL128 expected to be defined in /usr/lib/libmx.A.dylib /Volumes/haase/PrMacNPPC/numpy/core/umath.so undefined reference to _asinhl$LDBL128 expected to be After asking around in my lab I was told that "long double" might pose a problem when going backwards from 10.4 to 10.3. (I used gfortran gcc 4.x) Can I tell setup.py to skip "long double" support ? Thanks, Sebastian Haase From gregwillden at gmail.com Wed Oct 4 23:05:31 2006 From: gregwillden at gmail.com (Greg Willden) Date: Wed, 4 Oct 2006 22:05:31 -0500 Subject: [Numpy-discussion] Hello and my first patch Message-ID: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> Hello All, I introduced myself on the Scipy list and I have a feeling that most of the subscribers here are also on Scipy-devel. Anyway I just submitted my first patch to numpy (ticket #316). I added the blackman-harris, Nuttall and Flat Top windowing functions and added "See also" sections to the docstrings for all the window functions. I probably didn't do the patch right though because I just did an "svn diff >patch". If there is a way you'd rather see it please let me know and I'll fix it. I look forward to contributing a lot more in the future. Have a nice day. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From oliphant at ee.byu.edu Thu Oct 5 00:15:50 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 04 Oct 2006 22:15:50 -0600 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> Message-ID: <452486F6.3060508@ee.byu.edu> Greg Willden wrote: > Hello All, > I introduced myself on the Scipy list and I have a feeling that most > of the subscribers here are also on Scipy-devel. Anyway I just > submitted my first patch to numpy (ticket #316). I added the > blackman-harris, Nuttall and Flat Top windowing functions and added > "See also" sections to the docstrings for all the window functions. Great contribution. Thanks a bunch. I think this will probably go into the scipy package, though. There is already a lot of windows available in the scipy.signal.window function. The window functions that are in NumPy are there only for historical purposes only (i.e. compatibility with old MLab). On the other hand, the other thought to consider is that since we have window functions in NumPy already. Let's just move them all from scipy.signal into NumPy. -Travis From haase at msg.ucsf.edu Thu Oct 5 00:35:49 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Wed, 04 Oct 2006 21:35:49 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452486F6.3060508@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> Message-ID: <45248BA5.6010709@msg.ucsf.edu> Travis Oliphant wrote: > Greg Willden wrote: > >> Hello All, >> I introduced myself on the Scipy list and I have a feeling that most >> of the subscribers here are also on Scipy-devel. Anyway I just >> submitted my first patch to numpy (ticket #316). I added the >> blackman-harris, Nuttall and Flat Top windowing functions and added >> "See also" sections to the docstrings for all the window functions. > > Great contribution. Thanks a bunch. I think this will probably go into > the scipy package, though. There is already a lot of windows available > in the scipy.signal.window function. > > The window functions that are in NumPy are there only for historical > purposes only (i.e. compatibility with old MLab). > > On the other hand, the other thought to consider is that since we have > window functions in NumPy already. Let's just move them all from > scipy.signal into NumPy. If scipy is going to be installable as separate sub-packages maybe all window functions can be moved to scipy ;-) In other words, if the ones in numpy are there only for "historical reasons" maybe they should be cleaned out before the 1.0 release. All arguments seem similar to ndimage (which was is numarray and is now in scipy) -Sebastian > > -Travis > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 5 00:54:51 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 04 Oct 2006 22:54:51 -0600 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <45248BA5.6010709@msg.ucsf.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <45248BA5.6010709@msg.ucsf.edu> Message-ID: <4524901B.3090505@ee.byu.edu> Sebastian Haase wrote: >If scipy is going to be installable as separate sub-packages maybe >all window functions can be moved to scipy ;-) > >In other words, if the ones in numpy are there only for "historical >reasons" maybe they should be cleaned out before the 1.0 release. >All arguments seem similar to ndimage (which was is numarray and is now >in scipy) > > Not really, because these functions were in *both* Numeric and numarray. That's the trouble. And the multiple scipy packages situation needs more discussion.... We are all ears... -Travis From david at ar.media.kyoto-u.ac.jp Thu Oct 5 02:08:33 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 05 Oct 2006 15:08:33 +0900 Subject: [Numpy-discussion] speeding up y[:, i] for y.shape = (big number, small number) In-Reply-To: References: <452241A3.3000702@ar.media.kyoto-u.ac.jp> <45228EAA.9040301@ieee.org> <452393D1.2020308@ar.media.kyoto-u.ac.jp> Message-ID: <4524A161.8060708@ar.media.kyoto-u.ac.jp> Bruce Southey wrote: > Hi, > I think what you are after is the multivariate normal distribution. > Indeed > Assuming I have it correctly, it is clearer to see (and probably more > accurate to compute) in the log form as: > > -(N/2)*log(2*PI) - 0.5*log(determinant of V) - 0.5*(transpose of > (x-mu))*inverse(V)*(x-mu) > > where N is the number of observations, PI is math constant, V is the > known variance co-variance matrix, x is vector of values, mu is the > known mean. > Sure, but I need the exponential form at the end (actually, in the real code, you can choose between log form and 'standard' form, but I removed this to make the example as simple as possible). > If so, then you can vectorize the density calculation. I am not sure to understand what you mean: the computation is already vectorized in _diag_gauss_den; there is no loop there, and the function expects x to be of shape (n, d), where d is the dimension and n the number of samples. The loop in multiple_gaussian in not on samples, but on densities, that is I need to compute the normal multivariate densities on the same data but with different (vector) means and (diagonal ) variances, for which I don't see any easy way to vectorize without huge memory usage (using rank 3 arrays). Anyway, the problem I try to understand here is not related to gaussian kernel computation, but rather on cost difference for accessing row and columns depending on the underlying storage (C or Fortran). Don't try to find flaws in _diag_gauss_den, as it is just a toy example to make my point clearer. David From Mailer-Daemon at lists.sourceforge.net Thu Oct 5 02:46:13 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Wed, 04 Oct 2006 23:46:13 -0700 Subject: [Numpy-discussion] Warning: message 1GUFPK-0001wW-5k delayed 72 hours Message-ID: This message was created automatically by mail delivery software. A message that you sent has not yet been delivered to one or more of its recipients after more than 72 hours on the queue on externalmx-1.sourceforge.net. The message identifier is: 1GUFPK-0001wW-5k The date of the message is: Mon, 02 Oct 2006 09:32:13 -0200 The subject of the message is: doctor, at the revolt is located treating the planet he gold; The address to which the message has not yet been delivered is: numpy-discussion at lists.sourceforge.net Delay reason: SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is No action is required on your part. Delivery attempts will continue for some time, and this warning may be repeated at intervals if the message remains undelivered. Eventually the mail delivery software will give up, and when that happens, the message will be returned to you. From ckkart at hoc.net Thu Oct 5 04:41:25 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Thu, 5 Oct 2006 08:41:25 +0000 (UTC) Subject: [Numpy-discussion] compatibility of extension modules Message-ID: Hi, i've got problems running a numpy/scipy extension module (scipy.sandbox.odr) built with cygwin/mingw32 on XP on other machines (windows 2000). I get those very informative 'windows encountered a problem' messages when calling the extension module - importing seems to work. Is there any optimization done in the build process which creates incompatibilities between different systems? And if yes, how can I turn them of? Regards, Christian From ivilata at carabos.com Thu Oct 5 04:42:05 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 05 Oct 2006 10:42:05 +0200 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523ED0C.4070306@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> Message-ID: <4524C55D.6050001@carabos.com> En/na Tim Hochberg ha escrit:: > Ivan Vilata i Balaguer wrote: >> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >> so, but I have come across some alignment or striding problems which can >> be seen with the following code:: > I looked at this just a little bit and clearly this bit from interp_body > cannot work in the presence of recor arrays: > > //.... > intp sf1 = sb1 / sizeof(double); \ > //... > #define f1 ((double *)x1)[j*sf1] > > There are clearly some assumptions that sb1 is evenly divisible by > sizeof(double). [...] I noticed something strange in those statements when implementing support for strings, and I must confess that I didn't grasp their meaning, so I implemented it a little differently for strings:: #define s1 ((char *)x1 + j*params.memsteps[arg1]) That seemed to work, but it might not be right (though I tested a bit), and certainly it may not be efficient enough. Here you have my previous patches if you want to have a look at how I (try to) do it: 1.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg01551.html 2.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg02261.html 3.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg02644.html :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: From gregwillden at gmail.com Thu Oct 5 09:24:35 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 08:24:35 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <4524901B.3090505@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <45248BA5.6010709@msg.ucsf.edu> <4524901B.3090505@ee.byu.edu> Message-ID: <903323ff0610050624n7e6949d6gb566030590fc2271@mail.gmail.com> On 10/4/06, Travis Oliphant wrote: > > Not really, because these functions were in *both* Numeric and > numarray. That's the trouble. > > And the multiple scipy packages situation needs more discussion.... We > are all ears... Well I started here because I have been using the Matplotlib package and it uses these compatibility functions. So I wanted more window functions available for calls to specgram. This does bring up a frustration for users new to scipy/numpy/matplotlib. In short, that there are three packages. I know the historical reasons why but I hope that the ultimate destination is a tight integration among these packages. I hope to help out toward that goal. Thanks Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.hochberg at ieee.org Thu Oct 5 09:24:42 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 05 Oct 2006 06:24:42 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4523F289.901@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> Message-ID: <4525079A.4080801@ieee.org> Tim Hochberg wrote: > David M. Cooke wrote: >> On Wed, 04 Oct 2006 10:19:08 -0700 >> Tim Hochberg wrote: >> >> >>> Ivan Vilata i Balaguer wrote: >>> >>>> It seemed that discontiguous arrays worked OK in Numexpr since >>>> r1977 or >>>> so, but I have come across some alignment or striding problems >>>> which can >>>> be seen with the following code:: >>>> >>> I looked at this just a little bit and clearly this bit from >>> interp_body cannot work in the presence of recor arrays: >>> >>> //.... >>> intp sf1 = sb1 / sizeof(double); \ >>> //... >>> #define f1 ((double *)x1)[j*sf1] >>> >>> >>> There are clearly some assumptions that sb1 is evenly divisible by >>> sizeof(double). Blech!. This is likely my fault, and I expect it >>> won't be too horrible to fix, but I don't know that I'll have time >>> immediately. >>> >> >> My thinking is that this should be handled by a copy, so that the >> opcodes >> always work on contiguous data. The copy can be another opcode. One >> advantage >> of operating on contiguous data is that it's easier to use the >> processor's >> vector instructions, if applicable. >> > > That would be easy to do. Right now the opcodes should work correctly > on data that is spaced in multiples of the itemsize on the last axis. > Other arrays are copied (no opcode required, it's embedded at the top > of interp_body lines 64-80). The record array case apparently slips > through the cracks when we're checking whether an array is suitable to > be used correctly (interpreter.c 1086-1103). It would certainly not be > any harder to only allow contiguous arrays than to correctly deal with > record arrays. Only question I have is whether the extra copy will > overwhelm the savings of that operating on contiguous data gives. The > thing to do is probably try it and see what happens. OK, I've checked in a fix for this that makes a copy when the array is not strided in an even multiple of the itemsize. I first tried copying for all discontiguous array, but this resulted in a large speed hit for vanilla strided arrays (a=arange(10)[::2], etc.), so I was more frugal with my copying. I'm not entirely certain that I caught all of the problematic cases, so let me know if you run into any more issues like this. -tim From oliphant at ee.byu.edu Thu Oct 5 09:34:45 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 07:34:45 -0600 Subject: [Numpy-discussion] Vectorizing code, for loops, and all that In-Reply-To: <45227AC6.4010801@ieee.org> References: <45227AC6.4010801@ieee.org> Message-ID: <452509F5.4050702@ee.byu.edu> Travis Oliphant wrote: >Albert Strasheim wrote: > > >>>>[1] 12.97% of function time >>>>[2] 8.65% of functiont ime >>>>[3] 62.14% of function time >>>> >>>>If statistics from elsewhere in the code would be helpful, let me >>>>know, >>>> >>>> >>>> >>>and >>> >>> >>> >>>>I'll see if I can convince Quantify to cough it up. >>>> >>>> >>>> >>>> >>>Please run the same test but using >>> >>>x1 = N.random.rand(39,2000) >>>x2 = N.random.rand(39,64,1) >>> >>>z1 = x1[:,N.newaxis,:] - x2 >>> >>> >>> >>Very similar results to what I had previously: >> >>[1] 10.88% >>[2] 7.25% >>[3] 68.25% >> >> >> >> >Thanks, > >I've got some ideas about how to speed this up by eliminating some of >the unnecessary calculations going on outside of the function loop, but >there will still be some speed issues depending on how the array is >traversed once you get above a certain size. I'm not sure there anyway >around that, ultimately, due to memory access being slow on most hardware. > > Well, I tried out my ideas and didn't get much improvement (8-10%). Then, I finally realized more fully that the slowness was due to the loop taking place over an axis which had a very large stride so that the memory access was taking a long time. Thus, instead of picking the loop axis to correspond to the axis with the longest dimension, I've picked the loop axis to be one with the smallest sum of strides. In this particular example, the speed-up is about 6-7 times... -Travis From david at ar.media.kyoto-u.ac.jp Thu Oct 5 09:37:33 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 05 Oct 2006 22:37:33 +0900 Subject: [Numpy-discussion] On loop and broadcasting (again) Message-ID: <45250A9D.5040403@ar.media.kyoto-u.ac.jp> Hi, The email from Albert made me look again on some surprising results I got a few months ago when starting my first "serious" numpy project. I noticed that when computing multivariate gaussian densities, centering the data was more expensive than everything else, including exponentiation. Now that I have some experience with numpy, and following the previous discussion, I tried the following script: import numpy from numpy.random import randn import numpy as N def center_broadcast(X, mu): return X - mu def center_broadcast2(X, mu): return N.transpose(X.T - mu.T) def center_repmat(X, mu): n = X.shape[0] return X - N.repmat(mu, n, 1) def center_outer(X, mu): n = X.shape[0] return X - N.outer(N.ones(n), mu) def center_dot(X, mu): n = X.shape[0] return X - N.dot(N.ones((n, 1)), mu[N.newaxis, :]) def center_manual(X, mu): return X - mu def bench(): n = 1e5 d = 30 niter = 10 X = randn(n, d) mu = randn(d) va = randn(d) ** 2 mur = N.repmat(mu, n, 1) for i in range(niter): y1 = center_broadcast(X, mu) y2 = center_repmat(X, mu) y3 = center_dot(X, mu) y4 = center_outer(X, mu) y5 = center_manual(X, mur) y6 = center_broadcast2(X, mur) if __name__ == '__main__': import hotshot, hotshot.stats profile_file = 'storage.prof' prof = hotshot.Profile(profile_file, lineevents=1) prof.runcall(bench) p = hotshot.stats.load(profile_file) print p.sort_stats('cumulative').print_stats(20) prof.close() If X is an array containing n samples of d dimension, and mu an array with d elements, I want to compute x - mu, for each row x of X. The idea is to compare timing of "manual broadcasting" using repmat vs broadcasting, and storage influence: - center_broadcast expect mu to be a d elements array, and uses the numpy broadcast rules - center_broadcast2 expect same args, but tranpose to force a C-like storage. - center_repmat uses repmat instead of the automatic broadcast to make mu a (n, d) array with n times the same row - center_outer and center_dot implements manually the repmat using matrix product (under matlab, this was much faster on large matrices) - center_manual expects mu to be already in a (n, d) form with n times the same row: this is to see the cost of the substraction itself. The results are the following: 10 4.204 0.420 4.204 0.420 storage_email.py:8(center_broadcast) 10 0.475 0.048 3.449 0.345 storage_email.py:18(center_outer) 10 2.964 0.296 2.964 0.296 /usr/lib/python2.4/site-packages/numpy/core/numeric.py:227(outer) 10 2.767 0.277 2.768 0.277 storage_email.py:11(center_broadcast2) 10 0.485 0.049 1.288 0.129 storage_email.py:14(center_repmat) 10 1.217 0.122 1.227 0.123 storage_email.py:22(center_dot) 11 0.883 0.080 0.883 0.080 /usr/lib/python2.4/site-packages/numpy/lib/shape_base.py:522(repmat) 10 0.457 0.046 0.457 0.046 storage_email.py:26(center_manual) 5 0.137 0.027 0.137 0.027 /usr/lib/python2.4/site-packages/numpy/core/fromnumeric.py:375(sum) 20 0.020 0.001 0.020 0.001 /usr/lib/python2.4/site-packages/numpy/core/numeric.py:527(ones) 31 0.000 0.000 0.000 0.000 /usr/lib/python2.4/site-packages/numpy/core/numeric.py:125(asarray) 10 0.000 0.000 0.000 0.000 /usr/lib/python2.4/site-packages/numpy/core/fromnumeric.py:116(transpose) As you can see, there is a quite a difference between implementations: using repmat is the fastest, with repmat taking twice as much time as the substraction itself. Replacing repmat with dot does not give any advantage (I guess the underlying implementation uses the same core C functions ?). Is it expected than automatic broadcast is so much expensive ? And why using tranpose gives a 40% speed improvement ? What makes automatic broadcast so expensive compared to repmat ? For what it worths, the substraction itself is as fast as in matlab, whereas the repmat is twice slower (this quick script is not meant as a comparison against matlab of course, but I just wanted to check how matlab behaves in those situations), David From oliphant at ee.byu.edu Thu Oct 5 09:38:34 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 07:38:34 -0600 Subject: [Numpy-discussion] On loop and broadcasting (again) In-Reply-To: <45250A9D.5040403@ar.media.kyoto-u.ac.jp> References: <45250A9D.5040403@ar.media.kyoto-u.ac.jp> Message-ID: <45250ADA.105@ee.byu.edu> David Cournapeau wrote: >Hi, > > The email from Albert made me look again on some surprising results I >got a few months ago when starting my first "serious" numpy project. I >noticed that when computing multivariate gaussian densities, centering >the data was more expensive than everything else, including >exponentiation. Now that I have some experience with numpy, and >following the previous discussion, I tried the following script: > > Try it again with the new code in SVN. -Travis From oliphant at ee.byu.edu Thu Oct 5 09:39:53 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 07:39:53 -0600 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4525079A.4080801@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> <4525079A.4080801@ieee.org> Message-ID: <45250B29.4040001@ee.byu.edu> Tim Hochberg wrote: >>> >>> >>> >>That would be easy to do. Right now the opcodes should work correctly >>on data that is spaced in multiples of the itemsize on the last axis. >>Other arrays are copied (no opcode required, it's embedded at the top >>of interp_body lines 64-80). The record array case apparently slips >>through the cracks when we're checking whether an array is suitable to >>be used correctly (interpreter.c 1086-1103). It would certainly not be >>any harder to only allow contiguous arrays than to correctly deal with >>record arrays. Only question I have is whether the extra copy will >>overwhelm the savings of that operating on contiguous data gives. The >>thing to do is probably try it and see what happens. >> >> > >OK, I've checked in a fix for this that makes a copy when the array is >not strided in an even multiple of the itemsize. I first tried copying >for all discontiguous array, but this resulted in a large speed hit for >vanilla strided arrays (a=arange(10)[::2], etc.), so I was more frugal >with my copying. I'm not entirely certain that I caught all of the >problematic cases, so let me know if you run into any more issues like this. > > > There is an ElementStrides check and similar requirement flag you can use to make sure that you have an array whose strides are multiples of it's itemsize. -Travis From gregwillden at gmail.com Thu Oct 5 10:02:25 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 09:02:25 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452486F6.3060508@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> Message-ID: <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> On 10/4/06, Travis Oliphant wrote: > > Great contribution. Thanks a bunch. I think this will probably go into > the scipy package, though. There is already a lot of windows available > in the scipy.signal.window function. > BTW. Is there some sort of clear statement about what goes in Scipy versus what goes in Numpy? It's a bit confusing for this newbie. Thanks Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 5 11:34:13 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 08:34:13 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> Message-ID: Greg Willden wrote: > On 10/4/06, *Travis Oliphant* > wrote: > > Great contribution. Thanks a bunch. I think this will probably go into > the scipy package, though. There is already a lot of windows available > in the scipy.signal.window function. > > BTW. Is there some sort of clear statement about what goes in Scipy > versus what goes in Numpy? It's a bit confusing for this newbie. IMO, I'd rather see this and similar functions go into scipy. New functions that apply semantics to arrays (in this case, treating them as time series), I think should go into scipy. New functions that treat arrays simply as arrays and are generally useful can probably go into numpy. There's some grey area in this scheme, of course. New non-uniform random number generators might need to go into numpy, for now, simply due to technical reasons. Fleshing out the masked array and matrix classes would be similar, I imagine. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From gregwillden at gmail.com Thu Oct 5 11:58:17 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 10:58:17 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> Message-ID: <903323ff0610050858l57681adbmeeb75bf874043dcf@mail.gmail.com> On 10/5/06, Robert Kern wrote: > > IMO, I'd rather see this and similar functions go into scipy. New > functions that > apply semantics to arrays (in this case, treating them as time series), I > think > should go into scipy. New functions that treat arrays simply as arrays and > are > generally useful can probably go into numpy. > Okay I'll take a look at the Scipy parts tonight. So do you cancel that ticket I created or do I attach a new patch against scipy or what? Thanks Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivilata at carabos.com Thu Oct 5 11:59:45 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 05 Oct 2006 17:59:45 +0200 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4525079A.4080801@ieee.org> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> <4525079A.4080801@ieee.org> Message-ID: <45252BF1.8070705@carabos.com> En/na Tim Hochberg ha escrit:: > Tim Hochberg wrote: >>>> Ivan Vilata i Balaguer wrote: >>>>> It seemed that discontiguous arrays worked OK in Numexpr since >>>>> r1977 or >>>>> so, but I have come across some alignment or striding problems >>>>> which can >>>>> be seen with the following code:: > OK, I've checked in a fix for this that makes a copy when the array is > not strided in an even multiple of the itemsize. I first tried copying > for all discontiguous array, but this resulted in a large speed hit for > vanilla strided arrays (a=arange(10)[::2], etc.), so I was more frugal > with my copying. I'm not entirely certain that I caught all of the > problematic cases, so let me know if you run into any more issues like this. Great! For the moment I just can say that the problems I had with that have disappeared, but I will keep you up to date if I find something else. Thank you very much! :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: From jdhunter at ace.bsd.uchicago.edu Thu Oct 5 12:33:22 2006 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Thu, 05 Oct 2006 11:33:22 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: (Robert Kern's message of "Thu, 05 Oct 2006 08:34:13 -0700") References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> Message-ID: <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> >>>>> "Robert" == Robert Kern writes: Robert> IMO, I'd rather see this and similar functions go into Robert> scipy. New functions that apply semantics to arrays (in Robert> this case, treating them as time series), I think should Robert> go into scipy. New functions that treat arrays simply as Robert> arrays and are generally useful can probably go into Robert> numpy. I prefer Perry's longstanding suggestion: things that do not add to distribution complexity should go into numpy. If it compiles as easily as numpy itself, it should go into numpy where sensible. It remains a fact of life that numpy gets a wider distribution than scipy, and some packages are hesitant to require scipy as a prereq because of the additional complexity or building fortran, etc. I would be nice to have as much as possible in the most widely distributed package IMO. JDH From tim.hochberg at ieee.org Thu Oct 5 12:52:14 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 05 Oct 2006 09:52:14 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <4524C55D.6050001@carabos.com> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <4524C55D.6050001@carabos.com> Message-ID: <4525383E.2090209@ieee.org> Ivan Vilata i Balaguer wrote: > En/na Tim Hochberg ha escrit:: > > >> Ivan Vilata i Balaguer wrote: >> >>> It seemed that discontiguous arrays worked OK in Numexpr since r1977 or >>> so, but I have come across some alignment or striding problems which can >>> be seen with the following code:: >>> >> I looked at this just a little bit and clearly this bit from interp_body >> cannot work in the presence of recor arrays: >> >> //.... >> intp sf1 = sb1 / sizeof(double); \ >> //... >> #define f1 ((double *)x1)[j*sf1] >> >> There are clearly some assumptions that sb1 is evenly divisible by >> sizeof(double). [...] >> > > I noticed something strange in those statements when implementing > support for strings, and I must confess that I didn't grasp their > meaning, so I implemented it a little differently for strings:: > > #define s1 ((char *)x1 + j*params.memsteps[arg1]) > I believe that these approaches are the same as long as memstep is a multiple of itemsize. I chose the indexing rather than the pointer foo version[1] because there's a rumor that compilers will sometimes generate faster code for it. One additional potential slowdown in the above is the compiler may not be able to tell that memsteps[arg1]) is constant and may do that lookup repeatedly. Or maybe not, I try not to second guess compilers too much. -tim [1] I'm pretty sure I used the pointer foo version at least for a while. and may have gone back and forth several times. > That seemed to work, but it might not be right (though I tested a bit), > and certainly it may not be efficient enough. Here you have my previous > patches if you want to have a look at how I (try to) do it: > > 1.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg01551.html > 2.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg02261.html > 3.http://www.mail-archive.com/numpy-discussion%40lists.sourceforge.net/msg02644.html > > :: > > Ivan Vilata i Balaguer >qo< http://www.carabos.com/ > C?rabos Coop. V. V V Enjoy Data > "" > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From bhendrix at enthought.com Thu Oct 5 13:34:24 2006 From: bhendrix at enthought.com (Bryce Hendrix) Date: Thu, 05 Oct 2006 12:34:24 -0500 Subject: [Numpy-discussion] zip safe egg? Message-ID: <45254220.2010001@enthought.com> Is anyone using the numpy egg compressed? Recent versions of setuptools seem to think it is zip safe, but this causes builds to fail which compile with the numpy headers. Can we change the setup.py to default to not zip safe (by adding zip_safe=False as a keyword arg to the setup function call)? Bryce From robert.kern at gmail.com Thu Oct 5 13:41:53 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 10:41:53 -0700 Subject: [Numpy-discussion] zip safe egg? In-Reply-To: <45254220.2010001@enthought.com> References: <45254220.2010001@enthought.com> Message-ID: Bryce Hendrix wrote: > Is anyone using the numpy egg compressed? Recent versions of setuptools > seem to think it is zip safe, but this causes builds to fail which > compile with the numpy headers. Can we change the setup.py to default to > not zip safe (by adding zip_safe=False as a keyword arg to the setup > function call)? It is not zip-safe if you want to compile against the headers. That keyword can't be added to the setup() call in the trunk's setup.py because numpy cannot depend on setuptools, at the moment. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From bhendrix at enthought.com Thu Oct 5 13:52:27 2006 From: bhendrix at enthought.com (Bryce Hendrix) Date: Thu, 05 Oct 2006 12:52:27 -0500 Subject: [Numpy-discussion] zip safe egg? In-Reply-To: References: <45254220.2010001@enthought.com> Message-ID: <4525465B.9050608@enthought.com> Robert Kern wrote: > It is not zip-safe if you want to compile against the headers. That keyword > can't be added to the setup() call in the trunk's setup.py because numpy cannot > depend on setuptools, at the moment. > > Adding the keyword does not break builds not using setuptools, the build just prints a warning that its not a valid keyword. I just discovered this is also an issue for scipy eggs when building with weave. Perhaps the best solution is to complain to the setuptools list, I'm just looking for a quick fix for now. Bryce From robert.kern at gmail.com Thu Oct 5 14:25:55 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 11:25:55 -0700 Subject: [Numpy-discussion] zip safe egg? In-Reply-To: <4525465B.9050608@enthought.com> References: <45254220.2010001@enthought.com> <4525465B.9050608@enthought.com> Message-ID: Bryce Hendrix wrote: > Robert Kern wrote: >> It is not zip-safe if you want to compile against the headers. That keyword >> can't be added to the setup() call in the trunk's setup.py because numpy cannot >> depend on setuptools, at the moment. >> > Adding the keyword does not break builds not using setuptools, the build > just prints a warning that its not a valid keyword. I just discovered > this is also an issue for scipy eggs when building with weave. A warning is not really acceptable in the trunk, either. We've found that warnings during the build process tend to make people think that something went wrong. > Perhaps the best solution is to complain to the setuptools list, I'm > just looking for a quick fix for now. Patch setup.py in our build system, I would think. What revision of numpy and what version of setuptools are you using? setuptools 0.7a1 and numpy r3261 correctly recognizes numpy as not zip-safe. zip_safe flag not set; analyzing archive contents... numpy._import_tools: module references __file__ numpy._import_tools: module references __path__ numpy.version: module references __file__ numpy.core.generate_array_api: module references __file__ numpy.core.setup: module references __file__ numpy.distutils.exec_command: module references __file__ numpy.distutils.misc_util: module references __file__ numpy.distutils.system_info: module references __file__ numpy.distutils.command.build_src: module references __file__ numpy.f2py.diagnose: module references __file__ numpy.f2py.f2py2e: module references __file__ numpy.f2py.setup: module references __file__ numpy.f2py.lib.wrapper_base: module references __file__ numpy.lib.utils: module MAY be using inspect.getsource numpy.lib.utils: module MAY be using inspect.getsourcefile numpy.numarray.util: module references __file__ numpy.testing.numpytest: module references __file__ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From bhendrix at enthought.com Thu Oct 5 14:51:52 2006 From: bhendrix at enthought.com (Bryce Hendrix) Date: Thu, 05 Oct 2006 13:51:52 -0500 Subject: [Numpy-discussion] zip safe egg? In-Reply-To: References: <45254220.2010001@enthought.com> <4525465B.9050608@enthought.com> Message-ID: <45255448.9010100@enthought.com> Robert Kern wrote: >> Perhaps the best solution is to complain to the setuptools list, I'm >> just looking for a quick fix for now. >> > > Patch setup.py in our build system, I would think. > > Thats what I did, but patching working copies of files has been troublesome in the past when svn conflicts have wiggled their way into shipped builds. > What revision of numpy and what version of setuptools are you using? setuptools > 0.7a1 and numpy r3261 correctly recognizes numpy as not zip-safe. > > 0.6c3. I think 0.6c1 did _not_ tag it as zip safe either. Guess I'll just update setuptools on our build system. Bryce -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregwillden at gmail.com Thu Oct 5 15:25:16 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 14:25:16 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> On 10/5/06, John Hunter wrote: > > I prefer Perry's longstanding suggestion: things that do not add to > distribution complexity should go into numpy. If it compiles as > easily as numpy itself, it should go into numpy where sensible. It > remains a fact of life that numpy gets a wider distribution than > scipy, and some packages are hesitant to require scipy as a prereq > because of the additional complexity or building fortran, etc. I > would be nice to have as much as possible in the most widely > distributed package IMO. That is a much better policy in my view. I (gently) encourage this group (Travis?) to make this the policy for Numpy/Scipy. >From my view as a newbie to numpy/scipy/matplotlib it isn't clear where I should look for what functionality. Matplotlib plots the spectrogram but it only supports two or three window functions. Numpy supports 4 or 5 window functions and Scipy apparently supports more but Matplotlib doesn't support Scipy. Of course this is a minor example and I could just write the window function myself and then use it in Matplotlib but I want to give back so that the project can grow. I'd really like to be able to leave Matlab behind and encourage everyone else to do the same but there are still these annoyances that need to be worked out. Thanks Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 5 15:39:35 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 12:39:35 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> Message-ID: Greg Willden wrote: > From my view as a newbie to numpy/scipy/matplotlib it isn't clear where > I should look for what functionality. Matplotlib plots the spectrogram > but it only supports two or three window functions. Numpy supports 4 or > 5 window functions and Scipy apparently supports more but Matplotlib > doesn't support Scipy. That's not true. specgram() takes a windowing function. It doesn't matter where that function comes from. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From peridot.faceted at gmail.com Thu Oct 5 15:46:48 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Thu, 5 Oct 2006 15:46:48 -0400 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> Message-ID: On 05/10/06, Greg Willden wrote: > That is a much better policy in my view. > > I (gently) encourage this group (Travis?) to make this the policy for > Numpy/Scipy. > > From my view as a newbie to numpy/scipy/matplotlib it isn't clear where I > should look for what functionality. Matplotlib plots the spectrogram but it > only supports two or three window functions. Numpy supports 4 or 5 window > functions and Scipy apparently supports more but Matplotlib doesn't support > Scipy. Of course this is a minor example and I could just write the window > function myself and then use it in Matplotlib but I want to give back so > that the project can grow. I'd really like to be able to leave Matlab > behind and encourage everyone else to do the same but there are still these > annoyances that need to be worked out. Unfortunately, that policy (put it in numpy if it doesn't make the build dependencies any worse) makes it even harder for the user to figure out what is where. Say I want a fast matrix product. Do I look in numpy or scipy? It'll run faster if it uses a tuned BLAS, so it ought to have external requirements, so I'd look in scipy, but maybe there's a simple non-tuned implementation in numpy instead... Frankly, I tend to prefer the other approach to solving all these issues: distribute numpy, scipy, and matplotlib as one bundle. The requirements for scipy are not particularly onerous, particularly if it comes as part of your distribution. There are currently some problems successfully finding optimized versions of LAPACK and BLAS, but to me the benefits of bundling the packages together outweigh the difficulties: * routines and objects can be in the package in which they make the most semantic sense, rather than the one with the correct external dependencies (how is a user supposed to know whether convolution uses an external library or not?) * documentation can be cross-referenced between packages (so the Matrix class can tell people to look in scipy.linalg for inverses, for example) * users can more easily find what's available rather than rewriting from scratch * derived packages (ipython, IDEs, MATLAB-alikes) have many fewer possibilities to support I'm not arguing that the development processes need to be connected, just that making the primary distributed object a package containing all the components will make life easier for everyone involved. A. M. Archibald From oliphant at ee.byu.edu Thu Oct 5 15:53:58 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 13:53:58 -0600 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <452562D6.7010605@ee.byu.edu> John Hunter wrote: >>>>>>"Robert" == Robert Kern writes: >>>>>> >>>>>> > > Robert> IMO, I'd rather see this and similar functions go into > Robert> scipy. New functions that apply semantics to arrays (in > Robert> this case, treating them as time series), I think should > Robert> go into scipy. New functions that treat arrays simply as > Robert> arrays and are generally useful can probably go into > Robert> numpy. > >I prefer Perry's longstanding suggestion: things that do not add to >distribution complexity should go into numpy. If it compiles as >easily as numpy itself, it should go into numpy where sensible. > I don't think this is as feasible as it sounds at first. Some people complain that NumPy is too big already. SciPy is very easy to install on Windows (there is a binary available). The only major platform that still gives some trouble is Intel Mac (due to the fortran compiler situation). But, all you need is one person who can build it and distribute a binary. I think a better long-term solution is to understand how to package things better by working with people at Enthought so that when you advertise to the ex-Matlab user you point him to a "super-package" that installs a bunch of other small packages. This is a more maintainable solution as long as we set standards for 1) documentation 2) tests 3) some kind of problem-domain hierarchy The idea of just lumping more an more things into NumPy itself is not a good idea. What most users want is something that installs easily (like Enthon). How it is packaged is not as important. What developers need is a sane multi-namespace system that can be maintained separately if needed. I think we decided a while ago, that the package approach should contain indicators as to whether or not a fortran compiler was needed to build the system so that dependency on those things could be eliminated if needed. Do we want to pull scipy apart into two components: one that needs Fortran to build and another that doesn't? Perhaps that is the best way to move forward along with the work on a "pylab" super-package. -Travis From gregwillden at gmail.com Thu Oct 5 15:56:38 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 14:56:38 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> Message-ID: <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> On 10/5/06, Robert Kern wrote: > > Greg Willden wrote: > > > From my view as a newbie to numpy/scipy/matplotlib it isn't clear where > > I should look for what functionality. Matplotlib plots the spectrogram > > but it only supports two or three window functions. Numpy supports 4 or > > 5 window functions and Scipy apparently supports more but Matplotlib > > doesn't support Scipy. > > That's not true. specgram() takes a windowing function. It doesn't matter > where > that function comes from. The next sentence (that you snipped) afirms what you said. Of course this is a minor example and I could just write the window function myself and then use it in Matplotlib The details of my off-the-cuff example would probably be better addressed by the Matplotlib folks. (i.e. why they don't have builtin functions for more windows). You can see how it's all very confusing to someone new. "Matplotlib can plot a spectrogram but I need to use a window function from SciPy because Matplotlib only supports NumPy and NumPy doesn't have the one I want?" You get the idea. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregwillden at gmail.com Thu Oct 5 16:04:57 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 5 Oct 2006 15:04:57 -0500 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452562D6.7010605@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> Message-ID: <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> On 10/5/06, Travis Oliphant wrote: > > Perhaps that is the best way to move forward along with the work on a > "pylab" super-package. That is exactly what I want. In the end I want a nice collection of functions, logically organized, that let me analyze/filter/plot etc. etc. etc. The key for me is "logically organized". And right now that means "So a newbie can find the function I need and the function I need is there" I'm not criticising. I'd like to help get there. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 5 16:02:52 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 13:02:52 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> Message-ID: Greg Willden wrote: > On 10/5/06, *Robert Kern* > wrote: > > Greg Willden wrote: > > > From my view as a newbie to numpy/scipy/matplotlib it isn't > clear where > > I should look for what functionality. Matplotlib plots the > spectrogram > > but it only supports two or three window functions. Numpy > supports 4 or > > 5 window functions and Scipy apparently supports more but Matplotlib > > doesn't support Scipy. > > That's not true. specgram() takes a windowing function. It doesn't > matter where > that function comes from. > > The next sentence (that you snipped) afirms what you said. > > Of course this is a minor example and I could just write the window > function myself and then use it in Matplotlib > > > The details of my off-the-cuff example would probably be better > addressed by the Matplotlib folks. (i.e. why they don't have builtin > functions for more windows). > > You can see how it's all very confusing to someone new. > "Matplotlib can plot a spectrogram but I need to use a window function > from SciPy because Matplotlib only supports NumPy and NumPy doesn't have > the one I want?" > > You get the idea. No, I'm afraid I don't. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From aisaac at american.edu Thu Oct 5 16:12:44 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 5 Oct 2006 16:12:44 -0400 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com><452486F6.3060508@ee.byu.edu><903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com><87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> Message-ID: > On 10/5/06, John Hunter wrote: >> I would be nice to have as much as possible in the most >> widely distributed package IMO. On Thu, 5 Oct 2006, Greg Willden apparently wrote: > That is a much better policy in my view. A user's perspective: Well yes, all else equal, I'd like to have as much as possible in the easiest to install package, BUT my top priority is that numpy be completely bulletproof. Do these goals conflict? Cheers, Alan Isaac From peridot.faceted at gmail.com Thu Oct 5 16:13:13 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Thu, 5 Oct 2006 16:13:13 -0400 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> Message-ID: On 05/10/06, Greg Willden wrote: > On 10/5/06, Travis Oliphant wrote: > > Perhaps that is the best way to move forward along with the work on a > > "pylab" super-package. > > That is exactly what I want. What is unsatisfactory about installing numpy+scipy+matplotlib? I've found they're generally pretty complete (except where no decent python alternative exists). > In the end I want a nice collection of functions, logically organized, that > let me analyze/filter/plot etc. etc. etc. > > The key for me is "logically organized". For the most part, the organization is pretty logical: * Basic array and matrix operations in numpy * linear algebra, differential equation, interpolation, etc. tools are in scipy, each in their own subpackage * weave is mysteriously in scipy * plotting tools are in matplotlib There are a few historical quirks, like window functions in numpy (they really belong in scipy), and some of the less-used scipy subpackages are a bit of a mess internally (scipy.interpolate for example), but otherwise I'm not sure what you want to be different. > And right now that means "So a newbie can find the function I need and the > function I need is there" > > I'm not criticising. I'd like to help get there. Install all three major packages. Use the window functions from scipy in matplotlib. Task-oriented documentation is so far a bit scant, although the scipy cookbook (http://www.scipy.org/Cookbook ) and the numpy examples list (http://www.scipy.org/Numpy_Example_List ) are a good start. A. M. Archibald From martin.wiechert at gmx.de Thu Oct 5 16:09:03 2006 From: martin.wiechert at gmx.de (Martin Wiechert) Date: Thu, 5 Oct 2006 22:09:03 +0200 Subject: [Numpy-discussion] flat indexing of object arrays Message-ID: <200610052209.04341.martin.wiechert@gmx.de> Hi list, when I try to assign a sequence as an element of an object array via flat indexing only the first element of the sequence is assigned: >>> import numpy >>> numpy.version.version '1.0rc1.dev3171' >>> from numpy import * >>> a = ndarray ((2,2), object) >>> a.flat [2] = (1, 2, 3) >>> a.flat [2] 1 >>> a array([[None, None], [1, None]], dtype=object) Is this a feature? Wouldn't a naive user like me expect a.flat [2] == (1, 2, 3)? Thanks in advance, Martin Wiechert From fullung at gmail.com Thu Oct 5 16:19:33 2006 From: fullung at gmail.com (Albert Strasheim) Date: Thu, 5 Oct 2006 22:19:33 +0200 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452562D6.7010605@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> Message-ID: <20061005201932.GA31182@dogbert.sdsl.sun.ac.za> Hello all Some comments from a Windows user's perspective. On Thu, 05 Oct 2006, Travis Oliphant wrote: > John Hunter wrote: > > >>>>>>"Robert" == Robert Kern writes: > > > > Robert> IMO, I'd rather see this and similar functions go into > > Robert> scipy. New functions that apply semantics to arrays (in > > Robert> this case, treating them as time series), I think should > > Robert> go into scipy. New functions that treat arrays simply as > > Robert> arrays and are generally useful can probably go into > > Robert> numpy. > > > >I prefer Perry's longstanding suggestion: things that do not add to > >distribution complexity should go into numpy. If it compiles as > >easily as numpy itself, it should go into numpy where sensible. > > I don't think this is as feasible as it sounds at first. Some people > complain that NumPy is too big already. I agree here. Focus NumPy on doing array library things. > SciPy is very easy to install on Windows (there is a binary available). > The only major platform that still gives some trouble is Intel Mac (due > to the fortran compiler situation). But, all you need is one person > who can build it and distribute a binary. So far, I've been shying away from SciPy, because, if I encounter a problem, I have no easy way of building from SVN on Windows. I don't think I'm the only one: few Windows users have a proper Fortran compiler. Sure, there's MinGW, but that breaks all my other tools, most notably the Visual Studio debugger and other useful things like profilers (e.g. IBM Rational Quantify). That being said, Enthought's nightly builds obviate the need of most Windows users to build from source. (Enthought rocks.) Two feature requests at this point, which would make NumPy/SciPy/Matplotlib dead easy to use on Windows, even if you're trying to stay close to trunk: 1. Please consider setting up a buildbot(*) that builds and runs the tests on every checkin. I've set up a buildbot for NumPy on my own machine; it takes a matter of minutes. Probably they already have something like this in place. (*) http://buildbot.sourceforge.net/ 2. Please consider doing separate builds per CPU with ATLAS 3.7.11, Intel MKL and ACML. By all means, make a generic build available that runs everywhere. This will require some reading of the MKL license agreement, but I think Enthought should be able to distribute Windows builds based on MKL without problems. Why go to this trouble? MATLAB R2006b uses Intel MKL on my CPU, and it is much faster than NumPy with ATLAS 3.6.0. Core Duo users also have the option of enabling OpenMP, to spread calculations to multiple cores. > I think a better long-term solution is to understand how to package > things better by working with people at Enthought so that when you > advertise to the ex-Matlab user you point him to a "super-package" that > installs a bunch of other small packages. This is a more maintainable > solution as long as we set standards for > > 1) documentation > 2) tests > 3) some kind of problem-domain hierarchy Agreed. If Enthought is willing to provide the resources, Enthon could be the perfect solution to many of the issues that we currently encounter to get decent builds on Windows. > The idea of just lumping more an more things into NumPy itself is not a > good idea. What most users want is something that installs easily (like > Enthon). How it is packaged is not as important. What developers need > is a sane multi-namespace system that can be maintained separately if > needed. > > I think we decided a while ago, that the package approach should contain > indicators as to whether or not a fortran compiler was needed to build > the system so that dependency on those things could be eliminated if > needed. > > Do we want to pull scipy apart into two components: one that needs > Fortran to build and another that doesn't? Maybe. Maybe not. On Linux this doesn't make much difference to me if I check out 3 projects or 10 -- builds are easy. On Windows, getting the support libraries, build tools and configuration right is much harder. Hard enough, that I don't think anybody wants to do it regularly. > Perhaps that is the best way to move forward along with the work on a > "pylab" super-package. Yes, please. Regards, Albert From tim.hochberg at ieee.org Thu Oct 5 16:13:53 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 05 Oct 2006 13:13:53 -0700 Subject: [Numpy-discussion] Problems with Numexpr and discontiguous arrays In-Reply-To: <45250B29.4040001@ee.byu.edu> References: <452399C8.7050805@carabos.com> <4523ED0C.4070306@ieee.org> <20061004132736.00cbca52@arbutus.physics.mcmaster.ca> <4523F289.901@ieee.org> <4525079A.4080801@ieee.org> <45250B29.4040001@ee.byu.edu> Message-ID: <45256781.9090401@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > > >>>> >>>> >>>> >>>> >>> That would be easy to do. Right now the opcodes should work correctly >>> on data that is spaced in multiples of the itemsize on the last axis. >>> Other arrays are copied (no opcode required, it's embedded at the top >>> of interp_body lines 64-80). The record array case apparently slips >>> through the cracks when we're checking whether an array is suitable to >>> be used correctly (interpreter.c 1086-1103). It would certainly not be >>> any harder to only allow contiguous arrays than to correctly deal with >>> record arrays. Only question I have is whether the extra copy will >>> overwhelm the savings of that operating on contiguous data gives. The >>> thing to do is probably try it and see what happens. >>> >>> >>> >> OK, I've checked in a fix for this that makes a copy when the array is >> not strided in an even multiple of the itemsize. I first tried copying >> for all discontiguous array, but this resulted in a large speed hit for >> vanilla strided arrays (a=arange(10)[::2], etc.), so I was more frugal >> with my copying. I'm not entirely certain that I caught all of the >> problematic cases, so let me know if you run into any more issues like this. >> >> >> >> > There is an ElementStrides check and similar requirement flag you can > use to make sure that you have an array whose strides are multiples of > it's itemsize. > > Thanks Travis, I'll make a note; next time I look at this code I'll see if that can be used to simplify the code in question. -tim From Chris.Barker at noaa.gov Thu Oct 5 16:22:52 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 05 Oct 2006 13:22:52 -0700 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <903323ff0610051225k6a9215f0ud1a9a0c1160f1bd5@mail.gmail.com> <903323ff0610051256g30471327ve106c01781b3567d@mail.gmail.com> Message-ID: <4525699C.1050707@noaa.gov> The situation is confusing, we all know that, and we all want to move toward a better way. Key to that is that SciPy needs to be easier to build and install -- that's happening, but I don't know that it's there yet. Maybe it can be built on Fedora Core 4 now, but last I tried it couldn't be. Anyway, few thoughts on comments made here: > Matplotlib plots the spectrogram Here's a key problem -- matplotlib includes WAY too much. There are reasons, and there is history, but as a goal, I think matplotlib should be just what it says it is -- a plotting library. I don't know that MPL has been declared the "official" plotting package for SciPy, but it would be nice if it were. SciPy has suffered for a very long time without a full-featured, cross-platform, "official" plotting package. AS far as I know, MPL comes the closest (except it doesn't do 3-d -- darn!) A. M. Archibald wrote: > Frankly, I tend to prefer the other approach to solving all these > issues: distribute numpy, scipy, and matplotlib as one bundle. This is really the only way to set things up for someone that want what could be used as a "matlab replacement". If we ever get settools working just right, we should be able to do: easy_install scipy and have it all! woo hoo! However, as we work to that goal, i do think it makes sense that numpy, and matplotlib be packages unto themselves -- developed separately, etc. In fact, SciPy should be a collection of distinct packages as well. I think there is a real benefit to be able to install just what you need. Not very user of numpy and/or MPL is looking for a full scientific software package. I'm a big advocate of people using numpy arrays for all sorts of thinks that fit well into an n-d array, that have nothing to do with Scientific programming. Matplotlib is also very useful for people who need a quick plot for a web site or something. These people don't want to install the entirety of SciPy. > * routines and objects can be in the package in which they make the > most semantic sense, exactly. If it's plotting (but not computing stuff to plot) put it in MPL, if it's generic computation (if you can't understand it with high school math, it's not generic), put it in numpy. Of course, these aren't clear definitions, but can still be useful as guidelines. > * documentation can be cross-referenced between packages (so the > Matrix class can tell people to look in scipy.linalg for inverses, for > example) If it were me, I'd probably have the Matrix package depend on a linear algebra package, and have only one of those. Travis Oliphant wrote: > 3) some kind of problem-domain hierarchy +1 > Do we want to pull scipy apart into two components: one that needs > Fortran to build and another that doesn't? yup -- I don't like it, but the state of Fortran compilers really gives little choice. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From matthew.brett at gmail.com Thu Oct 5 17:30:16 2006 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu, 5 Oct 2006 22:30:16 +0100 Subject: [Numpy-discussion] flat indexing of object arrays In-Reply-To: <200610052209.04341.martin.wiechert@gmx.de> References: <200610052209.04341.martin.wiechert@gmx.de> Message-ID: <1e2af89e0610051430t1ed6da8ayd6471313ccd6002d@mail.gmail.com> Hi, On 10/5/06, Martin Wiechert wrote: > Hi list, > > when I try to assign a sequence as an element of an object array via flat > indexing only the first element of the sequence is assigned: I've also been having trouble with flat on object arrays. Is this intended? In [1]: from numpy import * In [2]: a = arange(2) In [3]: a[1] Out[3]: 1 In [4]: a.flat[1] Out[4]: 1 In [5]: b = array([a], dtype=object) In [6]: b[1] --------------------------------------------------------------------------- exceptions.IndexError Traceback (most recent call last) /home/mb312/devel_trees/scipy/Lib/io/ IndexError: index out of bounds In [7]: b.flat[1] Out[7]: 1 Best, Matthew From oliphant at ee.byu.edu Thu Oct 5 18:11:34 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 16:11:34 -0600 Subject: [Numpy-discussion] flat indexing of object arrays In-Reply-To: <200610052209.04341.martin.wiechert@gmx.de> References: <200610052209.04341.martin.wiechert@gmx.de> Message-ID: <45258316.60803@ee.byu.edu> Martin Wiechert wrote: >Hi list, > >when I try to assign a sequence as an element of an object array via flat >indexing only the first element of the sequence is assigned: > > > >>>>import numpy >>>>numpy.version.version >>>> >>>> >'1.0rc1.dev3171' > > >>>>from numpy import * >>>>a = ndarray ((2,2), object) >>>>a.flat [2] = (1, 2, 3) >>>>a.flat [2] >>>> >>>> >1 > > >>>>a >>>> >>>> >array([[None, None], > [1, None]], dtype=object) > >Is this a feature? Wouldn't a naive user like me expect >a.flat [2] == (1, 2, 3)? > > > You are probably right. This should be changed. -Travis From oliphant at ee.byu.edu Thu Oct 5 18:19:26 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 16:19:26 -0600 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> Message-ID: <452584EE.70106@ee.byu.edu> A. M. Archibald wrote: >On 05/10/06, Greg Willden wrote: > > >>On 10/5/06, Travis Oliphant wrote: >> >> >>>Perhaps that is the best way to move forward along with the work on a >>>"pylab" super-package. >>> >>> >>That is exactly what I want. >> >> > >What is unsatisfactory about installing numpy+scipy+matplotlib? I've >found they're generally pretty complete (except where no decent python >alternative exists). > > > >>In the end I want a nice collection of functions, logically organized, that >>let me analyze/filter/plot etc. etc. etc. >> >>The key for me is "logically organized". >> >> > > > There is a structure to it, but it's more organic because of the multiple contributors. weave should be in NumPy but nobody was willing to step up to maintain it a year ago. I may be willing to step up at this point. I would like to see weave in NumPy (maybe not the blitz libraries though...) I think a hybrid for weave / f2py / ctypes that allows "inlining in multiple languages" as well as automatic extension module generation for "already-written" code is in order. -Travis From oliphant at ee.byu.edu Thu Oct 5 18:42:01 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 16:42:01 -0600 Subject: [Numpy-discussion] flat indexing of object arrays In-Reply-To: <1e2af89e0610051430t1ed6da8ayd6471313ccd6002d@mail.gmail.com> References: <200610052209.04341.martin.wiechert@gmx.de> <1e2af89e0610051430t1ed6da8ayd6471313ccd6002d@mail.gmail.com> Message-ID: <45258A39.6080201@ee.byu.edu> Matthew Brett wrote: >Hi, > >On 10/5/06, Martin Wiechert wrote: > > >>Hi list, >> >>when I try to assign a sequence as an element of an object array via flat >>indexing only the first element of the sequence is assigned: >> >> > >I've also been having trouble with flat on object arrays. > >Is this intended? > >In [1]: from numpy import * > >In [2]: a = arange(2) > >In [3]: a[1] >Out[3]: 1 > >In [4]: a.flat[1] >Out[4]: 1 > >In [5]: b = array([a], dtype=object) > >In [6]: b[1] >--------------------------------------------------------------------------- >exceptions.IndexError Traceback (most >recent call last) > >/home/mb312/devel_trees/scipy/Lib/io/ > >IndexError: index out of bounds > >In [7]: b.flat[1] >Out[7]: 1 > > > This is correct behavior. Look at the shape of b. It is being indexed correctly. The problem is that it is ambiguous as to what is wanted when you write b = array([a], dtype=object). We have gone through the rounds on this one and the current behavior is our best compromise. -Travis From peridot.faceted at gmail.com Thu Oct 5 19:02:03 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Thu, 5 Oct 2006 19:02:03 -0400 Subject: [Numpy-discussion] Hello and my first patch In-Reply-To: <452584EE.70106@ee.byu.edu> References: <903323ff0610042005x8f7841y9405d6af65a17d8c@mail.gmail.com> <452486F6.3060508@ee.byu.edu> <903323ff0610050702p404983eawd1b5a2b7bbe2989c@mail.gmail.com> <87bqoqen8t.fsf@peds-pc311.bsd.uchicago.edu> <452562D6.7010605@ee.byu.edu> <903323ff0610051304w7aedb22au8cadbc1160ff1bb9@mail.gmail.com> <452584EE.70106@ee.byu.edu> Message-ID: On 05/10/06, Travis Oliphant wrote: > I think a hybrid for weave / f2py / ctypes that allows "inlining in > multiple languages" as well as automatic extension module generation for > "already-written" code is in order. It might make sense to also include SWIG (since that seems to be a popular choice for wrapping "already-written" C and C++ code). A. M. Archibald From vjetly at i3.com Thu Oct 5 19:27:52 2006 From: vjetly at i3.com (Vikalpa Jetly) Date: Thu, 5 Oct 2006 17:27:52 -0600 Subject: [Numpy-discussion] Memory errors In-Reply-To: Message-ID: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> I am reading a very large array (~9000,11000) of 1 byte image values. I need to change values in the array that meet a certain condition so I am running something like: b = numpy.where(a>200,0,1) to create a new array with the changed values. However, I get a "MemoryError" everytime I try this. I have over 3gb of RAM on my machine (most of which is available). The process runs fine on smaller datasets. Is there a maximum array size that numpy handles? Any alternatives/workarounds? Thanks. Vikalpa -----Original Message----- From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy-discussion-bounces at lists.sourceforge.net] On Behalf Of A. M. Archibald Sent: Thursday, October 05, 2006 5:02 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Hello and my first patch On 05/10/06, Travis Oliphant wrote: > I think a hybrid for weave / f2py / ctypes that allows "inlining in > multiple languages" as well as automatic extension module generation for > "already-written" code is in order. It might make sense to also include SWIG (since that seems to be a popular choice for wrapping "already-written" C and C++ code). A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 5 19:44:05 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 05 Oct 2006 17:44:05 -0600 Subject: [Numpy-discussion] Memory errors In-Reply-To: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> References: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> Message-ID: <452598C5.3060004@ee.byu.edu> Vikalpa Jetly wrote: >I am reading a very large array (~9000,11000) of 1 byte image values. I need >to change values in the array that meet a certain condition so I am running >something like: > >b = numpy.where(a>200,0,1) > >to create a new array with the changed values. However, I get a >"MemoryError" everytime I try this. I have over 3gb of RAM on my machine >(most of which is available). The process runs fine on smaller datasets. Is >there a maximum array size that numpy handles? Any alternatives/workarounds? > > > The MemoryError is a direct result when system malloc fails. Rather than use where with two scalars (you're resulting array will be int32 and therefore 4-times larger). Use b = zeros_like(a) b[a>200] = 1 which will consume less memory. -Travis From robert.kern at gmail.com Thu Oct 5 19:46:15 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Oct 2006 16:46:15 -0700 Subject: [Numpy-discussion] Memory errors In-Reply-To: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> References: <002a01c6e8d5$e10a6230$1b28c80a@i3.local> Message-ID: If you are going to start a new topic, please start a new thread as well, instead of replying to a message. Thanks. Vikalpa Jetly wrote: > I am reading a very large array (~9000,11000) of 1 byte image values. I need > to change values in the array that meet a certain condition so I am running > something like: > > b = numpy.where(a>200,0,1) > > to create a new array with the changed values. However, I get a > "MemoryError" everytime I try this. I have over 3gb of RAM on my machine > (most of which is available). The process runs fine on smaller datasets. Is > there a maximum array size that numpy handles? Any alternatives/workarounds? There is no predefined limit on the array size, just your memory. However, note that what you are doing here is creating an int32 (or int64 if you are on a 64-bit machine) array since you are using Python integers in your where() function. You can save quite a bit of memory by using the array from (a>200) and simply casting it to the uint8 type. That way, you only have two arrays in memory at any time, a and b. b = (a > 200).astype(numpy.uint8) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From wbaxter at gmail.com Thu Oct 5 21:30:43 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 6 Oct 2006 10:30:43 +0900 Subject: [Numpy-discussion] repmat Message-ID: [There seem to have been some gmail delivery problems that prevented my previous mail on this subject from being delivered] I've proposed that we fix repmat handle arbitrary dimensions before 1.0. http://projects.scipy.org/scipy/numpy/ticket/292 I don't think this is particularly controversial, just I'm guessing no-one's found the time to look at my proposed fixes. And gmail/sourceforge issues haven't helped either. In short, repmat(A, M,N) is an oddball. It only deals with 2D arrays. We should do some combination of: 1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability to it, while maintaining backwards compatibility. 2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability to it, and bring it into line with the "numpy standard" of shapes being passed as one tuple arg. 3) add a new method like reparray(A, shape) that has the multidim ability, and matches numpy conventions. 4) if 3), move repmat out of the top-level namespace to indicate deprecation The tracker item has all the code necessary to implement these fixes, it's just a matter of deciding which one to go with. My personal preference would be a combination of 1 and 3. Add reparray(A,shape) , and change repmat to repmat(A, *shape). Then just implement repmat as a call to reparray. If someone will decide which route to go, I'd be happy to provide an actual patch against the latest SVN to implement that decision. If this is somehow controversial for some reason then let's discuss it. But so far the only response has been "copying data is a bad idea", which is really a separate issue. --bb From tim.hochberg at ieee.org Thu Oct 5 21:51:50 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 05 Oct 2006 18:51:50 -0700 Subject: [Numpy-discussion] repmat In-Reply-To: References: Message-ID: <4525B6B6.7050104@ieee.org> Bill Baxter wrote: > [There seem to have been some gmail delivery problems that prevented > my previous mail on this subject from being delivered] > > I've proposed that we fix repmat handle arbitrary dimensions before 1.0. > > http://projects.scipy.org/scipy/numpy/ticket/292 > > I don't think this is particularly controversial, just I'm guessing > no-one's found the time to look at my proposed fixes. And > gmail/sourceforge issues haven't helped either. > > In short, repmat(A, M,N) is an oddball. It only deals with 2D arrays. > > We should do some combination of: > 1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability > to it, while maintaining backwards compatibility. > Doesn't repmat imply matrix and thus 2d arrays? It is pretty horrible though. > 2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability > to it, and bring it into line with the "numpy standard" of shapes > being passed as one tuple arg. > I presume shape actually means number of repetitions, and not shape. If so, shape is the wrong name for the second argument. Anyway as long as your breaking backward compatibility you might as well improve the name. > 3) add a new method like reparray(A, shape) that has the multidim > ability, and matches numpy conventions. > Same comment on the name shape. > 4) if 3), move repmat out of the top-level namespace to indicate deprecation > I'm not completely sold on the whole reparray thing: does repmat have any use outside of the linear algebra convention of squashing a 3D array into a 2D matrix (or however that goes)? If not, perhaps it should just get left alone and exciled to matrix specific functions namespace. So: +1 on deprecating, moving or otherwise getting rid of repmat. Neutral on whether to replace it with something else, but if it is replaced #3 looks like the correct route, but not with a different argument name than 'shape'. Perhaps 'reps'. -tim > The tracker item has all the code necessary to implement these fixes, > it's just a matter of deciding which one to go with. > > My personal preference would be a combination of 1 and 3. Add > reparray(A,shape) , and change repmat to repmat(A, *shape). Then just > implement repmat as a call to reparray. > > If someone will decide which route to go, I'd be happy to provide an > actual patch against the latest SVN to implement that decision. > > If this is somehow controversial for some reason then let's discuss > it. But so far the only response has been "copying data is a bad > idea", which is really a separate issue. > > --bb > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > From wbaxter at gmail.com Thu Oct 5 22:18:22 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 6 Oct 2006 11:18:22 +0900 Subject: [Numpy-discussion] repmat In-Reply-To: <4525B6B6.7050104@ieee.org> References: <4525B6B6.7050104@ieee.org> Message-ID: On 10/6/06, Tim Hochberg wrote: > Bill Baxter wrote: > > In short, repmat(A, M,N) is an oddball. It only deals with 2D arrays. > > > > We should do some combination of: > > 1) change repmat(A, M,N) to repmat(A, *dims) to add multidim ability > > to it, while maintaining backwards compatibility. > > > Doesn't repmat imply matrix and thus 2d arrays? It is pretty horrible > though. It does kind of imply that (which is why I think something like 'reparray' would be a better name) but even matlab's repmat() function doesn't restrict you to 2D arrays, and I'm pretty sure numpy.repmat was intended to be a clone of that. > > 2) change repmat(A, M,N) to repmat(A, shape) to add multidim ability > > to it, and bring it into line with the "numpy standard" of shapes > > being passed as one tuple arg. > > > I presume shape actually means number of repetitions, and not shape. If > so, shape is the wrong name for the second argument. Anyway as long as > your breaking backward compatibility you might as well improve the name. Yeh, that did occur to me. Maybe 'reps' would be a better name for the argument. > > 3) add a new method like reparray(A, shape) that has the multidim > > ability, and matches numpy conventions. > > > Same comment on the name shape. > > > > 4) if 3), move repmat out of the top-level namespace to indicate deprecation > > > I'm not completely sold on the whole reparray thing: does repmat have > any use outside of the linear algebra convention of squashing a 3D array > into a 2D matrix (or however that goes)? If not, perhaps it should just > get left alone and exciled to matrix specific functions namespace. So: Well, I'm not sure about N-d, but I've definitely had a need for 1-d version of it. That's what prompted me to look into this. Also I think I read a comment in the other thread about pyEM that repmat() turned out to be one of the faster ways to implement something-or-other (?). At the very least you could think of tiling 3D image data as one application, in fact 'tile()' is probably a better name for the function than 'reparray'. I was initially suprised that 'repeat()' featured so prominently in numpy. I initially assumed it meant "repmat", and was surprised when it didn't do that. I thought "why on earth would you want to do that?". But it does turn out to have lots of uses (like implementing "repmat" ;-)) > +1 on deprecating, moving or otherwise getting rid of repmat. > Neutral on whether to replace it with something else, but if it is > replaced #3 looks like the correct route, but not with a different > argument name than 'shape'. Perhaps 'reps'. Hey! Looks like we agree on the name of the argument, apparently. :-) I do think there is a strong connection with "shape" though, that 'reps' hides. Maybe 'repshape' or 'tileshape'? I'd be happy with reps, though. Agreed that 'shape' is bad. Makes it sound like you should specify the actual .shape of the output. --bb From fperez.net at gmail.com Thu Oct 5 22:26:54 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 5 Oct 2006 20:26:54 -0600 Subject: [Numpy-discussion] [OT] Interesting interview with Pearu (f2py and others...) Message-ID: Hi all, since this is not about me, I think it's fair to plug an interview :) http://www.pythonthreads.com/articles/interviews/once-i-learned-about-python,-i-stopped-trying-out-different-languages.html contains a very nice interview with Pearu, where in fact I learned a number of interesting things not only about him but also about what's happening with f2py. Good job, Pearu :) Best, f From pfdubois at gmail.com Thu Oct 5 23:00:55 2006 From: pfdubois at gmail.com (Paul Dubois) Date: Thu, 5 Oct 2006 20:00:55 -0700 Subject: [Numpy-discussion] contribution about history of Numeric Message-ID: I was reading the 'History of SciPy' page and noticed the discussion about Numeric. Here's the true story about why the various names for the original: numpy, Numeric, Numerical. At the time Source Forge was pretty young, and I decided to put the project there. We all said 'numpy' informally not Numerical Python but the module name was Numeric. I created the project as numpy. I have no memory of why I didn't call it Numeric, but if it wasn't a conflict, probably I was focused on making it clear it was for Python and/or easy to type. (the FTP's etc. all had to go through a long directory path that involved the name). The documentation for the CVS stuff was confusing, and I made a mistake with my first submit of 'Numeric' (I think it was ending up with everything in Numeric/Numeric) and then discovered I could not delete it; you had to ask the Source Forge staff. Impatient, I did a second submit as Numerical. In short, all my fault, but then again, SF was so security-minded that it was hard to do anything. That's why I soon gave up on their website and hosted it at my own site for so long. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Oct 6 00:08:28 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 5 Oct 2006 22:08:28 -0600 Subject: [Numpy-discussion] contribution about history of Numeric In-Reply-To: References: Message-ID: On 10/5/06, Paul Dubois wrote: > I was reading the 'History of SciPy' page and noticed the discussion about > Numeric. Here's the true story about why the various names for the original: > numpy, Numeric, Numerical. I added this to the wiki page (http://www.scipy.org/History_of_SciPy), though I don't know how to make a block quote in wiki markup, so I left it without any visual clue. Thanks for the info! Best, f From david at ar.media.kyoto-u.ac.jp Fri Oct 6 02:04:44 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 06 Oct 2006 15:04:44 +0900 Subject: [Numpy-discussion] repmat In-Reply-To: References: <4525B6B6.7050104@ieee.org> Message-ID: <4525F1FC.8070908@ar.media.kyoto-u.ac.jp> Bill Baxter wrote: > >> I'm not completely sold on the whole reparray thing: does repmat have >> any use outside of the linear algebra convention of squashing a 3D array >> into a 2D matrix (or however that goes)? If not, perhaps it should just >> get left alone and exciled to matrix specific functions namespace. So: >> > > Well, I'm not sure about N-d, but I've definitely had a need for 1-d > version of it. That's what prompted me to look into this. Also I > think I read a comment in the other thread about pyEM that repmat() > turned out to be one of the faster ways to implement > something-or-other (?). It was in a very specific case, and it could have been replaced by an other function in this case without any loss of performance. The following is my experience with matlab/repmat: generally, repmat was necessary in many cases in matlab either for efficiency (big scalar matrix were slow to generate using ones and zeros, for example) and because matlab had almost no broadcasting rules, repmat was almost mandatory for matlab. In numpy, the syntax at least does not require repmat (at least, I never *had* to use it in numpy, and I used it a lot with matlab). Moreoever, the slowness of some broadcasting seem solved in recent subversion (see my answer on the specific thread), David > At the very least you could think of tiling > 3D image data as one application, in fact 'tile()' is probably a > better name for the function than 'reparray'. > > I was initially suprised that 'repeat()' featured so prominently in > numpy. I initially assumed it meant "repmat", and was surprised when > it didn't do that. I thought "why on earth would you want to do > that?". But it does turn out to have lots of uses (like implementing > "repmat" ;-)) > > >> +1 on deprecating, moving or otherwise getting rid of repmat. >> Neutral on whether to replace it with something else, but if it is >> replaced #3 looks like the correct route, but not with a different >> argument name than 'shape'. Perhaps 'reps'. >> > > Hey! Looks like we agree on the name of the argument, apparently. :-) > I do think there is a strong connection with "shape" though, that > 'reps' hides. Maybe 'repshape' or 'tileshape'? I'd be happy with > reps, though. Agreed that 'shape' is bad. Makes it sound like you > should specify the actual .shape of the output. > > --bb > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > From david at ar.media.kyoto-u.ac.jp Fri Oct 6 02:33:11 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 06 Oct 2006 15:33:11 +0900 Subject: [Numpy-discussion] On loop and broadcasting (again) In-Reply-To: <45250ADA.105@ee.byu.edu> References: <45250A9D.5040403@ar.media.kyoto-u.ac.jp> <45250ADA.105@ee.byu.edu> Message-ID: <4525F8A7.6030306@ar.media.kyoto-u.ac.jp> Travis Oliphant wrote: > David Cournapeau wrote: > >> Hi, >> >> The email from Albert made me look again on some surprising results I >> got a few months ago when starting my first "serious" numpy project. I >> noticed that when computing multivariate gaussian densities, centering >> the data was more expensive than everything else, including >> exponentiation. Now that I have some experience with numpy, and >> following the previous discussion, I tried the following script: >> >> > > Try it again with the new code in SVN. > It looks like your modification solved this particular issue !: 10 0.645 0.065 0.645 0.065 storage_email.py:8(center_broadcast) 10 0.625 0.062 0.625 0.062 storage_email.py:26(center_manual) 1 0.333 0.333 0.333 0.333 /usr/lib/python2.4/site-packages/numpy/lib/shape_base.py:530(repmat) Now, using broadcast is as fast as doing the substraction itself (which does not include the necessary repmat). I tried it on my laptop, where I can safely use beta code (python 2.4 + SVN numpy), which explains the timing differences compared to my previous email; as the memory is limited on the laptop, I only benchmarked the brodcasting. For smaller arrays, I couldn't see major differences in relative timing for the other implementations. Thank you very much ! David From stefan at sun.ac.za Fri Oct 6 04:46:01 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 6 Oct 2006 10:46:01 +0200 Subject: [Numpy-discussion] repmat In-Reply-To: References: Message-ID: <20061006084601.GJ20486@mentat.za.net> On Fri, Oct 06, 2006 at 10:30:43AM +0900, Bill Baxter wrote: > If this is somehow controversial for some reason then let's discuss > it. But so far the only response has been "copying data is a bad > idea", which is really a separate issue. An interesting issue, though. I've often wondered about an array view where data is visible in more than one place. For example, you have an array x = N.arange((100)) on which you'd like to apply a filter, which, say, finds the moving variance of the 5 closest elements. Here, a new representation of x would be useful: In [23]: indices = N.arange(5) + N.arange(96).reshape((96,1)) In [24]: indices Out[24]: array([[ 0, 1, 2, 3, 4], [ 1, 2, 3, 4, 5], ... [95, 96, 97, 98, 99]]) In [25]: y = x.index_view(indices) # returns array with same values as # y = x[indices] # except that no values are copied In [29]: y Out[29]: array([[ 0, 1, 2, 3, 4], [ 1, 2, 3, 4, 5], [ 2, 3, 4, 5, 6], ... after which you would be able to do In [30]: y.var(axis=1) Out[30]: array([ 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., While there would be a lot of jumping around in indices, at the loops can be done in C, which should speed up the process. Of course, with small arrays I can simply copy the data, but it becomes troublesome with images and other such large signals. I am not sure if providing the indices is the best way to initialise such an array (otherwise maybe a rule for index calculation?) or how feasible it would be to implement, so I'd be glad to hear anyone's thoughts on the topic. Regards St?fan From Mailer-Daemon at lists.sourceforge.net Fri Oct 6 04:47:03 2006 From: Mailer-Daemon at lists.sourceforge.net (Mail Delivery System) Date: Fri, 06 Oct 2006 01:47:03 -0700 Subject: [Numpy-discussion] Mail delivery failed: returning message to sender Message-ID: This message was created automatically by mail delivery software. A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed: numpy-discussion at lists.sourceforge.net SMTP error from remote mailer after RCPT TO:: host mail.sourceforge.net [66.35.250.206]: 451-Could not complete sender verify callout 451-Could not complete sender verify callout for 451-. 451-The mail server(s) for the domain may be temporarily unreachable, or 451-they may be permanently unreachable from this server. In the latter case, 451-you need to change the address or create an MX record for its domain 451-if it is : retry timeout exceeded ------ This is a copy of the message, including all the headers. ------ From knfpts at allwrite.be Mon Oct 2 07:32:13 2006 From: knfpts at allwrite.be (Daniel Davis) Date: Mon, 02 Oct 2006 09:32:13 -0200 Subject: doctor, at the revolt is located treating the planet he gold; Message-ID: <560583873365.2006100209321346524648@allwrite.be> HOT STOCK ALERT - THIS ONE IS STILL CLIMBING THE STOCK CHARTS ALERT -- BREAKING MARKET NEWS REPORT ---- PGCN.OB Company: Pingchuan Pharmaceutical Inc. Ticker: PGCN.OB Current Price: $.26 5-Day Target: $1.5 - $2 Rating: 10(10) Recommendation: S T R 0 N G - B U Y !! This quick rising stock is a good long term winner. This stock is going high due to superb business solutions and creative partnerships in the business world. Could this company be the next PFIZER?? Pingchaun Pharmaceuticals disributes its medications to over 50 percent of CHINA and has a tremendous presence via exports to U.S.A, Russia, Japan and South-Eastern Asia. They have partnerships with the biggest drugstores, equivalent in the US to SAV-ON, CVS, RITE-AID, etc. With BIG NEWS anticipated this coming WEEK we strongly advise you to get in EARLY and issue a 10/10 Alert. About Pingchuan Pharmaceutical Inc. Pingchuan Pharmaceutical Inc. ("PINGCHUAN") is a modernized pharmaceutical manufacturer with first-class medical R&D ability, pioneered medicine products, and well-established marketing network. Since its establishment, PINGCHUAN has focused its businesses on diabetes medicine and its medical products. The products of PINGCHUAN include health care products, varieties of medicine, as well as medical apparatus. Kang Da Glycosuria Capsule is a successful product of PINGCHUAN for treating diabetes, which was first developed in 1998 and introduced to the market in 2001. PINGCHUAN has well-established marketing network and sale branches, which include about 180,000 retail pharmacies /drugstores across China, agency network in major cities like Beijing, Shanghai, Guangzhou, and Xi'an, special counter sale at drug chain stores in key regions, residence community clinic sale and promotion, as well as internet marketing through the company's website. The marketing network of PINGCHUAN covers more than 50 percent districts of China and exports to US, Japan, Russia, and south-eastern Asia. Dont miss the boat, this is a new issue, is thinly traded and could move up quickly. We anticipate that shares of PGCN will be much higher in the short-term. ACT NOW! Remember the gains from our recent recommendations... Do not miss out, Put PGCN.OB on your radar's now. Good luck and trade out at the top!! From Jonathan.Peirce at nottingham.ac.uk Fri Oct 6 06:55:18 2006 From: Jonathan.Peirce at nottingham.ac.uk (Jon Peirce) Date: Fri, 06 Oct 2006 11:55:18 +0100 Subject: [Numpy-discussion] super-packages [was "Hello and my first patch"] In-Reply-To: References: Message-ID: <45263616.7090809@psychology.nottingham.ac.uk> > Perhaps that is the best way to move forward along with the work on a > "pylab" super-package. A few years ago it seemed that scipy was developing to BE the super-package. It was to have plotting and some numpy flavour and all sorts of goodies for scientists. The name still implies that. People are now talking about a super-package that includes scipy as one of three packages. Definitely it seems that pylab is a bad name since it's already used by matplotlib and the last thing we want is another confusing name convention that becomes hard to google. But why not scipy? It seems the right name for a super-package. Jon This message has been checked for viruses but the contents of an attachment may still contain software viruses, which could damage your computer system: you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation. From vjetly at i3.com Fri Oct 6 09:30:51 2006 From: vjetly at i3.com (Vikalpa Jetly) Date: Fri, 6 Oct 2006 07:30:51 -0600 Subject: [Numpy-discussion] Memory errors In-Reply-To: <452598C5.3060004@ee.byu.edu> Message-ID: <003401c6e94b$a4a23270$1b28c80a@i3.local> Thanks Travis and Robert. I am just getting my feet wet in numpy. Both approaches i.e: b = zeros_like(a) b[a>200] = 1 or b = (a > 200).astype(numpy.uint8) avoid the memory error. Related question is that I need to test for multiple conditions on the same array and set values to 1 or 0. I see that statements like b[a>200 or a<50] = 1 do not work. So is the way to do this simply to run a separate statement in the form b[condition]= 1 for each test? Also since my output has to be a binary array, can the new array be defined as binary type or nibble, potentially reducing memory overhead? Thanks. Vikalpa -----Original Message----- From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy-discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant Sent: Thursday, October 05, 2006 5:44 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Memory errors > The MemoryError is a direct result when system malloc fails. Rather than use where with two scalars (you're resulting array will be int32 and therefore 4-times larger). Use b = zeros_like(a) b[a>200] = 1 which will consume less memory. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From faltet at carabos.com Fri Oct 6 09:46:43 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 6 Oct 2006 15:46:43 +0200 Subject: [Numpy-discussion] Memory errors In-Reply-To: <003401c6e94b$a4a23270$1b28c80a@i3.local> References: <003401c6e94b$a4a23270$1b28c80a@i3.local> Message-ID: <200610061546.44463.faltet@carabos.com> A Divendres 06 Octubre 2006 15:30, Vikalpa Jetly va escriure: > avoid the memory error. Related question is that I need to test for > multiple conditions on the same array and set values to 1 or 0. I see that > statements like b[a>200 or a<50] = 1 do not work. So is the way to do this > simply to run a separate statement in the form b[condition]= 1 for each > test? For this, you have to use the binary operators instead of logical ones. For example: In [43]: a=numpy.arange(10) In [44]: b=numpy.arange(10) In [45]: b[(a < 2) | (a > 8)] = 1 In [46]: b Out[46]: array([1, 1, 2, 3, 4, 5, 6, 7, 8, 1]) [be sure to use parentesizes appropriately] Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From emsellem at obs.univ-lyon1.fr Fri Oct 6 10:31:04 2006 From: emsellem at obs.univ-lyon1.fr (Eric Emsellem) Date: Fri, 06 Oct 2006 16:31:04 +0200 Subject: [Numpy-discussion] ide for python/numpy/scipy/mpl development ? Message-ID: <452668A8.7000904@obs.univ-lyon1.fr> Hi, I am looking for an IDE to develop python programs and I am not sure what to take. The two critical items for me are 1/ a good debugger (simple and efficient) 2/ something simple to manage the files. I would also very much like to keep some basic things such as (if possible): - editing with gvim - running things with Ipython -pylab So the question is: does such an IDE exists? I have looked through a number of them, but they look either very complex or too simple. Any advice here is very welcome! thanks, and cheers Eric From robert.kern at gmail.com Fri Oct 6 10:41:29 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 06 Oct 2006 07:41:29 -0700 Subject: [Numpy-discussion] ide for python/numpy/scipy/mpl development ? In-Reply-To: <452668A8.7000904@obs.univ-lyon1.fr> References: <452668A8.7000904@obs.univ-lyon1.fr> Message-ID: Eric Emsellem wrote: > Hi, > > I am looking for an IDE to develop python programs and I am not sure > what to take. > The two critical items for me are 1/ a good debugger (simple and > efficient) 2/ something simple to manage the files. > > I would also very much like to keep some basic things such as (if possible): > - editing with gvim This probably is the most limiting factor. I use pida because it embeds gvim into a PyGTK frame with all of the IDE goodies around it. http://pida.berlios.de/ I believe it can use one of the PyGTK debugger GUIs, but I've never used it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From gnurser at googlemail.com Fri Oct 6 10:53:40 2006 From: gnurser at googlemail.com (George Nurser) Date: Fri, 6 Oct 2006 15:53:40 +0100 Subject: [Numpy-discussion] [OT] Interesting interview with Pearu (f2py and others...) In-Reply-To: References: Message-ID: <1d1e6ea70610060753j6c954232xf9d4d63a4cc01231@mail.gmail.com> On 06/10/06, Fernando Perez wrote: > Hi all, > > since this is not about me, I think it's fair to plug an interview :) > > http://www.pythonthreads.com/articles/interviews/once-i-learned-about-python,-i-stopped-trying-out-different-languages.html > > contains a very nice interview with Pearu, where in fact I learned a > number of interesting things not only about him but also about what's > happening with f2py. > > Good job, Pearu :) Great to hear that f2py is being developed so actively. Feature suggestion: Fortran 90 has interface blocks that allow the calling routine to pick up whichever of a set of routines has the appropriate datatype. So one has foo_real32,foo_real64 etc held within the interface block, but calling foo will automatically choose the correct routine. I don't know whether it's possible to make this work with f2py, but it would be very useful if it could be. --George Nurser. From peridot.faceted at gmail.com Fri Oct 6 11:01:28 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 6 Oct 2006 11:01:28 -0400 Subject: [Numpy-discussion] super-packages [was "Hello and my first patch"] In-Reply-To: <45263616.7090809@psychology.nottingham.ac.uk> References: <45263616.7090809@psychology.nottingham.ac.uk> Message-ID: On 06/10/06, Jon Peirce wrote: > > Perhaps that is the best way to move forward along with the work on a > > "pylab" super-package. > > [...] > > But why not scipy? It seems the right name for a super-package. It does, particularly if (as is being discussed) the current content of scipy might get subdivided into separately buildable modules (weave, fortran, non-fortran, say). scipy would then refer to the lot - numpy, matplotlib, weave, scipy-fortran, scipy-nonfortran, and any others that seem essential. Is setting out to create a "MATLAB replacement" a good goal? I think so. Really the goal should be to create a useful tool for scientists, but I think MATLAB is clearly already that, so (at first) it can serve as a definite objective. Of course, anyone who wants to make scipy *better* than MATLAB should be warmly encouraged... A. M. Archibald From faltet at carabos.com Fri Oct 6 13:08:20 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 6 Oct 2006 19:08:20 +0200 Subject: [Numpy-discussion] PyTables passes all tests with NumPy at its core Message-ID: <200610061908.21964.faltet@carabos.com> Hi, I'm very happy to announce that, after several weeks of effort, PyTables is able to run its complete test suite (made of more than 3000 test units) using *NumPy* at its core (now numarray is just an optional package :-). However, and before the new PyTables will hit the public (most probably released as PyTables 2.0), we should still find more time to add more tests (specially for numarray, which we plan to continue supporting), update manual, and making sure that the new package deploys well on every supported plaform. So there is a lot of work to be done yet, but things are in good shape. I have to say that although NumPy was an already well tested package for homogeneous data containers, PyTables has strong requirements for heterogeneous arrays as well, and most of my efforts lately have been to locate the flaws in that area and reporting them properly to the NumPy issue tracker. Fortunately, at the NumPy end there was a formidable team (and in particular, a formidable Travis Oliphant) ready for eat all bugs that I was able to provide. As a result, I'd say that NumPy is quite mature for dealing with arrays of records as well, which is great news :-) Cheers! -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From stefan at sun.ac.za Fri Oct 6 13:33:40 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 6 Oct 2006 19:33:40 +0200 Subject: [Numpy-discussion] "Manually" broadcasting arrays in Python In-Reply-To: References: Message-ID: <20061006173340.GA1929@mentat.za.net> On Wed, Oct 04, 2006 at 01:37:55AM -0400, A. M. Archibald wrote: > Would it be useful for me to contribute the tiny script I wrote to > trigger it as a regression test? > > A. M. Archibald > > from numpy import vectorize, zeros > > vt = vectorize(lambda *args: args) > # Removing either of the following lines cures the segfault > vt(zeros((1,2,1)), zeros((2,1,1)), zeros((1,1,2))) > vt(zeros((1,2,1)), zeros((2,1,1)), zeros((1,1,2)), zeros((2,2))) I also get a segfault when running this, but the strange thing is that I can't catch it with valgrind (which also segfaults). I've filed a ticket at http://projects.scipy.org/scipy/numpy/ticket/325 Regards St?fan From Chris.Barker at noaa.gov Fri Oct 6 14:08:25 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Fri, 06 Oct 2006 11:08:25 -0700 Subject: super-packages [was "Hello and my first patch"] In-Reply-To: References: <45263616.7090809@psychology.nottingham.ac.uk> Message-ID: <45269B99.2000104@noaa.gov> A. M. Archibald wrote: > Is setting out to create a "MATLAB replacement" a good goal? No. > the goal should be to create a useful tool for scientists. Yes. > but I think MATLAB is clearly already that, so (at first) it can serve > as a definite objective. Of course, anyone who wants to make scipy > *better* than MATLAB should be warmly encouraged... It already *is* better! I say this as a former, committed MATLAB user. I personally think Python is a much better foundation language, and numpy a better array implementation. MATLAB two advantages (and they are big ones!): integrated plotting and a large and broad library of ready-to-use functions for a wide variety of computing. MPL has almost solved one, and SciPy is getting better an better -- so we're really on the right track! -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From peridot.faceted at gmail.com Fri Oct 6 16:35:34 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 6 Oct 2006 16:35:34 -0400 Subject: super-packages [was "Hello and my first patch"] In-Reply-To: <45269B99.2000104@noaa.gov> References: <45263616.7090809@psychology.nottingham.ac.uk> <45269B99.2000104@noaa.gov> Message-ID: On 06/10/06, Christopher Barker wrote: > A. M. Archibald wrote: > > Is setting out to create a "MATLAB replacement" a good goal? > > No. I see what you mean, but I still think that using MATLAB as a yardstick ("Can users do everything with the python superpackage that they could with MATLAB?" "Is it of comparable or lesser difficulty?") is a good idea, at least until we've clearly outstripped it in all categories. > MATLAB two advantages (and they are big ones!): integrated plotting and > a large and broad library of ready-to-use functions for a wide variety > of computing. This is what I mean about using it as a yardstick. As a MATLAB survivor, would you be interested in going to http://www.scipy.org/NumPy_for_Matlab_Users and adding numerical tools that MATLAB has, so that we can use it as a checklist for what tools we'd like to have (first)? I put a short list there, but it's years since I last used MATLAB. > MPL has almost solved one, and SciPy is getting better an better -- so > we're really on the right track! I agree. A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant at ee.byu.edu Fri Oct 6 21:52:55 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 06 Oct 2006 19:52:55 -0600 Subject: repmat In-Reply-To: References: Message-ID: <45270877.7030903@ee.byu.edu> Bill Baxter wrote: >[There seem to have been some gmail delivery problems that prevented >my previous mail on this subject from being delivered] > >I've proposed that we fix repmat handle arbitrary dimensions before 1.0. > > http://projects.scipy.org/scipy/numpy/ticket/292 > >I don't think this is particularly controversial, just I'm guessing >no-one's found the time to look at my proposed fixes. And >gmail/sourceforge issues haven't helped either. > > Thanks for reminding us again. I don't think this is too bad of a deal. I propose to move repmat(A, M,N) to matlib.py and replace it in numpy with a function named tile(a, reps) That is more general-purpose. If this is not acceptable, please speak up. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Sat Oct 7 00:16:26 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 6 Oct 2006 22:16:26 -0600 Subject: ide for python/numpy/scipy/mpl development ? In-Reply-To: References: <452668A8.7000904@obs.univ-lyon1.fr> Message-ID: On 10/6/06, Robert Kern wrote: > > Eric Emsellem wrote: > > Hi, > > > > I am looking for an IDE to develop python programs and I am not sure > > what to take. > > The two critical items for me are 1/ a good debugger (simple and > > efficient) 2/ something simple to manage the files. > > > > I would also very much like to keep some basic things such as (if > possible): > > - editing with gvim > > This probably is the most limiting factor. I use pida because it embeds > gvim > into a PyGTK frame with all of the IDE goodies around it. > > http://pida.berlios.de/ > > I believe it can use one of the PyGTK debugger GUIs, but I've never used > it. Pida looks nice, thanks for the pointer. Unfortunately, the mercurial support needs editing to make it work at a basic level. To make pida really work right for distributed repositories looks trickier because distributed version control doesn't necessarily have a central repository while the current pida interface looks to be based around that concept. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From boae at kobtv.com Sat Oct 7 03:26:29 2006 From: boae at kobtv.com (Calvagh Moldenhauer) Date: Sat, 7 Oct 2006 00:26:29 -0700 Subject: Hi Message-ID: <000001c6e9e1$e83293e0$88faa8c0@dzkyk> Hi, VrAGRA for less http://www.ruishinimdesin.com _____ time-what had it done for my personal poisonous deadline! I groped for He was gray-haired, stern-faced, dressed in the same combat fatigues ask the right question. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From karol.langner at kn.pl Sat Oct 7 08:06:57 2006 From: karol.langner at kn.pl (Karol Langner) Date: Sat, 7 Oct 2006 14:06:57 +0200 Subject: problem with lapack_lite / BLAS Message-ID: <200610071406.57647.karol.langner@kn.pl> Dear list, I'm trying to compile ATLAS 3.6.0 and LAPACK 3.0 (+ update) and use them with the latest numpy (checked out today). Installation seems to be fine, all the libraries are detected. When I import numpy, however, I get this: Python 2.5 (r25:51908, Sep 27 2006, 17:40:29) [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "", line 1, in File "/home/langner/apps/python/lib/python2.5/site-packages/numpy/__init__.py", line 40, in import linalg File "/home/langner/apps/python/lib/python2.5/site-packages/numpy/linalg/__init__.py", line 4, in from linalg import * File "/home/langner/apps/python/lib/python2.5/site-packages/numpy/linalg/linalg.py", line 25, in from numpy.linalg import lapack_lite ImportError: /home/langner/apps/python/lib/python2.5/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: xerbla_ Does anyone know where this comes from? Cheers, Karol -- written by Karol Langner Sat Oct 7 14:02:46 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From matthew.brett at gmail.com Sat Oct 7 08:15:36 2006 From: matthew.brett at gmail.com (Matthew Brett) Date: Sat, 7 Oct 2006 13:15:36 +0100 Subject: Numpy types and object comparison Message-ID: <1e2af89e0610070515v10fec160saa352335c86c53f6@mail.gmail.com> Hi, I am sorry, I am still struggling with object arrays - and here - numpy data types. Is there any easy way to explain why comparisons with numpy types give a different output from comparisons with non-numpy types: In [108]:oa = zeros((1,), dtype=object) In [109]:oa == int Out[109]:array([False], dtype=bool) In [110]:oa == int32 Out[110]:False Many thanks for your patience, Matthew ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From fullung at gmail.com Sat Oct 7 08:20:06 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sat, 7 Oct 2006 14:20:06 +0200 Subject: problem with lapack_lite / BLAS In-Reply-To: <200610071406.57647.karol.langner@kn.pl> References: <200610071406.57647.karol.langner@kn.pl> Message-ID: Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Karol Langner > Sent: 07 October 2006 14:07 > To: NumPy List > Subject: [Numpy-discussion] problem with lapack_lite / BLAS > > Dear list, > > I'm trying to compile ATLAS 3.6.0 and LAPACK 3.0 (+ update) and use them > with > the latest numpy (checked out today). Installation seems to be fine, all > the > libraries are detected. When I import numpy, however, I get this: > > > ImportError: /home/langner/apps/python/lib/python2.5/site- > packages/numpy/linalg/lapack_lite.so: > undefined symbol: xerbla_ > > Does anyone know where this comes from? xerbla is an error handler for the LAPACK routines. Make sure you included xerbla.f when building your LAPACK library. Check the symbols included in the library with strings or objdump. Hope this helps. Cheers, Albert ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From yuki at yahoo.co.jp Sat Oct 7 08:41:59 2006 From: yuki at yahoo.co.jp (=?iso-2022-jp?B?eXVraQ==?=) Date: Sat, 7 Oct 2006 05:41:59 -0700 (PDT) Subject: (no subject) Message-ID: :?? INFORMATION ?????????????????????????: ?????????????????????? ???????????? http://love-match.bz/pc/01 :??????????????????????????????????: *????*:.?. .?.:*????*:.?..?:*????*:.?..?:**????* ??????????????????????????????????? ??? ???????????????????Love?Match? ?----------------------------------------------------------------- ??????????????????????? ??????????????????????? ??????????????????????? ??????????????????????? ??????????????????????? ??????????????????????? ?----------------------------------------------------------------- ????????????????http://love-match.bz/pc/01 ??????????????????????????????????? ??? ?????????????????????? ?----------------------------------------------------------------- ???????????????????????????? ?----------------------------------------------------------------- ????????????????????????????? ??????????????????????????????? ?http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????????????????????????????? ?----------------------------------------------------------------- ???????????????????????????????? ????????????????????? ?http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????????????????? ?----------------------------------------------------------------- ???????????????????????? ?????????????????????????????????? ?http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????????????????????????? ?----------------------------------------------------------------- ??????????????????????????? ????????????????????????????????? ?http://love-match.bz/pc/01 ?----------------------------------------------------------------- ????????????????????????? ?----------------------------------------------------------------- ????????????????????????? ????????????????????????????????? ?http://love-match.bz/pc/01 ??????????????????????????????????? ??? ??500???????????????? ?----------------------------------------------------------------- ???????/???? ???????????????????? ????????????????????????????????? ???????????????????????????????? ?????????????????????????? ?????????????????????????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????/?????? ?????????????????????????????????? ??????????????????????????????????? ?????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????/????? ?????????????????????????????????? ???????????????????????????????? ?????????????????????????(^^) ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????/???? ??????????????????????????????? ?????????????????????????????? ?????????????????????????????? ???????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ????????/??? ???????????????1??? ????????????????????????? ????????????????????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????/??????? ????18?????????????????????????? ????????????????????????????? ????????????????????????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???`????/??? ????????????????????? ?????????????????????? ?????????????? ?[????] http://love-match.bz/pc/01 ?----------------------------------------------------------------- ???????????????????? ?????????????????????????????????? ????????????? ??------------------------------------------------------------- ???????????????????????????????? ??[??????????]?http://love-match.jp/pc/?1 ??------------------------------------------------------------- ????????????????????? ??????????????????????????? ??????????????????? ??????????????????????????????? ??[??????????]?http://love-match.bz/pc/01 ?????????????????????????????????? ??????????3-6-4-533 ?????? 139-3668-7892 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From karol.langner at kn.pl Sat Oct 7 09:00:08 2006 From: karol.langner at kn.pl (Karol Langner) Date: Sat, 7 Oct 2006 15:00:08 +0200 Subject: problem with lapack_lite / BLAS In-Reply-To: References: Message-ID: <200610071500.08199.karol.langner@kn.pl> On Saturday 07 of October 2006 14:20, Albert Strasheim wrote: > Hello all > > > -----Original Message----- > > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > > discussion-bounces at lists.sourceforge.net] On Behalf Of Karol Langner > > Sent: 07 October 2006 14:07 > > To: NumPy List > > Subject: [Numpy-discussion] problem with lapack_lite / BLAS > > > > Dear list, > > > > I'm trying to compile ATLAS 3.6.0 and LAPACK 3.0 (+ update) and use them > > with > > the latest numpy (checked out today). Installation seems to be fine, all > > the > > libraries are detected. When I import numpy, however, I get this: > > > > > > ImportError: /home/langner/apps/python/lib/python2.5/site- > > packages/numpy/linalg/lapack_lite.so: > > undefined symbol: xerbla_ > > > > Does anyone know where this comes from? > > xerbla is an error handler for the LAPACK routines. Make sure you included > xerbla.f when building your LAPACK library. Check the symbols included in > the library with strings or objdump. > > Hope this helps. > > Cheers, > > Albert Yes, I know, and I checked this, and xerbla.o is in my lapack library, ad provides "xerbla_": langner@~/apps/linalg/lib: objdump -x liblapack.a | grep "xerbla.o" -A 100 xerbla.o: file format elf32-i386 rw-r--r-- 1000/1000 1380 Oct 6 17:51 2006 xerbla.o architecture: i386, flags 0x00000011: HAS_RELOC, HAS_SYMS start address 0x00000000 Sections: Idx Name Size VMA LMA File off Algn 0 .text 00000061 00000000 00000000 00000040 2**4 CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE 1 .data 0000006a 00000000 00000000 000000c0 2**5 CONTENTS, ALLOC, LOAD, RELOC, DATA 2 .bss 00000000 00000000 00000000 0000012c 2**2 ALLOC 3 .rodata 00000000 00000000 00000000 0000012c 2**0 CONTENTS, ALLOC, LOAD, READONLY, DATA 4 .rodata.cst4 00000004 00000000 00000000 0000012c 2**2 CONTENTS, ALLOC, LOAD, READONLY, DATA 5 .note.GNU-stack 00000000 00000000 00000000 00000130 2**0 CONTENTS, READONLY 6 .comment 00000026 00000000 00000000 00000130 2**0 CONTENTS, READONLY SYMBOL TABLE: 00000000 l df *ABS* 00000000 xerbla.f 00000000 l d .text 00000000 00000000 l d .data 00000000 00000000 l d .bss 00000000 00000000 l O .data 00000014 __g77_cilist_0.1 00000020 l O .data 0000004a __g77_format_9999.0 00000000 l d .rodata 00000000 00000000 l d .rodata.cst4 00000000 00000000 l d .note.GNU-stack 00000000 00000000 l d .comment 00000000 00000000 g F .text 00000061 xerbla_ 00000000 *UND* 00000000 s_wsfe 00000000 *UND* 00000000 do_fio 00000000 *UND* 00000000 e_wsfe 00000000 *UND* 00000000 s_stop RELOCATION RECORDS FOR [.text]: OFFSET TYPE VALUE (...) And this _is_ the library I point to in site.cfg. Karol -- written by Karol Langner Sat Oct 7 14:53:15 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From gsmith at alumni.uwaterloo.ca Sat Oct 7 10:51:35 2006 From: gsmith at alumni.uwaterloo.ca (Greg Smith) Date: Sat, 07 Oct 2006 10:51:35 -0400 Subject: paypal won't take my money for scipy docs Message-ID: I fill in the credit-card form, press continue, and it comes back with "Some required information is missing"; the indicated missing info is the 'security code' you copy in to prove you are human, which I had entered. This happened 3 times in a row, and I am certain I got it right at least the last two (I even listened to the audio read-out the third time). So I am starting to get concerned that something is being intercepted.. I'm guessing that a paypal merchant is more likely to get attention from paypal than an accountless purchaser - any idea what's going on here? ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From karol.langner at kn.pl Sat Oct 7 14:01:47 2006 From: karol.langner at kn.pl (Karol Langner) Date: Sat, 7 Oct 2006 20:01:47 +0200 Subject: problem with lapack_lite / BLAS In-Reply-To: <200610071500.08199.karol.langner@kn.pl> References: <200610071500.08199.karol.langner@kn.pl> Message-ID: <200610072001.47603.karol.langner@kn.pl> On Saturday 07 of October 2006 15:00, Karol Langner wrote: > > > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > > > discussion-bounces at lists.sourceforge.net] On Behalf Of Karol Langner > > > Sent: 07 October 2006 14:07 > > > To: NumPy List > > > Subject: [Numpy-discussion] problem with lapack_lite / BLAS > > > > > > Dear list, > > > > > > I'm trying to compile ATLAS 3.6.0 and LAPACK 3.0 (+ update) and use > > > them with > > > the latest numpy (checked out today). Installation seems to be fine, > > > all the > > > libraries are detected. When I import numpy, however, I get this: > > > > > > > > > ImportError: /home/langner/apps/python/lib/python2.5/site- > > > packages/numpy/linalg/lapack_lite.so: > > > undefined symbol: xerbla_ > > > > > > Does anyone know where this comes from? OK... I am so angry at myself... I spent around 2 hours fiddling around with blas/lapack and numpy, and the problem was that I wasn't removing the 'build' directory between compiilations. Apparently it's not enough to run "setup.py clean" (I read this and forgot about it) to recompile lapack_lite.so. Now numpy imports fine and everything I use it for works as before (only a bit faster). Karol P.S. I still get a floating point exception when running the numpy tests, though. I haven't checked out numpy for some time, so I don't now if it's a bug, or if it's my setup. The same thing happens when I use my manually built atlas/lapack and the built-in debian atlas/lapack libraries. I'd be grateful for a comment on if this is just me: >>numpy.test(10,10) [output] Check reading the top fields of a nested array ... ok Check reading the nested fields of a nested array (1st level) ... ok Check access nested descriptors of a nested array (1st level) ... ok Check reading the nested fields of a nested array (2nd level) ... ok Check access nested descriptors of a nested array (2nd level) ... ok check_access_fields (numpy.core.tests.test_numerictypes.test_read_values_plain_multiple) ... ok check_access_fields (numpy.core.tests.test_numerictypes.test_read_values_plain_single) ... ok check_cdouble (numpy.tests.test_linalg.test_det)Floating point exception thanks! -- written by Karol Langner Sat Oct 7 19:41:58 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From pav at iki.fi Sat Oct 7 14:32:21 2006 From: pav at iki.fi (Pauli Virtanen) Date: Sat, 07 Oct 2006 21:32:21 +0300 Subject: problem with lapack_lite / BLAS In-Reply-To: <200610072001.47603.karol.langner@kn.pl> References: <200610071500.08199.karol.langner@kn.pl> <200610072001.47603.karol.langner@kn.pl> Message-ID: <1160245941.3098.5.camel@localhost.localdomain> la, 2006-10-07 kello 20:01 +0200, Karol Langner kirjoitti: > I still get a floating point exception when running the numpy tests, though. I > haven't checked out numpy for some time, so I don't now if it's a bug, or if > it's my setup. The same thing happens when I use my manually built > atlas/lapack and the built-in debian atlas/lapack libraries. I'd be grateful > for a comment on if this is just me: > > >>numpy.test(10,10) > [output] > Check reading the top fields of a nested array ... ok > Check reading the nested fields of a nested array (1st level) ... ok > Check access nested descriptors of a nested array (1st level) ... ok > Check reading the nested fields of a nested array (2nd level) ... ok > Check access nested descriptors of a nested array (2nd level) ... ok > check_access_fields > (numpy.core.tests.test_numerictypes.test_read_values_plain_multiple) ... ok > check_access_fields > (numpy.core.tests.test_numerictypes.test_read_values_plain_single) ... ok > check_cdouble (numpy.tests.test_linalg.test_det)Floating point exception If you are using Debian stable (sarge), you might need to read http://www.its.caltech.edu/~astraw/coding.html#libc-patched-for-debian-sarge-to-fix-floating-point-exceptions-on-sse2 In short, libc in Debian stable has a bug that makes programs crash with SIGFPE when SSE instructions are invoked. The solution is to recompile libc from patched sources, and replace libm.so.6. At least for me this fixed crashes in numpy. Pauli Virtanen ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From aisaac at american.edu Sat Oct 7 14:45:10 2006 From: aisaac at american.edu (Alan G Isaac) Date: Sat, 7 Oct 2006 14:45:10 -0400 Subject: ide for python/numpy/scipy/mpl development ? In-Reply-To: References: <452668A8.7000904@obs.univ-lyon1.fr> Message-ID: On Fri, 06 Oct 2006, Robert Kern apparently wrote: > http://pida.berlios.de/ Can this be installed under Windows? (Looks unlikely.) Cheers, Alan Isaac ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From mattknox_ca at hotmail.com Sat Oct 7 16:11:56 2006 From: mattknox_ca at hotmail.com (Matt Knox) Date: Sat, 7 Oct 2006 16:11:56 -0400 Subject: user-defined type example Message-ID: Could someone please point me to/provide me with a basic example of creating a user defined type? Here is my completely naive attempt which obviously doesn't work...import numpy class myType(numpy.void): def __init__(self,val): self.val = val testType = numpy.dtype(myType) val1 = myType(5)val2 = myType(6) foo = numpy.array([val1,val2],testType) Any help would be greatly appreciated. Thanks, - Matt Knox _________________________________________________________________ Be one of the first to try Windows Live Mail. http://ideas.live.com/programpage.aspx?versionId=5d21c51a-b161-4314-9b0e-4911fb2b2e6d -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gnchen at cortechs.net Sat Oct 7 19:16:16 2006 From: gnchen at cortechs.net (Gennan Chen) Date: Sat, 7 Oct 2006 16:16:16 -0700 Subject: MKL on Intel OS X Message-ID: <08ED7810-A788-49E8-8072-EABE3A077E9B@cortechs.net> Hi! We just bought MKL and icc for OS X. I don't think I will try icc for numpy and scipy. But anyone ever tried compiled with os x' MKL? How to add options to make it work?? Gen -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Sun Oct 8 00:29:39 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 07 Oct 2006 22:29:39 -0600 Subject: ***[Possible UCE]*** user-defined type example In-Reply-To: References: Message-ID: <45287EB3.6080709@ieee.org> Matt Knox wrote: > Could someone please point me to/provide me with a basic example of > creating a user defined type? > > Here is my completely naive attempt which obviously doesn't work... > > import numpy > > class myType(numpy.void): > def __init__(self,val): > self.val = val > > testType = numpy.dtype(myType) > > val1 = myType(5) > val2 = myType(6) > > foo = numpy.array([val1,val2],testType) > > > Any help would be greatly appreciated. Thanks, > A "true" user-defined data-type can only be done in C. However. You can use the "VOID" data-type to create a type with multiple fields. What do you want to do with your user-defined data-type. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From robert.kern at gmail.com Sun Oct 8 00:41:09 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 08 Oct 2006 00:41:09 -0400 Subject: ide for python/numpy/scipy/mpl development ? In-Reply-To: References: <452668A8.7000904@obs.univ-lyon1.fr> Message-ID: Alan G Isaac wrote: > On Fri, 06 Oct 2006, Robert Kern apparently wrote: >> http://pida.berlios.de/ > > Can this be installed under Windows? > (Looks unlikely.) It might be possible if you have a Windows build of PyGTK, but not with the gvim integration. However, pida also supports other PyGTK-accessible editor widgets. See its documentation for details. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Sun Oct 8 00:54:45 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 7 Oct 2006 22:54:45 -0600 Subject: tensor product Message-ID: Hmmm, I notice that there is no longer a tensor product. As it was the only one of the outer, kron bunch that I really wanted, l miss it. In fact, I always thought outer should act like the tensor product for the other binary operators too. Anyway, mind if I put it back? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From mattknox_ca at hotmail.com Sun Oct 8 01:27:02 2006 From: mattknox_ca at hotmail.com (Matt Knox) Date: Sun, 8 Oct 2006 01:27:02 -0400 Subject: ***[Possible UCE]*** user-defined type example Message-ID: > Matt Knox wrote:> > Could someone please point me to/provide me with a basic example of > > creating a user defined type?> > > > Here is my completely naive attempt which obviously doesn't work...> >> > import numpy> > > > class myType(numpy.void):> > def __init__(self,val):> > self.val = val> > > > testType = numpy.dtype(myType)> >> > val1 = myType(5)> > val2 = myType(6)> > > > foo = numpy.array([val1,val2],testType)> > > > > > Any help would be greatly appreciated. Thanks,> > > A "true" user-defined data-type can only be done in C.> > However. You can use the "VOID" data-type to create a type with > multiple fields. What do you want to do with your user-defined data-type.>> -Travis Thanks for the reply Travis. I have a custom Date class which allows creation of different frequency Dates (annual, monthly, etc..). Two dates of the same frequency can be subtracted, and integers can be added to a Date, with the obvious results in each case. I store these dates in arrays currently using the "object" dtype. However, the problem with this is that I can't tell the frequency of the Dates in the array without inspecting invidual elements of the array, or subclassing the array object itself. I'd like to be able to have each frequency of date as a data type (DateAnnual, DateMonthly, etc...). It just seems like a cleaner approach to have these dates defined as data types. I would be open to the idea of defining this in C, but I don't really have any examples to work with for that either. Many thanks for any advice/suggestions you can provide. - Matt _________________________________________________________________ Be one of the first to try Windows Live Mail. http://ideas.live.com/programpage.aspx?versionId=5d21c51a-b161-4314-9b0e-4911fb2b2e6d -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dmahler at gmail.com Sun Oct 8 01:36:19 2006 From: dmahler at gmail.com (Daniel Mahler) Date: Sun, 8 Oct 2006 06:36:19 +0100 Subject: can this be made faster? Message-ID: Is there a 'loop free' way to do this in Numeric for i in arange(l): a[b[i]]+=c[i] where l == len(b) == len(c) thanks Daniel ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From wbaxter at gmail.com Sun Oct 8 01:48:59 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 8 Oct 2006 14:48:59 +0900 Subject: can this be made faster? In-Reply-To: References: Message-ID: Yes, that'd be a[b] += c On 10/8/06, Daniel Mahler wrote: > Is there a 'loop free' way to do this in Numeric > > for i in arange(l): > a[b[i]]+=c[i] > > where l == len(b) == len(c) > > thanks > Daniel ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From dmahler at gmail.com Sun Oct 8 02:03:38 2006 From: dmahler at gmail.com (Daniel Mahler) Date: Sun, 8 Oct 2006 07:03:38 +0100 Subject: can this be made faster? In-Reply-To: References: Message-ID: Thanks Bill. Thats what I was hoping for, but I get >>> a array([0, 0]) >>> b array([0, 1, 0, 1, 0]) >>> c array([1, 1, 1, 1, 1]) >>> a[b]+=c Traceback (most recent call last): File "", line 1, in ? IndexError: invalid index whereas i would like to get array([3, 2]) On 10/8/06, Bill Baxter wrote: > Yes, that'd be > a[b] += c > > On 10/8/06, Daniel Mahler wrote: > > Is there a 'loop free' way to do this in Numeric > > > > for i in arange(l): > > a[b[i]]+=c[i] > > > > where l == len(b) == len(c) > > > > thanks > > Daniel > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From wbaxter at gmail.com Sun Oct 8 02:17:29 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 8 Oct 2006 15:17:29 +0900 Subject: can this be made faster? In-Reply-To: References: Message-ID: Yeh, I spoke too soon. Tried a little example and it seemed to work. I don't get a traceback here, but your example doesn't work as expected, either. I get [1,1] as the answer with numpy 1.0rc1. Probably it should be an exception, though. It seems to work if len(b)<=len(a) and when no indices are repeated in b. --bb On 10/8/06, Daniel Mahler wrote: > Thanks Bill. > Thats what I was hoping for, > but I get > > >>> a > array([0, 0]) > >>> b > array([0, 1, 0, 1, 0]) > >>> c > array([1, 1, 1, 1, 1]) > >>> a[b]+=c > Traceback (most recent call last): > File "", line 1, in ? > IndexError: invalid index > > whereas i would like to get > > array([3, 2]) > > > > On 10/8/06, Bill Baxter wrote: > > Yes, that'd be > > a[b] += c > > > > On 10/8/06, Daniel Mahler wrote: > > > Is there a 'loop free' way to do this in Numeric > > > > > > for i in arange(l): > > > a[b[i]]+=c[i] > > > > > > where l == len(b) == len(c) > > > > > > thanks > > > Daniel > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From nadavh at visionsense.com Sun Oct 8 02:55:42 2006 From: nadavh at visionsense.com (Nadav Horesh) Date: Sun, 8 Oct 2006 08:55:42 +0200 Subject: tensor product Message-ID: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> There is a "tensortdot" function in numpy1.0rc1 Nadav -----Original Message----- From: numpy-discussion-bounces at lists.sourceforge.net on behalf of Charles R Harris Sent: Sun 08-Oct-06 06:54 To: numpy-discussion at lists.sourceforge.net Cc: Subject: [Numpy-discussion] tensor product Hmmm, I notice that there is no longer a tensor product. As it was the only one of the outer, kron bunch that I really wanted, l miss it. In fact, I always thought outer should act like the tensor product for the other binary operators too. Anyway, mind if I put it back? Chuck -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 2754 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cabwxna at utahgrizz.com Sun Oct 8 12:33:12 2006 From: cabwxna at utahgrizz.com (Your search) Date: Sun, 8 Oct 2006 14:33:12 -0200 Subject: Make sure words are Message-ID: <000f01c6ead5$ebac9e90$f3c1f552@nico> Spelled try different keywords. Videonew News Maps. Fewer your on can. Create new group About Searched all of groups Your search in! Maps is more raquo Advanced Search. Are spelled in try. Create new of group About Searched all is. Sure words is are is spelled try different! Fewer is your a on can try is Answers for expert help with search get the latest messages is. Members users Join Alerts Create new of group About Searched is all groups Your search. Create new or group About Searched all groups Your search did not. Alerts Create new group About Searched all groups am Your search. More raquo Advanced. Images Videonew News? Messages emailed to in Alerts Terms? Messages. Users Join Alerts Create. Search Members users Join. Users Join Alerts of Create ne -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sure.gif Type: image/gif Size: 5009 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From aisaac at american.edu Sun Oct 8 12:06:05 2006 From: aisaac at american.edu (Alan G Isaac) Date: Sun, 8 Oct 2006 12:06:05 -0400 Subject: =?iso-8859-1?q?***=5BPossible_UCE=5D***_user-d?= =?iso-8859-1?q?efined_type=09example?= In-Reply-To: References: Message-ID: On Sun, 8 Oct 2006, Matt Knox apparently wrote: > I have a custom Date class which allows creation of > different frequency Dates (annual, monthly, etc..). Two > dates of the same frequency can be subtracted, and > integers can be added to a Date, with the obvious results > in each case. I store these dates in arrays currently > using the "object" dtype. Just making sure you know about http://www.egenix.com/files/python/mxDateTime.html If you need something that is not there, Marc-Andr? Lemburg may be willing to add it, if others might need it too. Cheers, Alan Isaac ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From gregwillden at gmail.com Sun Oct 8 18:20:40 2006 From: gregwillden at gmail.com (Greg Willden) Date: Sun, 8 Oct 2006 17:20:40 -0500 Subject: can this be made faster? In-Reply-To: References: Message-ID: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> On 10/8/06, Daniel Mahler wrote: > > >>> a > array([0, 0]) > >>> b > array([0, 1, 0, 1, 0]) > >>> c > array([1, 1, 1, 1, 1]) > Well for this particular example you could do a=array([len(b)-sum(b), sum(b)]) Since you are just counting the ones and zeros. This next one is a little closer for the case when c is not just a bunch of 1's but you still have to know how the highest number in b. a=array([sum(c[b==0]), sum(c[b==1]), ... sum(c[b==N]) ] ) So it sort of depends on your ultimate goal. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Sun Oct 8 18:59:19 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 08 Oct 2006 18:59:19 -0400 Subject: can this be made faster? In-Reply-To: References: Message-ID: Bill Baxter wrote: > Yes, that'd be > a[b] += c No, I'm afraid that fancy indexing does not do the loop that you are thinking it would (and for reasons that we've discussed previously on this list, *can't* do that loop). That statement reduces to something like the following: tmp = a[b] tmp = tmp.__iadd__(c) a[b] = tmp In [1]: from numpy import * In [2]: a = array([0, 0]) In [3]: b = array([0, 1, 0, 1, 0]) In [4]: c = array([1, 1, 1, 1, 1]) In [5]: a[b] += c In [6]: a Out[6]: array([1, 1]) In [7]: a = array([0, 0]) In [8]: tmp = a[b] In [9]: tmp Out[9]: array([0, 0, 0, 0, 0]) In [10]: tmp = tmp.__iadd__(c) In [11]: tmp Out[11]: array([1, 1, 1, 1, 1]) In [12]: a[b] = tmp In [13]: a Out[13]: array([1, 1]) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From wbaxter at gmail.com Sun Oct 8 19:07:02 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Mon, 9 Oct 2006 08:07:02 +0900 Subject: can this be made faster? In-Reply-To: References: Message-ID: So what's the answer then? Can it be made faster? --bb On 10/9/06, Robert Kern wrote: > > Bill Baxter wrote: > > Yes, that'd be > > a[b] += c > > No, I'm afraid that fancy indexing does not do the loop that you are > thinking it > would (and for reasons that we've discussed previously on this list, > *can't* do > that loop). That statement reduces to something like the following: > > tmp = a[b] > tmp = tmp.__iadd__(c) > a[b] = tmp > > > In [1]: from numpy import * > > In [2]: a = array([0, 0]) > > In [3]: b = array([0, 1, 0, 1, 0]) > > In [4]: c = array([1, 1, 1, 1, 1]) > > In [5]: a[b] += c > > In [6]: a > Out[6]: array([1, 1]) > > In [7]: a = array([0, 0]) > > In [8]: tmp = a[b] > > In [9]: tmp > Out[9]: array([0, 0, 0, 0, 0]) > > In [10]: tmp = tmp.__iadd__(c) > > In [11]: tmp > Out[11]: array([1, 1, 1, 1, 1]) > > In [12]: a[b] = tmp > > In [13]: a > Out[13]: array([1, 1]) > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though > it had > an underlying truth." > -- Umberto Eco > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Sun Oct 8 19:08:31 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Sun, 8 Oct 2006 19:08:31 -0400 Subject: can this be made faster? In-Reply-To: References: Message-ID: On 08/10/06, Robert Kern wrote: > Bill Baxter wrote: > > Yes, that'd be > > a[b] += c > > No, I'm afraid that fancy indexing does not do the loop that you are thinking it > would (and for reasons that we've discussed previously on this list, *can't* do > that loop). That statement reduces to something like the following: So the question remains, is there a for-loop-free way to do this? (This, specifically, is: for i in range(len(b)): a[b[i]]+=c[i] where b[i] may contain repetitions.) I didn't find one, but came to the conclusion that for loops are not necessarily slower than fancy indexing, so the way to do this one is just to use a for loop. A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Sun Oct 8 22:19:02 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 8 Oct 2006 20:19:02 -0600 Subject: tensor product In-Reply-To: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> Message-ID: Hi Nadav, On 10/8/06, Nadav Horesh wrote: > > > There is a "tensortdot" function in numpy1.0rc1 The tensordot is not the same thing as a tensor product. What I want is the following: def tensor(a, b) : """Tensor product of a and b """ a = asarray(a) b = asarray(b) return outer(a, b).reshape(a.shape + b.shape) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dmahler at gmail.com Mon Oct 9 01:23:18 2006 From: dmahler at gmail.com (Daniel Mahler) Date: Mon, 9 Oct 2006 06:23:18 +0100 Subject: can this be made faster? In-Reply-To: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: On 10/8/06, Greg Willden wrote: > On 10/8/06, Daniel Mahler wrote: > > > > >>> a > > array([0, 0]) > > >>> b > > array([0, 1, 0, 1, 0]) > > >>> c > > array([1, 1, 1, 1, 1]) > > > > > Well for this particular example you could do > a=array([len(b)-sum(b), sum(b)]) > Since you are just counting the ones and zeros. > > This next one is a little closer for the case when c is not just a bunch of > 1's but you still have to know how the highest number in b. > a=array([sum(c[b==0]), sum(c[b==1]), ... sum(c[b==N]) ] ) > > So it sort of depends on your ultimate goal. > Greg > Linux. Because rebooting is for adding hardware. In my case all a, b, c are large with b and c being orders of magnitude lareger than a. b is known to contain only, but potentially any, a-indexes, reapeated many times. c contains arbitray floats. essentially it is to compute class totals as in total[class[i]] += value[i] Daniel ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From robert.kern at gmail.com Mon Oct 9 01:40:41 2006 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 09 Oct 2006 01:40:41 -0400 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: Daniel Mahler wrote: > On 10/8/06, Greg Willden wrote: >> This next one is a little closer for the case when c is not just a bunch of >> 1's but you still have to know how the highest number in b. >> a=array([sum(c[b==0]), sum(c[b==1]), ... sum(c[b==N]) ] ) >> >> So it sort of depends on your ultimate goal. >> Greg >> Linux. Because rebooting is for adding hardware. > > In my case all a, b, c are large with b and c being orders of > magnitude lareger than a. > b is known to contain only, but potentially any, a-indexes, reapeated > many times. > c contains arbitray floats. > essentially it is to compute class totals > as in total[class[i]] += value[i] In that case, a slight modification to Greg's suggestion will probably be fastest: import numpy as np # Set up the problem. lena = 10 lenc = 10000 a = np.zeros(lena, dtype=float) b = np.random.randint(lena, size=lenc) c = np.random.uniform(size=lenc) idx = np.arange(lena, dtype=int)[:, np.newaxis] mask = (b == idx) for i in range(lena): a[i] = c[b[i]].sum() -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant.travis at ieee.org Mon Oct 9 04:02:56 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Oct 2006 02:02:56 -0600 Subject: tensor product In-Reply-To: References: Message-ID: <452A0230.5060106@ieee.org> Charles R Harris wrote: > Hmmm, > > I notice that there is no longer a tensor product. As it was the only > one of the outer, kron bunch that I really wanted, l miss it. In fact, > I always thought outer should act like the tensor product for the > other binary operators too. Anyway, mind if I put it back? I'm not opposed to the idea, necessarily. But, when and where did this exist? I don't remember it. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From emsellem at obs.univ-lyon1.fr Mon Oct 9 04:10:55 2006 From: emsellem at obs.univ-lyon1.fr (Eric Emsellem) Date: Mon, 09 Oct 2006 10:10:55 +0200 Subject: ide for python/numpy/scipy/mpl, development ? Message-ID: <452A040F.3050708@obs.univ-lyon1.fr> Hi, thanks for the pointer to pida. I installed it successfully but I didn't manage to make it really work, meaning having my python file opened with gvim and include some debugger (I guess either the installation is not complete or there is something I don't understand when opening a new python file). And the docs is really scarce at the moment so it is quite difficult to pursue along that line. Indeed keeping gvim may be too tight a constraint here. Someone else suggested spe (http://stani.be/python/spe/blog/), but there I have problems getting binaries for wxPython (Suse10.1) and I don't want to attempt a full compilation of that package... (I installed Suse10.1 from the downloadable version and many many packages are missing there - the commercial DVD being double layer... and it seems that only half of it is available on the web). I tried to install eric3, and after checking QScintilla, python-qt, etc, I launch the installation of eric3 with python install.py and I get a seg fault.. frustrating. Eric On 10/6/06, Robert Kern wrote: > > > > Eric Emsellem wrote: > >> > > Hi, >> > > >> > > I am looking for an IDE to develop python programs and I am not sure >> > > what to take. >> > > The two critical items for me are 1/ a good debugger (simple and >> > > efficient) 2/ something simple to manage the files. >> > > >> > > I would also very much like to keep some basic things such as (if >> > > possible): > >> > > - editing with gvim >> > > > > This probably is the most limiting factor. I use pida because it embeds > > gvim > > into a PyGTK frame with all of the IDE goodies around it. > > > > http://pida.berlios.de/ > > > > I believe it can use one of the PyGTK debugger GUIs, but I've never used > > it. > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From wbaxter at gmail.com Mon Oct 9 04:20:25 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Mon, 9 Oct 2006 17:20:25 +0900 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <452A040F.3050708@obs.univ-lyon1.fr> References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: If you don't mind going commercial, then the WingIDE has been working very well for me. And it apparently has a VIM mode (and emacs). I'm an emacs guy, though, so I haven't tried out the VIM support. Wing is based on GTK so the interface on Windows doesn't look so nice, or act so much like a native Windows app (F10 pops up menus -- weird?), but it works well. I tried a bunch of free IDEs first and gave up out of frustration after a while, too. Wing just worked for me and worked quite well. Documentation is pretty good too. www.wingware.com. Free trial so you can see if you like it first. --bb On 10/9/06, Eric Emsellem wrote: > Hi, > > thanks for the pointer to pida. I installed it successfully but I didn't manage to make it really work, meaning having my python file opened with gvim and include some debugger (I guess either the installation is not complete or there is something I don't understand when opening a new python file). And the docs is really scarce at the moment so it is quite difficult to pursue along that line. > > Indeed keeping gvim may be too tight a constraint here. > > Someone else suggested spe (http://stani.be/python/spe/blog/), but there I have problems getting binaries for wxPython (Suse10.1) and I don't want to attempt a full compilation of that package... (I installed Suse10.1 from the downloadable version and many many packages are missing there - the commercial DVD being double layer... and it seems that only half of it is available on the web). > > I tried to install eric3, and after checking QScintilla, python-qt, etc, I launch the installation of eric3 with python install.py and I get a seg fault.. > > frustrating. > Eric > > > On 10/6/06, Robert Kern wrote: > > > > > > > Eric Emsellem wrote: > > > >> > > Hi, > >> > > > >> > > I am looking for an IDE to develop python programs and I am not sure > >> > > what to take. > >> > > The two critical items for me are 1/ a good debugger (simple and > >> > > efficient) 2/ something simple to manage the files. > >> > > > >> > > I would also very much like to keep some basic things such as (if > >> > > > possible): > > > >> > > - editing with gvim > >> > > > > > > This probably is the most limiting factor. I use pida because it embeds > > > gvim > > > into a PyGTK frame with all of the IDE goodies around it. > > > > > > http://pida.berlios.de/ > > > > > > I believe it can use one of the PyGTK debugger GUIs, but I've never used > > > it. > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From nagyebhzlv at westhillshonda.com Sun Oct 8 17:17:41 2006 From: nagyebhzlv at westhillshonda.com (Nokia) Date: Mon, 9 Oct 2006 04:17:41 +0700 Subject: VideoNew Maps Members: Message-ID: <000b01c6eb94$8955da30$1d28d97c@vertex5> Answers expert a search get latest. Democracy in in January led Kabul meet with our am? Watchgot moustache Aviators. from query new a Array Features am National is the is Final a Word. Required Feeds list one am pos create. Larger Where add. Games these ones keyring. Strut your stuff Almost in else Beta Discuss versions in Latest am. Sylla not working Ozzmosis List Google Groups Help a Sign. Regularly publishes place share well providing Knowing someone gone similar or situation syndromes! Cant seem find anything of decent in though Benjamin dab radio Desktop. Levine graduate am Yale a College currently Fellow in! Families. May changed is. Power wants am Most Popular. This cache. Coolest. Inspired winging store. Muscula -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: has.gif Type: image/gif Size: 5012 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Mon Oct 9 09:01:01 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 09 Oct 2006 06:01:01 -0700 Subject: tensor product In-Reply-To: References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> Message-ID: <452A480D.2010300@ieee.org> Charles R Harris wrote: > Hi Nadav, > > On 10/8/06, *Nadav Horesh* > wrote: > > > There is a "tensortdot" function in numpy1.0rc1 > > > > The tensordot is not the same thing as a tensor product. What I want > is the following: > > def tensor(a, b) : > """Tensor product of a and b > > """ > a = asarray(a) > b = asarray(b) > return outer(a, b).reshape(a.shape + b.shape) > Is this not the same things as numpy.multiply.outer(a, b)? (as opposed to outer(a, b), which appears to pretend that everything is a vector -- I'm not sure what the point of that is). -tim ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From tim.hochberg at ieee.org Mon Oct 9 10:10:09 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 09 Oct 2006 07:10:09 -0700 Subject: vectorized linalg Message-ID: <452A5841.6050802@ieee.org> I periodically need to perform various linalg operations on a large number of small matrices. The normal numpy approach of stacking up the data into an extra dimension and doing the operations in parallel doesn't work because the linalg functions are only designed to work on one matrix at a time. At first glance, this restriction seems harmless enough since the underlying LAPACK functions are also designed to work one matrix at a time and it seem that there would be little to be gained by parallelizing the Python level code. However, this turns out to be the case. I have some code, originally written for numarray, but I recently ported it over to numpy, that rewrites the linalg function determinant, inverse and solve_linear_equations[1] to operate of 2 or 3D arrays, treating 3D arrays as stacks of 2D arrays. For operating on large numbers of small arrays (for example 1000 2x2 arrays), this approach is over an order of magnitude faster than the obvious map(inverse, threeDArray) approach. So, some questions: 1. Is there any interest in this code for numpy.linalg? I'd be willing to clean it up and work on the other functions in linalg so that they were all consistent. Since these changes should all be backwards compatible (unless your counting on catching an error from one of the linalg functions), I don't see this as a problem to put in after 1.0, so there's really no rush on this. I'm only bringing it up now since I just ported it over to numpy and it's fresh in my mind. 2. If 1., what to do about norm? I think the other functions in linalg stack naturally, but norm, since it is meant to work on both vectors and matrices is problematic. Norm already seems a bit problematic in that, for some values of 'ord', norm can return different values for a shape [N] and a shape [1,N] array containing the same values: >>> linalg.norm(a[:1], 1) 0.78120069791102054 >>> linalg.norm(a[0], 1) 1.4588102446677758 My inclination is to introduce stackable 1 and 2D version s of norm, vnorm and mnorm for lack of better names. Ideally we'd deprecate the use of norm for ord!=None (Froebenius norm) or at least in cases where the result depends on the rank [1,N] versus [N]. 3. A similar issue arises with dot. One would like to be able to do stacked matrix products, vector products and matrix-vector products. If we are making the rest of linalg stackable, linalg would be a sensible place for a stackable version of dot to live. However, it's immediately clear how to do this without introducing a whole pile (4) of stacking dot function to handle the various cases, plus broadcasting, cleanly. This would require some more thought. Thoughts? -tim [1] Those are actually the numarray names, which the current code still uses, the numpy names are 'det', 'inv' and 'solve' [2] Treatment of stacking insolve linear equations is a bit more complicated, but I'll ignore that for now. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From David.L.Goldsmith at noaa.gov Mon Oct 9 10:32:12 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Mon, 09 Oct 2006 07:32:12 -0700 Subject: vectorized linalg In-Reply-To: <452A5841.6050802@ieee.org> References: <452A5841.6050802@ieee.org> Message-ID: <452A5D6C.9040204@noaa.gov> Tim Hochberg wrote: > I periodically need to perform various linalg operations on a large > number of small matrices. The normal numpy approach of stacking up the > data into an extra dimension and doing the operations in parallel > doesn't work because the linalg functions are only designed to work on > one matrix at a time. At first glance, this restriction seems harmless > enough since the underlying LAPACK functions are also designed to work > one matrix at a time and it seem that there would be little to be gained > by parallelizing the Python level code. However, this turns out to be > the case. > > I have some code, originally written for numarray, but I recently ported > it over to numpy, that rewrites the linalg function determinant, inverse > and solve_linear_equations[1] to operate of 2 or 3D arrays, treating 3D > arrays as stacks of 2D arrays. For operating on large numbers of small > arrays (for example 1000 2x2 arrays), this approach is over an order of > magnitude faster than the obvious map(inverse, threeDArray) approach. > > So, some questions: > > 1. Is there any interest in this code for numpy.linalg? I'd be > willing to clean it up and work on the other functions in linalg so that > they were all consistent. Since these changes should all be backwards > compatible (unless your counting on catching an error from one of the > linalg functions), I don't see this as a problem to put in after 1.0, so > there's really no rush on this. I'm only bringing it up now since I just > ported it over to numpy and it's fresh in my mind. > Say "no" to someone offering to make a Python module more feature rich? Blasphemy! :-) But I am very curious as to the source/basis of the performance improvement - did you figure this out? DG > 2. If 1., what to do about norm? I think the other functions in > linalg stack naturally, but norm, since it is meant to work on both > vectors and matrices is problematic. Norm already seems a bit > problematic in that, for some values of 'ord', norm can return different > values for a shape [N] and a shape [1,N] array containing the same values: > > >>> linalg.norm(a[:1], 1) > 0.78120069791102054 > >>> linalg.norm(a[0], 1) > 1.4588102446677758 > > My inclination is to introduce stackable 1 and 2D version s of norm, > vnorm and mnorm for lack of better names. Ideally we'd deprecate the use > of norm for ord!=None (Froebenius norm) or at least in cases where the > result depends on the rank [1,N] versus [N]. > > 3. A similar issue arises with dot. One would like to be able to do > stacked matrix products, vector products and matrix-vector products. If > we are making the rest of linalg stackable, linalg would be a sensible > place for a stackable version of dot to live. However, it's immediately > clear how to do this without introducing a whole pile (4) of stacking > dot function to handle the various cases, plus broadcasting, cleanly. > This would require some more thought. Thoughts? > > -tim > > > [1] Those are actually the numarray names, which the current code still > uses, the numpy names are 'det', 'inv' and 'solve' > > [2] Treatment of stacking insolve linear equations is a bit more > complicated, but I'll ignore that for now. > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 10:43:49 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 08:43:49 -0600 Subject: tensor product In-Reply-To: <452A480D.2010300@ieee.org> References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> <452A480D.2010300@ieee.org> Message-ID: On 10/9/06, Tim Hochberg wrote: > > Is this not the same things as numpy.multiply.outer(a, b)? (as opposed > to outer(a, b), which appears to pretend that everything is a vector -- > I'm not sure what the point of that is). Hmmm, yes, multiply.outer does do that. I thought that outer was short for multiply.outer and that the behaviour had changed. So the question is why outer does what it does. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Mon Oct 9 10:45:37 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 09 Oct 2006 07:45:37 -0700 Subject: vectorized linalg In-Reply-To: <452A5D6C.9040204@noaa.gov> References: <452A5841.6050802@ieee.org> <452A5D6C.9040204@noaa.gov> Message-ID: <452A6091.3080609@ieee.org> David Goldsmith wrote: > Tim Hochberg wrote: > >> I periodically need to perform various linalg operations on a large >> number of small matrices. The normal numpy approach of stacking up the >> data into an extra dimension and doing the operations in parallel >> doesn't work because the linalg functions are only designed to work on >> one matrix at a time. At first glance, this restriction seems harmless >> enough since the underlying LAPACK functions are also designed to work >> one matrix at a time and it seem that there would be little to be gained >> by parallelizing the Python level code. However, this turns out to be >> the case. >> >> I have some code, originally written for numarray, but I recently ported >> it over to numpy, that rewrites the linalg function determinant, inverse >> and solve_linear_equations[1] to operate of 2 or 3D arrays, treating 3D >> arrays as stacks of 2D arrays. For operating on large numbers of small >> arrays (for example 1000 2x2 arrays), this approach is over an order of >> magnitude faster than the obvious map(inverse, threeDArray) approach. >> >> So, some questions: >> >> 1. Is there any interest in this code for numpy.linalg? I'd be >> willing to clean it up and work on the other functions in linalg so that >> they were all consistent. Since these changes should all be backwards >> compatible (unless your counting on catching an error from one of the >> linalg functions), I don't see this as a problem to put in after 1.0, so >> there's really no rush on this. I'm only bringing it up now since I just >> ported it over to numpy and it's fresh in my mind. >> >> > Say "no" to someone offering to make a Python module more feature rich? > Blasphemy! :-) > > But I am very curious as to the source/basis of the performance > improvement - did you figure this out? > Yes and no. Here's what the begining of linalg.det looks lik: a = asarray(a) _assertRank2(a) _assertSquareness(a) t, result_t = _commonType(a) a = _fastCopyAndTranspose(t, a) n = a.shape[0] if isComplexType(t): lapack_routine = lapack_lite.zgetrf else: lapack_routine = lapack_lite.dgetrf # ... There's quite a few function calls here, since the actual call to dgetrf probably takes neglible time for a 2x2 array, the cost of computation here is dominated by function call overhead, creating the return arrays and such not. Which is the biggest culprit, I'm not sure; by vectorizing it, much of that overhead is only incurred once rather than hundreds or thousands of times, so it all becomes negligible at that point. I didn't bother to track down the worst offender. -tim ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From emsellem at obs.univ-lyon1.fr Mon Oct 9 10:50:33 2006 From: emsellem at obs.univ-lyon1.fr (Eric Emsellem) Date: Mon, 09 Oct 2006 16:50:33 +0200 Subject: ide for python/numpy/scipy/mpl, , development ? Message-ID: <452A61B9.2090100@obs.univ-lyon1.fr> Hi, thanks for the tip: very nice but really too expensive for me... (the personal version having too many constraints). Thanks again anyway. Will search for something else... Maybe I'll manage to get pida or spe to work ? cheers Eric > From: "Bill Baxter" > Subject: Re: [Numpy-discussion] ide for python/numpy/scipy/mpl, > development ? > To: "Discussion of Numerical Python" > > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > If you don't mind going commercial, then the WingIDE has been working > very well for me. > And it apparently has a VIM mode (and emacs). I'm an emacs guy, > though, so I haven't tried out the VIM support. > > Wing is based on GTK so the interface on Windows doesn't look so nice, > or act so much like a native Windows app (F10 pops up menus -- > weird?), but it works well. I tried a bunch of free IDEs first and > gave up out of frustration after a while, too. Wing just worked for > me and worked quite well. Documentation is pretty good too. > www.wingware.com. Free trial so you can see if you like it first. > > --bb -- ==================================================================== Eric Emsellem emsellem at obs.univ-lyon1.fr Centre de Recherche Astrophysique de Lyon 9 av. Charles-Andre tel: +33 (0)4 78 86 83 84 69561 Saint-Genis Laval Cedex fax: +33 (0)4 78 86 83 86 France http://www-obs.univ-lyon1.fr/eric.emsellem ==================================================================== ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From perry at stsci.edu Mon Oct 9 11:00:28 2006 From: perry at stsci.edu (Perry Greenfield) Date: Mon, 9 Oct 2006 11:00:28 -0400 Subject: ide for python/numpy/scipy/mpl, , development ? In-Reply-To: <452A61B9.2090100@obs.univ-lyon1.fr> References: <452A61B9.2090100@obs.univ-lyon1.fr> Message-ID: On Oct 9, 2006, at 10:50 AM, Eric Emsellem wrote: > Hi, > > thanks for the tip: very nice but really too expensive for me... (the > personal version having too many constraints). Thanks again anyway. > Will search for something else... Maybe I'll manage to get pida or spe > to work ? > > cheers > > Eric I believe that they are likely to give you the educational discount if you work in a research institution. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From pgmdevlist at mailcan.com Mon Oct 9 11:11:54 2006 From: pgmdevlist at mailcan.com (Pierre GM) Date: Mon, 9 Oct 2006 11:11:54 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: <200610091111.54778.pgmdevlist@mailcan.com> On Monday 09 October 2006 04:20, Bill Baxter wrote: > If you don't mind going commercial, then the WingIDE has been working > very well for me. Did anybody mention Eclipse+pydev ? I've been a regular user for the last few months, and I'm quite happy about it. True, Eclipse can be a bit overkill, but it's quite performant. Actually, I use pydev for the development and an interactive shell like ipython for testing/playing around... ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 12:04:28 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 10:04:28 -0600 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <200610091111.54778.pgmdevlist@mailcan.com> References: <452A040F.3050708@obs.univ-lyon1.fr> <200610091111.54778.pgmdevlist@mailcan.com> Message-ID: On 10/9/06, Pierre GM wrote: > > On Monday 09 October 2006 04:20, Bill Baxter wrote: > > If you don't mind going commercial, then the WingIDE has been working > > very well for me. > > Did anybody mention Eclipse+pydev ? I've been a regular user for the last > few > months, and I'm quite happy about it. True, Eclipse can be a bit overkill, > but it's quite performant. Actually, I use pydev for the development and > an > interactive shell like ipython for testing/playing around... There is also a vim emulator plugin for eclipse that costs 15 euro. Here: http://www.satokar.com/viplugin/ Chuck ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jnbeckman at lesarnaques.net Mon Oct 9 12:31:21 2006 From: jnbeckman at lesarnaques.net (Pearl) Date: Mon, 09 Oct 2006 18:31:21 +0200 Subject: see4 urself Message-ID: <000b01c6ebc0$5ae51a90$0200a8c0@2.0.168.192> dozen words to a mouthful; and as every one else talked equally fast, and allfalls, and had them, too.that a word would bring the ally she needed; and, with a courage as native to ben! i'll be just as good as ever was, and i'll take care of sanch all the way,"that perfectly well, and particularly desired to prevent it, for the mood inthis heavy. the march family had enjoyed her when she had newly cleaned his cage, made mop bark by pecking at him whileat the close of the dance and"that'ssteve well,is absent templatea ring from his pocket and gently drawing a warm, bare hand out of the muff a very short trip by rail and the rest of the journey a very short trip by rail and the rest of the journeyas if i'd given her a check for a million, to be spent in charity. you are not"i never shall outgrow my foolish'let her come,' called dan; 'she climbs like but before we buy any more pictures, let's get the ones we have framed.soon die a natural death," said mr. shaw, who seemed to find it difficultso it was decided, and without saying much polly scoffed at this sort of thing the barn, and see if you don't howl then." about in the chair as if the process of getting words into his memoryma'am," returned betty, sharply, for her usually amiable spirit hadwent without another word, but she wore a curiously anxious look, "can'tdidn't think i ever should want to play again, but i do. will youaltering fast in other ways and watched the "distinguished-looking suspected it," said tip. "did you get hurt? are you cracked at all?" deposited, and was promised that he might break the bank as soon as"don't"not this morning, i 'm rather in a hurry." and away went pollybreezy hair, which stood up like the fur of an angry kitten. f: peter mustn't eat lunch in class. mir - li - ton.""my dear man, it's a bonnet! my very best business with all their might, as if they feared that some trace of is absent templatehad never allowed her to bring to see him, and the proud old man was to him. He took it tenderly in his hands, and from its folds there fellis absent templateFBI Special Agent Richard Kolko said in Washington. the sparkling sunshine; when the first wild roses are spilling their is absent templatewithout hesitation: It is never right, Maggie.is absent template -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 3Y0nwivhlt.gif Type: image/gif Size: 8579 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From aisaac at american.edu Mon Oct 9 12:49:23 2006 From: aisaac at american.edu (Alan G Isaac) Date: Mon, 9 Oct 2006 12:49:23 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <200610091111.54778.pgmdevlist@mailcan.com> References: <452A040F.3050708@obs.univ-lyon1.fr><200610091111.54778.pgmdevlist@mailcan.com> Message-ID: On Mon, 9 Oct 2006, Pierre GM apparently wrote: > Did anybody mention Eclipse+pydev? It used to be a problem to get Eclipse to talk to gvim on Windows (via OLE). Has that changed? Cheers, Alan Isaac ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From karol.langner at kn.pl Mon Oct 9 13:02:59 2006 From: karol.langner at kn.pl (Karol Langner) Date: Mon, 9 Oct 2006 19:02:59 +0200 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: <200610091902.59198.karol.langner@kn.pl> On Monday 09 of October 2006 10:20, Bill Baxter wrote: > If you don't mind going commercial, then the WingIDE has been working > very well for me. > And it apparently has a VIM mode (and emacs). I'm an emacs guy, > though, so I haven't tried out the VIM support. I also have WingIDE installed and like it alot! I don't actually use it as my main ide, but might in the future. Currently I use Kile (the LaTeX front-end from KDE), but it doesn't have any python-specific features. Karol -- written by Karol Langner pon pa? 9 19:00:27 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From kwgoodman at gmail.com Mon Oct 9 13:04:40 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Mon, 9 Oct 2006 10:04:40 -0700 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <452A040F.3050708@obs.univ-lyon1.fr> References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: On 10/9/06, Eric Emsellem wrote: > thanks for the pointer to pida. I installed it successfully but I didn't manage to > makeit really work [snip] I was excited to see that pida can use meld. But when I try to do a diff, pida complains that it cannot find meldembed.py. Has anyone else run into that? I installed pida from binary on debian etch. And meld is installed and working. Hey, wait a minute, isn't this a numpy list? ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From David.L.Goldsmith at noaa.gov Mon Oct 9 13:09:39 2006 From: David.L.Goldsmith at noaa.gov (David L Goldsmith) Date: Mon, 09 Oct 2006 10:09:39 -0700 Subject: vectorized linalg In-Reply-To: <452A6091.3080609@ieee.org> References: <452A5841.6050802@ieee.org> <452A5D6C.9040204@noaa.gov> <452A6091.3080609@ieee.org> Message-ID: <452A8253.7020000@noaa.gov> Tim Hochberg wrote: > David Goldsmith wrote: > >> Tim Hochberg wrote: >> >> >>> I periodically need to perform various linalg operations on a large >>> number of small matrices. The normal numpy approach of stacking up the >>> data into an extra dimension and doing the operations in parallel >>> doesn't work because the linalg functions are only designed to work on >>> one matrix at a time. At first glance, this restriction seems harmless >>> enough since the underlying LAPACK functions are also designed to work >>> one matrix at a time and it seem that there would be little to be gained >>> by parallelizing the Python level code. However, this turns out to be >>> the case. >>> >>> I have some code, originally written for numarray, but I recently ported >>> it over to numpy, that rewrites the linalg function determinant, inverse >>> and solve_linear_equations[1] to operate of 2 or 3D arrays, treating 3D >>> arrays as stacks of 2D arrays. For operating on large numbers of small >>> arrays (for example 1000 2x2 arrays), this approach is over an order of >>> magnitude faster than the obvious map(inverse, threeDArray) approach. >>> >>> So, some questions: >>> >>> 1. Is there any interest in this code for numpy.linalg? I'd be >>> willing to clean it up and work on the other functions in linalg so that >>> they were all consistent. Since these changes should all be backwards >>> compatible (unless your counting on catching an error from one of the >>> linalg functions), I don't see this as a problem to put in after 1.0, so >>> there's really no rush on this. I'm only bringing it up now since I just >>> ported it over to numpy and it's fresh in my mind. >>> >>> >>> >> Say "no" to someone offering to make a Python module more feature rich? >> Blasphemy! :-) >> >> But I am very curious as to the source/basis of the performance >> improvement - did you figure this out? >> >> > Yes and no. Here's what the begining of linalg.det looks lik: > > a = asarray(a) > _assertRank2(a) > _assertSquareness(a) > t, result_t = _commonType(a) > a = _fastCopyAndTranspose(t, a) > n = a.shape[0] > if isComplexType(t): > lapack_routine = lapack_lite.zgetrf > else: > lapack_routine = lapack_lite.dgetrf > # ... > > There's quite a few function calls here, since the actual call to dgetrf > probably takes neglible time for a 2x2 array, the cost of computation > here is dominated by function call overhead, creating the return arrays > and such not. Which is the biggest culprit, I'm not sure; by vectorizing > it, much of that overhead is only incurred once rather than hundreds or > thousands of times, so it all becomes negligible at that point. I didn't > bother to track down the worst offender. > > Ah, that all makes sense (including not bothering to profile the code). DG > -tim > > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- HMRD/ORR/NOS/NOAA ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant.travis at ieee.org Mon Oct 9 13:51:38 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Oct 2006 11:51:38 -0600 Subject: tensor product In-Reply-To: References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> <452A480D.2010300@ieee.org> Message-ID: <452A8C2A.8000004@ieee.org> Charles R Harris wrote: > > > On 10/9/06, *Tim Hochberg* > wrote: > > > > Is this not the same things as numpy.multiply.outer(a, b)? (as > opposed > to outer(a, b), which appears to pretend that everything is a > vector -- > I'm not sure what the point of that is). > > > Hmmm, yes, multiply.outer does do that. I thought that outer was short > for multiply.outer and that the behaviour had changed. So the question > is why outer does what it does. Unfortunately, I don't know the answer to that. numpy.outer is the same as Numeric.outerproduct and does the same thing. I'm not sure of the reason behind it. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 13:59:16 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 11:59:16 -0600 Subject: tensor product In-Reply-To: <452A8C2A.8000004@ieee.org> References: <07C6A61102C94148B8104D42DE95F7E8C8F0CD@exchange2k.envision.co.il> <452A480D.2010300@ieee.org> <452A8C2A.8000004@ieee.org> Message-ID: On 10/9/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > > On 10/9/06, *Tim Hochberg* > > wrote: > > > > > > > > Is this not the same things as numpy.multiply.outer(a, b)? (as > > opposed > > to outer(a, b), which appears to pretend that everything is a > > vector -- > > I'm not sure what the point of that is). > > > > > > Hmmm, yes, multiply.outer does do that. I thought that outer was short > > for multiply.outer and that the behaviour had changed. So the question > > is why outer does what it does. > Unfortunately, I don't know the answer to that. > > numpy.outer is the same as Numeric.outerproduct and does the same > thing. I'm not sure of the reason behind it. Looks sorta like a matrix thing. Maybe it should be called outerflat or some such. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From karol.langner at kn.pl Mon Oct 9 14:15:58 2006 From: karol.langner at kn.pl (Karol Langner) Date: Mon, 9 Oct 2006 20:15:58 +0200 Subject: AMD & UMFpack in numpy Message-ID: <200610092015.58322.karol.langner@kn.pl> Can someone give me a hint as to where in numpy the AMD and UMFpack libraries are used, if at all? I ask, because they have their respective sections in site.cfg.example in the trunk. Karol -- written by Karol Langner pon pa? 9 20:13:25 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From peridot.faceted at gmail.com Mon Oct 9 14:30:03 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Mon, 9 Oct 2006 14:30:03 -0400 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: On 09/10/06, Robert Kern wrote: > Daniel Mahler wrote: > > In my case all a, b, c are large with b and c being orders of > > magnitude lareger than a. > > b is known to contain only, but potentially any, a-indexes, reapeated > > many times. > > c contains arbitray floats. > > essentially it is to compute class totals > > as in total[class[i]] += value[i] > > In that case, a slight modification to Greg's suggestion will probably be fastest: If a is even moderately large and you don't care what's left behind in b and c you will probably accelerate the process by sorting b and c together (for cache coherency in a) This seems like a rather common operation - I know I've needed it on at least two occasions - is it worth creating some sort of C implementation? What is the appropriate generalization? A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 14:57:43 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 12:57:43 -0600 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: On 10/9/06, A. M. Archibald wrote: > > On 09/10/06, Robert Kern wrote: > > Daniel Mahler wrote: > > > In my case all a, b, c are large with b and c being orders of > > > magnitude lareger than a. > > > b is known to contain only, but potentially any, a-indexes, reapeated > > > many times. > > > c contains arbitray floats. > > > essentially it is to compute class totals > > > as in total[class[i]] += value[i] > > > > In that case, a slight modification to Greg's suggestion will probably > be fastest: > > If a is even moderately large and you don't care what's left behind in > b and c you will probably accelerate the process by sorting b and c > together (for cache coherency in a) > > This seems like a rather common operation - I know I've needed it on > at least two occasions - is it worth creating some sort of C > implementation? What is the appropriate generalization? Some sort of indirect addressing infrastructure. But it looks like this could be tricky to make safe, it would need to do bounds checking at the least and would probably work best with a contiguous array as the target. I could see some sort of low-level function called argassign(target, indirect index, source) that could be used to build more complicated things in python. Chuck A. M. Archibald > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Mon Oct 9 15:59:58 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Mon, 9 Oct 2006 15:59:58 -0400 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: > > > > c contains arbitray floats. > > > > essentially it is to compute class totals > > > > as in total[class[i]] += value[i] > > This seems like a rather common operation - I know I've needed it on > > at least two occasions - is it worth creating some sort of C > > implementation? What is the appropriate generalization? > > Some sort of indirect addressing infrastructure. But it looks like this > could be tricky to make safe, it would need to do bounds checking at the > least and would probably work best with a contiguous array as the target. I > could see some sort of low-level function called argassign(target, indirect > index, source) that could be used to build more complicated things in > python. If it were only assignment that was needed, fancy indexing could already handle it. The problem is that this is something that can't *quite* be done with the current fancy indexing infrastructure - every time an index comes up we want to add the value to what's there, rather than replacing it. I suppose histogram covers one major application; in fact if histogram allowed weightings ("count this point as -0.6") it would solve the OP's problem. A. M. Archibald ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Mon Oct 9 16:14:52 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 9 Oct 2006 14:14:52 -0600 Subject: can this be made faster? In-Reply-To: References: <903323ff0610081520w52ce4abm3d0ce2cac3b50b90@mail.gmail.com> Message-ID: On 10/9/06, A. M. Archibald wrote: > > > > > > c contains arbitray floats. > > > > > essentially it is to compute class totals > > > > > as in total[class[i]] += value[i] > > > > This seems like a rather common operation - I know I've needed it on > > > at least two occasions - is it worth creating some sort of C > > > implementation? What is the appropriate generalization? > > > > Some sort of indirect addressing infrastructure. But it looks like this > > could be tricky to make safe, it would need to do bounds checking at the > > least and would probably work best with a contiguous array as the > target. I > > could see some sort of low-level function called argassign(target, > indirect > > index, source) that could be used to build more complicated things in > > python. > > If it were only assignment that was needed, fancy indexing could > already handle it. The problem is that this is something that can't > *quite* be done with the current fancy indexing infrastructure - every > time an index comes up we want to add the value to what's there, > rather than replacing it. I suppose histogram covers one major > application; in fact if histogram allowed weightings ("count this > point as -0.6") it would solve the OP's problem. Sure, just add functions arg_addassign, etc., which means dest[ind[i]] += src[i], just as arg_assign would mean dest[ind[i]] = src[i]. If you covered all the assign variants I think you could do most everything. Upper level python routines could deal with shaping and such while the lower level routines dealt with flat, contiguous arrays. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Mon Oct 9 16:59:38 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 09 Oct 2006 14:59:38 -0600 Subject: Numpy rc2 is out Message-ID: <452AB83A.7010701@ieee.org> Release Candidate 2.0 is now out. Thanks to all the great testing and fixes that were done between 1.0rc1 and 1.0rc2 The release date for NumPy 1.0 is Oct. 17. There will be a freeze on the trunk starting Monday Oct. 16 so any changes should be in by then. If significant changes are made then we will release 1.0rc3 on Oct. 17 and push the release date of NumPy 1.0 to Oct 24. -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From roma at public6.sta.net.cn Mon Oct 9 14:43:26 2006 From: roma at public6.sta.net.cn (roma at public6.sta.net.cn) Date: Mon, 9 Oct 2006 12:43:26 -0600 Subject: (no subject) Message-ID: <000b01c6ebed$f0ae5050$690a010a@74-93-25-77-Minnesota.hfc.comcastbusiness.net> Hi look button, and just got out ahead of the closing doors. The streaming down my face from the fringes of the gas, and He looked surprised at that, giving me some small left newspapers containing the article. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 7WnXXkubAx.gif Type: image/gif Size: 11165 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From parlar at gmail.com Mon Oct 9 21:26:27 2006 From: parlar at gmail.com (Jay Parlar) Date: Mon, 9 Oct 2006 21:26:27 -0400 Subject: Problem buliding rc2 on OS X 10.3.9/Python2.5 Message-ID: In the process of finally switching over to Python 2.5, and am trying to build numpy. Unfortunately, it dies during the build: Jay-Computer:~/Desktop/numpy-1.0rc2 jayparlar$ python setup.py build Running from numpy source directory. F2PY Version 2_3296 blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] running build running config_fc running build_src building py_modules sources building extension "numpy.core.multiarray" sources Generating build/src.macosx-10.3-fat-2.5/numpy/core/config.h customize NAGFCompiler customize AbsoftFCompiler customize IbmFCompiler Could not locate executable g77 Could not locate executable f77 Could not locate executable gfortran Could not locate executable f95 customize GnuFCompiler customize Gnu95FCompiler customize G95FCompiler customize GnuFCompiler customize Gnu95FCompiler customize NAGFCompiler customize NAGFCompiler using config C compiler: gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 compile options: '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -Inumpy/core/src -Inumpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c' gcc: _configtest.c gcc: cannot specify -o with -c or -S and multiple compilations gcc: cannot specify -o with -c or -S and multiple compilations failure. removing: _configtest.c _configtest.o numpy/core/setup.py:50: DeprecationWarning: raising a string exception is deprecated raise "ERROR: Failed to test configuration" Traceback (most recent call last): File "setup.py", line 89, in setup_package() File "setup.py", line 82, in setup_package configuration=configuration ) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/core.py", line 174, in setup return old_setup(**new_attr) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", line 112, in run self.run_command(cmd_name) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 87, in run self.build_sources() File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 106, in build_sources self.build_extension_sources(ext) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 212, in build_extension_sources sources = self.generate_sources(sources, ext) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 270, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 50, in generate_config_h raise "ERROR: Failed to test configuration" ERROR: Failed to test configuration This is with the Universal 2.5 binary, and OS X 10.3.9. Any ideas? Sorry if this one has been asked before, but I can't seem to find a solution anywhere. Jay P. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From ckkart at hoc.net Mon Oct 9 21:42:52 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Tue, 10 Oct 2006 01:42:52 +0000 (UTC) Subject: compatibility of extension modules References: Message-ID: Christian Kristukat hoc.net> writes: > > Hi, > i've got problems running a numpy/scipy extension module (scipy.sandbox.odr) > built with cygwin/mingw32 on XP on other machines (windows 2000). I get those > very informative 'windows encountered a problem' messages when calling the > extension module - importing seems to work. More precisely, a binary extension module built on a Celeron D crashes on an Athlon Thunderbird, whereas the Atholon binary runs on the Celereon. So, I'm wondering if I can turn off some processor specific optimization and if this might help here. Christian ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From sz13544007750 at 163.com Tue Oct 10 01:06:35 2006 From: sz13544007750 at 163.com (=?GB2312?B?xvPStdPQwPs=?=) Date: Tue, 10 Oct 2006 13:06:35 +0800 Subject: =?GB2312?B?xvPStdPQwPs=?= Message-ID: ???????(??/??)? ??? ??????????????????????????????????2002?? ??????????????????????????????????????????,?????????????????????????????????????????????????? ???????????????????????????? ??????????????????????????????????????? ?????????????????????????????? ????????? 1.?????????????(????????)??????? 2.??????(???????????????????????????)? 3.????????????(????????????????)? 4.?????????????????????????????????????? ???????????????????????(24???48??????)? 5.???????????????????????????????????? ??????????????????????(??????????????)? ????????! ??????? ????? 0755--81590701 13544007750 E-mail?sz13544007750 at 163.com ??????????????????4009?????? ????????????????????????????????????????????????????????????????????????????????????????????? ???????????????? -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From karol.langner at kn.pl Tue Oct 10 02:11:30 2006 From: karol.langner at kn.pl (Karol Langner) Date: Tue, 10 Oct 2006 08:11:30 +0200 Subject: problem with lapack_lite / BLAS In-Reply-To: <1160245941.3098.5.camel@localhost.localdomain> References: <200610072001.47603.karol.langner@kn.pl> <1160245941.3098.5.camel@localhost.localdomain> Message-ID: <200610100811.30847.karol.langner@kn.pl> On Saturday 07 of October 2006 20:32, Pauli Virtanen wrote: > la, 2006-10-07 kello 20:01 +0200, Karol Langner kirjoitti: > > I still get a floating point exception when running the numpy tests, > > though. I haven't checked out numpy for some time, so I don't now if it's > > a bug, or if it's my setup. The same thing happens when I use my manually > > built atlas/lapack and the built-in debian atlas/lapack libraries. I'd be > > grateful > > > > for a comment on if this is just me: > > >>numpy.test(10,10) > > > > [output] > > Check reading the top fields of a nested array ... ok > > Check reading the nested fields of a nested array (1st level) ... ok > > Check access nested descriptors of a nested array (1st level) ... ok > > Check reading the nested fields of a nested array (2nd level) ... ok > > Check access nested descriptors of a nested array (2nd level) ... ok > > check_access_fields > > (numpy.core.tests.test_numerictypes.test_read_values_plain_multiple) ... > > ok check_access_fields > > (numpy.core.tests.test_numerictypes.test_read_values_plain_single) ... ok > > check_cdouble (numpy.tests.test_linalg.test_det)Floating point exception > > If you are using Debian stable (sarge), you might need to read > > http://www.its.caltech.edu/~astraw/coding.html#libc-patched-for-debian-sarg >e-to-fix-floating-point-exceptions-on-sse2 > > In short, libc in Debian stable has a bug that makes programs crash with > SIGFPE when SSE instructions are invoked. The solution is to recompile > libc from patched sources, and replace libm.so.6. At least for me this > fixed crashes in numpy. > > Pauli Virtanen Big thanks for that link. I took the leap and it does fix that SIGFPE. As a note, after building the glibc sources, you only have to install the libc6 binary deb. There should be a link to that page somewhere on the wiki. Karol -- written by Karol Langner Tue Oct 10 08:07:04 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From josh8912 at yahoo.com Tue Oct 10 02:17:28 2006 From: josh8912 at yahoo.com (JJ) Date: Mon, 9 Oct 2006 23:17:28 -0700 (PDT) Subject: suggestions for slices of matrices Message-ID: <20061010061728.12419.qmail@web51709.mail.yahoo.com> Hello. I haven't been following development too closely lately, but I did just download and reinstall the current svn version. For what its worth, I would like to again suggest two changes: -- If M is a nxm matrix and P and Z are nx1 (or 1xn) matrices, then it would be nice if we could write M[P==Z,:] to obtain all columns and only those rows where P==Z. Likewise, for 1xm (or mx1) matrices U and V, it would be nice to be able to use M[P==Z,U==V]. Also, it would be nice to use M[P==Z,U==2], for example, to obtain selected rows where matrix U is equal to a constant. -- It would be nice to slice a matrix by using M[[1,2,3],[3,5,7]], for example. I believe this would help make indexing more user friendly. In my humble opinion, I think indexing is a weak spot in numpy. I find that most of my debugging involves finding the right code to make a matrix slice the way I want it. John __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From karol.langner at kn.pl Tue Oct 10 02:46:23 2006 From: karol.langner at kn.pl (Karol Langner) Date: Tue, 10 Oct 2006 08:46:23 +0200 Subject: AMD & UMFpack in numpy In-Reply-To: <200610092015.58322.karol.langner@kn.pl> References: <200610092015.58322.karol.langner@kn.pl> Message-ID: <200610100846.23381.karol.langner@kn.pl> On Monday 09 of October 2006 20:15, Karol Langner wrote: > Can someone give me a hint as to where in numpy the AMD and UMFpack > libraries are used, if at all? I ask, because they have their respective > sections in site.cfg.example in the trunk. > > Karol One more thing - should there be a [fft] section in this file? Karol -- written by Karol Langner Tue Oct 10 08:45:05 CEST 2006 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant.travis at ieee.org Tue Oct 10 02:52:22 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 10 Oct 2006 00:52:22 -0600 Subject: suggestions for slices of matrices In-Reply-To: <20061010061728.12419.qmail@web51709.mail.yahoo.com> References: <20061010061728.12419.qmail@web51709.mail.yahoo.com> Message-ID: <452B4326.4080003@ieee.org> JJ wrote: > Hello. > I haven't been following development too closely > lately, but I did just download and reinstall the > current svn version. For what its worth, I would like > to again suggest two changes: > Suggestions are nice. Working code is better. Many ideas are just too difficult to implement (and still work with the system as it exists) and so never get done. I'm not saying these ideas fit into this category, but generally if a suggestion is not taken it's very likely seen in that light. > -- If M is a nxm matrix and P and Z are nx1 (or 1xn) > matrices, then it would be nice if we could write > M[P==Z,:] to obtain all columns and only those rows > where P==Z. This works already if p and z are 1-d arrays. That seems to be the only issue. you want this to work with p and z being 2-d arrays (i.e. matrices). The problem is there is already a defined behavior for this case that would have to be ignored (i.e. special-cased to get what you want). This could be done within the special matrix sub-class of course, but I'm not sure it is wise. Too many special cases make life difficult down the road. It is better to just un-think the ambiguity between 1-d and 2-d arrays that was inspired by Matlab and recognize a 1-d situation when you have it. But, that's just my opinion. I'm not dead-set against special-casing in the matrix object if enough matrix-oriented people are in favor of it. But, it would be a feature for a later NumPy (not 1.0). > Likewise, for 1xm (or mx1) matrices U and > V, it would be nice to be able to use M[P==Z,U==V]. > Same issue as before + cross-product versus element-by-element. > Also, it would be nice to use M[P==Z,U==2], for > example, to obtain selected rows where matrix U is > equal to a constant. > Again. Form the cross-product using ix_(). > -- It would be nice to slice a matrix by using > M[[1,2,3],[3,5,7]], for example. > You can get the cross-product using M[ix_([1,2,3],[3,5,7])]. This was a design choice and I think a good one. It's been discussed before. > I believe this would help make indexing more user > friendly. In my humble opinion, I think indexing is a > weak spot in numpy. I'm sorry you see it that way. I think indexing is a strength of numpy. It's a little different then what you are used to with Matlab, perhaps, but it is much more general-purpose and capable (there is one weak-spot in that a certain boolean indexing operations uses more memory than it needs to, but that is a separate issue...). The Matlab behavior can always be created in a sub-class. Best regards, -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From a.u.r.e.l.i.a.n at gmx.net Tue Oct 10 04:03:27 2006 From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert) Date: Tue, 10 Oct 2006 10:03:27 +0200 Subject: can this be made faster? In-Reply-To: References: Message-ID: <200610101003.28188.a.u.r.e.l.i.a.n@gmx.net> Hi, > > This seems like a rather common operation - I know I've needed it on > > at least two occasions - is it worth creating some sort of C > > implementation? What is the appropriate generalization? > > Some sort of indirect addressing infrastructure. But it looks like this > could be tricky to make safe, it would need to do bounds checking at the > least and would probably work best with a contiguous array as the target. I > could see some sort of low-level function called argassign(target, indirect > index, source) that could be used to build more complicated things in > python. This looks somehow like the behaviour of builtin map. One could do map(fn, index) with appropriate fn. But iirc this is not faster than a for loop if fn is not a builtin function. An infrastructure like you imagine might use a similar syntax (with underlying C funcs). The main point is, how to tell it which operation to perform (add, multiply, average, whatever). Implementing a bunch of functions add_argassign, ... whatever_argassign contradicts my understanding of "generalized". ;) Maybe it would be simpler to just have functions which handle the index arrays in advance. An example will show it best: index = array([1, 2, 4, 2, 3, 1]) # 1 and 2 occur twice data = array([1, 1, 1, 1, 1, 1]) newindex, newdata = filter_and_add(index, data) # the kind of function I mean print newindex --> array([1, 2, 4, 3]) # duplicates have been removed print newdata --> array([2, 2, 1, 1]) # corresponding entries have been added a[newindex] += newdata Johannes ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From Peter.Bienstman at ugent.be Tue Oct 10 04:31:12 2006 From: Peter.Bienstman at ugent.be (Peter Bienstman) Date: Tue, 10 Oct 2006 10:31:12 +0200 Subject: 1.0rc2: imag attribute of numpy.float64 causes segfault Message-ID: <200610101031.12595.Peter.Bienstman@ugent.be> This is on an AMD64 platform: Python 2.4.3 (#1, Sep 27 2006, 14:14:48) [GCC 4.1.1 (Gentoo 4.1.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> print numpy.__version__ 1.0rc2 >>> a = numpy.float64(1.0) >>> a 1.0 >>> a.real 1.0 >>> a.imag Segmentation fault Thanks! Peter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Tue Oct 10 04:54:52 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 10 Oct 2006 02:54:52 -0600 Subject: 1.0rc2: imag attribute of numpy.float64 causes segfault In-Reply-To: <200610101031.12595.Peter.Bienstman@ugent.be> References: <200610101031.12595.Peter.Bienstman@ugent.be> Message-ID: <452B5FDC.3050106@ieee.org> Peter Bienstman wrote: > This is on an AMD64 platform: > > Python 2.4.3 (#1, Sep 27 2006, 14:14:48) > [GCC 4.1.1 (Gentoo 4.1.1)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>>> import numpy >>>> print numpy.__version__ >>>> > 1.0rc2 > >>>> a = numpy.float64(1.0) >>>> a >>>> > 1.0 > >>>> a.real >>>> > 1.0 > >>>> a.imag >>>> > Segmentation fault > > Thanks for the test. Fixed in SVN r3299 -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From cimrman3 at ntc.zcu.cz Tue Oct 10 05:48:17 2006 From: cimrman3 at ntc.zcu.cz (Robert Cimrman) Date: Tue, 10 Oct 2006 11:48:17 +0200 Subject: AMD & UMFpack in numpy In-Reply-To: <200610092015.58322.karol.langner@kn.pl> References: <200610092015.58322.karol.langner@kn.pl> Message-ID: <452B6C61.6070603@ntc.zcu.cz> Karol Langner wrote: > Can someone give me a hint as to where in numpy the AMD and UMFpack libraries > are used, if at all? I ask, because they have their respective sections in > site.cfg.example in the trunk. AMD and UMFpack are optional parts of scipy.linsolve, so if you do not want them you can freely ignore the entries in site.cfg (or remove them). r. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From c.hanot at septentrio.com Tue Oct 10 07:11:28 2006 From: c.hanot at septentrio.com (Charles Hanot) Date: Tue, 10 Oct 2006 13:11:28 +0200 Subject: Numpy with Py2exe module Message-ID: <8B7FAE730B4A3646B51C94CD6C8784B7B28F48@serversbs2003.Septentrio.local> Hi, my question is the following one. Is it possible to use Numpy with py2exe in order to compile my program? In fact I'm trying to compile a program using py2exe but the problem is that the numpy function cannot be loaded by py2exe. I've already seen on the web that other users have the same problem but I've never seen any answer. Could you tell me if there is something to do? Is there any solution? Thank you very much, Best Regards, Charles Hanot This e-mail communication contains information that is confidential and may also be privileged. It is intended for the exclusive use of the addressees. If you are not the person or organization to whom it is addressed, you must not copy, distribute or take any action in reliance upon it. If you received this communication in error, please notify Septentrio nv immediately [ telephone +32 [0] 16 300800 ]. Septentrio nv will not accept liability for contractual commitments made by individuals employed by this company outside the scope of our business. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.douard at logilab.fr Tue Oct 10 08:35:53 2006 From: david.douard at logilab.fr (David Douard) Date: Tue, 10 Oct 2006 14:35:53 +0200 Subject: Numpy with Py2exe module In-Reply-To: <8B7FAE730B4A3646B51C94CD6C8784B7B28F48@serversbs2003.Septentrio.local> References: <8B7FAE730B4A3646B51C94CD6C8784B7B28F48@serversbs2003.Septentrio.local> Message-ID: <20061010123553.GA12527@crater.logilab.fr> On Tue, Oct 10, 2006 at 01:11:28PM +0200, Charles Hanot wrote: > Hi, Hi > > > > my question is the following one. Is it possible to use Numpy with > py2exe in order to compile my program? > I am right now finishing a dev (GUI app un pyGtk) that uses numpy, and it have to be released as a win32 app. I used to have problems packing it using py2exe with enthough version of numpy. Now I use numpy 1.0RC1 and it works fine (I'm only using numpy, not scipy). I use mpl too, and packaging it using py2exe is a little more problematic. One thing I had to do is to set the "optimize" py2exe option to 0 (because mpl does automatic docstring stuffs, and docstrings are removed when optimizing is used). Note that I am usiing the latest py2exe too. David > > > In fact I'm trying to compile a program using py2exe but the problem is > that the numpy function cannot be loaded by py2exe. I've already seen on > the web that other users have the same problem but I've never seen any > answer. Could you tell me if there is something to do? Is there any > solution? > > > > Thank you very much, > > > > Best Regards, > > > > Charles Hanot > > > > This e-mail communication contains information that is confidential and may also be privileged. It is intended for the exclusive use of the addressees. If you are not the person or organization to whom it is addressed, you must not copy, distribute or take any action in reliance upon it. If you received this communication in error, please notify Septentrio nv immediately [ telephone +32 [0] 16 300800 ]. Septentrio nv will not accept liability for contractual commitments made by individuals employed by this company outside the scope of our business. > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- David Douard LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian : http://www.logilab.fr/formations D?veloppement logiciel sur mesure : http://www.logilab.fr/services Informatique scientifique : http://www.logilab.fr/science -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From josh8912 at yahoo.com Tue Oct 10 09:19:30 2006 From: josh8912 at yahoo.com (jj) Date: Tue, 10 Oct 2006 13:19:30 +0000 (UTC) Subject: suggestions for slices of matrices References: <20061010061728.12419.qmail@web51709.mail.yahoo.com> <452B4326.4080003@ieee.org> Message-ID: > > -- If M is a nxm matrix and P and Z are nx1 (or 1xn) > > matrices, then it would be nice if we could write > > M[P==Z,:] to obtain all columns and only those rows > > where P==Z. > This works already if p and z are 1-d arrays. That seems to be the > only issue. you want this to work with p and z being 2-d arrays (i.e. > matrices). The problem is there is already a defined behavior for this > case that would have to be ignored (i.e. special-cased to get what you > want). This could be done within the special matrix sub-class of > course, but I'm not sure it is wise. Too many special cases make life > difficult down the road. > Thanks for the thoughtful reply Travis. I see I will just have to get used to using code such as M[(P==Z).A.ravel(),:], which I can live with. Your comments helped put this in perspective for me. By the way, is there a web site that lists all the current modules (as does http://www.scipy.org/doc/api_docs/scipy.html)? Best, John ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From aisaac at american.edu Tue Oct 10 09:23:17 2006 From: aisaac at american.edu (Alan G Isaac) Date: Tue, 10 Oct 2006 09:23:17 -0400 Subject: suggestions for slices of matrices In-Reply-To: <452B4326.4080003@ieee.org> References: <20061010061728.12419.qmail@web51709.mail.yahoo.com><452B4326.4080003@ieee.org> Message-ID: > JJ wrote: >> In my humble opinion, I think indexing is a weak spot in >> numpy. On Tue, 10 Oct 2006, Travis Oliphant apparently wrote: > I'm sorry you see it that way. I think indexing is a strength of > numpy. It's a little different then what you are used to > with Matlab, perhaps, but it is much more general-purpose > and capable I am finding numpy indexing to be great, but it would be helpful perhaps to new users to have a few of the examples from this thread make it to the Cookbook http://www.scipy.org/Cookbook/BuildingArrays Sorry, I cannot do that at the moment. Maybe JJ would find this a profitable exercise? Cheers, Alan Isaac ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From c.hanot at septentrio.com Tue Oct 10 11:23:46 2006 From: c.hanot at septentrio.com (Charles Hanot) Date: Tue, 10 Oct 2006 17:23:46 +0200 Subject: Using Py2exe with numpy Message-ID: <8B7FAE730B4A3646B51C94CD6C8784B7B28F4B@serversbs2003.Septentrio.local> Hello, I'm trying to use py2exe but I have a problem :-) . I've already compiled some program with it but this time I have a proble because my program is using Numpy modules. My setup.py is the following from distutils.core import setup import py2exe, sys sys.path.append("tools") sys.path.append("report") sys.path.append("spirent") sys.path.append("numpydir") setup( console = ['checkfile.py'], ) But when I enter the command "python setup.py py2exe", lots of numpy files appear to be missing. Therefore, my executable file does not work. Is there something special to do with the setup.py in order to import numpy modules? Thank you very much for you help Best Regards This e-mail communication contains information that is confidential and may also be privileged. It is intended for the exclusive use of the addressees. If you are not the person or organization to whom it is addressed, you must not copy, distribute or take any action in reliance upon it. If you received this communication in error, please notify Septentrio nv immediately [ telephone +32 [0] 16 300800 ]. Septentrio nv will not accept liability for contractual commitments made by individuals employed by this company outside the scope of our business. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ddrake at brontes3d.com Tue Oct 10 12:34:50 2006 From: ddrake at brontes3d.com (Daniel Drake) Date: Tue, 10 Oct 2006 12:34:50 -0400 Subject: Constructing an array from memory address Message-ID: <1160498090.31210.12.camel@systems03.lan.brontes3d.com> Hi, I have an area of memory which is shared between processes (it is actually a shared memory segment). The address of this memory is stored in a python long variable, which I pass to various custom C/C++ python modules. I would like to construct a numpy array in this area. Is there any way I can tell numpy to use a specific address (stored as a long) to use as storage for the array? Is there any interest in adding this? Thanks, Daniel ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From Chris.Barker at noaa.gov Tue Oct 10 12:47:15 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Tue, 10 Oct 2006 09:47:15 -0700 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <452A040F.3050708@obs.univ-lyon1.fr> References: <452A040F.3050708@obs.univ-lyon1.fr> Message-ID: <452BCE93.60605@noaa.gov> Eric Emsellem wrote: > I have problems getting binaries for wxPython (Suse10.1) and I > don't want to attempt a full compilation of that package... Di you try the rpms at sourceforge? one of them may well be compatible. If not, I've had good luck with building the source rpms: rpmbuild --rebuild NameOfRPM.srpm > installed Suse10.1 from the downloadable version and many many > packages are missing there - That would be a trick -- you'll need relevant the dev packages. But that sounds like an issue you'll be coming up against over and over again anyway. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From Chris.Barker at noaa.gov Tue Oct 10 12:59:53 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Tue, 10 Oct 2006 09:59:53 -0700 Subject: PIL with numpy support! Message-ID: <452BD189.5080300@noaa.gov> Hi all: Fredrik Lundh wrote: > A little later that planned, but PIL 1.1.6 beta 2 is now available from SVN: > > http://svn.effbot.python-hosting.com/tags/pil-1.1.6b2/ > > A tarball will appear on effbot.org shortly: > > http://effbot.org/downloads/#Imaging > > As usual, PIL 1.1.6 supports all Python versions from 1.5.2 and onwards, > including 2.5. > > For a hopefully complete list of changes, see: > > http://effbot.org/zone/pil-changes-116.htm From there: """ * Added "fromarray" function, which takes an object implementing the NumPy array interface and creates a PIL Image from it. (from Travis Oliphant). * Added NumPy array interface support (__array_interface__) to the Image class (based on code by Travis Oliphant). This allows you to easily convert between PIL image memories and NumPy arrays: import numpy, Image i = Image.open('lena.jpg') a = numpy.asarray(i) # a is readonly i = Image.fromarray(a) """ I hope some of us numpy users will be able to test this new functionality while it's in beta. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From charlesr.harris at gmail.com Tue Oct 10 13:26:01 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 10 Oct 2006 11:26:01 -0600 Subject: PIL with numpy support! In-Reply-To: <452BD189.5080300@noaa.gov> References: <452BD189.5080300@noaa.gov> Message-ID: On 10/10/06, Christopher Barker wrote: > > Hi all: > > Fredrik Lundh wrote: > > A little later that planned, but PIL 1.1.6 beta 2 is now available from > SVN: > > > > http://svn.effbot.python-hosting.com/tags/pil-1.1.6b2/ > > > > A tarball will appear on effbot.org shortly: > > > > http://effbot.org/downloads/#Imaging > > > > As usual, PIL 1.1.6 supports all Python versions from 1.5.2 and onwards, > > including 2.5. > > > > For a hopefully complete list of changes, see: > > > > http://effbot.org/zone/pil-changes-116.htm > > From there: > > """ > * Added "fromarray" function, which takes an object implementing the > NumPy array interface and creates a PIL Image from it. (from Travis > Oliphant). > > * Added NumPy array interface support (__array_interface__) to the Image > class (based on code by Travis Oliphant). This allows you to easily > convert between PIL image memories and NumPy arrays: > import numpy, Image > > i = Image.open('lena.jpg') > a = numpy.asarray(i) # a is readonly > i = Image.fromarray(a) > """ Fromarray wasn't there for me running latest PIL from svn last week. I had to use another function whose name escapes me at the moment (I don't use PIL very often), but yes, there is a way to use numpy arrays in PIL. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Oct 10 13:30:12 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 10 Oct 2006 11:30:12 -0600 Subject: PIL with numpy support! In-Reply-To: References: <452BD189.5080300@noaa.gov> Message-ID: On 10/10/06, Charles R Harris wrote: > > > > On 10/10/06, Christopher Barker wrote: > > > > Hi all: > > > > Fredrik Lundh wrote: > > > A little later that planned, but PIL 1.1.6 beta 2 is now available > > from SVN: > > > > > > http://svn.effbot.python-hosting.com/tags/pil-1.1.6b2/ > > > > > > A tarball will appear on effbot.org shortly: > > > > > > http://effbot.org/downloads/#Imaging > > > > > > As usual, PIL 1.1.6 supports all Python versions from 1.5.2 and > > onwards, > > > including 2.5. > > > > > > For a hopefully complete list of changes, see: > > > > > > http://effbot.org/zone/pil-changes-116.htm > > > > From there: > > > > """ > > * Added "fromarray" function, which takes an object implementing the > > NumPy array interface and creates a PIL Image from it. (from Travis > > Oliphant). > > > > * Added NumPy array interface support (__array_interface__) to the Image > > class (based on code by Travis Oliphant). This allows you to easily > > convert between PIL image memories and NumPy arrays: > > import numpy, Image > > > > i = Image.open('lena.jpg') > > a = numpy.asarray(i) # a is readonly > > i = Image.fromarray(a) > > """ > > > Fromarray wasn't there for me running latest PIL from svn last week. I had > to use another function whose name escapes me at the moment (I don't use PIL > very often), but yes, there is a way to use numpy arrays in PIL. > Ah, I recall now, I used Image.frombuffer(...) or some such. This is probably a problem if the numpy array is not of the right type. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez.net at gmail.com Tue Oct 10 15:33:36 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 10 Oct 2006 13:33:36 -0600 Subject: A mean docstring? Message-ID: Hi all, I was updating some old Numeric code to be more 'numpythonic' (it works, but uses a lot of Numeric-style cruft), and read the a.mean() docstring that says: If axis is None, this equals: a.sum(axis, dtype) * 1.0 / product(a.shape,axis=0) Is this convoluted form of the denominator really needed? Isn't it equivalent to a.size? A few quick tests (and my understanding of numpy's new attributes) seem to indicate so: In [78]: a=rand(1,2,3,4); product(a.shape,axis=0)==a.size Out[78]: True In [79]: a=rand(2,3,4,2); product(a.shape,axis=0)==a.size Out[79]: True In [80]: a=rand(2); product(a.shape,axis=0)==a.size Out[80]: True In [81]: a=rand(2,4,4); product(a.shape,axis=0)==a.size Out[81]: True So my question is two-fold (assuming the above is correct in all cases): 1. Should the docstring be changed to simply say If axis is None, this equals: a.sum(axis, dtype) * 1.0 / a.size This is clearer to readers, and promotes the use of .size as a quick and correct way of getting an array's size. 2. Does the implementation actually used the more complicated product() approach, or does it just read .size? I'm wondering about efficiency. Regards, f ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From humufr at yahoo.fr Tue Oct 10 15:41:28 2006 From: humufr at yahoo.fr (humufr at yahoo.fr) Date: Tue, 10 Oct 2006 15:41:28 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <452BCE93.60605@noaa.gov> References: <452A040F.3050708@obs.univ-lyon1.fr> <452BCE93.60605@noaa.gov> Message-ID: <200610101541.28438.humufr@yahoo.fr> I asked if that will be possible to use ipython instead of the python console in eric4 (I know that it's not possible with eric3) but it's seems that eric4 does have it's own console. So, at least for me, it's not anymore an option. N. Le mardi 10 octobre 2006 12:47, Christopher Barker a ?crit?: > Eric Emsellem wrote: > > I have problems getting binaries for wxPython (Suse10.1) and I > > don't want to attempt a full compilation of that package... > > Di you try the rpms at sourceforge? one of them may well be compatible. > > If not, I've had good luck with building the source rpms: > > rpmbuild --rebuild NameOfRPM.srpm > > > installed Suse10.1 from the downloadable version and many many > > packages are missing there - > > That would be a trick -- you'll need relevant the dev packages. But that > sounds like an issue you'll be coming up against over and over again > anyway. > > -Chris ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From dd55 at cornell.edu Tue Oct 10 16:30:34 2006 From: dd55 at cornell.edu (Darren Dale) Date: Tue, 10 Oct 2006 16:30:34 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <200610101541.28438.humufr@yahoo.fr> References: <452A040F.3050708@obs.univ-lyon1.fr> <452BCE93.60605@noaa.gov> <200610101541.28438.humufr@yahoo.fr> Message-ID: <200610101630.35272.dd55@cornell.edu> On Tuesday 10 October 2006 15:41, humufr at yahoo.fr wrote: > I asked if that will be possible to use ipython instead of the python > console in eric4 (I know that it's not possible with eric3) but it's seems > that eric4 does have it's own console. So, at least for me, it's not > anymore an option. Why do you say that? I just installed the latest development snapshots, and Eric4 does have its own console. I haven't been able to set ipython as the custom interpretter, but I will ask the developer about it. Darren ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From ndarray at mac.com Tue Oct 10 17:13:55 2006 From: ndarray at mac.com (Sasha) Date: Tue, 10 Oct 2006 17:13:55 -0400 Subject: A mean docstring? In-Reply-To: References: Message-ID: On 10/10/06, Fernando Perez wrote: > ... > 1. Should the docstring be changed to simply say > > If axis is None, this equals: > a.sum(axis, dtype) * 1.0 / a.size > I would think so. In addition, at the expense of being slightly inaccurate, I would change that to simply a.sum(axis, dtype)/a.size . This will only be inaccurate if dtype is integer and true division is not enabled. If you want to be pedantic, a.sum(axis, dtype)/float(a.size) would still read better. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From oliphant at ee.byu.edu Tue Oct 10 20:36:49 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 10 Oct 2006 18:36:49 -0600 Subject: Constructing an array from memory address In-Reply-To: <1160498090.31210.12.camel@systems03.lan.brontes3d.com> References: <1160498090.31210.12.camel@systems03.lan.brontes3d.com> Message-ID: <452C3CA1.8050608@ee.byu.edu> Daniel Drake wrote: >Hi, > >I have an area of memory which is shared between processes (it is >actually a shared memory segment). The address of this memory is stored >in a python long variable, which I pass to various custom C/C++ python >modules. > >I would like to construct a numpy array in this area. Is there any way I >can tell numpy to use a specific address (stored as a long) to use as >storage for the array? > This functionality is already available. There are two ways to do it. 1) Create an object with an __array_interface__ attribute that returns a dictionary with shape, typestr, and data fields. The data field should return the tuple (memory-address-as-long, True-if-read-only-otherwise-False). Passing this object into the asarray constructor will use the memory given 2) Use the asbuffer function in numpy.core.multiarray (it's not exposed to numpy yet) to create a buffer object from the memory address and then use frombuffer to create an array from the memory. Good luck, -Travis ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV From ktrxclfi at sysalli.com Wed Oct 11 01:18:58 2006 From: ktrxclfi at sysalli.com (GRX Graphics) Date: Wed, 11 Oct 2006 03:18:58 -0200 Subject: hereLose Message-ID: <000f01c6ecd3$3a0ccd90$46853551@nomordinateur> Iconutils complete cursor single listing. Rocker Roll Fame of reopened attempt is manner Bands? Who Onlineour usersthe. Unsigned Glasgow sigend Scotland songs Winif vocals gives a. Gale Tristan Roetsch Gross Alpine Skiing Short Speed Skating a. Andprint font a Rowe filesif! Your island getaway valcohol xx cdabbyy. Xerces xsv Libxml Msxml Msxmlnet. Darkness Forces Chaos Marines Eldar Imperial Guard Necrons Orks is Empire Tyranids Hunters eye Terror Medusa. Jose is Mercury Mike Dorning in Three polls Monday political House. Wikipedia continued to of bein computing console of physical operate consoles typical terminal. Help am relevant focused dialog Type Token scheme configure a tokens Saxonb. Talk post. Vincent? Piano womanby of Niki Dandrea Followed Ledby. Hide. Listings siteif unsure in shareware? Nba of. Cnet Tucows Cows Zdnet in Juke -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: allthe.gif Type: image/gif Size: 12199 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From steve at shrogers.com Tue Oct 10 23:38:27 2006 From: steve at shrogers.com (Steven H. Rogers) Date: Tue, 10 Oct 2006 21:38:27 -0600 Subject: libblas error with NumPy 1.0rc2 on Python 2.5 and Red Hat 9 Message-ID: <452C6733.40705@shrogers.com> Is there a specific libblas required by NumPy? I just installed 1.0rc2 and get this error. Python 2.5 (r25:51908, Sep 21 2006, 17:18:24) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/site-packages/numpy/__init__.py", line 40, in import linalg File "/usr/local/lib/python2.5/site-packages/numpy/linalg/__init__.py", line 4, in from linalg import * File "/usr/local/lib/python2.5/site-packages/numpy/linalg/linalg.py", line 25, in from numpy.linalg import lapack_lite ImportError: /usr/lib/libblas.so.3: undefined symbol: e_wsfe Regards, Steve -- Steven H. Rogers, Ph.D., steve at shrogers.com Weblog: http://shrogers.com/weblog "He who refuses to do arithmetic is doomed to talk nonsense." -- John McCarthy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From karol.langner at kn.pl Wed Oct 11 03:43:44 2006 From: karol.langner at kn.pl (Karol Langner) Date: Wed, 11 Oct 2006 09:43:44 +0200 Subject: libblas error with NumPy 1.0rc2 on Python 2.5 and Red Hat 9 In-Reply-To: <452C6733.40705@shrogers.com> References: <452C6733.40705@shrogers.com> Message-ID: <200610110943.44245.karol.langner@kn.pl> On Wednesday 11 of October 2006 05:38, Steven H. Rogers wrote: > Is there a specific libblas required by NumPy? I just installed 1.0rc2 > and get this error. > > Python 2.5 (r25:51908, Sep 21 2006, 17:18:24) > [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > > >>> import numpy > > Traceback (most recent call last): > File "", line 1, in > File "/usr/local/lib/python2.5/site-packages/numpy/__init__.py", line > 40, in > import linalg > File > "/usr/local/lib/python2.5/site-packages/numpy/linalg/__init__.py", line > 4, in > from linalg import * > File "/usr/local/lib/python2.5/site-packages/numpy/linalg/linalg.py", > line 25, in > from numpy.linalg import lapack_lite > ImportError: /usr/lib/libblas.so.3: undefined symbol: e_wsfe > > Regards, > Steve I've seen this a couple of times - e_wsfe is a Fortran I/O symbol from the gcc system and should be made available by libg2c. My guess if that you have inconsistencies in the compilers used, i .e. a different one was used for building the blas libraries by RH and by you for numpy. Try adding g2c to the list of libraries. There's a comment about it in the scipy FAQ: http://www.scipy.org/FAQ#head-26562f0a9e046b53eae17de300fc06408f9c91a8 Karol -- written by Karol Langner Wed Oct 11 09:35:15 CEST 2006 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at distip.com.au Wed Oct 11 04:08:57 2006 From: david at distip.com.au (David Novakovic) Date: Wed, 11 Oct 2006 18:08:57 +1000 Subject: incrementing along a diagonal Message-ID: <452CA699.4030305@distip.com.au> Hi, i'm moving some old perl PDL code to python. I've come across a line which changes values in a diagonal line accross a matrix. matrix.diagonal() returns a list of values, but making changes to these does not reflect in the original (naturally). I'm just wondering if there is a way that i can increment all the values along a diagonal? Cheers Dave P.S i wasnt sure how to sign to the mailing list - so i'd appreciate being CC'd in any replies ;) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.u.r.e.l.i.a.n at gmx.net Wed Oct 11 04:35:32 2006 From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert) Date: Wed, 11 Oct 2006 10:35:32 +0200 Subject: incrementing along a diagonal In-Reply-To: <452CA699.4030305@distip.com.au> References: <452CA699.4030305@distip.com.au> Message-ID: <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> > I'm just wondering if there is a way that i can increment all the values > along a diagonal? Assume you want to change mat. # min() only necessary for non-square matrices index = arange(min(mat.shape[0], mat.shape[1])) # add 1 to each diagonal element matrix[index, index] += 1 # add some other stuff matrix[index, index] += some_array_shaped_like_index HTH, Johannes ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From markbak at gmail.com Wed Oct 11 05:06:24 2006 From: markbak at gmail.com (Mark Bakker) Date: Wed, 11 Oct 2006 11:06:24 +0200 Subject: select part of array using two conditions Message-ID: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> Hello - I want to select part of an array using two conditions. I know how to do it with one condition (and it works great), but when I use two conditions I get an error message? This is probably easy, but I cannot figure it out. Thanks for any help, Mark >>> a = arange(10) >>> a array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) >>> a[ a>2 ] array([3, 4, 5, 6, 7, 8, 9]) >>> a[ a>2 and a<8 ] Traceback (most recent call last): File "", line 1, in ? a[ a>2 and a<8 ] ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From faltet at carabos.com Wed Oct 11 05:20:50 2006 From: faltet at carabos.com (Francesc Altet) Date: Wed, 11 Oct 2006 11:20:50 +0200 Subject: select part of array using two conditions In-Reply-To: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> References: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> Message-ID: <1160558451.3967.17.camel@localhost.localdomain> El dc 11 de 10 del 2006 a les 11:06 +0200, en/na Mark Bakker va escriure: > Hello - > > I want to select part of an array using two conditions. > I know how to do it with one condition (and it works great), but when > I use two conditions I get an error message? > This is probably easy, but I cannot figure it out. > Thanks for any help, Mark > > >>> a = arange(10) > >>> a > array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) > >>> a[ a>2 ] > array([3, 4, 5, 6, 7, 8, 9]) > >>> a[ a>2 and a<8 ] > Traceback (most recent call last): > File "", line 1, in ? > a[ a>2 and a<8 ] > ValueError: The truth value of an array with more than one element is > ambiguous. Use a.any() or a.all() Yeah, this is a common error on people that is not used to these kind of selections. You have to use the boolean binary operators (and not the logical operators) for doing this: >>> a[(a>2) & (a<8)] Notice the parenthesis. They are necessary because the operator & has more precedence than < or >. -- >0,0< Francesc Altet http://www.carabos.com/ V V C?rabos Coop. V. Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From nwagner at iam.uni-stuttgart.de Wed Oct 11 05:29:26 2006 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Wed, 11 Oct 2006 11:29:26 +0200 Subject: select part of array using two conditions In-Reply-To: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> References: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> Message-ID: <452CB976.6050809@iam.uni-stuttgart.de> Mark Bakker wrote: > Hello - > > I want to select part of an array using two conditions. > I know how to do it with one condition (and it works great), but when > I use two conditions I get an error message? > This is probably easy, but I cannot figure it out. > Thanks for any help, Mark > > >>> a = arange(10) > >>> a > array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) > >>> a[ a>2 ] > array([3, 4, 5, 6, 7, 8, 9]) > >>> a[ a>2 and a<8 ] > Traceback (most recent call last): > File "", line 1, in ? > a[ a>2 and a<8 ] > ValueError: The truth value of an array with more than one element is > ambiguous. Use a.any() or a.all() > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > a[ (a>2) & (a<8) ] Nils ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From humufr at yahoo.fr Wed Oct 11 09:49:04 2006 From: humufr at yahoo.fr (humufr at yahoo.fr) Date: Wed, 11 Oct 2006 09:49:04 -0400 Subject: ide for python/numpy/scipy/mpl, development ? In-Reply-To: <200610101630.35272.dd55@cornell.edu> References: <452A040F.3050708@obs.univ-lyon1.fr> <200610101541.28438.humufr@yahoo.fr> <200610101630.35272.dd55@cornell.edu> Message-ID: <200610110949.04852.humufr@yahoo.fr> Le mardi 10 octobre 2006 16:30, Darren Dale a ?crit?: > On Tuesday 10 October 2006 15:41, humufr at yahoo.fr wrote: > > I asked if that will be possible to use ipython instead of the python > > console in eric4 (I know that it's not possible with eric3) but it's > > seems that eric4 does have it's own console. So, at least for me, it's > > not anymore an option. > > Why do you say that? I just installed the latest development snapshots, and > Eric4 does have its own console. I haven't been able to set ipython as the > custom interpretter, but I will ask the developer about it. > > Darren I asked on the mail list and this is the answer from the developer: On Tuesday 10 October 2006 17:33, humufr at yahoo.fr wrote: > Hi, > > I would like to know if that will be possible to use ipython with eric4 > instead of the normal python console? > > Thanks, > > N. > _______________________________________________ No. Eric uses it's own console, which talks to a remote Python (or Ruby) interpreter via a network socket. Regards, Detlev -- Detlev Offenbach detlev at die-offenbachs.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Michael_OKeefe at nrel.gov Wed Oct 11 09:50:25 2006 From: Michael_OKeefe at nrel.gov (O'Keefe, Michael) Date: Wed, 11 Oct 2006 07:50:25 -0600 Subject: warning upon running numpy rc02 tests In-Reply-To: References: Message-ID: Just FYI, I got the following warning while running the unittests from RC02: Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information. **************************************************************** Personal firewall software may warn about the connection IDLE makes to its subprocess using this computer's internal loopback interface. This connection is not visible on any external interface and no data is sent to or received from the Internet. **************************************************************** IDLE 1.2 >>> from numpy import test >>> from numpy import __version__ >>> __version__ '1.0rc2' >>> test() Found 5 tests for numpy.distutils.misc_util Found 3 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 32 tests for numpy.linalg Found 13 tests for numpy.core.umath Found 4 tests for numpy.core.scalarmath Found 9 tests for numpy.lib.arraysetops Found 4 tests for numpy.ctypeslib Found 42 tests for numpy.lib.type_check Warning: FAILURE importing tests for C:\Python25\Lib\site-packages\numpy\core\tests\test_multiarray.py:355: ImportError: No module named test_unicode (in ) Found 3 tests for numpy.fft.helper Found 36 tests for numpy.core.ma Found 1 tests for numpy.lib.ufunclike Found 12 tests for numpy.lib.twodim_base Found 10 tests for numpy.core.defmatrix Found 41 tests for numpy.lib.function_base Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 26 tests for numpy.core.numeric Found 4 tests for numpy.lib.index_tricks Found 47 tests for numpy.lib.shape_base Found 0 tests for __main__ ............................................................................................................................................................................................................................................................................................................................................. ---------------------------------------------------------------------- Ran 333 tests in 2.687s OK ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From klemm at phys.ethz.ch Wed Oct 11 10:24:24 2006 From: klemm at phys.ethz.ch (Hanno Klemm) Date: Wed, 11 Oct 2006 16:24:24 +0200 Subject: Compile with atlas 3.7.17 fails Message-ID: Hi, I don't know if this is a bug or just me doing something wrong (I suspect the latter). I try to compile numpy-1.0rc1 with python2.5 and atlas 3.7.17. I have build the atlas library myself, it doesn't give any errors under make test or make pttest, so it seems to be okay. if I try to build numpy I get the following error: creating build/temp.linux-x86_64-2.5/numpy/core/blasdot compile options: '-DATLAS_INFO="\"3.7.17\"" -Inumpy/core/blasdot -I/scratch/python2.5/include -Inumpy/core/include -Ibuild/src.linux-x86_64-2.5/numpy/core -Inumpy/core/src -Inumpy/core/include -I/scratch/python2.5/include/python2.5 -c' gcc: numpy/core/blasdot/_dotblas.c gcc -pthread -shared build/temp.linux-x86_64-2.5/numpy/core/blasdot/_dotblas.o -L/scratch/python2.5/lib -lcblas -latlas -o build/lib.linux-x86_64-2.5/numpy/core/_dotblas.so /usr/bin/ld: /scratch/python2.5/lib/libcblas.a(cblas_dgemm.o): relocation R_X86_64_32 can not be used when making a shared object; recompile with -fPIC /scratch/python2.5/lib/libcblas.a: could not read symbols: Bad value collect2: ld returned 1 exit status /usr/bin/ld: /scratch/python2.5/lib/libcblas.a(cblas_dgemm.o): relocation R_X86_64_32 can not be used when making a shared object; recompile with -fPIC /scratch/python2.5/lib/libcblas.a: could not read symbols: Bad value collect2: ld returned 1 exit status error: Command "gcc -pthread -shared build/temp.linux-x86_64-2.5/numpy/core/blasdot/_dotblas.o -L/scratch/python2.5/lib -lcblas -latlas -o build/lib.linux-x86_64-2.5/numpy/core/_dotblas.so" failed with exit status 1 I then tried to recompile atlas with the configure options as: configure --cflags=-fPIC --with-netlib-lapack=/scratch/src/python_dist/LAPACK --prefix=/scratch/python2.5 but the error still remains. I'm on a 2 CPU Intel Xeon machine with gcc 3.2.3. Any hints? Hanno -- Hanno Klemm klemm at phys.ethz.ch ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Wed Oct 11 12:01:11 2006 From: aisaac at american.edu (Alan G Isaac) Date: Wed, 11 Oct 2006 12:01:11 -0400 Subject: naive RNG question Message-ID: Python's MT documentation exmphasize the period of the MT19937 algorithm but discusses not at all the seed size. The numpy documentation contains no commentary (I believe). Speaking from a position of utter RNG ignorance, seed size seems really important too: why is it not discussed? I noticed this algorithm http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/emt19937ar.html provides explicitly for seed size choice, which suggests that some documentation in this area would not be irrelevant (and also addresses some other interesting technical issues). Just trying to understand a bit better. Thanks in advance for any helpful comments or URLs, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Wed Oct 11 12:00:02 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Wed, 11 Oct 2006 09:00:02 -0700 Subject: asmatrix and asarray exception Message-ID: This works: >> M.asmatrix(['a', 'b', None]) matrix([[a, b, None]], dtype=object) But this doesn't: >> M.asmatrix(['a', 'b', None, 'c']) TypeError: expected a readable buffer object >> M.__version__ '1.0rc1' It also doesn't work for asarray and for tuples. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Wed Oct 11 12:06:32 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Wed, 11 Oct 2006 09:06:32 -0700 Subject: asmatrix and asarray exception In-Reply-To: References: Message-ID: On 10/11/06, Keith Goodman wrote: > This works: > > >> M.asmatrix(['a', 'b', None]) > matrix([[a, b, None]], dtype=object) > > But this doesn't: > > >> M.asmatrix(['a', 'b', None, 'c']) > TypeError: expected a readable buffer object > > >> M.__version__ > '1.0rc1' > > It also doesn't work for asarray and for tuples. > Here's a workaround: >> M.asmatrix(['a', 'b', None, 'c'], dtype=object) matrix([[a, b, None, c]], dtype=object) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 11 12:43:47 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 10:43:47 -0600 Subject: naive RNG question In-Reply-To: References: Message-ID: On 10/11/06, Alan G Isaac wrote: > > Python's MT documentation exmphasize the period of the > MT19937 algorithm but discusses not at all the seed size. > The numpy documentation contains no commentary (I believe). > Speaking from a position of utter RNG ignorance, seed size > seems really important too: why is it not discussed? > I noticed this algorithm > http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/emt19937ar.html > provides explicitly for seed size choice, which suggests > that some documentation in this area would not be irrelevant > (and also addresses some other interesting technical issues). The maximum seed size is 624 32bit words. No doubt the documentation should mention this. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 11 12:46:16 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 10:46:16 -0600 Subject: asmatrix and asarray exception In-Reply-To: References: Message-ID: <452D1FD8.3030203@ee.byu.edu> Keith Goodman wrote: >On 10/11/06, Keith Goodman wrote: > > >>This works: >> >> >> >>>>M.asmatrix(['a', 'b', None]) >>>> >>>> >>matrix([[a, b, None]], dtype=object) >> >>But this doesn't: >> >> >> >>>>M.asmatrix(['a', 'b', None, 'c']) >>>> >>>> >>TypeError: expected a readable buffer object >> >> >> >>>>M.__version__ >>>> >>>> >>'1.0rc1' >> >>It also doesn't work for asarray and for tuples. >> >> >> > > > It is pretty fragile to rely on NumPy's "detection" of object arrays. The problem is that with the introduction of string, unicode, and record array styles, what is supposed to be an object array is harder to detect. The type-error propagates up from trying to create a record-array (apparently that's what was detected). You can only create record-array items from tuples or objects exposing the buffer interface. It's interesting that the detection algorithm gets thrown off by the addition of an other element. There may be an easy fix there. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From carlwenrich at yahoo.com Wed Oct 11 12:48:33 2006 From: carlwenrich at yahoo.com (Carl Wenrich) Date: Wed, 11 Oct 2006 09:48:33 -0700 (PDT) Subject: cannot import Numeric Message-ID: <20061011164833.95005.qmail@web32908.mail.mud.yahoo.com> The installation of Numpy went well, and numeric.py is in the python site-packages/numpy/core directory. But when I run python, and enter import Numeric, it says no module named Numeric. Please advise. Carl -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 11 12:50:51 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 10:50:51 -0600 Subject: Compile with atlas 3.7.17 fails In-Reply-To: References: Message-ID: <452D20EB.7030506@ee.byu.edu> Hanno Klemm wrote: >Hi, > >I don't know if this is a bug or just me doing something wrong (I >suspect the latter). I try to compile numpy-1.0rc1 with python2.5 and >atlas 3.7.17. > >I have build the atlas library myself, it doesn't give any errors >under make test or make pttest, so it seems to be okay. if I try to >build numpy I get the following error: > >creating build/temp.linux-x86_64-2.5/numpy/core/blasdot >compile options: '-DATLAS_INFO="\"3.7.17\"" -Inumpy/core/blasdot >-I/scratch/python2.5/include -Inumpy/core/include >-Ibuild/src.linux-x86_64-2.5/numpy/core -Inumpy/core/src >-Inumpy/core/include -I/scratch/python2.5/include/python2.5 -c' >gcc: numpy/core/blasdot/_dotblas.c >gcc -pthread -shared >build/temp.linux-x86_64-2.5/numpy/core/blasdot/_dotblas.o >-L/scratch/python2.5/lib -lcblas -latlas -o >build/lib.linux-x86_64-2.5/numpy/core/_dotblas.so >/usr/bin/ld: /scratch/python2.5/lib/libcblas.a(cblas_dgemm.o): >relocation R_X86_64_32 can not be used when making a shared object; >recompile with -fPIC > > This may be part of your problem. It's looks like the linker is having a hard time making use of your compiled extension in a shared library. Perhaps you should make sure -fPIC is on when you compile atlas (I'm not sure how to do that --- perhaps setting CCFLAGS environment variable to include -fPIC would help). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dd55 at cornell.edu Wed Oct 11 12:52:15 2006 From: dd55 at cornell.edu (Darren Dale) Date: Wed, 11 Oct 2006 12:52:15 -0400 Subject: cannot import Numeric In-Reply-To: <20061011164833.95005.qmail@web32908.mail.mud.yahoo.com> References: <20061011164833.95005.qmail@web32908.mail.mud.yahoo.com> Message-ID: <200610111252.15991.dd55@cornell.edu> On Wednesday 11 October 2006 12:48, Carl Wenrich wrote: > The installation of Numpy went well, and numeric.py is in the python > site-packages/numpy/core directory. But when I run python, and enter import > Numeric, it says no module named Numeric. Please advise. import numpy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 11 12:52:49 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 10:52:49 -0600 Subject: asmatrix and asarray exception In-Reply-To: References: Message-ID: On 10/11/06, Keith Goodman wrote: > > On 10/11/06, Keith Goodman wrote: > > This works: > > > > >> M.asmatrix(['a', 'b', None]) > > matrix([[a, b, None]], dtype=object) > > > > But this doesn't: > > > > >> M.asmatrix(['a', 'b', None, 'c']) > > TypeError: expected a readable buffer object > > As a side observation, I note that the 'None' is also non-printing: >>> a = asarray(['a', 'b', None, 'c'], dtype=object) >>> a[2] >>> str(a[2]) 'None' I wonder if this should be changed? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From carlwenrich at yahoo.com Wed Oct 11 12:54:54 2006 From: carlwenrich at yahoo.com (Carl Wenrich) Date: Wed, 11 Oct 2006 09:54:54 -0700 (PDT) Subject: cannot import Numeric In-Reply-To: <200610111252.15991.dd55@cornell.edu> References: <200610111252.15991.dd55@cornell.edu> Message-ID: <20061011165454.668.qmail@web32904.mail.mud.yahoo.com> thanks, but actually it's the other applications i want to use that have the 'import Numeric' line in them. i'm sure others have noted this before. what's the normal procedure? Darren Dale wrote: On Wednesday 11 October 2006 12:48, Carl Wenrich wrote: > The installation of Numpy went well, and numeric.py is in the python > site-packages/numpy/core directory. But when I run python, and enter import > Numeric, it says no module named Numeric. Please advise. import numpy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 11 13:04:17 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 11:04:17 -0600 Subject: asmatrix and asarray exception In-Reply-To: References: Message-ID: <452D2411.2010707@ee.byu.edu> Charles R Harris wrote: > > > On 10/11/06, *Keith Goodman* > wrote: > > On 10/11/06, Keith Goodman > wrote: > > This works: > > > > >> M.asmatrix(['a', 'b', None]) > > matrix([[a, b, None]], dtype=object) > > > > But this doesn't: > > > > >> M.asmatrix(['a', 'b', None, 'c']) > > TypeError: expected a readable buffer object > > > > > As a side observation, I note that the 'None' is also non-printing: > > >>> a = asarray(['a', 'b', None, 'c'], dtype=object) > >>> a[2] > >>> str(a[2]) > 'None' > > I wonder if this should be changed? That's Python's decision. You are getting back the None object when you access element a[2]. Thus, there is no way to change it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 13:05:38 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 11:05:38 -0600 Subject: cannot import Numeric In-Reply-To: <20061011165454.668.qmail@web32904.mail.mud.yahoo.com> References: <20061011165454.668.qmail@web32904.mail.mud.yahoo.com> Message-ID: <452D2462.1060007@ee.byu.edu> Carl Wenrich wrote: > thanks, but actually it's the other applications i want to use that > have the 'import Numeric' line in them. i'm sure others have noted > this before. what's the normal procedure? You must install Numeric if a package needs Numeric. As far as Python is concerned NumPy is a separate package. Packages must be "ported" to use numpy. Please encourage the package author to port. Help is available for open source packages. Just ask on the list. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From carlwenrich at yahoo.com Wed Oct 11 13:06:57 2006 From: carlwenrich at yahoo.com (Carl Wenrich) Date: Wed, 11 Oct 2006 10:06:57 -0700 (PDT) Subject: cannot import Numeric In-Reply-To: <452D2462.1060007@ee.byu.edu> References: <452D2462.1060007@ee.byu.edu> Message-ID: <20061011170657.70128.qmail@web32907.mail.mud.yahoo.com> thanks. Travis Oliphant wrote: Carl Wenrich wrote: > thanks, but actually it's the other applications i want to use that > have the 'import Numeric' line in them. i'm sure others have noted > this before. what's the normal procedure? You must install Numeric if a package needs Numeric. As far as Python is concerned NumPy is a separate package. Packages must be "ported" to use numpy. Please encourage the package author to port. Help is available for open source packages. Just ask on the list. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Wed Oct 11 14:04:27 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 12 Oct 2006 03:04:27 +0900 Subject: select part of array using two conditions In-Reply-To: <452CB976.6050809@iam.uni-stuttgart.de> References: <6946b9500610110206j36253aa9u9f775c43d801d5a1@mail.gmail.com> <452CB976.6050809@iam.uni-stuttgart.de> Message-ID: On 10/11/06, Nils Wagner wrote: > Mark Bakker wrote: > > Hello - > > > > I want to select part of an array using two conditions. > > I know how to do it with one condition (and it works great), but when > > I use two conditions I get an error message? > > This is probably easy, but I cannot figure it out. > > Thanks for any help, Mark > > > > >>> a = arange(10) > > >>> a > > array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) > > >>> a[ a>2 ] > > array([3, 4, 5, 6, 7, 8, 9]) > > >>> a[ a>2 and a<8 ] > > Traceback (most recent call last): > > File "", line 1, in ? > > a[ a>2 and a<8 ] > > ValueError: The truth value of an array with more than one element is > > ambiguous. Use a.any() or a.all() > > > > ------------------------------------------------------------------------ > > > a[ (a>2) & (a<8) ] > & is bitwiase and which works fine for the booleans you get back from comparisons like (a>2). So in this case & is ok. For arrays with non-boolean values (any non-zero is True) use logical_and: a[ logical_and(c, d) ] Logical_and works always to give you the boolean result. '&' gives you the bitwise result, which is sometimes equivalent to the boolean result. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 14:41:11 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 12:41:11 -0600 Subject: Things to address for Py3K Message-ID: <452D3AC7.7000805@ee.byu.edu> Hi all, Py3K is undergoing active development. This gives us an opportunity to discuss more significant changes to the language that might improve the experience of NumPy users. We should form a list and start commenting on the py3k mailing lists about what changes would be most helpful for our community. Please provide examples of changes to Python that you think might help us. A couple on my short list 1) Adding a *few* new infix operators. a) an extra multiplication operator to distinguish between element-by-element and dot b) extending 'and' and 'or' to allow element-by-element logical operations or adding && and || 2) Lowering the precedence of & so that a > 8 & a < 10 works as you would expect. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Wed Oct 11 15:09:06 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Wed, 11 Oct 2006 12:09:06 -0700 Subject: Things to address for Py3K In-Reply-To: <452D3AC7.7000805@ee.byu.edu> References: <452D3AC7.7000805@ee.byu.edu> Message-ID: <452D4152.5090709@noaa.gov> Travis Oliphant wrote: > A couple on my short list > > 1) Adding a *few* new infix operators. > > a) an extra multiplication operator to distinguish between > element-by-element and dot > b) extending 'and' and 'or' to allow element-by-element logical > operations or adding && and || > > 2) Lowering the precedence of & so that a > 8 & a < 10 works as you > would expect. Maybe this goes without saying, but: 3) Inclusion of an nd-array type in the standard lib! (or at the very least, an nd-array protocol) -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 15:15:24 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 13:15:24 -0600 Subject: Things to address for Py3K In-Reply-To: <452D4152.5090709@noaa.gov> References: <452D3AC7.7000805@ee.byu.edu> <452D4152.5090709@noaa.gov> Message-ID: <452D42CC.9020404@ee.byu.edu> Christopher Barker wrote: >Travis Oliphant wrote: > > >>A couple on my short list >> >>1) Adding a *few* new infix operators. >> >> a) an extra multiplication operator to distinguish between >>element-by-element and dot >> b) extending 'and' and 'or' to allow element-by-element logical >>operations or adding && and || >> >>2) Lowering the precedence of & so that a > 8 & a < 10 works as you >>would expect. >> >> > >Maybe this goes without saying, but: > >3) Inclusion of an nd-array type in the standard lib! > >(or at the very least, an nd-array protocol) > > Work on an nd-array protocol to extend the buffer protocol is occurring right now. It think this will be better in the end then a standard nd-array type. I think a multi-dimensional object array would at least be a nice step. There are enough differences between lists and 1-d arrays though, that I'm not sure the accepted multi-dimensional object array would just be the NumPy version. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From bssjqdbrg at thefa.com Wed Oct 11 18:03:59 2006 From: bssjqdbrg at thefa.com (Our Partners) Date: Wed, 11 Oct 2006 21:03:59 -0100 Subject: sheet wizard create truly Message-ID: <001401c6ed70$64334700$0c40fe3e@your46d615e07a> Here log in you are raquo Magazine Home Todays Compact din Rail. Just anything in several skins choose between own style Creating takes few seconds click get. Create truly unique layout Change every of image Template system Thread a count Total. Modules Devices Embedded Systems Displays Packaging Passive Discrete Power Sources ics is Sensors or Actuators Test or Search Welcome. Charts Webcasts or Tools ecards Search Reviews Infolink Events Peer rss Contact us for. Reviews Infolink Events Peer rss Contact is us for Boards amp Modules is. Sector but recognize they in must meet of stringent Slightly Semi a Forecast With signs of major dropoff this? Profile page Exco simple message advanced features Your hosted by. Archives Related Links web Exclusive Wall Charts Webcasts Tools ecards Search Reviews Infolink in Events in Peer a. Hosted by dontt worry about am backups bandwidth a usage community building is customize just anything several skins in choose between own style. Extras Articles or Plus Features Rohs Central Robust Data key Success ada Managing Complex Clock Networks Satellite Readers. Template system Thread count a Total visitors Whos a online number or users Last Member Personal gallery of images account profile page a Exco. Difficult sector but recognize is they must meet stringent Slightly Semi Forecast With a signs major dropoff a this market research. Made Simple of Rising energy prices focusing consumers design features home operation or. Open door of entry is into difficult is sector but recognize they is must meet stringent Slightly is Semi Forecast With signs is. For Boards amp am Modules Devices Embedded Systems Displays Packaging Passive Discrete in Power Sources or. Made Simple Rising is energy am prices focusing consumers design features home operation a such Consumers Lift Finger many years writers. Message advanced features Your hosted is by of dontt worry or about backups bandwidth usage community building or customize is just anything. Satellite Readers Most or Accessed with or low in Frequency am Usbee Extractor users Visualize -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Memory.gif Type: image/gif Size: 12073 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 11 16:04:13 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 14:04:13 -0600 Subject: Things to address for Py3K In-Reply-To: <452D3AC7.7000805@ee.byu.edu> References: <452D3AC7.7000805@ee.byu.edu> Message-ID: On 10/11/06, Travis Oliphant wrote: > > > Hi all, > > Py3K is undergoing active development. This gives us an opportunity to > discuss more significant changes to the language that might improve the > experience of NumPy users. > > We should form a list and start commenting on the py3k mailing lists > about what changes would be most helpful for our community. > > Please provide examples of changes to Python that you think might help us. > > A couple on my short list > > 1) Adding a *few* new infix operators. > > a) an extra multiplication operator to distinguish between > element-by-element and dot > b) extending 'and' and 'or' to allow element-by-element logical > operations or adding && and || > > 2) Lowering the precedence of & so that a > 8 & a < 10 works as you > would expect. Yes on the extra operators. No on changing the precedence of &, that would just confuse the heck out of all us c/c++ programmers; && and || would be good. Yes to something that improves on the buffer interface, although it is certainly usable right now. Speaking long term, what about data types? The basic 80 bit extended precision float now occurs in 80, 96, and 128 bit versions depending on alignment. So what happens when quad precision, which will probably be in the next IEEE standard, comes down the pike and is 128 bits long? The length of a float will no longer be sufficient to distinguish the various types. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pearu at cens.ioc.ee Wed Oct 11 16:06:14 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Wed, 11 Oct 2006 23:06:14 +0300 (EEST) Subject: Should numpy.sqrt(-1) return 1j rather than nan? Message-ID: Hi, I have recieved the following note from a user: """ In SciPy 0.3.x the ufuncs were overloaded by more "intelligent" versions. A very attractive feature was that sqrt(-1) would yield 1j as in Matlab. Then you can program formulas directly (e.g., roots of a 2nd order polynomial) and the right answer is always achieved. In the Matlab-Python battle in mathematics education, this feature is important. Now in SciPy 0.5.x sqrt(-1) yields nan. A lot of code we have, especially for introductory numerics and physics courses, is now broken. This has already made my colleagues at the University skeptical to Python as "this lack of backward compatibility would never happen in Matlab". Another problem related to Numeric and numpy is that in these courses we use ScientificPython several places, which applies Numeric and will continue to do so. You then easily get a mix of numpy and Numeric in scripts, which may cause problems and at least extra overhead. Just converting to numpy in your own scripts isn't enough if you call up libraries using and returning Numeric. """ I wonder, what are the reasons that numpy.sqrt(-1) returns nan? Could sqrt(-1) made to return 1j again? If not, shouldn't numpy.sqrt(-1) raise a ValueError instead of returning silently nan? Thanks, Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Wed Oct 11 16:13:19 2006 From: gregwillden at gmail.com (Greg Willden) Date: Wed, 11 Oct 2006 15:13:19 -0500 Subject: round Message-ID: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Hi All, I've read discussions in the archives about how round() "rounds to even" and how that is supposedly better. But what I haven't been able to find is "What do I use if I want the regular old round that you learn in school?" Sorry for the likely FAQ. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 11 16:28:19 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 14:28:19 -0600 Subject: round In-Reply-To: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> References: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Message-ID: On 10/11/06, Greg Willden wrote: > > Hi All, > > I've read discussions in the archives about how round() "rounds to even" > and how that is supposedly better. > > But what I haven't been able to find is "What do I use if I want the > regular old round that you learn in school?" Perhaps you could explain *why* you want the schoolbook round? Given that floating point is inherently inaccurate you would have to expect to produce a lot of numbers exactly of the form x.5 *without errors*, which means you probably don't need round to deal with it. Anyway, absent a flag somewhere, you can do something like (x + sign(x)*.5).astype(int). Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From msubotin at gmail.com Wed Oct 11 16:30:42 2006 From: msubotin at gmail.com (Michael Subotin) Date: Wed, 11 Oct 2006 16:30:42 -0400 Subject: Library problem on installation Message-ID: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> Hi, I know that it's a perennial topic on the list, but I haven't been able to find my answer in the archives. After running the installation on a RedHat Linux machine, I'm getting the import error: "/usr/lib/libblas.so.3: undefined symbol: e_wsfe". Judging from earlier exchanges here, it seems that I need to add libg2c (which this machine does have in /usr/lib, unlike g2c) somewhere between 'f77blas' and 'cblas', but I'm not sure where I should make this change. Not being well versed in Python distributions, I tried my luck with a few candidates and the import error remains. The machine should be running gcc. Thanks for any help. Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Wed Oct 11 16:34:03 2006 From: gregwillden at gmail.com (Greg Willden) Date: Wed, 11 Oct 2006 15:34:03 -0500 Subject: round In-Reply-To: References: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Message-ID: <903323ff0610111334m622effedu6c0f2d811cd47794@mail.gmail.com> On 10/11/06, Charles R Harris wrote: > > Perhaps you could explain *why* you want the schoolbook round? Given that > floating point is inherently inaccurate you would have to expect to produce > a lot of numbers exactly of the form x.5 *without errors*, which means you > probably don't need round to deal with it. Anyway, absent a flag somewhere, > you can do something like (x + sign(x)*.5).astype(int). > Yeah. Forget it. That was stupid of me to step into that one. This has obviously been discussed before and you have good reasons for doing it the way you do it. Carry on. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Wed Oct 11 16:35:13 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 12 Oct 2006 05:35:13 +0900 Subject: round In-Reply-To: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> References: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Message-ID: Hmm. I learned "round to even" in school... But another formula that should get you what you want is: floor(x + 0.5).astype(int) --bb On 10/12/06, Greg Willden wrote: > Hi All, > > I've read discussions in the archives about how round() "rounds to even" and > how that is supposedly better. > > But what I haven't been able to find is "What do I use if I want the regular > old round that you learn in school?" > > Sorry for the likely FAQ. > Greg > -- > Linux. Because rebooting is for adding hardware. > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 16:41:25 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 14:41:25 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: <452D56F5.9000404@ee.byu.edu> pearu at cens.ioc.ee wrote: >Hi, > >I have recieved the following note from a user: > >""" >In SciPy 0.3.x the ufuncs were overloaded by more "intelligent" versions. >A very attractive feature was that sqrt(-1) would yield 1j as in Matlab. >Then you can program formulas directly (e.g., roots of a 2nd order >polynomial) and the right answer is always achieved. In the Matlab-Python >battle in mathematics education, this feature is important. > >Now in SciPy 0.5.x sqrt(-1) yields nan. A lot of code we have, especially >for introductory numerics and physics courses, is now broken. >This has already made my colleagues at the University skeptical to >Python as "this lack of backward compatibility would never happen in Matlab". > > This was a consequence of moving scipy_base into NumPy but not exposing the scimath library in NumPy. It would be a very easy thing to put from numpy.lib.scimath import * into the scipy name-space. I'm supportive of that as a backward-compatibility measure. >Another problem related to Numeric and numpy is that in these courses we >use ScientificPython several places, which applies Numeric and will >continue to do so. You then easily get a mix of numpy and Numeric >in scripts, which may cause problems and at least extra overhead. >Just converting to numpy in your own scripts isn't enough if you call >up libraries using and returning Numeric. > > >""" > >I wonder, what are the reasons that numpy.sqrt(-1) returns nan? > > Because that is backwards compatible. You have to construct a function-wrapper in order to handle the negative case correctly. The function wrapper is going to be slower. Thus, it is placed in a separate library. >Could sqrt(-1) made to return 1j again? > Not in NumPy. But, in scipy it could. >If not, shouldn't > > >numpy.sqrt(-1) raise a ValueError instead of returning silently nan? > > This is user adjustable. You change the error mode to raise on 'invalid' instead of pass silently which is now the default. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Wed Oct 11 16:46:59 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Wed, 11 Oct 2006 22:46:59 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D56F5.9000404@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> Message-ID: <452D5843.4050007@gmx.net> Travis Oliphant schrieb: > >> If not, shouldn't >> >> >> numpy.sqrt(-1) raise a ValueError instead of returning silently nan? >> >> > This is user adjustable. You change the error mode to raise on > 'invalid' instead of pass silently which is now the default. > > -Travis > Could you please explain how this adjustment is done, or point to the relevant documentation. Thank you, Sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 17:02:50 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 15:02:50 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D5843.4050007@gmx.net> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> Message-ID: <452D5BFA.7000603@ee.byu.edu> Sven Schreiber wrote: >>This is user adjustable. You change the error mode to raise on >>'invalid' instead of pass silently which is now the default. >> >>-Travis >> >> >> > >Could you please explain how this adjustment is done, or point to the >relevant documentation. > > numpy.sqrt(-1) old = seterr(invalid='raise') numpy.sqrt(-1) # should raise an error seterr(**old) # restores error-modes for current thread numpy.sqrt(-1) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Wed Oct 11 17:34:02 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Wed, 11 Oct 2006 14:34:02 -0700 Subject: Things to address for Py3K In-Reply-To: <452D42CC.9020404@ee.byu.edu> References: <452D3AC7.7000805@ee.byu.edu> <452D4152.5090709@noaa.gov> <452D42CC.9020404@ee.byu.edu> Message-ID: <452D634A.1000607@noaa.gov> Travis Oliphant wrote: >> 3) Inclusion of an nd-array type in the standard lib! >> (or at the very least, an nd-array protocol) >> > Work on an nd-array protocol to extend the buffer protocol is occurring > right now. It think this will be better in the end then a standard > nd-array type. Well, yes, if you have to choose just one. I'd like to see both. > I think a multi-dimensional object array would at least be a nice step. Me too. n-d slicing is so handy. > There are enough differences between lists and 1-d arrays though, that > I'm not sure the accepted multi-dimensional object array would just be > the NumPy version. I guess the question is whether the goal is an n-d list or an n-d array. The key difference that I see is that numpy arrays are not re-sizable. But I have to wonder how practical it is to make an nd-array re-sizable, and, indeed, how that would work. I'm not sure what the n-d version of append() would be. Numpy provides three things that I think are key: 1) An n-d array data structure. These are very useful for lots of things where an multi-dimensional data structure just makes sense. For these cases, you need the object array type (class?), and n-d slicing. 2) array arithmetic: I'm a big fan of list comprehensions, but: A = [x * 2 for x in B] really is a LOT klunkier, and far less clear, than: A = 2 * B or even worse: A = [] for x in B: A.append(2*x) While array arithmetic is often seen primarily as a performance advantage (which, of course, it is) I also think it is a real boon to clear, error-free coding, and Python could use that. 3) A bunch of other really "mathy" or "scientific" functions: (looking in the numpy book here) polynomial functions set operations (hmm -- there is a Python set type) bessel functions smoothing windows fft llinalg etc. etc. These really do belong in numpy (scipy?), rather than the standard lib. However this turn out, thanks for keeping numpy in the py3k loop. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 11 17:37:42 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 11 Oct 2006 17:37:42 -0400 Subject: Things to address for Py3K In-Reply-To: References: <452D3AC7.7000805@ee.byu.edu> Message-ID: On 11/10/06, Charles R Harris wrote: > Speaking long term, what about data types? The basic 80 bit extended > precision float now occurs in 80, 96, and 128 bit versions depending on > alignment. So what happens when quad precision, which will probably be in > the next IEEE standard, comes down the pike and is 128 bits long? The length > of a float will no longer be sufficient to distinguish the various types. This doesn't require python 3K; it can happn in numpy with no language support. But perhaps python 3K can be viewed as a chance to ditch backwards-compatibility baggage? (cough, Numeric compatibility, cough) Would it be of interest to have numeric datatypes integrated with python datatypes? How about IEEE floats in python proper, at least? It can be rather confusing when doing a calculation yields different results for arrays than for ordinary scalars... A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Wed Oct 11 17:37:34 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 11 Oct 2006 15:37:34 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D56F5.9000404@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> Message-ID: On 10/11/06, Travis Oliphant wrote: > pearu at cens.ioc.ee wrote: > >Could sqrt(-1) made to return 1j again? > > > Not in NumPy. But, in scipy it could. Without taking sides on which way to go, I'd like to -1 the idea of a difference in behavior between numpy and scipy. IMHO, scipy should be within reason a strict superset of numpy. Gratuitious differences in behavior like this one are going to drive us all mad. There are people who import scipy for everything, others distinguish between numpy and scipy, others use numpy alone and at some point in their life's code they do import numpy as N -> import scipy as N because they start needing stuff not in plain numpy. Having different APIs and behaviors appear there is, I think, a Seriously Bad Idea (TM). Just my 1e-2j, Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Wed Oct 11 17:52:59 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Wed, 11 Oct 2006 14:52:59 -0700 Subject: Things to address for Py3K In-Reply-To: References: <452D3AC7.7000805@ee.byu.edu> Message-ID: <452D67BB.9090108@noaa.gov> A. M. Archibald wrote: > IEEE floats in python proper +1 -CHB -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 18:19:58 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 16:19:58 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> Message-ID: <452D6E0E.7020503@ee.byu.edu> Fernando Perez wrote: >On 10/11/06, Travis Oliphant wrote: > > > >>pearu at cens.ioc.ee wrote: >> >> >>>Could sqrt(-1) made to return 1j again? >>> >>> >>> >>Not in NumPy. But, in scipy it could. >> >> > >Without taking sides on which way to go, I'd like to -1 the idea of a >difference in behavior between numpy and scipy. > >IMHO, scipy should be within reason a strict superset of numpy. > > This was not the relationship of scipy to Numeric. For me, it's the fact that scipy *used* to have the behavior that scipy.sqrt(-1) return 1j and now doesn't that is the kicker. On the other hand requiring all calls to numpy.sqrt to go through an "argument-checking" wrapper is a bad idea as it will slow down other uses. So, I committed a change to scipy to bring it back into compatibility with 0.3.2 >Gratuitious differences in behavior like this one are going to drive >us all mad. > >There are people who import scipy for everything, others distinguish >between numpy and scipy, others use numpy alone and at some point in >their life's code they do > >import numpy as N -> import scipy as N > >because they start needing stuff not in plain numpy. Having different >APIs and behaviors appear there is, I think, a Seriously Bad Idea >(TM). > > I think the SBI is mixing numpy and scipy gratuitously (which I admit I have done in the past). I'm trying to repent.... -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Wed Oct 11 18:31:21 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 11 Oct 2006 15:31:21 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D5BFA.7000603@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> <452D5BFA.7000603@ee.byu.edu> Message-ID: <452D70B9.7070209@ieee.org> Travis Oliphant wrote: > Sven Schreiber wrote: > > >>> This is user adjustable. You change the error mode to raise on >>> 'invalid' instead of pass silently which is now the default. >>> >>> -Travis >>> >>> >>> >>> >> Could you please explain how this adjustment is done, or point to the >> relevant documentation. >> >> >> > > numpy.sqrt(-1) > > old = seterr(invalid='raise') > numpy.sqrt(-1) # should raise an error > > seterr(**old) # restores error-modes for current thread > numpy.sqrt(-1) > > With python 2.5 out now, perhaps it's time to come up with a with statement context manager. Something like: from __future__ import with_statement import numpy class errstate(object): def __init__(self, **kwargs): self.kwargs = kwargs def __enter__(self): self.oldstate = numpy.seterr(**self.kwargs) def __exit__(self, *exc_info): numpy.seterr(**self.oldstate) a = numpy.arange(10) a/a # ignores divide by zero with errstate(divide='raise'): a/a # raise exception on divide by zer # Would ignore divide by zero again if we got here. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From efiring at hawaii.edu Wed Oct 11 18:46:56 2006 From: efiring at hawaii.edu (Eric Firing) Date: Wed, 11 Oct 2006 12:46:56 -1000 Subject: Things to address for Py3K In-Reply-To: References: <452D3AC7.7000805@ee.byu.edu> Message-ID: <452D7460.6060304@hawaii.edu> Charles R Harris wrote: [...] > b) extending 'and' and 'or' to allow element-by-element logical > operations or adding && and || > > 2) Lowering the precedence of & so that a > 8 & a < 10 works as you > would expect. > > > Yes on the extra operators. No on changing the precedence of &, that > would just confuse the heck out of all us c/c++ programmers; && and || > would be good. Travis's suggestion 2 is consistent with c/c++, where precedence increases from logical to bitwise to relational. (e.g., http://www.cs.niu.edu/~duffin/csci241/precedence.html). Python precedence now increases from logical to relational to bitwise, so it is inconsistent with c/c++. (http://docs.python.org/ref/summary.html#l2h-456) Eric ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Wed Oct 11 18:59:27 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 11 Oct 2006 16:59:27 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D6E0E.7020503@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452D6E0E.7020503@ee.byu.edu> Message-ID: On 10/11/06, Travis Oliphant wrote: > Fernando Perez wrote: > >IMHO, scipy should be within reason a strict superset of numpy. > > > > > This was not the relationship of scipy to Numeric. > > For me, it's the fact that scipy *used* to have the behavior that > > scipy.sqrt(-1) return 1j > > and now doesn't that is the kicker. That's fine, my only point was that we should really strive for consitency between the two. I think most users should be able to expect that numpy.foo(x) == scipy.foo(x) for all cases where foo exists in both. The scipy.foo() call might be faster, or take extra arguments for flexibility, and the above might only be true within floating point accuracy (since a different algorithm may be used), but hopefully functions with the same name do the same thing in both. I really think breaking this will send quite a few potential users running for the hills, and this is what I meant by 'superset'. Perhaps I wasn't clear enough. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Wed Oct 11 19:02:16 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 11 Oct 2006 17:02:16 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D6E0E.7020503@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452D6E0E.7020503@ee.byu.edu> Message-ID: On 10/11/06, Travis Oliphant wrote: > Fernando Perez wrote: > >There are people who import scipy for everything, others distinguish > >between numpy and scipy, others use numpy alone and at some point in > >their life's code they do > > > >import numpy as N -> import scipy as N > > > >because they start needing stuff not in plain numpy. Having different > >APIs and behaviors appear there is, I think, a Seriously Bad Idea > >(TM). > > > > > I think the SBI is mixing numpy and scipy gratuitously (which I admit I > have done in the past). I'm trying to repent.... Well, the problem is that it may not be so easy not to do so, esp. for new users. The fact that scipy absorbs and exposes many numpy functions makes this a particularly easy trap for anyone to fall into. The fact that even seasoned users do it should be an indicator that the 'right thing to do' is anything but obvious, IMHO. Once the dust settles on numpy 1.0, I think that the issues of how scipy plays with it, API consistency, coding best practices, etc, will need serious attention. But let's cross one bridge at a time :) Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Wed Oct 11 19:07:05 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 01:07:05 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> Message-ID: <20061011230705.GA30571@mentat.za.net> On Wed, Oct 11, 2006 at 03:37:34PM -0600, Fernando Perez wrote: > On 10/11/06, Travis Oliphant wrote: > > > pearu at cens.ioc.ee wrote: > > >Could sqrt(-1) made to return 1j again? > > > > > Not in NumPy. But, in scipy it could. > > Without taking sides on which way to go, I'd like to -1 the idea of a > difference in behavior between numpy and scipy. > > IMHO, scipy should be within reason a strict superset of numpy. > Gratuitious differences in behavior like this one are going to drive > us all mad. > > There are people who import scipy for everything, others distinguish > between numpy and scipy, others use numpy alone and at some point in > their life's code they do > > import numpy as N -> import scipy as N > > because they start needing stuff not in plain numpy. Having different > APIs and behaviors appear there is, I think, a Seriously Bad Idea > (TM). I agree with Fernando on this one. Further, if I understand correctly, changing sqrt and power to give the right answer by default will slow things down somewhat. But is it worth sacrificing intuitive usage for speed? N.power(2,-2) == 0 and N.sqrt(-1) == nan just doesn't feel right. Why not then have N.power(2,-2) == 0.24 N.sqrt(-1) == 1j and write a special function that does fast calculation of square-roots for positive values? Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Wed Oct 11 19:13:16 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Thu, 12 Oct 2006 02:13:16 +0300 (EEST) Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D6E0E.7020503@ee.byu.edu> References: <452D6E0E.7020503@ee.byu.edu> Message-ID: On Wed, 11 Oct 2006, Travis Oliphant wrote: > On the other hand requiring all calls to numpy.sqrt to go through an > "argument-checking" wrapper is a bad idea as it will slow down other uses. Interestingly, in worst cases numpy.sqrt is approximately ~3 times slower than scipy.sqrt on negative input but ~2 times faster on positive input: In [47]: pos_input = numpy.arange(1,100,0.001) In [48]: %timeit -n 1000 b=numpy.sqrt(pos_input) 1000 loops, best of 3: 4.68 ms per loop In [49]: %timeit -n 1000 b=scipy.sqrt(pos_input) 1000 loops, best of 3: 10 ms per loop In [50]: neg_input = -pos_input In [52]: %timeit -n 1000 b=numpy.sqrt(neg_input) 1000 loops, best of 3: 99.3 ms per loop In [53]: %timeit -n 1000 b=scipy.sqrt(neg_input) 1000 loops, best of 3: 29.2 ms per loop nan's are making things really slow, Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 19:21:44 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 17:21:44 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <20061011230705.GA30571@mentat.za.net> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> Message-ID: <452D7C88.3090506@ee.byu.edu> Stefan van der Walt wrote: >I agree with Fernando on this one. > >Further, if I understand correctly, changing sqrt and power to give >the right answer by default will slow things down somewhat. But is it >worth sacrificing intuitive usage for speed? > > For NumPy, yes. This is one reason that NumPy by itself is not a MATLAB replacement. >N.power(2,-2) == 0 > >and > >N.sqrt(-1) == nan > >just doesn't feel right. > Only because your expectations are that NumPy *be* a MATLAB replacement. The problem is that it sacrifices too much for that to be the case. And we all realize that NumPy needs more stuff added to it to be like IDL/MATLAB such as SciPy, Matplotlib, IPython, etc. >Why not then have > >N.power(2,-2) == 0.24 >N.sqrt(-1) == 1j > >and write a special function that does fast calculation of >square-roots for positive values? > > We've already done this. The special functions are called numpy.power numpy.sqrt (notice that if you do numpy.sqrt(-1+0j) you get the "expected" answer emphasizing that numpy does no "argument" checking to determine the output). The "intuitive" functions (which must do argument checking) are (in numpy.lib.scimath) but exported as scipy.power (actually I need to check that one...) scipy.sqrt What could be simpler? ;-) -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 19:24:35 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 17:24:35 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: <452D7D33.6070709@ee.byu.edu> pearu at cens.ioc.ee wrote: > >On Wed, 11 Oct 2006, Travis Oliphant wrote: > > > >>On the other hand requiring all calls to numpy.sqrt to go through an >>"argument-checking" wrapper is a bad idea as it will slow down other uses. >> >> > >Interestingly, in worst cases numpy.sqrt is approximately ~3 times slower >than scipy.sqrt on negative input but ~2 times faster on positive input: > >In [47]: pos_input = numpy.arange(1,100,0.001) > >In [48]: %timeit -n 1000 b=numpy.sqrt(pos_input) >1000 loops, best of 3: 4.68 ms per loop > >In [49]: %timeit -n 1000 b=scipy.sqrt(pos_input) >1000 loops, best of 3: 10 ms per loop > > This is the one that concerns me. Slowing everybody down who knows they have positive values just for people that don't seems problematic. >In [50]: neg_input = -pos_input > >In [52]: %timeit -n 1000 b=numpy.sqrt(neg_input) >1000 loops, best of 3: 99.3 ms per loop > >In [53]: %timeit -n 1000 b=scipy.sqrt(neg_input) >1000 loops, best of 3: 29.2 ms per loop > >nan's are making things really slow, > > Yeah, they do. This actually makes the case for masked arrays, rather than using NAN's. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Wed Oct 11 19:24:54 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Wed, 11 Oct 2006 16:24:54 -0700 Subject: RC2 - f2py no workee Message-ID: <452D7D46.6040100@jpl.nasa.gov> I'm running the following python c:\Python24\Scripts\f2py.py --fcompiler=absoft -c foo.pyf foo.f and it seems that the compiler info isn't being passed down. When distutils tries to compile I get the error --------------------------------------------------- File "C:\Python24\Lib\site-packages\numpy\distutils\command\build_ext.py", li e 260, in build_extension f_objects += self.fcompiler.compile(f_sources, AttributeError: 'NoneType' object has no attribute 'compile' ------------------------------------------------ so the fcompiler isn't being set. Any help? Mathew Here is the complete stack trace Traceback (most recent call last): File "c:\Python24\Scripts\f2py.py", line 26, in ? main() File "C:\Python24\Lib\site-packages\numpy\f2py\f2py2e.py", line 552, in main run_compile() File "C:\Python24\Lib\site-packages\numpy\f2py\f2py2e.py", line 539, in run_co mpile setup(ext_modules = [ext]) File "C:\Python24\Lib\site-packages\numpy\distutils\core.py", line 174, in set up return old_setup(**new_attr) File "C:\Python24\lib\distutils\core.py", line 149, in setup dist.run_commands() File "C:\Python24\lib\distutils\dist.py", line 946, in run_commands self.run_command(cmd) File "C:\Python24\lib\distutils\dist.py", line 966, in run_command cmd_obj.run() File "C:\Python24\lib\distutils\command\build.py", line 112, in run self.run_command(cmd_name) File "C:\Python24\lib\distutils\cmd.py", line 333, in run_command self.distribution.run_command(command) File "C:\Python24\lib\distutils\dist.py", line 966, in run_command cmd_obj.run() File "C:\Python24\Lib\site-packages\numpy\distutils\command\build_ext.py", lin e 121, in run self.build_extensions() File "C:\Python24\lib\distutils\command\build_ext.py", line 405, in build_exte nsions self.build_extension(ext) File "C:\Python24\Lib\site-packages\numpy\distutils\command\build_ext.py", lin e 260, in build_extension f_objects += self.fcompiler.compile(f_sources, AttributeError: 'NoneType' object has no attribute 'compile' ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at distip.com.au Wed Oct 11 19:34:07 2006 From: david at distip.com.au (David Novakovic) Date: Thu, 12 Oct 2006 09:34:07 +1000 Subject: incrementing along a diagonal In-Reply-To: <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> References: <452CA699.4030305@distip.com.au> <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> Message-ID: <452D7F6F.2090908@distip.com.au> Johannes Loehnert wrote: >> I'm just wondering if there is a way that i can increment all the values >> along a diagonal? >> > > Assume you want to change mat. > > # min() only necessary for non-square matrices > index = arange(min(mat.shape[0], mat.shape[1])) > # add 1 to each diagonal element > matrix[index, index] += 1 > # add some other stuff > matrix[index, index] += some_array_shaped_like_index > > > HTH, Johannes > > Thank you very much for the prompt reply, I'm just having a problem with this method: This method appears to only work if the matrix is mxm for example: >>> zeros((5,5)) array([[0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]) >>> x = zeros((5,5)) >>> index = arange(min(x.shape[0],x.shape[1])) >>> x[index,index] += 1 >>> x array([[1, 0, 0, 0, 0], [0, 1, 0, 0, 0], [0, 0, 1, 0, 0], [0, 0, 0, 1, 0], [0, 0, 0, 0, 1]]) >>> This is very nice, exactly what i want, but it doesnt work for mxn matricies: >>> x = zeros((5,3)) >>> x array([[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0]]) >>> index = arange(min(x.shape[0],x.shape[1])) >>> x[index,index] += 1 >>> x array([[1, 0, 0], [0, 1, 0], [0, 0, 1], [0, 0, 0], [0, 0, 0]]) >>> So the min part is right for mxn matrices - but perhaps there is a way to use the index differently. I'm very new to numpy, so excuse my noobness :) Just for reference, this is the line of perl i'm trying to port: (my $dummy = $ones->diagonal(0,1))++; # ones is a matrix created with zeroes() Yes, i know it is horribly ugly, but in this case, diagonal returns a list of references to the values in the original matrix, so values can be changed in place. I much prefer python - hence the port, but it seems like a hard thing to replicate. Perhaps there could be a function that returns an iterator over the values in a matrix and returns the index's. like: for index in diag_iter(matrix,*axes): matrix[index] +=1 Once again, cheers - i hope we can figure something out :) Dave Novakovic PS: If anyone would care to link me to the subscription page for the mailing list so you dont have to CC me all the time :) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Wed Oct 11 19:39:57 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Thu, 12 Oct 2006 02:39:57 +0300 (EEST) Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7D33.6070709@ee.byu.edu> References: <452D7D33.6070709@ee.byu.edu> Message-ID: On Wed, 11 Oct 2006, Travis Oliphant wrote: > >Interestingly, in worst cases numpy.sqrt is approximately ~3 times slower > >than scipy.sqrt on negative input but ~2 times faster on positive input: > > > >In [47]: pos_input = numpy.arange(1,100,0.001) > > > >In [48]: %timeit -n 1000 b=numpy.sqrt(pos_input) > >1000 loops, best of 3: 4.68 ms per loop > > > >In [49]: %timeit -n 1000 b=scipy.sqrt(pos_input) > >1000 loops, best of 3: 10 ms per loop > > > > > > This is the one that concerns me. Slowing everybody down who knows they > have positive values just for people that don't seems problematic. I think the code in scipy.sqrt can be optimized from def _fix_real_lt_zero(x): x = asarray(x) if any(isreal(x) & (x<0)): x = _tocomplex(x) return x def sqrt(x): x = _fix_real_lt_zero(x) return nx.sqrt(x) to (untested) def _fix_real_lt_zero(x): x = asarray(x) if not isinstance(x,(nt.csingle,nt.cdouble)) and any(x<0): x = _tocomplex(x) return x def sqrt(x): x = _fix_real_lt_zero(x) return nx.sqrt(x) or def sqrt(x): old = nx.seterr(invalid='raises') try: r = nx.sqrt(x) except FloatingPointError: x = _tocomplex(x) r = nx.sqrt(x) nx.seterr(**old) return r I haven't timed these cases yet.. Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 11 19:47:03 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 11 Oct 2006 17:47:03 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D70B9.7070209@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> <452D5BFA.7000603@ee.byu.edu> <452D70B9.7070209@ieee.org> Message-ID: <452D8277.3080306@ee.byu.edu> Tim Hochberg wrote: >With python 2.5 out now, perhaps it's time to come up with a with >statement context manager. Something like: > > from __future__ import with_statement > import numpy > > class errstate(object): > def __init__(self, **kwargs): > self.kwargs = kwargs > def __enter__(self): > self.oldstate = numpy.seterr(**self.kwargs) > def __exit__(self, *exc_info): > numpy.seterr(**self.oldstate) > > a = numpy.arange(10) > a/a # ignores divide by zero > with errstate(divide='raise'): > a/a # raise exception on divide by zer > # Would ignore divide by zero again if we got here. > >-tim > > > This looks great. I think most people aren't aware of the with statement and what it can do (I'm only aware because of your posts, for example). So, what needs to be added to your example in order to just add it to numpy? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Wed Oct 11 19:58:45 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 12 Oct 2006 08:58:45 +0900 Subject: incrementing along a diagonal In-Reply-To: <452D7F6F.2090908@distip.com.au> References: <452CA699.4030305@distip.com.au> <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> <452D7F6F.2090908@distip.com.au> Message-ID: On 10/12/06, David Novakovic wrote: > Johannes Loehnert wrote: > This is very nice, exactly what i want, but it doesnt work for mxn > matricies: > > >>> x = zeros((5,3)) > >>> x > array([[0, 0, 0], > [0, 0, 0], > [0, 0, 0], > [0, 0, 0], > [0, 0, 0]]) > >>> index = arange(min(x.shape[0],x.shape[1])) > >>> x[index,index] += 1 > >>> x > array([[1, 0, 0], > [0, 1, 0], > [0, 0, 1], > [0, 0, 0], > [0, 0, 0]]) Exactly what output are you expecting? That is the definition of the 'diagonal' for a non-square matrix. If you're expecting something else then what you want is not the diagonal. > Just for reference, this is the line of perl i'm trying to port: > > > like: > > for index in diag_iter(matrix,*axes): > matrix[index] +=1 That's not going to change the mathematical definition of the diagonal of a non-square matrix. > PS: If anyone would care to link me to the subscription page for the > mailing list so you dont have to CC me all the time :) Check the bottom of this message. > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pierregm at engr.uga.edu Wed Oct 11 19:51:18 2006 From: pierregm at engr.uga.edu (Pierre GM) Date: Wed, 11 Oct 2006 19:51:18 -0400 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7D33.6070709@ee.byu.edu> References: <452D7D33.6070709@ee.byu.edu> Message-ID: <200610111951.19154.pierregm@engr.uga.edu> > >nan's are making things really slow, > > Yeah, they do. This actually makes the case for masked arrays, rather > than using NAN's. Travis, Talking about masked arrays, I'm about being done rewriting numpy.core.ma, mainly transforming MaskedArray as a subclass of ndarray (it should be OK by the end of the week), and allowing for an easy subclassing of MaskedArrays (which is far from being the case right now) What would be the best procedure to submit it ? Ticket on SVN ? Wiki on scipy.org ? Thanks again for your time ! Pierre ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Wed Oct 11 20:17:27 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 02:17:27 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7C88.3090506@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> Message-ID: <20061012001727.GQ20657@mentat.za.net> On Wed, Oct 11, 2006 at 05:21:44PM -0600, Travis Oliphant wrote: > Stefan van der Walt wrote: > > >I agree with Fernando on this one. > > > >Further, if I understand correctly, changing sqrt and power to give > >the right answer by default will slow things down somewhat. But is it > >worth sacrificing intuitive usage for speed? > > > > > For NumPy, yes. > > This is one reason that NumPy by itself is not a MATLAB > replacement. Intuitive usage is hopefully not a MATLAB-only feature. > >N.power(2,-2) == 0 > > > >and > > > >N.sqrt(-1) == nan > > > >just doesn't feel right. > > > > Only because your expectations are that NumPy *be* a MATLAB > replacement. The problem is that it sacrifices too much for that to be > the case. And we all realize that NumPy needs more stuff added to it > to be like IDL/MATLAB such as SciPy, Matplotlib, IPython, etc. I have none such expectations -- I havn't used MATLAB in over 5 years. All I'm saying is that, since the value of the square-root of -1 is not nan and 2^(-2) is not 0, it doesn't surprise me that this behaviour confuses people. > The "intuitive" functions (which must do argument checking) are (in > numpy.lib.scimath) but exported as > > scipy.power (actually I need to check that one...) > scipy.sqrt > > What could be simpler? ;-) I'm sure this is going to come back and haunt us. We have two libraries, one depends on and exposes the API of the other, yet it also overrides some functions with its own behaviour, while keeping the same names. I'll shut up now :) Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at distip.com.au Wed Oct 11 20:20:29 2006 From: david at distip.com.au (David Novakovic) Date: Thu, 12 Oct 2006 10:20:29 +1000 Subject: incrementing along a diagonal In-Reply-To: References: <452CA699.4030305@distip.com.au> <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> <452D7F6F.2090908@distip.com.au> Message-ID: <452D8A4D.4060708@distip.com.au> Thanks for the help, i've learnt a lot and also figured out something that does what I want, i'll paste an interactive session below: x = zeros((4,7)) x array([[0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0]]) index = arange(min(x.shape[0], x.shape[1])) index2 = copy.deepcopy(index) #deep copy may be overkill for a,b in enumerate(index): ... index2[a] += a ... if len(x[:,0]) > len(x[0]): ... x[index2,index] +=1 ... else: ... x[index,index2] +=1 ... x array([[1, 0, 0, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0, 0], [0, 0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 0, 0, 1]]) Thanks for the tips Dave Novakovic P.S. subscribed to the list now Bill Baxter wrote: > Forgot to CC you... > > ---------- Forwarded message ---------- > From: Bill Baxter > Date: Oct 12, 2006 8:58 AM > Subject: Re: [Numpy-discussion] incrementing along a diagonal > To: Discussion of Numerical Python > > > > On 10/12/06, David Novakovic wrote: >> Johannes Loehnert wrote: >> This is very nice, exactly what i want, but it doesnt work for mxn >> matricies: >> >> >>> x = zeros((5,3)) >> >>> x >> array([[0, 0, 0], >> [0, 0, 0], >> [0, 0, 0], >> [0, 0, 0], >> [0, 0, 0]]) >> >>> index = arange(min(x.shape[0],x.shape[1])) >> >>> x[index,index] += 1 >> >>> x >> array([[1, 0, 0], >> [0, 1, 0], >> [0, 0, 1], >> [0, 0, 0], >> [0, 0, 0]]) > > Exactly what output are you expecting? That is the definition of the > 'diagonal' for a non-square matrix. If you're expecting something > else then what you want is not the diagonal. > >> Just for reference, this is the line of perl i'm trying to port: >> >> >> like: >> >> for index in diag_iter(matrix,*axes): >> matrix[index] +=1 > > That's not going to change the mathematical definition of the diagonal > of a non-square matrix. > >> PS: If anyone would care to link me to the subscription page for the >> mailing list so you dont have to CC me all the time :) > > Check the bottom of this message. > >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > > --bb > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 11 20:24:01 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 11 Oct 2006 20:24:01 -0400 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: On 11/10/06, pearu at cens.ioc.ee wrote: > """ > In SciPy 0.3.x the ufuncs were overloaded by more "intelligent" versions. > A very attractive feature was that sqrt(-1) would yield 1j as in Matlab. > Then you can program formulas directly (e.g., roots of a 2nd order > polynomial) and the right answer is always achieved. In the Matlab-Python > battle in mathematics education, this feature is important. > > Now in SciPy 0.5.x sqrt(-1) yields nan. A lot of code we have, especially > for introductory numerics and physics courses, is now broken. > This has already made my colleagues at the University skeptical to > Python as "this lack of backward compatibility would never happen in Matlab". > > Another problem related to Numeric and numpy is that in these courses we > use ScientificPython several places, which applies Numeric and will > continue to do so. You then easily get a mix of numpy and Numeric > in scripts, which may cause problems and at least extra overhead. > Just converting to numpy in your own scripts isn't enough if you call > up libraries using and returning Numeric. > """ > > I wonder, what are the reasons that numpy.sqrt(-1) returns nan? > Could sqrt(-1) made to return 1j again? If not, shouldn't > numpy.sqrt(-1) raise a ValueError instead of returning silently nan? What is the desired behaviour of sqrt? Should sqrt always return a complex array, regardless of the type of its input? This will be extremely surprising to many users, whose memory usage suddenly doubles and for whom many functions no longer work the way they're accustomed to. Should it return a complex array only when any entry in its input is negative? This will be even *more* surprising when a negative (perhaps even -0) value appears in their matrix (for example, does a+min(a) yield -0s in the minimal values?) and suddenly it's complex. A ValueError is also surprising, and it forces the user to sanitize her array before taking the square root, instead of whenever convenient. If you want MATLAB behaviour, use only complex arrays. If the problem is backward incompatibility, there's a reason 1.0 hasn't been released yet... A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Wed Oct 11 20:41:08 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 02:41:08 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: <20061012004108.GR20657@mentat.za.net> On Wed, Oct 11, 2006 at 08:24:01PM -0400, A. M. Archibald wrote: > What is the desired behaviour of sqrt? [...] > Should it return a complex array only when any entry in its input is > negative? This will be even *more* surprising when a negative (perhaps > even -0) value appears in their matrix (for example, does a+min(a) > yield -0s in the minimal values?) and suddenly it's complex. Luckily sqrt(-0.) gives -0.0 and not nan ;) Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at distip.com.au Wed Oct 11 20:35:35 2006 From: david at distip.com.au (David Novakovic) Date: Thu, 12 Oct 2006 10:35:35 +1000 Subject: SPAM-LOW: Re: incrementing along a diagonal In-Reply-To: <452D8A4D.4060708@distip.com.au> References: <452CA699.4030305@distip.com.au> <200610111035.32889.a.u.r.e.l.i.a.n@gmx.net> <452D7F6F.2090908@distip.com.au> <452D8A4D.4060708@distip.com.au> Message-ID: <452D8DD7.6060601@distip.com.au> David Novakovic wrote: > Thanks for the help, i've learnt a lot and also figured out something > that does what I want, i'll paste an interactive session below: > > x = zeros((4,7)) > x > array([[0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0]]) > index = arange(min(x.shape[0], x.shape[1])) > index2 = copy.deepcopy(index) #deep copy may be overkill > for a,b in enumerate(index): > ... index2[a] += a > Turns out this is not good at all. I guess I'm still open to suggestions then :( Dave > ... > if len(x[:,0]) > len(x[0]): > ... x[index2,index] +=1 > ... else: > ... x[index,index2] +=1 > ... > x > array([[1, 0, 0, 0, 0, 0, 0], > [0, 0, 1, 0, 0, 0, 0], > [0, 0, 0, 0, 1, 0, 0], > [0, 0, 0, 0, 0, 0, 1]]) > > > Thanks for the tips > > Dave Novakovic > > P.S. subscribed to the list now > > Bill Baxter wrote: > >> Forgot to CC you... >> >> ---------- Forwarded message ---------- >> From: Bill Baxter >> Date: Oct 12, 2006 8:58 AM >> Subject: Re: [Numpy-discussion] incrementing along a diagonal >> To: Discussion of Numerical Python >> >> >> >> On 10/12/06, David Novakovic wrote: >> >>> Johannes Loehnert wrote: >>> This is very nice, exactly what i want, but it doesnt work for mxn >>> matricies: >>> >>> >>>>>> x = zeros((5,3)) >>>>>> x >>>>>> >>> array([[0, 0, 0], >>> [0, 0, 0], >>> [0, 0, 0], >>> [0, 0, 0], >>> [0, 0, 0]]) >>> >>>>>> index = arange(min(x.shape[0],x.shape[1])) >>>>>> x[index,index] += 1 >>>>>> x >>>>>> >>> array([[1, 0, 0], >>> [0, 1, 0], >>> [0, 0, 1], >>> [0, 0, 0], >>> [0, 0, 0]]) >>> >> Exactly what output are you expecting? That is the definition of the >> 'diagonal' for a non-square matrix. If you're expecting something >> else then what you want is not the diagonal. >> >> >>> Just for reference, this is the line of perl i'm trying to port: >>> >>> >>> like: >>> >>> for index in diag_iter(matrix,*axes): >>> matrix[index] +=1 >>> >> That's not going to change the mathematical definition of the diagonal >> of a non-square matrix. >> >> >>> PS: If anyone would care to link me to the subscription page for the >>> mailing list so you dont have to CC me all the time :) >>> >> Check the bottom of this message. >> >> >>> _______________________________________________ >>> Numpy-discussion mailing list >>> Numpy-discussion at lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >>> >>> >> --bb >> >> > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pfdubois at gmail.com Wed Oct 11 21:35:58 2006 From: pfdubois at gmail.com (Paul Dubois) Date: Wed, 11 Oct 2006 18:35:58 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <20061012004108.GR20657@mentat.za.net> References: <20061012004108.GR20657@mentat.za.net> Message-ID: This is a meta-statement about this argument. We already had it. Repeatedly. Whether you choose it one way or the other, for Numeric the community chose it the way it did for a reason. It is a good reason. It isn't stupid. There were good reasons for the other way. Those reasons weren't stupid. It was a 'choice amongst equals'. Being compatible with some other package is all very nice but it is simply a different choice and the choice was already made 10 years ago. If scipy chose to do this differently then you now have an intractable problem; somebody is going to get screwed. So, next time somebody tells you that some different choice amongst equals should be made for this and that good reason, just say no. This is why having a project leader who is mean like me is better than having a nice guy like Travis. (:-> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Wed Oct 11 21:40:51 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 11 Oct 2006 18:40:51 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D8277.3080306@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> <452D5BFA.7000603@ee.byu.edu> <452D70B9.7070209@ieee.org> <452D8277.3080306@ee.byu.edu> Message-ID: <452D9D23.5030700@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > > >> With python 2.5 out now, perhaps it's time to come up with a with >> statement context manager. Something like: >> >> from __future__ import with_statement >> import numpy >> >> class errstate(object): >> def __init__(self, **kwargs): >> self.kwargs = kwargs >> def __enter__(self): >> self.oldstate = numpy.seterr(**self.kwargs) >> def __exit__(self, *exc_info): >> numpy.seterr(**self.oldstate) >> >> a = numpy.arange(10) >> a/a # ignores divide by zero >> with errstate(divide='raise'): >> a/a # raise exception on divide by zer >> # Would ignore divide by zero again if we got here. >> >> -tim >> >> >> >> > > This looks great. I think most people aren't aware of the with > statement and what it can do (I'm only aware because of your posts, for > example). > > So, what needs to be added to your example in order to just add it to > numpy? > As far as I know, just testing and documentation -- however testing was so minimal that I may find some other stuff. I'll try to clean it up tomorrow so that I'm a little more confident that it works correctly and I'll send another note out then. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 11 22:44:54 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 11 Oct 2006 20:44:54 -0600 Subject: incrementing along a diagonal In-Reply-To: <452CA699.4030305@distip.com.au> References: <452CA699.4030305@distip.com.au> Message-ID: <452DAC26.5010805@ieee.org> David Novakovic wrote: > Hi, > > i'm moving some old perl PDL code to python. I've come across a line > which changes values in a diagonal line accross a matrix. > > matrix.diagonal() returns a list of values, but making changes to these > does not reflect in the original (naturally). > > I'm just wondering if there is a way that i can increment all the values > along a diagonal? > You can refer to a diagonal using flattened index with a element-skip equal to the number of columns+1. Thus, a.flat[::a.shape[1]+1] += 1 will increment the elements of a along the main diagonal. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Wed Oct 11 23:17:13 2006 From: gregwillden at gmail.com (Greg Willden) Date: Wed, 11 Oct 2006 22:17:13 -0500 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7C88.3090506@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> Message-ID: <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> On 10/11/06, Travis Oliphant wrote: > > Stefan van der Walt wrote: > >Further, if I understand correctly, changing sqrt and power to give > >the right answer by default will slow things down somewhat. But is it > >worth sacrificing intuitive usage for speed? > > > For NumPy, yes. > > This is one reason that NumPy by itself is not a MATLAB replacement. This is not about being a Matlab replacement. This is about correctness. Numpy purports to handle complex numbers. Mathematically, sqrt(-1) is a complex number. Therefore Numpy *must* return a complex number. Speed should not take precedence over correctness. If Numpy doesn't return a complex number then it shouldn't pretend to support complex numbers. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Wed Oct 11 23:48:45 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 12 Oct 2006 12:48:45 +0900 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> Message-ID: On 10/12/06, Greg Willden wrote: > On 10/11/06, Travis Oliphant wrote: > > Stefan van der Walt wrote: > > >Further, if I understand correctly, changing sqrt and power to give > > >the right answer by default will slow things down somewhat. But is it > > >worth sacrificing intuitive usage for speed? > > > > > For NumPy, yes. > > > > This is one reason that NumPy by itself is not a MATLAB replacement. > > This is not about being a Matlab replacement. > This is about correctness. > Numpy purports to handle complex numbers. > Mathematically, sqrt(-1) is a complex number. > Therefore Numpy *must* return a complex number. > Speed should not take precedence over correctness. Unless your goal is speed. Then speed should take precedence over correctness. And unless you're a fan of quaternions, in which case *which* square root of -1 should it return? It's interesting to note that although python has had complex numbers pretty much from the beginning, math.sqrt(-1) returns an error. If you want to work with complex square roots you need to use cmath.sqrt(). Basically you have to tell python that compex numbers are something you care about by using the module designed for complex math. This scimath module is a similar deal. But perhaps the name could be a little more obvious / short? Right now it seems it only deals with complex numbers, so maybe having "complex" or "cmath" in the name would make it clearer. Hmm there is a numpy.math, why not a numpy.cmath. > If Numpy doesn't return a complex number then it shouldn't pretend to > support complex numbers. That's certainly being overdramatic. Lot's of folks are doing nifty stuff with complex FFTs every day using numpy/scipy. And I'm sure they will continue to no matter what numpy.sqrt(-1) returns. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 00:02:21 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 11 Oct 2006 22:02:21 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> Message-ID: <452DBE4D.4000906@ieee.org> Greg Willden wrote: > On 10/11/06, *Travis Oliphant* > wrote: > > Stefan van der Walt wrote: > >Further, if I understand correctly, changing sqrt and power to give > >the right answer by default will slow things down somewhat. But > is it > >worth sacrificing intuitive usage for speed? > > > For NumPy, yes. > > This is one reason that NumPy by itself is not a MATLAB replacement. > > > This is not about being a Matlab replacement. > This is about correctness. I disagree. NumPy does the "correct" thing when you realize that sqrt is a function that returns the same type as it's input. The field over-which the operation takes place is defined by the input data-type and not the input "values". Either way can be considered correct mathematically. As Paul said it was a design decision not to go searching through the array to determine whether or not there are negative numbers in the array. Of course you can do that if you want and that's what scipy.sqrt does. > Numpy purports to handle complex numbers. > Mathematically, sqrt(-1) is a complex number. Or, maybe it's undefined if you are in the field of real numbers. It all depends. > Therefore Numpy *must* return a complex number. Only if the input is complex. That is a reasonable alternative to your specification. > > If Numpy doesn't return a complex number then it shouldn't pretend to > support complex numbers. Of course it supports complex numbers, it just doesn't support automatic conversion to complex numbers. It supports complex numbers the same way Python supports them (i.e. you have to use cmath to get sqrt(-1) == 1j) People can look at this many ways without calling the other way of looking at it unreasonable. I don't see a pressing need to change this in NumPy, and in fact see many reasons to leave it the way it is. This discussion should move to the scipy list because that is the only place where a change could occur. -Travis. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 00:08:27 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 11 Oct 2006 21:08:27 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> Message-ID: <452DBFBB.8090403@ieee.org> Greg Willden wrote: > On 10/11/06, *Travis Oliphant* > wrote: > > Stefan van der Walt wrote: > >Further, if I understand correctly, changing sqrt and power to give > >the right answer by default will slow things down somewhat. But > is it > >worth sacrificing intuitive usage for speed? > > > For NumPy, yes. > > This is one reason that NumPy by itself is not a MATLAB replacement. > > > This is not about being a Matlab replacement. > This is about correctness. > Numpy purports to handle complex numbers. > Mathematically, sqrt(-1) is a complex number. That's vastly oversimplified. If you are working with the reals, then sqrt(-1) is undefined (AKA nan). If you are working in the complex plane, then sqrt(-1) is indeed *a* complex number; of course you don't know *which* complex number it is unless you also specify the branch. > Therefore Numpy *must* return a complex number. No I don't think that it must. I've found it a very useful tool for the past decade plus without it returning complex numbers from sqrt. > Speed should not take precedence over correctness. The current behavior is not incorrect. > > If Numpy doesn't return a complex number then it shouldn't pretend to > support complex numbers. Please relax. Personally I think that the default error mode should be tightened up. Then people would only see these sort of things if they really care about them. Using Python 2.5 and the errstate class I posted earlier: # This is what I like for the default error state numpy.seterr(invalid='raise', divide='raise', over='raise', under='ignore') a = -numpy.arange(10) with errstate(invalid='ignore'): print numpy.sqrt(a) # This happily returns a bunch of NANs; and one zero. print numpy.sqrt(a.astype(complex)) # This returns a bunch of complex values print numpy.sqrt(a) # This raise a floating point error. No silent NANs returned This same error state make the vagaries of dividing by zero less surprising as well. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 12 00:49:45 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 11 Oct 2006 22:49:45 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DBFBB.8090403@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <452DBFBB.8090403@ieee.org> Message-ID: On 10/11/06, Tim Hochberg wrote: > > Greg Willden wrote: > > On 10/11/06, *Travis Oliphant* > > wrote: > > > > Stefan van der Walt wrote: > > >Further, if I understand correctly, changing sqrt and power to give > > >the right answer by default will slow things down somewhat. But > > is it > > >worth sacrificing intuitive usage for speed? > > > > > For NumPy, yes. > > > > This is one reason that NumPy by itself is not a MATLAB replacement. > > > > > > This is not about being a Matlab replacement. > > This is about correctness. > > Numpy purports to handle complex numbers. > > Mathematically, sqrt(-1) is a complex number. > That's vastly oversimplified. If you are working with the reals, then > sqrt(-1) is undefined (AKA nan). If you are working in the complex > plane, then sqrt(-1) is indeed *a* complex number; And if you are working over the rationals, sqrt(-1) and sqrt(-2) lead to different field extensions ;) Of course, numpy doesn't *have* rationals, so I'm just being cute. Personally I think that the default error mode should be tightened up. > Then people would only see these sort of things if they really care > about them. Using Python 2.5 and the errstate class I posted earlier: > > # This is what I like for the default error state > numpy.seterr(invalid='raise', divide='raise', over='raise', > under='ignore') I like these choices too. Overflow, division by zero, and sqrt(-x) are usually errors, indicating bad data or programming bugs. Underflowed floats, OTOH, are just really, really small numbers and can be treated as zero. An exception might be if the result is used in division and no error is raised, resulting in a loss of accuracy. If complex results are *expected*, then this should be made explicit by using complex numbers. Numpy allows finegrained control of data types and array ordering, it is a bit closer to the metal than Matlab. This extra control allows greater efficiency, both in speed and in storage, at the cost of a bit more care on the programmers side. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From David.L.Goldsmith at noaa.gov Thu Oct 12 02:03:02 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:03:02 -0700 Subject: round In-Reply-To: References: <903323ff0610111313xcf1807p26df966d69924bdb@mail.gmail.com> Message-ID: <452DDA96.3010005@noaa.gov> Charles R Harris wrote: > > > On 10/11/06, *Greg Willden* > wrote: > > Hi All, > > I've read discussions in the archives about how round() "rounds to > even" and how that is supposedly better. > > But what I haven't been able to find is "What do I use if I want > the regular old round that you learn in school?" > > > Perhaps you could explain *why* you want the schoolbook round? Given > that floating point is inherently inaccurate you would have to expect > to produce a lot of numbers exactly of the form x.5 *without errors*, > which means you probably don't need round to deal with it. Anyway, > absent a flag somewhere, you can do something like (x + > sign(x)*.5).astype(int). > > Chuck Also, where did you go to school? In Fairfax County, VA in the 80's at least, they were teaching "round to even"; since that time, I've taught math in a variety of locations and settings and with a variety of texts, and where I have seen the issue addressed, it's always "round to even" (or, as I learned it, "the rule of fives"). DG > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:10:19 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:10:19 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D56F5.9000404@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> Message-ID: <452DDC4B.1050304@noaa.gov> Travis Oliphant wrote: > pearu at cens.ioc.ee wrote: > > >> Could sqrt(-1) made to return 1j again? >> >> > Not in NumPy. But, in scipy it could. > > Ohmigod!!! You are definitely going to scare away many, many potential users - if I wasn't obliged to use open source at work, you'd be scaring me away. I was thinking about translating all my personal fractal-generating Matlab code into Python, but I certainly won't be doing that now! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:15:33 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:15:33 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D5843.4050007@gmx.net> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> Message-ID: <452DDD85.5010106@noaa.gov> Sven Schreiber wrote: > Travis Oliphant schrieb: > > >>> If not, shouldn't >>> >>> >>> numpy.sqrt(-1) raise a ValueError instead of returning silently nan? >>> >>> >>> >> This is user adjustable. You change the error mode to raise on >> 'invalid' instead of pass silently which is now the default. >> >> -Travis >> >> > > Could you please explain how this adjustment is done, or point to the > relevant documentation. > Thank you, > Sven > I'm glad you asked this, Sven, 'cause I was thinking that if making this "user adjustment" is this advanced (I too have no idea what you're talking about, Travis), then this would be another significant strike against numpy (but I was holding my tongue, since I'd just let fly in my previous email). DG ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 02:26:17 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 00:26:17 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DDC4B.1050304@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> Message-ID: <452DE009.3070100@ieee.org> David Goldsmith wrote: > Travis Oliphant wrote: > >> pearu at cens.ioc.ee wrote: >> >> >> >>> Could sqrt(-1) made to return 1j again? >>> >>> >>> >> Not in NumPy. But, in scipy it could. >> >> >> > Ohmigod!!! You are definitely going to scare away many, many potential > users - if I wasn't obliged to use open source at work, you'd be scaring > me away. Why in the world does it scare you away. This makes no sense to me. If you don't like the scipy version don't use it. NumPy and SciPy are not the same thing. The problem we have is that the scipy version (0.3.2) already had this feature (and Numeric didn't). What is so new here that is so scary ? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:27:19 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:27:19 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7C88.3090506@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> Message-ID: <452DE047.6080501@noaa.gov> Travis Oliphant wrote: > What could be simpler? ;-) Having sqrt(-1) return 1j (without having to remember that in order to get this, you have to write sqrt(-1+0j) instead). DG ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:31:35 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:31:35 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D7D33.6070709@ee.byu.edu> References: <452D7D33.6070709@ee.byu.edu> Message-ID: <452DE147.8010609@noaa.gov> Travis Oliphant wrote: > > This is the one that concerns me. Slowing everybody down who knows they > have positive values just for people that don't seems problematic. > > Then have a "sqrtp" function for those users who are fortunate enough to know ahead of time that they'll only be talking square roots of nonnegatives. DG ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Thu Oct 12 02:35:48 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 12 Oct 2006 15:35:48 +0900 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE047.6080501@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <452DE047.6080501@noaa.gov> Message-ID: <452DE244.8030608@ar.media.kyoto-u.ac.jp> David Goldsmith wrote: > Travis Oliphant wrote: > >> What could be simpler? ;-) >> > Having sqrt(-1) return 1j (without having to remember that in order to > get this, you have to write sqrt(-1+0j) instead). > > But this can sometimes lead to confusing errors hard to track when you don't want to treat complex numbers. That's one of the thing I hated in matlab, actually: if you don't want to handle complex numbers, you have to check regularly for it. So I don't think this is simpler. David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 02:38:51 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 00:38:51 -0600 Subject: Please test the SVN branch Message-ID: <452DE2FB.3070408@ieee.org> I made some fixes to the "asbuffer" code which let me feel better about exposing it in NumPy (where it is now named int_asbuffer). This code takes a Python integer and a size and returns a buffer object that points to that memory. A little test is performed by trying to read (and possibly write if a writeable buffer is requested) to the first and last elements of the buffer. Any segfault is trapped and used to raise a Python error indicating you can't use that area of memory. It doesn't guarantee you won't shoot yourself, but it does make it more difficult to segfault Python. Previously a simple int_asbuffer(3423423, 5) would have segfaulted (unless by chance you the memory area 3423423 happens to be owned by Python). I have not tested the code on other platforms to make sure it works as expected, so please try and compiled it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 02:37:30 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Wed, 11 Oct 2006 23:37:30 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <20061012001727.GQ20657@mentat.za.net> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <20061012001727.GQ20657@mentat.za.net> Message-ID: <452DE2AA.6020605@noaa.gov> Stefan van der Walt wrote: >> This is one reason that NumPy by itself is not a MATLAB >> replacement. >> > Intuitive usage is hopefully not a MATLAB-only feature. > Here, here! (Or is it Hear, hear! I don't think I've ever seen it written out before. :-) ) > I'll shut up now :) But why? To my mind, you're making a lot more sense than Travis. DG > Cheers > St?fan > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 02:40:18 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 00:40:18 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE047.6080501@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <452DE047.6080501@noaa.gov> Message-ID: <452DE352.80304@ieee.org> David Goldsmith wrote: > Travis Oliphant wrote: > >> What could be simpler? ;-) >> > Having sqrt(-1) return 1j (without having to remember that in order to > get this, you have to write sqrt(-1+0j) instead). > > That's exactly what scipy.sqrt(-1) does. That was my point. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.u.r.e.l.i.a.n at gmx.net Thu Oct 12 02:40:49 2006 From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert) Date: Thu, 12 Oct 2006 08:40:49 +0200 Subject: incrementing along a diagonal In-Reply-To: <452D8DD7.6060601@distip.com.au> References: <452CA699.4030305@distip.com.au> <452D8A4D.4060708@distip.com.au> <452D8DD7.6060601@distip.com.au> Message-ID: <200610120840.49968.a.u.r.e.l.i.a.n@gmx.net> Hi, I absolutely do not know perl, so I do not know what the expression you posted does. However, the key is just to understand indexing in numpy: if you have a matrix mat and index arrays index1, index2 with, lets say, index1 = array([ 17, 19, 29]) index2 = array([ 12, 3, 9]) then the entries of the index arrays are used as row and column indices respectively, and the result will be an array shaped like the index arrays. So doing mat[index1, index2] will give you --> array([ mat[17, 12], mat[19, 3], mat[29, 9]]). Now if you want the diagonal of a 3x3-mat, you need index1=index2=array([ 0, 1, 2]). mat[index1, index2] --> array([ mat[0,0], mat[1,1], mat[2,2]]) That is what my code does. If you need other, arbitrary subsets of mat, you just have to fill the index arrays accordingly. Johannes ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Thu Oct 12 02:49:39 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 12 Oct 2006 00:49:39 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE009.3070100@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: On 10/12/06, Travis Oliphant wrote: > Why in the world does it scare you away. This makes no sense to me. > If you don't like the scipy version don't use it. NumPy and SciPy are > not the same thing. I'd like to pitch in (again) on this issue, but I'll try to make sure that it's clear that I'm NOT arguing about sqrt() in particular, one way or another. It's perfectly clear that numpy != scipy to all of us. And yet, I think it is equally clear that the two are /very/ tightly related. Scipy builds on top of numpy and it directly exposes a LOT of the numpy API as scipy functions: In [21]: import numpy as n, scipy as s In [22]: common_names = set(dir(n)) & set(dir(s)) In [23]: [getattr(n,x) is getattr(s,x) for x in common_names ].count(True) Out[23]: 450 In [24]: len(common_names) Out[24]: 462 That's 450 objects from numpy which are directly exposed in Scipy, while only 12 names are in both top-level namespaces and yet are different objects. Put another way, scipy is a direct wrap of 97% of the numpy top-level namespace. While /we/ know they are distinct entities, to the casual user a 97% match looks pretty close to being the same, especially when the non-identical things are all non-numerical: In [27]: [x for x in common_names if getattr(n,x) is not getattr(s,x)] Out[27]: ['pkgload', 'version', '__config__', '__file__', '__all__', '__doc__', 'show_config', '__version__', '__path__', '__name__', 'info', 'test'] In [32]: n.__version__,s.__version__ Out[32]: ('1.0.dev3306', '0.5.2.dev2252') Basically, at least for these versions, the top-level API of scipy is a strict superset of the numpy one for all practical purposes. I think it's fair to say that if we start sprinkling special cases where certain objects happen to have the same name but produce different results for the same inputs, confusion will arise. Please note that I see a valid reason for scipy.foo != numpy.foo when the scipy version uses code with extra features, is faster, has additional options, etc. But as I said in a previous message, I think that /for the same input/, we should really try to satisfy that numpy.foo(x) == scipy.foo(x) (which is NOT the same as 'numpy.foo is scipy.foo') within reason. Obviously the scipy version may succeed where the numpy one fails due to better algorithms, or be faster, etc. I'm talking about a general principle here. I doubt I'll be able to state my point with any more clarity, so I'll stop now. But I really believe that this particular aspect of consistency between numpy and scipy is a /very/ important one for its adoption in wider communities. Best, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nwagner at iam.uni-stuttgart.de Thu Oct 12 02:50:23 2006 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Thu, 12 Oct 2006 08:50:23 +0200 Subject: Please test the SVN branch In-Reply-To: <452DE2FB.3070408@ieee.org> References: <452DE2FB.3070408@ieee.org> Message-ID: <452DE5AF.5010500@iam.uni-stuttgart.de> Travis Oliphant wrote: > I made some fixes to the "asbuffer" code which let me feel better about > exposing it in NumPy (where it is now named int_asbuffer). > > This code takes a Python integer and a size and returns a buffer object > that points to that memory. A little test is performed by trying to > read (and possibly write if a writeable buffer is requested) to the > first and last elements of the buffer. Any segfault is trapped and used > to raise a Python error indicating you can't use that area of memory. > > It doesn't guarantee you won't shoot yourself, but it does make it more > difficult to segfault Python. Previously a simple int_asbuffer(3423423, > 5) would have segfaulted (unless by chance you the memory area 3423423 > happens to be owned by Python). > > I have not tested the code on other platforms to make sure it works as > expected, so please try and compiled it. > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > Numpy version 1.0.dev3315 Scipy version 0.5.2.dev2254 Works fine here. All tests passed ! x86_64 x86_64 x86_64 GNU/Linux Nils ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 03:06:50 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 01:06:50 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <452DBFBB.8090403@ieee.org> Message-ID: <452DE98A.4090404@ieee.org> > > Personally I think that the default error mode should be tightened > up. > Then people would only see these sort of things if they really care > about them. Using Python 2.5 and the errstate class I posted earlier: > > # This is what I like for the default error state > numpy.seterr (invalid='raise', divide='raise', over='raise', > under='ignore') > > > I like these choices too. Overflow, division by zero, and sqrt(-x) are > usually errors, indicating bad data or programming bugs. Underflowed > floats, OTOH, are just really, really small numbers and can be treated > as zero. An exception might be if the result is used in division and > no error is raised, resulting in a loss of accuracy. > I'm fine with this. I've hesitated because error checking is by default slower. But, I can agree that it is "less surprising" to new-comers. People that don't mind no-checking can simply set their defaults back to 'ignore' -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sinclaird at ukzn.ac.za Thu Oct 12 03:11:48 2006 From: sinclaird at ukzn.ac.za (Scott Sinclair) Date: Thu, 12 Oct 2006 09:11:48 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452DEAB4.3030506@ukzn.ac.za> Fernando Perez wrote: > Please note that I see a valid reason for scipy.foo != numpy.foo when > the scipy version uses code with extra features, is faster, has > additional options, etc. But as I said in a previous message, I think > that /for the same input/, we should really try to satisfy that > > numpy.foo(x) == scipy.foo(x) (which is NOT the same as 'numpy.foo is scipy.foo') > > within reason. As far as I can tell this is exactly what happens. Consider the issue under discussion... ---------------------------------- >>> import numpy as np >>> np.sqrt(-1) -1.#IND >>> np.sqrt(-1+0j) 1j >>> a = complex(-1) >>> np.sqrt(a) 1j >>> import scipy as sp >>> sp.sqrt(-1) -1.#IND >>> np.sqrt(-1+0j) 1j >>> sp.sqrt(a) 1j >>> np.__version__ '1.0rc1' >>> sp.__version__ '0.5.1' >>> ---------------------------------- I'm sure that this hasn't changed in the development versions. Surely the point is that when your algorithm can potentially produce a complex result, the logical thing to do is to use a complex data type. In this case Numpy and Scipy behave in a way which is intuitive. If complex results are surprising and unexpected then the algorithm is probably in error or poorly understood ;-) Cheers, Scott ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 03:12:45 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 00:12:45 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE009.3070100@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452DEAED.9070408@noaa.gov> Travis Oliphant wrote: > David Goldsmith wrote: > >> Travis Oliphant wrote: >> >> >>> pearu at cens.ioc.ee wrote: >>> >>> >>> >>> >>>> Could sqrt(-1) made to return 1j again? >>>> >>>> >>>> >>>> >>> Not in NumPy. But, in scipy it could. >>> >>> >>> >>> >> Ohmigod!!! You are definitely going to scare away many, many potential >> users - if I wasn't obliged to use open source at work, you'd be scaring >> me away. >> > Why in the world does it scare you away. This makes no sense to me. > If you don't like the scipy version don't use it. NumPy and SciPy are > not the same thing. > I don't use scipy (and don't want to because of the overhead) but it sounds like I should because if I'm taking the square root of a variable whose value at run time happens to be real but less than zero, I *want* the language I'm using to return an imaginary; in other words, it's not the scipy behavior which "scares" me, its the numpy (which I do/have been using) behavior. To which you might say, "Well, if that's what you want, and you have Matlab (as you've said you do), then just use that." But that's precisely the point: people who don't want to be bothered with having to be "a bit more care[ful]" (as Chuck put it) - and who can afford it - are going to be inclined to choose Matlab over numpy. Perhaps one doesn't care - in the grand scheme of things, it certainly doesn't matter - but I think that you all should be aware that this numpy "feature" will be seen by many as more than just a nuisance. DG ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 03:18:57 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 00:18:57 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE352.80304@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <452DE047.6080501@noaa.gov> <452DE352.80304@ieee.org> Message-ID: <452DEC61.5050309@noaa.gov> Travis Oliphant wrote: > David Goldsmith wrote: > >> Travis Oliphant wrote: >> >> >>> What could be simpler? ;-) >>> >>> >> Having sqrt(-1) return 1j (without having to remember that in order to >> get this, you have to write sqrt(-1+0j) instead). >> >> >> > That's exactly what scipy.sqrt(-1) does. That was my point. > But I don't want to have to use scipy (which so far I haven't needed for any other reason) just to get this one behavior. But my personal preferences aside, I say again: if numpy doesn't behave this way, you *will* be "scaring" away many potential users. If you can live with that, so be it. DG DG > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jaqyhoga at bowenlaw.com Thu Oct 12 03:18:40 2006 From: jaqyhoga at bowenlaw.com (Natalee Remus) Date: Thu, 12 Oct 2006 00:18:40 -0700 Subject: VldAGRA Message-ID: <000001c6edce$a48567d0$45d8a8c0@lynzv> Hi, VldAGRA for LESS http://www.tikindefunkadesinmde.com Paskonjak, bag in hand and forged press credentials in pocket. Even angry and settled for a surly mutter and growl. Mata nodded blithely -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From David.L.Goldsmith at noaa.gov Thu Oct 12 03:23:10 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 00:23:10 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452DED5E.8090608@noaa.gov> (Very) well said, Fernando. Thanks! DG Fernando Perez wrote: > On 10/12/06, Travis Oliphant wrote: > > >> Why in the world does it scare you away. This makes no sense to me. >> If you don't like the scipy version don't use it. NumPy and SciPy are >> not the same thing. >> > > I'd like to pitch in (again) on this issue, but I'll try to make sure > that it's clear that I'm NOT arguing about sqrt() in particular, one > way or another. > > It's perfectly clear that numpy != scipy to all of us. And yet, I > think it is equally clear that the two are /very/ tightly related. > Scipy builds on top of numpy and it directly exposes a LOT of the > numpy API as scipy functions: > > In [21]: import numpy as n, scipy as s > > In [22]: common_names = set(dir(n)) & set(dir(s)) > > In [23]: [getattr(n,x) is getattr(s,x) for x in common_names ].count(True) > Out[23]: 450 > > In [24]: len(common_names) > Out[24]: 462 > > That's 450 objects from numpy which are directly exposed in Scipy, > while only 12 names are in both top-level namespaces and yet are > different objects. Put another way, scipy is a direct wrap of 97% of > the numpy top-level namespace. While /we/ know they are distinct > entities, to the casual user a 97% match looks pretty close to being > the same, especially when the non-identical things are all > non-numerical: > > In [27]: [x for x in common_names if getattr(n,x) is not getattr(s,x)] > Out[27]: > ['pkgload', > 'version', > '__config__', > '__file__', > '__all__', > '__doc__', > 'show_config', > '__version__', > '__path__', > '__name__', > 'info', > 'test'] > > In [32]: n.__version__,s.__version__ > Out[32]: ('1.0.dev3306', '0.5.2.dev2252') > > Basically, at least for these versions, the top-level API of scipy is > a strict superset of the numpy one for all practical purposes. > > I think it's fair to say that if we start sprinkling special cases > where certain objects happen to have the same name but produce > different results for the same inputs, confusion will arise. > > Please note that I see a valid reason for scipy.foo != numpy.foo when > the scipy version uses code with extra features, is faster, has > additional options, etc. But as I said in a previous message, I think > that /for the same input/, we should really try to satisfy that > > numpy.foo(x) == scipy.foo(x) (which is NOT the same as 'numpy.foo is scipy.foo') > > within reason. Obviously the scipy version may succeed where the > numpy one fails due to better algorithms, or be faster, etc. I'm > talking about a general principle here. > > I doubt I'll be able to state my point with any more clarity, so I'll > stop now. But I really believe that this particular aspect of > consistency between numpy and scipy is a /very/ important one for its > adoption in wider communities. > > Best, > > f > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 03:32:17 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 01:32:17 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452DEF81.4040208@ieee.org> > I'd like to pitch in (again) on this issue, but I'll try to make sure > that it's clear that I'm NOT arguing about sqrt() in particular, one > > way or another. > Fernando, I don't disagree with you in principle. I don't think anybody does. I think we should try to keep the interfaces and expectations of scipy and numpy the same. Unfortunately, we have competing issues in this particular case (in the case of the functions in numpy.lib.scimath). Nobody has suggested an alternative to the current situation in SVN that is satsifying to enough users. Here is the situation. 1) NumPy ufuncs never up-cast to complex numbers without the user explicitly requesting it so sqrt(-1) creates a floating-point error condition which is either caught or ignored according to the user's desires. To get complex results from sqrt you have put in complex numbers to begin with. That's inherent in the way ufuncs work. This is long-standing behavior that has good reasons for it's existence. I don't see this changing. That's why I suggested to move the discussion over to scipy (we have the fancy functions in NumPy, they are just not in the top-level name-space). Now, it would be possible to give ufuncs a dtype keyword argument that allowed you to specify which underlying loop was to be used for the calculation. That way you wouldn't have to convert inputs to complex numbers before calling the ufunc, but could let the ufunc do it in chunks during the loop. That is certainly a reasonable enhancement: sqrt(a, dtype=complex). This no-doubt has a "library-ish"-feeling, but that is what NumPy is. If such a change is desireable, I don't think it would be much to implement it. 2) In SciPy 0.3.2 the top-level math functions were overloaded with these fancy argument-checking versions. So that scipy.sqrt(-1) would return 1j. This was done precisely to attract users like David G. who don't mind data-type conversions on the fly, but prefer automatic conversion (funny being called non-sensical when I was the one who wrote those original scipy functions --- I guess I'm schizophrenic). We changed this in SciPy 0.5.1 by accident without any discussion. It was simply a by-product of moving scipy_base (including those special-handling functions) into NumPy and forgetting to import those functions again into top-level SciPy. It was an oversight that caused backwards compatibilty issues. So, I simply changed it back to what SciPy used to be in SVN. If we want to change SciPy, then fine, but let's move that discussion over to scipy-dev. In short, I appreciate all the comments and the differences of opinion they point out, but they are ultimately non-productive. We can't just change top-level sqrt to be the fancy function willy-nilly. Paul says I'm nice (he's not talked to my children recently), but I'm not that much of a push-over. There are very good reasons that NumPy has the behavior it does. In addition, the fancy functions are already there in numpy in numpy.lib.scimath. So, use them from there if you like them. Create your own little mynumpy module that does from numpy import * from numpy.lib.scimath import * and have a ball. Python is flexible enough that the sky is not going to fall if the library doesn't do things exactly the way you would do it. We can still cooperate in areas that we agree on. Again. Put this to rest on NumPy and move the discussion over to scipy-dev. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 03:41:46 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 01:41:46 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DEAED.9070408@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> Message-ID: <452DF1BA.9000906@ieee.org> David Goldsmith wrote: >> >> > I don't use scipy (and don't want to because of the overhead) but it > sounds like I should because if I'm taking the square root of a variable > whose value at run time happens to be real but less than zero, I *want* > the language I'm using to return an imaginary; in other words, it's not > the scipy behavior which "scares" me, its the numpy (which I do/have > been using) behavior. O.K. Well the functions you want are in numpy.lib.scimath. I should have directed you there. You actually don't need scipy installed at all. Just import sqrt from numpy.lib.scimath. I'm sorry I misunderstood the issue. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Thu Oct 12 03:45:37 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Thu, 12 Oct 2006 10:45:37 +0300 (EEST) Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE98A.4090404@ieee.org> References: <452DE98A.4090404@ieee.org> Message-ID: PS: I am still sending this message to numpy list only because the proposal below affects numpy code, not scipy one. I think Fernando points make sense, numpy.foo(x) != scipy.foo(x) can cause confusion and frustration both among new numpy/scipy users and developers (who need to find explanations for the choises made). So, let me propose the following solution so that all parties will get the same results without sacrifying numpy.sqrt speed on non-negative input and scipy.sqrt backward compatibility: Define numpy.sqrt as follows: def sqrt(x): r = nx.sqrt(x) if nx.nan in r: i = nx.where(nx.isnan(r)) r = _tocomplex(r) r[i] = nx.sqrt(_tocomplex(x[i])) return r and define numpy.sqrtp that takes only non-negative input, this is for those users who expect sqrt to fail on negative input (as Numeric.sqrt and math.sqrt do). Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Thu Oct 12 03:46:33 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 12 Oct 2006 01:46:33 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DEF81.4040208@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEF81.4040208@ieee.org> Message-ID: On 10/12/06, Travis Oliphant wrote: > > > I'd like to pitch in (again) on this issue, but I'll try to make sure > > that it's clear that I'm NOT arguing about sqrt() in particular, one > > > > way or another. > > > > Fernando, > > I don't disagree with you in principle. I don't think anybody does. I > think we should try to keep the interfaces and expectations of scipy and > numpy the same. OK, I'm glad to hear that. For a moment during the discussion I misunderstood your intent and thought you did disagree with this, which worried me. Sorry for the confusion and any noise caused. I realize the sqrt() topic is a tricky one which I've deliberately sidestepped, since I wasn't too interested in that particular case but rather in the general principle. I'll leave that discussion to continue on scipy-dev for those who are intereseted in it. Regards, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 12 03:51:23 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 12 Oct 2006 01:51:23 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DEF81.4040208@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEF81.4040208@ieee.org> Message-ID: <452DF3FB.8010201@ieee.org> Travis Oliphant wrote: > Now, it would be possible to give ufuncs a dtype keyword argument that > allowed you to specify which underlying loop was to be used for the > calculation. That way you wouldn't have to convert inputs to complex > numbers before calling the ufunc, but could let the ufunc do it in > chunks during the loop. That is certainly a reasonable enhancement: > > sqrt(a, dtype=complex). > > This no-doubt has a "library-ish"-feeling, but that is what NumPy is. > If such a change is desireable, I don't think it would be much to > implement it. > > This could be implemented, but only with a version number increase in the C-API (we would have to change the c-signature of the ufunc tp_call. This would mean that the next release of NumPy would be binary incompatible with packages built to previous NumPy releases. I've really been trying to avoid doing that. So, unless there are strong requests for this feature that outweigh the negatives of re-building dependent packages, then this feature will have to wait. OTOH: I suppose it could be implemented in a different way (using indexing or a method call): sqrt[complex](a) --- I remember Tim suggesting some use for indexing on ufuncs earlier though. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From klemm at phys.ethz.ch Thu Oct 12 03:53:05 2006 From: klemm at phys.ethz.ch (Hanno Klemm) Date: Thu, 12 Oct 2006 09:53:05 +0200 Subject: Compile with atlas 3.7.17 fails In-Reply-To: <452D20EB.7030506@ee.byu.edu> References: , Message-ID: Travis, thank you, setting the -fPIC option indeed solved my problem. Just for future reference (if anybody else needs it), I added the -fPIC flag to the compiler flags in Make.inc in the atlas build directory, as setting CCFLAGS somehwo didn't seem to be successful. This might not be elegant but it worked. Best regards, Hanno Travis Oliphant said: > Hanno Klemm wrote: > > >Hi, > > > >I don't know if this is a bug or just me doing something wrong (I > >suspect the latter). I try to compile numpy-1.0rc1 with python2.5 and > >atlas 3.7.17. > > > >I have build the atlas library myself, it doesn't give any errors > >under make test or make pttest, so it seems to be okay. if I try to > >build numpy I get the following error: > > > >creating build/temp.linux-x86_64-2.5/numpy/core/blasdot > >compile options: '-DATLAS_INFO="\"3.7.17\"" -Inumpy/core/blasdot > >-I/scratch/python2.5/include -Inumpy/core/include > >-Ibuild/src.linux-x86_64-2.5/numpy/core -Inumpy/core/src > >-Inumpy/core/include -I/scratch/python2.5/include/python2.5 -c' > >gcc: numpy/core/blasdot/_dotblas.c > >gcc -pthread -shared > >build/temp.linux-x86_64-2.5/numpy/core/blasdot/_dotblas.o > >-L/scratch/python2.5/lib -lcblas -latlas -o > >build/lib.linux-x86_64-2.5/numpy/core/_dotblas.so > >/usr/bin/ld: /scratch/python2.5/lib/libcblas.a(cblas_dgemm.o): > >relocation R_X86_64_32 can not be used when making a shared object; > >recompile with -fPIC > > > > > > This may be part of your problem. It's looks like the linker is having > a hard time making use of your compiled extension in a shared library. > Perhaps you should make sure -fPIC is on when you compile atlas (I'm not > sure how to do that --- perhaps setting CCFLAGS environment variable to > include -fPIC would help). > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- Hanno Klemm klemm at phys.ethz.ch ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 03:54:02 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 00:54:02 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DF1BA.9000906@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> <452DF1BA.9000906@ieee.org> Message-ID: <452DF49A.407@noaa.gov> Travis Oliphant wrote: > David Goldsmith wrote: > >>> >>> >>> >> I don't use scipy (and don't want to because of the overhead) but it >> sounds like I should because if I'm taking the square root of a variable >> whose value at run time happens to be real but less than zero, I *want* >> the language I'm using to return an imaginary; in other words, it's not >> the scipy behavior which "scares" me, its the numpy (which I do/have >> been using) behavior. >> > > O.K. Well the functions you want are in numpy.lib.scimath. I should > have directed you there. You actually don't need scipy installed at > all. Just import sqrt from numpy.lib.scimath. I'm sorry I > misunderstood the issue. > Got it. And if I understand correctly, the import order you specify in the little mynumpy example you included in your latest response to Fernando will result in any "overlap" between numpy and numpy.lib.scimath to call the latter's version of things rather than the former's, yes? DG > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From markbak at gmail.com Thu Oct 12 03:58:46 2006 From: markbak at gmail.com (Mark Bakker) Date: Thu, 12 Oct 2006 09:58:46 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? Message-ID: <6946b9500610120058l1a1d008agd19104ec23d7440a@mail.gmail.com> My vote is for consistency in numpy. But it is unclear what consistency is. What is truly confusing for newbie Python users (and a source for error even after 5 years of Python programming) is that >>> 2/3 0 In that respect, I would think >>> numpy.sqrt(2) should give 1, but it gives 1.4142135623730951 So numpy does typechecking anyway (it gets an integer and makes it a float). If that is the consistent behavior, then by all means >>> sqrt(-1) should return 1j. Wouldn't that be the consistent thing to do???? Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cristian.codorean at gmail.com Thu Oct 12 05:13:05 2006 From: cristian.codorean at gmail.com (Cristian Codorean) Date: Thu, 12 Oct 2006 10:13:05 +0100 Subject: Casting Message-ID: Hello list, I got some old code that uses Scientific and Numeric. Recently, these two packages were updated to the latest versions together with python and my code started failing. The first major problem I have is when downcasting. Code that used to work now fails because the array cannot be safely cast anymore to the required type. If anyone could help me please and direct me to some documentation about when this change happened (version), why it happened, is there any other solution but using astype, can numpy help me in anyway in this situation, because I got really lost ... Thank you in advance, Cristian -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From stefan at sun.ac.za Thu Oct 12 05:31:48 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 11:31:48 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: Message-ID: <20061012093148.GU20657@mentat.za.net> I've summarised this thread at http://www.scipy.org/NegativeSquareRoot Feel free to make adjustments, in case I missed something. Regards Stefan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mailer at phdjobs.com Thu Oct 12 06:12:54 2006 From: mailer at phdjobs.com (mailer at phdjobs.com) Date: Thu, 12 Oct 2006 11:12:54 +0100 Subject: =?iso-8859-1?q?PhDjobs=2Ecom_-_October_Newslet?= =?iso-8859-1?q?ter?= Message-ID: <20061012.XVYZAPTYBFJJJDWD@phdjobs.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From stefan at sun.ac.za Thu Oct 12 06:21:49 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 12:21:49 +0200 Subject: Please test the SVN branch In-Reply-To: <452DE2FB.3070408@ieee.org> References: <452DE2FB.3070408@ieee.org> Message-ID: <20061012102149.GV20657@mentat.za.net> On Thu, Oct 12, 2006 at 12:38:51AM -0600, Travis Oliphant wrote: > > I made some fixes to the "asbuffer" code which let me feel better about > exposing it in NumPy (where it is now named int_asbuffer). > > This code takes a Python integer and a size and returns a buffer object > that points to that memory. A little test is performed by trying to > read (and possibly write if a writeable buffer is requested) to the > first and last elements of the buffer. Any segfault is trapped and used > to raise a Python error indicating you can't use that area of memory. > > It doesn't guarantee you won't shoot yourself, but it does make it more > difficult to segfault Python. Previously a simple int_asbuffer(3423423, > 5) would have segfaulted (unless by chance you the memory area 3423423 > happens to be owned by Python). Some argument checking and a docstring would be helpful. I guessed wrong the first time: In [1]: x = N.array([1,2,3]) In [4]: y = N.int_asbuffer(x.size,x.__array_interface__['data'][0]) In [5]: y[0] Segmentation fault Out of curiosity, why does the following outputs differ: In [39]: y = N.int_asbuffer(0L,x.size) --------------------------------------------------------------------------- exceptions.SystemError Traceback (most recent call last) /home/stefan/ SystemError: error return without exception set In [40]: y = N.int_asbuffer(1L,x.size) --------------------------------------------------------------------------- exceptions.ValueError Traceback (most recent call last) /home/stefan/ ValueError: cannot use memory location as a buffer. Thanks for your work on this. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From zskhvgouoj at sncc.com Thu Oct 12 10:33:53 2006 From: zskhvgouoj at sncc.com (National) Date: Thu, 12 Oct 2006 12:33:53 -0200 Subject: Klaipeda Narva also comes Message-ID: <001201c6ede9$ea086760$e6cc1053@SZYMON> Grocery local Traffic am Maps Greater Orange co of Inland Empire Ventura Diego is Weather a Rain in. Smallest Four Yearswest is School Recalls in Exstudent is Disputes Claim war Deada Younger Unfazed Outrage Over. Slipups am sam Howe in Verhovek Using cheap video cameras a they shadow is opposing post or gotcha moments online n? Diego Weather Rain gtgt forecast Extreme house owner kindred in spirits then sue Wong restored. Byte of multibyte am auto indent of wheel mouse copy paste a line numbers unix format utf unicode? Downiraqi dreams acrobats Ryanair onto Curonian Pink girl suburbs a Movie or reviewout town Seventh festival in puppet. House a amp Garden Health Food am Culture Crossword Sudoku all Sections buy in Sell More Personals or Deals Stores Coupons Newspaper. Etbt Advanced newspaper This Week am Finance in Outlook Summer events am Cinema Cultural in agenda calendar Your Forum Todays Depth Budgets. Events Cinema Cultural agenda calendar Your Forum Todays Depth Budgets under fire increased or social Budgets offer. Seafood Stews Snapwest is and was Lynell George remembers Neils Words Tvcar! Guide fallen icons a horror Flash Most am Viewed False Choice Prompt Treason in Charge Southland am Mansmall of Plane am Pitcher. Athlete or theft Nokia planteu sees realistic date euro give military aidsenior State in official usa. Projects helping bridge gap of Galen Center Counts Opening Political am Trackers Capture Slipups sam Howe Verhovek Using cheap video cameras. Exodus a Vexes in Salinas usc Seeks Admission am Downtown by of Cara mia Dimassa. Mx Html Tidy of Cisco Rebol mirc Xslxslt Guicli a Fortran Autoit Pawl or dos Batch. Dxl Kixtart of Korn Shell Assembly Lisp or Maki is Script Nasm Nullsoft. Town Seventh in festival puppet of theatre Prima Donna Swamp Princess finally soul reviewhot royal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Trade.gif Type: image/gif Size: 11029 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cimrman3 at ntc.zcu.cz Thu Oct 12 07:25:11 2006 From: cimrman3 at ntc.zcu.cz (Robert Cimrman) Date: Thu, 12 Oct 2006 13:25:11 +0200 Subject: arraysetops update Message-ID: <452E2617.8020600@ntc.zcu.cz> I have just added 'See also' to all docstrings in arraysetops functions, replaced old numpy idioms with new ones (aka modernizing) and improved speed of numpy.unique1d, which now reflects numpy.unique. hoping no bugs were introduced, regards, r. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From c.hanot at septentrio.com Thu Oct 12 07:50:54 2006 From: c.hanot at septentrio.com (Charles Hanot) Date: Thu, 12 Oct 2006 13:50:54 +0200 Subject: Compiling Numpy with py2exe Message-ID: <8B7FAE730B4A3646B51C94CD6C8784B7B28F52@serversbs2003.Septentrio.local> Hello, I've a program using Numpy and I would like to compile it with Py2exe but I've some problems. When I try to compile my program, some modules generated by numpy are not loaded. I've already asked to the py2exe community if they have a solution for that and the answer was that it was possible to compile numpy modules with py2exe but I should ask to the Numpy community :-) So my question is, is there something special to do in order to compile a program using numpy module? This is the setup.py file that I use from distutils.core import setup import py2exe, sys sys.path.append("tools") sys.path.append("report") sys.path.append("spirent") sys.path.append("numpydir") setup( console = ['checkfile.py'], ) If I use this setup.py, lots of numpy file cannot be loaded. I get the following message copying c:\python24\lib\site-packages\py2exe\run.exe -> Y:\svn\copy\dist\checkfile.exe The following modules appear to be missing ['DateTime', 'FFT', 'Ft.Lib', 'Ft.Lib.DumpBgTuple', 'Ft.__init__', 'LinearAlgebra', 'MA', 'MLab', 'Matrix', 'Numeric', 'Pyrex', 'Pyrex.Compiler', 'XPathParserc' , '_curses', 'backends.draw_if_interactive', 'backends.new_figure_manager', 'backends.pylab_setup', 'backends.show', 'core.abs', 'core.max', 'core.min', 'core.r ound', 'ctypes', 'ext.IsDOMString', 'ext.SplitQName', 'fcompiler.FCompiler', 'fcompiler.show_fcompilers', 'lib.add_newdoc', 'mlab.amax', 'mlab.amin', 'numarray' , 'numarray.convolve', 'numarray.fft', 'numarray.ieeespecial', 'numarray.linear_algebra', 'numarray.linear_algebra.mlab', 'numarray.ma', 'numarray.numeric', 'nu merix.ArrayType', 'numerix.Complex', 'numerix.Complex32', 'numerix.Complex64', 'numerix.Float', 'numerix.Float32', 'numerix.Float64', 'numerix.Int', 'numerix.In t16', 'numerix.Int32', 'numerix.Int8', 'numerix.NewAxis', 'numerix.UInt16', 'numerix.UInt32', 'numerix.UInt8', 'numerix.absolute', 'numerix.add', 'numerix.all', 'numerix.allclose', 'numerix.alltrue', 'numerix.any', 'numerix.arange', 'numerix.arccos', 'numerix.arccosh', 'numerix.arcsin', 'numerix.arcsinh', 'numerix.arct an', 'numerix.arctan2', 'numerix.arctanh', 'numerix.argmax', 'numerix.argmin', 'numerix.argsort', 'numerix.around', 'numerix.array', 'numerix.arrayrange', 'nume rix.asarray', 'numerix.asum', 'numerix.bitwise_and', 'numerix.bitwise_or', 'numerix.bitwise_xor', 'numerix.ceil', 'numerix.choose', 'numerix.clip', 'numerix.com press', 'numerix.concatenate', 'numerix.conjugate', 'numerix.convolve', 'numerix.cos', 'numerix.cosh', 'numerix.cross_correlate', 'numerix.cumproduct', 'numerix .cumsum', 'numerix.diagonal', 'numerix.divide', 'numerix.dot', 'numerix.equal', 'numerix.exp', 'numerix.fabs', 'numerix.fft.fft', 'numerix.fft.inverse_fft', 'nu merix.floor', 'numerix.fmod', 'numerix.fromfunction', 'numerix.fromstring', 'numerix.greater', 'numerix.greater_equal', 'numerix.hypot', 'numerix.identity', 'nu merix.indices', 'numerix.innerproduct', 'numerix.iscontiguous', 'numerix.isnan', 'numerix.less', 'numerix.less_equal', 'numerix.log', 'numerix.log10', 'numerix. logical_and', 'numerix.logical_not', 'numerix.logical_or', 'numerix.logical_xor', 'numerix.matrixmultiply', 'numerix.maximum', 'numerix.minimum', 'numerix.mlab. amax', 'numerix.mlab.amin', 'numerix.mlab.cov', 'numerix.mlab.diff', 'numerix.mlab.hanning', 'numerix.mlab.rand', 'numerix.mlab.std', 'numerix.mlab.svd', 'numer ix.multiply', 'numerix.negative', 'numerix.newaxis', 'numerix.nonzero', 'numerix.not_equal', 'numerix.nx', 'numerix.ones', 'numerix.outerproduct', 'numerix.pi', 'numerix.power', 'numerix.product', 'numerix.put', 'numerix.putmask', 'numerix.rank', 'numerix.ravel', 'numerix.repeat', 'numerix.reshape', 'numerix.resize', ' numerix.searchsorted', 'numerix.shape', 'numerix.sin', 'numerix.sinh', 'numerix.size', 'numerix.sometrue', 'numerix.sort', 'numerix.sqrt', 'numerix.subtract', ' numerix.swapaxes', 'numerix.take', 'numerix.tan', 'numerix.tanh', 'numerix.trace', 'numerix.transpose', 'numerix.typecode', 'numerix.typecodes', 'numerix.where' , 'numerix.which', 'numerix.zeros', 'pre', 'serial', 'setuptools', 'setuptools.command', 'setuptools.command.egg_info', 'setuptools.command.install', 'testing.S cipyTest', 'win32api', 'win32con', 'win32pdh', 'xml.xslt', 'xml.xslt.ParsedPattern', 'numpy.Complex', 'numpy.Complex32', 'numpy.Complex64', 'numpy.Float', 'nump y.Float32', 'numpy.Float64', 'numpy.Int', 'numpy.Int16', 'numpy.Int32', 'numpy.Int8', 'numpy.UInt16', 'numpy.UInt32', 'numpy.UInt8', 'numpy.arccos', 'numpy.arcc osh', 'numpy.arcsin', 'numpy.arcsinh', 'numpy.arctan', 'numpy.arctanh', 'numpy.bitwise_and', 'numpy.bitwise_or', 'numpy.bitwise_xor', 'numpy.ceil', 'numpy.conju gate', 'numpy.core.add', 'numpy.core.bitwise_and', 'numpy.core.bitwise_or', 'numpy.core.bitwise_xor', 'numpy.core.cdouble', 'numpy.core.complexfloating', 'numpy .core.conjugate', 'numpy.core.csingle', 'numpy.core.divide', 'numpy.core.double', 'numpy.core.equal', 'numpy.core.float64', 'numpy.core.greater', 'numpy.core.gr eater_equal', 'numpy.core.inexact', 'numpy.core.intc', 'numpy.core.invert', 'numpy.core.left_shift', 'numpy.core.less', 'numpy.core.less_equal', 'numpy.core.max imum', 'numpy.core.not_equal', 'numpy.core.power', 'numpy.core.remainder', 'numpy.core.right_shift', 'numpy.core.sin', 'numpy.core.single', 'numpy.core.sqrt', ' numpy.core.subtract', 'numpy.cosh', 'numpy.dft.old', 'numpy.divide', 'numpy.e', 'numpy.fabs', 'numpy.floor', 'numpy.floor_divide', 'numpy.fmod', 'numpy.greater' , 'numpy.hypot', 'numpy.invert', 'numpy.left_shift', 'numpy.less', 'numpy.log', 'numpy.log10', 'numpy.logical_and', 'numpy.logical_not', 'numpy.logical_or', 'nu mpy.logical_xor', 'numpy.maximum', 'numpy.minimum', 'numpy.negative', 'numpy.not_equal', 'numpy.power', 'numpy.random.rand', 'numpy.random.randn', 'numpy.remain der', 'numpy.right_shift', 'numpy.sinh', 'numpy.tan', 'numpy.tanh', 'numpy.true_divide'] Do you know what to do to solve this problem? Thank you very much for your answers. This e-mail communication contains information that is confidential and may also be privileged. It is intended for the exclusive use of the addressees. If you are not the person or organization to whom it is addressed, you must not copy, distribute or take any action in reliance upon it. If you received this communication in error, please notify Septentrio nv immediately [ telephone +32 [0] 16 300800 ]. Septentrio nv will not accept liability for contractual commitments made by individuals employed by this company outside the scope of our business. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Thu Oct 12 09:58:21 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 08:58:21 -0500 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> Message-ID: <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> On 10/11/06, Bill Baxter wrote: > > On 10/12/06, Greg Willden wrote: > > Speed should not take precedence over correctness. > > Unless your goal is speed. Then speed should take precedence over > correctness. Huh? Person 1: "Hey you should use function X." Person 2: "No, it doesn't give me the correct answer." Person 1: "Who cares? It's fast!" What kind of logic is that? I'm having serious doubts about my conversion to Numpy. Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From aisaac at american.edu Thu Oct 12 10:13:46 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 12 Oct 2006 10:13:46 -0400 Subject: =?iso-8859-1?q?Should_numpy=2Esqrt=28-1=29_ret?= =?iso-8859-1?q?urn_1j_rather_than=09nan=3F?= In-Reply-To: <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu><20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu><903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> Message-ID: On Thu, 12 Oct 2006, Greg Willden apparently wrote: > I'm having serious doubts about my conversion to Numpy. That's fine, but be sure to treat the explanations you are receiving with the same seriousness that you treat your doubts. For example, in the speed vs. correctness discussion, be sure to understand the problems with the word "correct". Which have been explained. correct != expected when expectations lack context Cheers, Alan Isaac (just a happy user) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jg307 at cam.ac.uk Thu Oct 12 10:14:33 2006 From: jg307 at cam.ac.uk (James Graham) Date: Thu, 12 Oct 2006 15:14:33 +0100 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> Message-ID: <452E4DC9.8010805@cam.ac.uk> Greg Willden wrote: > On 10/11/06, *Bill Baxter* > wrote: > > On 10/12/06, Greg Willden > wrote: > > Speed should not take precedence over correctness. > > Unless your goal is speed. Then speed should take precedence over > correctness. > > > Huh? > Person 1: "Hey you should use function X." > Person 2: "No, it doesn't give me the correct answer." > Person 1: "Who cares? It's fast!" > > What kind of logic is that? > > I'm having serious doubts about my conversion to Numpy. It's more like Person 1: Function x is designed to be fast, have predictable memory usage and so be suitable for a wide variety of applications. Alternatively there is function y which is slower but handles a larger numerical domain. y Might be more sutiable for your needs; you should use it. Person 2: Numpy is broken! Function x should just do what I want! Really, given that both options are available, I don't see a big issue here. Claims that returning j for sqrt(-1) is "more correct" seem highly dubious as a) NumPy is a numerical library, not an algebraic one, hence the fact that it behaves in a type-dependent way is not so surprising, and b) no one complains that functions like sqrt return only positive floats, despite the fact that, mathematically, this function is multivalued. -- "Eternity's a terrible thought. I mean, where's it all going to end?" -- Tom Stoppard, Rosencrantz and Guildenstern are Dead ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 10:25:46 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 07:25:46 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <6946b9500610120058l1a1d008agd19104ec23d7440a@mail.gmail.com> References: <6946b9500610120058l1a1d008agd19104ec23d7440a@mail.gmail.com> Message-ID: <452E506A.6060306@ieee.org> Mark Bakker wrote: > My vote is for consistency in numpy. > But it is unclear what consistency is. > > What is truly confusing for newbie Python users (and a source for > error even after 5 years of Python programming) is that > > >>> 2/3 > 0 I recommend that you slap "from __future__ import division" into site.py or the top of your program: from __future__ import division import numpy print 3/2 a = numpy.arange(3) print a / (a+5) print a // a+5 >>>> 1.5 [ 0. 0.16666667 0.28571429] [5 6 6] > > In that respect, I would think > > >>> numpy.sqrt(2) > > should give 1, but it gives 1.4142135623730951 Is there any practical reason to return 1? If not, isn't this argument sort of silly? > So numpy does typechecking anyway (it gets an integer and makes it a > float). > > If that is the consistent behavior, then by all means > > >>> sqrt(-1) > > should return 1j. Well, it could also return: -1j or as someone mentioned one of six (I think) different quaternion values. It all depends on what the domain/range of the problem is. > Wouldn't that be the consistent thing to do???? No, there's a difference. In order to do the former, all that is required is that sqrt switches on the *type* of it argument. sqrt return a float for integer and floating point args and a complex for complex args. In order to the latter, sqrt needs to switch on the *value* of its argument, which is an entirely different beast both in theory and in practice. In particular: sqrt(some_big_array) would have to scan all the values in the array to determine if any were negative to decide whether to return a float or an imaginary number. It also means that the memory usage is unpredictable -- the returned array is double in size if any values are negative. In addition to the significant slowdown this introduces, the former approach (keeping the input and output domains the same) is somewhat more robust against error, particularly if one tightens up the error mode. I would guess that people are working the complex plane at most half the time, in the former case a negative square root signals a problem and promoting to complex is at best a nuisance and may result in a painful to track down error. In the latter case, it's easy enough for me to toss in a astype in situations where I'm mixing domains and need complex outputs. Ideally numpy and scipy would have chosen different names for forgiving and strict powers and square roots, square_root and power versus sqrt and pow for example, but it's probably too late to do anything about that now. Since it sounds like Travis is going to tighten up the default error mode, I think that this is a non issue. No ones going to run into NANs unexpectedly and the error when doing sqrt([1,2,3,-1]) should be confusing and most once. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 10:27:05 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 07:27:05 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DF3FB.8010201@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEF81.4040208@ieee.org> <452DF3FB.8010201@ieee.org> Message-ID: <452E50B9.80008@ieee.org> Travis Oliphant wrote: > Travis Oliphant wrote: > >> Now, it would be possible to give ufuncs a dtype keyword argument that >> allowed you to specify which underlying loop was to be used for the >> calculation. That way you wouldn't have to convert inputs to complex >> numbers before calling the ufunc, but could let the ufunc do it in >> chunks during the loop. That is certainly a reasonable enhancement: >> >> sqrt(a, dtype=complex). >> >> This no-doubt has a "library-ish"-feeling, but that is what NumPy is. >> If such a change is desireable, I don't think it would be much to >> implement it. >> >> >> > This could be implemented, but only with a version number increase in > the C-API (we would have to change the c-signature of the ufunc tp_call. > > This would mean that the next release of NumPy would be binary > incompatible with packages built to previous NumPy releases. > > I've really been trying to avoid doing that. So, unless there are > strong requests for this feature that outweigh the negatives of > re-building dependent packages, then this feature will have to wait. > > OTOH: I suppose it could be implemented in a different way (using > indexing or a method call): > > sqrt[complex](a) --- I remember Tim suggesting some use for indexing > on ufuncs earlier though. > It wouldn't surprise me if I did -- it sounds like the kind of thing I'd propose -- but I certainly can't remember what I was proposing. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 12 10:28:28 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 16:28:28 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> Message-ID: <20061012142828.GG32224@mentat.za.net> On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: > On 10/11/06, Bill Baxter wrote: > > On 10/12/06, Greg Willden wrote: > > Speed should not take precedence over correctness. > > Unless your goal is speed. Then speed should take precedence over > correctness. > > > Huh? > Person 1: "Hey you should use function X." > Person 2: "No, it doesn't give me the correct answer." > Person 1: "Who cares? It's fast!" > > What kind of logic is that? I tried to explain the argument at http://www.scipy.org/NegativeSquareRoot Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 10:31:05 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 07:31:05 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE98A.4090404@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <452DBFBB.8090403@ieee.org> <452DE98A.4090404@ieee.org> Message-ID: <452E51A9.5030403@ieee.org> Travis Oliphant wrote: >> Personally I think that the default error mode should be tightened >> up. >> Then people would only see these sort of things if they really care >> about them. Using Python 2.5 and the errstate class I posted earlier: >> >> # This is what I like for the default error state >> numpy.seterr (invalid='raise', divide='raise', over='raise', >> under='ignore') >> >> >> I like these choices too. Overflow, division by zero, and sqrt(-x) are >> usually errors, indicating bad data or programming bugs. Underflowed >> floats, OTOH, are just really, really small numbers and can be treated >> as zero. An exception might be if the result is used in division and >> no error is raised, resulting in a loss of accuracy. >> >> > > I'm fine with this. I've hesitated because error checking is by default > slower. But, I can agree that it is "less surprising" to new-comers. > People that don't mind no-checking can simply set their defaults back to > 'ignore' > > Great. One thing we may want to do (numarray had this), was add a pseudo argument 'all', that allows you to set all of the values at once. Then if you want the full-bore, ignore-all-errors computation (and your using 2.5 and "from __future__ import with_statement") you can just do: with errstate(all='ignore'): # computation here # back to being picky -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 10:43:13 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 07:43:13 -0700 Subject: Please test the SVN branch In-Reply-To: <452DE2FB.3070408@ieee.org> References: <452DE2FB.3070408@ieee.org> Message-ID: <452E5481.5070403@ieee.org> Travis Oliphant wrote: > I made some fixes to the "asbuffer" code which let me feel better about > exposing it in NumPy (where it is now named int_asbuffer). > > This code takes a Python integer and a size and returns a buffer object > that points to that memory. A little test is performed by trying to > read (and possibly write if a writeable buffer is requested) to the > first and last elements of the buffer. Any segfault is trapped and used > to raise a Python error indicating you can't use that area of memory. > > It doesn't guarantee you won't shoot yourself, but it does make it more > difficult to segfault Python. Previously a simple int_asbuffer(3423423, > 5) would have segfaulted (unless by chance you the memory area 3423423 > happens to be owned by Python). > > I have not tested the code on other platforms to make sure it works as > expected, so please try and compiled it. > FWIW, it compiles fine and passes all tests under Visual Studio and Python 2.4.3. I haven't tried to use it though. -tim > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Thu Oct 12 10:53:12 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 12 Oct 2006 10:53:12 -0400 Subject: =?iso-8859-1?q?Should_numpy=2Esqrt=28-1=29_ret?= =?iso-8859-1?q?urn_1j_rather_than=09nan=3F?= In-Reply-To: <20061012142828.GG32224@mentat.za.net> References: <452D56F5.9000404@ee.byu.edu><20061011230705.GA30571@mentat.za.net><452D7C88.3090506@ee.byu.edu><903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com><903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> <20061012142828.GG32224@mentat.za.net> Message-ID: On Thu, 12 Oct 2006, Stefan van der Walt apparently wrote: > I tried to explain the argument at > http://www.scipy.org/NegativeSquareRoot Helpful. But you start off by saying: In mathematics, the above assumption is true -- that the square root of -1 is 1j. Since square root is (here) a function, and part of the function definition is the domain and codomain, this statement is not correct. If the codomain is real numbers, the domain must correspondingly be (a subset of) the nonnegative reals. The nan output is the result of an **invalid** input. So the question is really: "Why is a negative real number an invalid input in this implementation", or "Why in this implementation is the type of the output restricted by the type of the input?" You get a good start on answering that. Cheers, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Thu Oct 12 10:54:22 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 09:54:22 -0500 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> Message-ID: <903323ff0610120754t399f3626o7fce374ad8d85582@mail.gmail.com> I'll try to quit being a pain and just work around it. Those wiki pages are really good to capture this information. I need to read all of them to know what other gotchas are out there. Thanks and Sorry, Greg On 10/12/06, Alan G Isaac wrote: > > That's fine, but be sure to treat the explanations you are > receiving with the same seriousness that you treat your > doubts. > > For example, in the speed vs. correctness discussion, be > sure to understand the problems with the word "correct". > Which have been explained. > > correct != expected > when expectations lack context > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Thu Oct 12 11:19:05 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 10:19:05 -0500 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610120754t399f3626o7fce374ad8d85582@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> <903323ff0610120754t399f3626o7fce374ad8d85582@mail.gmail.com> Message-ID: <903323ff0610120819t1a758a61y7f79e646b77266b@mail.gmail.com> May I suggest the following change to generate_umath.py? 'sqrt' : Ufunc(1, 1, None, 'square-root elementwise. For real x, the domain is restricted to x>=0.\n\ For complex results for x<0 see numpy.scimath.sqrt', TD(inexact, f='sqrt'), TD(M, f='sqrt'), ), ' When sqrt throws a ValueError would it be possible/appropriate for the error message to mention numpy.scimath.sqrt? I'm just trying to think of other ways to help make the transition as smooth as possible for new users. Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From a.h.jaffe at gmail.com Thu Oct 12 11:35:19 2006 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Thu, 12 Oct 2006 16:35:19 +0100 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452D9D23.5030700@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452D5843.4050007@gmx.net> <452D5BFA.7000603@ee.byu.edu> <452D70B9.7070209@ieee.org> <452D8277.3080306@ee.byu.edu> <452D9D23.5030700@ieee.org> Message-ID: <452E60B7.8030400@gmail.com> Tim Hochberg wrote: > Travis Oliphant wrote: >> Tim Hochberg wrote: >> >> >>> With python 2.5 out now, perhaps it's time to come up with a with >>> statement context manager. Something like: >>> >>> a = numpy.arange(10) >>> a/a # ignores divide by zero >>> with errstate(divide='raise'): >>> a/a # raise exception on divide by zer >>> # Would ignore divide by zero again if we got here. >>> >>> -tim >>> >> This looks great. I think most people aren't aware of the with >> statement and what it can do (I'm only aware because of your posts, for >> example). >> >> So, what needs to be added to your example in order to just add it to >> numpy? >> > As far as I know, just testing and documentation -- however testing was > so minimal that I may find some other stuff. I'll try to clean it up > tomorrow so that I'm a little more confident that it works correctly and > I'll send another note out then. For this particular application, why not a context manager which just substitutes in the appropriately-optimized version of sqrt? That is, don't change the error state, but actually change the value of the object pointed at by the name sqrt? Andrew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From hetland at tamu.edu Thu Oct 12 11:52:15 2006 From: hetland at tamu.edu (Rob Hetland) Date: Thu, 12 Oct 2006 10:52:15 -0500 Subject: Things to address for Py3K In-Reply-To: <452D3AC7.7000805@ee.byu.edu> References: <452D3AC7.7000805@ee.byu.edu> Message-ID: On Oct 11, 2006, at 1:41 PM, Travis Oliphant wrote: > a) an extra multiplication operator to distinguish between > element-by-element and dot One of the things I like about numpy is that element-by-element multiplication is default. Really, most of what I do is element by element, punctuated by a few matrix multiplications (or whatever other matrix operations) at the end. If I had a penny for every '.' I put in front of an operator in MATLAB, I would buy us all new computers -- expensive ones even. I hope that, if there is another multiplication operator like .*, that this will be shorthand for dot(A, B), and not for elem-by-elem multiplication. -Rob ---- Rob Hetland, Associate Professor Dept. of Oceanography, Texas A&M University http://pong.tamu.edu/~rob phone: 979-458-0096, fax: 979-845-6331 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Thu Oct 12 12:07:09 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Thu, 12 Oct 2006 12:07:09 -0400 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DEAED.9070408@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> Message-ID: On 12/10/06, David Goldsmith wrote: > I don't use scipy (and don't want to because of the overhead) but it > sounds like I should because if I'm taking the square root of a variable > whose value at run time happens to be real but less than zero, I *want* > the language I'm using to return an imaginary; in other words, it's not > the scipy behavior which "scares" me, its the numpy (which I do/have > been using) behavior. To which you might say, "Well, if that's what you > want, and you have Matlab (as you've said you do), then just use that." > But that's precisely the point: people who don't want to be bothered > with having to be "a bit more care[ful]" (as Chuck put it) - and who can > afford it - are going to be inclined to choose Matlab over numpy. > Perhaps one doesn't care - in the grand scheme of things, it certainly > doesn't matter - but I think that you all should be aware that this > numpy "feature" will be seen by many as more than just a nuisance. Scipy doesn't do what you want. What you want is to use complex numbers throughout; then numpy and scipy will do exactly what you request. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ndbecker2 at gmail.com Thu Oct 12 12:03:45 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Oct 2006 12:03:45 -0400 Subject: PATCH build numpy-1.0rc2 Message-ID: I have not been able to build any of the numpy-1.0 series. Finally I bit the bullet and fixed it. Here is the patch: diff -u numpy-1.0rc2/numpy/numarray/setup.py{~,} --- numpy-1.0rc2/numpy/numarray/setup.py~ 2006-07-26 16:04:32.000000000 -0400 +++ numpy-1.0rc2/numpy/numarray/setup.py 2006-10-12 11:58:23.000000000 -0400 @@ -5,7 +5,8 @@ from numpy.distutils.misc_util import Configuration config = Configuration('numarray',parent_package,top_path) - config.add_data_files('numpy/') +# config.add_data_files('numpy/') + config.add_data_dir('numpy/') # Configure fftpack_lite config.add_extension('_capi', ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 12 12:14:50 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 12 Oct 2006 18:14:50 +0200 Subject: Should numpy.sqrt(-1) return 1j rather than?nan? In-Reply-To: References: <20061012142828.GG32224@mentat.za.net> Message-ID: <20061012161450.GH32224@mentat.za.net> On Thu, Oct 12, 2006 at 10:53:12AM -0400, Alan G Isaac wrote: > On Thu, 12 Oct 2006, Stefan van der Walt apparently wrote: > > I tried to explain the argument at > > http://www.scipy.org/NegativeSquareRoot > > Helpful. But you start off by saying: > In mathematics, the above assumption is true -- that > the square root of -1 is 1j. > > Since square root is (here) a function, and part of the > function definition is the domain and codomain, this > statement is not correct. If the codomain is real numbers, > the domain must correspondingly be (a subset of) the > nonnegative reals. The nan output is the result of an > **invalid** input. > > So the question is really: "Why is a negative real number an > invalid input in this implementation", or "Why in this > implementation is the type of the output restricted by the > type of the input?" You get a good start on answering that. You are more than welcome to change the wiki if you can think of a simpler way to explain the problem... In fact, I would encourage that -- I just put it there as a starting point. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Thu Oct 12 12:24:40 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 09:24:40 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <903323ff0610120819t1a758a61y7f79e646b77266b@mail.gmail.com> References: <452D56F5.9000404@ee.byu.edu> <20061011230705.GA30571@mentat.za.net> <452D7C88.3090506@ee.byu.edu> <903323ff0610112017j26d15028nede93f8d3fa31100@mail.gmail.com> <903323ff0610120658w267d8211sc7eabceca3b3f93b@mail.gmail.com> <903323ff0610120754t399f3626o7fce374ad8d85582@mail.gmail.com> <903323ff0610120819t1a758a61y7f79e646b77266b@mail.gmail.com> Message-ID: <452E6C48.1030607@noaa.gov> Greg Willden wrote: > May I suggest the following change to generate_umath.py? > > > 'sqrt' : > Ufunc(1, 1, None, > 'square-root elementwise. For real x, the domain is > restricted to x>=0.\n\ > For complex results for x<0 see numpy.scimath.sqrt', > TD(inexact, f='sqrt'), > TD(M, f='sqrt'), > ), > ' > > When sqrt throws a ValueError would it be possible/appropriate for the > error message to mention numpy.scimath.sqrt? > > I'm just trying to think of other ways to help make the transition as > smooth as possible for new users. And helpful error messages are one of Python's best features - in general, numpy doesn't have enough of 'em, IMO. DG > > Greg > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Thu Oct 12 13:34:00 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 12 Oct 2006 10:34:00 -0700 Subject: Please test the SVN branch In-Reply-To: <452DE2FB.3070408@ieee.org> References: <452DE2FB.3070408@ieee.org> Message-ID: <452E7C88.7000205@noaa.gov> Travis Oliphant wrote: > I made some fixes to the "asbuffer" code which let me feel better about > exposing it in NumPy (where it is now named int_asbuffer). > > This code takes a Python integer and a size and returns a buffer object > that points to that memory. If I understand this correctly, maybe it should be called something like: int_asbuffer_pointer or something with "pointer" in it. I sure didn't expect that meaning from "asbuffer" or in-asbuffer" Otherwise -- nice idea! -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From vallis at vallis.org Thu Oct 12 13:41:37 2006 From: vallis at vallis.org (Michele Vallisneri) Date: Thu, 12 Oct 2006 10:41:37 -0700 Subject: Providing array interface from SWIG-wrapped C struct Message-ID: Does anybody here have experience about offering the array interface from a SWIG-wrapped C struct? I have tried the following, borrowing code from numpy's arrayobject.c: %extend real_vec_t { PyObject *__array_struct__() { /* From numpy/arrayobject.c/array_struct_get */ PyArrayInterface *inter; inter = (PyArrayInterface *)_pya_malloc(sizeof(PyArrayInterface)); inter->two = 2; inter->nd = 1; inter->typekind = 'f'; inter->itemsize = sizeof(float); inter->flags = (NPY_CONTIGUOUS | NPY_OWNDATA | NPY_ALIGNED | NPY_NOTSWAPPED); inter->shape = (intp *)_pya_malloc(2*sizeof(intp)); inter->strides = inter->shape + 1; *(inter->shape) = self->n; *(inter->strides) = 1; inter->data = (void *)(self->d); inter->descr = 0; return PyCObject_FromVoidPtr(inter,0); } %} When passing such a SWIG real_vec_t object to numpy.array, numpy gets a PyArrayInterface that fails to check correctly as a Python C object. Three comments: - The original numpy/arrayobject.c code actually gets its return value from PyCObject_FromVoidPtrAndDesc(inter,self,...); but in this context self does not point to a PyObject, but rather to the C struct, so I don't think I can use that form. - The code above does not deal with properly destructing the PyArrayInterface, setting refcounts, etc. But I was going to worry about those later. - My problem is probably with SWIG rather than numpy... but I thought the people in this forum (rather than those at the SWIG forum) would be most likely to have encountered a situation similar to these. Thanks a lot for any help, or just for reading this post. Michele ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Thu Oct 12 13:49:25 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 12 Oct 2006 10:49:25 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DE009.3070100@ieee.org> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> Message-ID: <452E8025.3060005@noaa.gov> As someone on this list (sorry to quick with the delete button) said: "numpy is closer to the metal than MATLAB" MATLAB aside, numpy is somewhat "close to the metal". I think this is clearly part of its design philosophy, and also something I personally like about out. Getting back to MATLAB (at least version 4, the last one I used regularly) -- everything in MATLAB is a matrix of double precision Floating points. (or complex - that does get weird!). This makes things simpler for a lot of use, but restricts the power and flexibility substantially. Everything in numpy is an array of arbitrary shape and many possible data types -- this is more to think about, but give you a lot of power and flexibility. In any case, that is what numpy is. Period. Given that, arrays of different types behave differently -- that's something you'd darn well better understand pretty soon in your numpy career: >>> a = N.array((1,2,3), dtype=N.uint8) >>> a *= 200 >>> a array([200, 144, 88], dtype=uint8) Oh my gosh! 2 * 200 isn't 144! what happened? This isn't any different than sqrt(-1) resulting in NaN or an error. In fact, there are all sorts of places where upcasting is not happening automagically -- in fact, I think that's become more consistent in the new numpy. So, in numpy you need to choose the datatype appropriate for your problem at hand. I know I always know whether I want to work in the complex plane or not. Another change recently made is to make doubles the default in more places -- I like that change. So, given the entire philosophy and current status of how numpy works, the ONLY question that is legitimate here is: Should the "default" datatype be complex or double? I vote double. Travis Oliphant wrote: > Now, it would be possible to give ufuncs a dtype keyword argument that > allowed you to specify which underlying loop was to be used for the > calculation. > sqrt(a, dtype=complex). I like this OK, and isn't is similar to the dtype argument in the accumulating methods? (sum, etc.). However, it's probably not worth the C-api change -- is it that different than calling: sqrt(a.astype(complex)) ? As for scipy, I heartily agree that scipy should be a superset of numpy, and NOT change the behavior of numpy functions and methods. We're taking the opportunity at this point for numpy to break backward compatibility with Numeric/numarray -- this is probably the best time to do the same for scipy. couldn't we at least introduce new names? numpy.scimath.csqrt() for instance? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 14:04:55 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 11:04:55 -0700 Subject: multiarray tests skipped under py2.5 Message-ID: <452E83C7.5040404@ieee.org> Is any one else seeing the multiarray tests all get skipped because of an import error when compiling under python 2.5. Everything else seems to work and all the tests go fine under 2.4. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From loredo at astro.cornell.edu Thu Oct 12 14:03:31 2006 From: loredo at astro.cornell.edu (Tom Loredo) Date: Thu, 12 Oct 2006 14:03:31 -0400 Subject: gcc/fortran compilers for building extensions with Tiger/MacPython In-Reply-To: References: Message-ID: <1160676211.452e837390848@astrosun2.astro.cornell.edu> Hi folks, I've just upgraded my G4 (PPC) from Pather to Tiger, and just installed MacPython. I now need to build a slew of numerical extensions. I'm building them from source (I have lots of custom extensions to build from source in any case). I see the binaries available for Tiger are built with gcc 3.3 for PPC and gcc 4 for Intel; I think they may be tied to ActivePython but I'm not sure. When I start MacPython, I see it is built with gcc 4. Is this what I should be using? I'll also be compiling fortran extensions; I presume I should use gfortran rather than g77 for this (g77 is what worked with gcc 3.3). Is this correct? Thanks, Tom ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gerard.vermeulen at grenoble.cnrs.fr Thu Oct 12 14:13:38 2006 From: gerard.vermeulen at grenoble.cnrs.fr (Gerard Vermeulen) Date: Thu, 12 Oct 2006 20:13:38 +0200 Subject: multiarray tests skipped under py2.5 In-Reply-To: <452E83C7.5040404@ieee.org> References: <452E83C7.5040404@ieee.org> Message-ID: <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> On Thu, 12 Oct 2006 11:04:55 -0700 Tim Hochberg wrote: > > Is any one else seeing the multiarray tests all get skipped because of > an import error when compiling under python 2.5. Everything else seems > to work and all the tests go fine under 2.4. Everything looks fine here: >>> sys.version '2.5 (r25:51908, Oct 8 2006, 07:23:15) \n[GCC 4.1.1 (Gentoo 4.1.1)]' >>> np.version.version '1.0.dev3317' >>> np.test() Found 13 tests for numpy.core.umath Found 9 tests for numpy.lib.arraysetops Found 36 tests for numpy.core.ma Found 3 tests for numpy.fft.helper Found 4 tests for numpy.ctypeslib Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 26 tests for numpy.core.numeric Found 5 tests for numpy.distutils.misc_util Found 3 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 4 tests for numpy.core.scalarmath Found 1 tests for numpy.lib.ufunclike Found 12 tests for numpy.lib.twodim_base Found 47 tests for numpy.lib.shape_base Found 4 tests for numpy.lib.index_tricks Found 32 tests for numpy.linalg.linalg Found 42 tests for numpy.lib.type_check Found 176 tests for numpy.core.multiarray Found 10 tests for numpy.core.defmatrix Found 41 tests for numpy.lib.function_base Found 0 tests for __main__ ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ---------------------------------------------------------------------- Ran 509 tests in 0.265s OK >>> Gerard ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 14:22:01 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 11:22:01 -0700 Subject: multiarray tests skipped under py2.5 In-Reply-To: <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> References: <452E83C7.5040404@ieee.org> <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> Message-ID: <452E87C9.5000503@ieee.org> Gerard Vermeulen wrote: > On Thu, 12 Oct 2006 11:04:55 -0700 > Tim Hochberg wrote: > > >> Is any one else seeing the multiarray tests all get skipped because of >> an import error when compiling under python 2.5. Everything else seems >> to work and all the tests go fine under 2.4. >> > > Everything looks fine here: > Thanks. I suppose that means it's either Windows specific or I gummed up my build tree somehow. I'll make a clean checkout and see if it goes away. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From haase at msg.ucsf.edu Thu Oct 12 14:30:34 2006 From: haase at msg.ucsf.edu (Sebastian Haase) Date: Thu, 12 Oct 2006 11:30:34 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452E8025.3060005@noaa.gov> References: <452DE009.3070100@ieee.org> <452E8025.3060005@noaa.gov> Message-ID: <200610121130.34936.haase@msg.ucsf.edu> Hi, On Thursday 12 October 2006 10:49, Christopher Barker wrote: > As someone on this list (sorry to quick with the delete button) said: I think I might agree with everything (!) you that - very well written . > Travis Oliphant wrote: > > Now, it would be possible to give ufuncs a dtype keyword argument that > > allowed you to specify which underlying loop was to be used for the > > calculation. > > > > sqrt(a, dtype=complex). > > I like this OK, and isn't is similar to the dtype argument in the > accumulating methods? (sum, etc.). However, it's probably not worth the > C-api change -- is it that different than calling: > > sqrt(a.astype(complex)) ? > It is very late in the game - obvious ! (one week before the announced final release). But I think adding the dtype argument to ufuncs would be a very useful addition !! It is very nice to look at ! And it saves (temporary) memory and should be noticeably faster than "sqrt(a.astype(complex))". This dtype addition might address many of the issues I have raised before on this list. Travis, be courageous and change the C-API one last time before putting it in stone for 1.0 final !!! > As for scipy, I heartily agree that scipy should be a superset of numpy, > and NOT change the behavior of numpy functions and methods. We're taking > the opportunity at this point for numpy to break backward compatibility > with Numeric/numarray -- this is probably the best time to do the same > for scipy. > > couldn't we at least introduce new names? > > numpy.scimath.csqrt() > > for instance? > > -Chris It has to be very well documented what the 12 functions (the last 3%) are were scipy and numpy differ. --- if at all possible I would prefer a name change like csqrt (or better sqrtc - I always like seen things together when sorted alphabetically ;-) ) [by the way: I like sqrt(-1) == nan just fine] Thanks for the great work. - Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Thu Oct 12 14:40:46 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 12 Oct 2006 15:40:46 -0300 Subject: Providing array interface from SWIG-wrapped C struct In-Reply-To: References: Message-ID: On 10/12/06, Michele Vallisneri wrote: > Does anybody here have experience about offering the array interface > from a SWIG-wrapped C struct? I have. > I have tried the following, borrowing code from numpy's arrayobject.c: > > %extend real_vec_t { > PyObject *__array_struct__() { > /* From numpy/arrayobject.c/array_struct_get */ You are extending real_vec_t with a new METHOD, but what numpy requests is an ATTRIBUTE. So, numpy simply queries your vec like: arrstr = vec.__array_struct__ and not with a method call like this arrstr = vec.__array_struct__() So here is what I would do (can fail with some SWIG optimizations) %extend Vec { PyObject* __array_struct__ () { /* ... */ } %pythoncode { __array_struct__ = property(__array_struct__, doc='Array protocol') } } Hope you got the idea. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ndbecker2 at gmail.com Thu Oct 12 13:54:16 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Oct 2006 13:54:16 -0400 Subject: numpy-1.0 problem with install --skip-build Message-ID: It appears the problem with getting a numpy-1.0 built with RPM is caused by using 2 step process: python setup.py build python setup.py install --skip-build --root $RPM_BUILD_ROOT This doesn't seem to work correctly! If I change it to just do: python setup.py install it works OK. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cookedm at physics.mcmaster.ca Thu Oct 12 14:30:47 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Thu, 12 Oct 2006 14:30:47 -0400 Subject: gcc/fortran compilers for building extensions with Tiger/MacPython In-Reply-To: <1160676211.452e837390848@astrosun2.astro.cornell.edu> References: <1160676211.452e837390848@astrosun2.astro.cornell.edu> Message-ID: <20061012143047.6fac6e27@arbutus.physics.mcmaster.ca> On Thu, 12 Oct 2006 14:03:31 -0400 Tom Loredo wrote: > Hi folks, > > I've just upgraded my G4 (PPC) from Pather to Tiger, and just > installed MacPython. I now need to build a slew of numerical extensions. > I'm building them from source (I have lots of custom extensions > to build from source in any case). I see the binaries available > for Tiger are built with gcc 3.3 for PPC and gcc 4 for Intel; I > think they may be tied to ActivePython but I'm not sure. When > I start MacPython, I see it is built with gcc 4. Is this what > I should be using? I'll also be compiling fortran extensions; > I presume I should use gfortran rather than g77 for this (g77 > is what worked with gcc 3.3). Is this correct? For Intel, yes, you need to use gfortran, and it has to be the prelease version (4.2): 4.1 and earlier don't build on Intel Macs. g77 (and gcc 3.x) won't build either, unfortunately. If you're using MacPorts (previously DarwinPorts), you'll want the gcc42 port. Or, you can grab binaries at http://hpc.sourceforge.net/ -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pfdubois at gmail.com Thu Oct 12 14:50:12 2006 From: pfdubois at gmail.com (Paul Dubois) Date: Thu, 12 Oct 2006 11:50:12 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452E8025.3060005@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452E8025.3060005@noaa.gov> Message-ID: Historical note. When Cleve Moler first created 'matlab' as a tool for teaching at the University of New Mexico, all numbers were complex. And, in fact, there was a limit of 10,000 numbers. There was a fixed array where these 10,000 numbers were stored. I got pretty excited about his project and wanted to use it to get eigenvalues etc. for matrices of interest from our codes. But I had a Cray 1 with an entire megaword of memory and my matrices wouldn't fit. So I set out to hack the code to add a memory manager so that we could use the entire memory. This was a big success. Then I decided it was sad that I had to write the matrices out of the code onto a disk and then read them into matlab, so I got the idea of making something similar as the front end of the code, only with data types (because I hated only being able to use half my Cray since my data was real, and our codes were often memory bound) and a full programming language like Fortran with array syntax for speed. This idea resulted in Basis (basis.llnl.gov). My interest in Python (and hence numpy) evolved from that as OOP arrived. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Thu Oct 12 14:35:35 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 11:35:35 -0700 Subject: multiarray tests skipped under py2.5 In-Reply-To: <452E87C9.5000503@ieee.org> References: <452E83C7.5040404@ieee.org> <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> <452E87C9.5000503@ieee.org> Message-ID: <452E8AF7.1040203@ieee.org> Tim Hochberg wrote: > Gerard Vermeulen wrote: > >> On Thu, 12 Oct 2006 11:04:55 -0700 >> Tim Hochberg wrote: >> >> >> >>> Is any one else seeing the multiarray tests all get skipped because of >>> an import error when compiling under python 2.5. Everything else seems >>> to work and all the tests go fine under 2.4. >>> >>> >> Everything looks fine here: >> >> > > Thanks. > > I suppose that means it's either Windows specific or I gummed up my > build tree somehow. I'll make a clean checkout and see if it goes away. > > Still there with a clean checkout. Blech! I hate weird import problems. I'll try to look into it, but if anyone has any good ideas, let me know -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 15:32:28 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 12:32:28 -0700 Subject: multiarray tests skipped under py2.5 In-Reply-To: <452E9726.9020405@ee.byu.edu> References: <452E83C7.5040404@ieee.org> <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> <452E87C9.5000503@ieee.org> <452E8AF7.1040203@ieee.org> <452E9726.9020405@ee.byu.edu> Message-ID: <452E984C.5000709@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > > >>> >>> >>> >>> >> Still there with a clean checkout. Blech! I hate weird import problems. >> I'll try to look into it, but if anyone has any good ideas, let me know >> >> >> > Show us the errors, I may be able to help. It's possible some recent > changes messed things up. > I just checked in a fix. I think it was some case problem in the paths that got set in set_local_path(). In any event, applying normpath to the merged path fixed the problem and I didn't feel like pursuing it further. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 15:24:28 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 12:24:28 -0700 Subject: More SVN testing Message-ID: <452E966C.1060300@ieee.org> I just checked in a couple of changes to SVN. I was going to check in errstate, but it looks like Travis beat me to it, so I contented myself with adding a docstring and some tests. These tests are only run under 2.5; things seem to work fine here, but if someone on a Linux box whose running Python 2.5 could try it out I'd appreciate it. I'm suspicous of Linux/Windows differences right now because of my run in with them earlier today in the tests for test_multiarray (which I believe it now fixed -- at least everything is working here OK). -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 12 15:27:34 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 12 Oct 2006 13:27:34 -0600 Subject: multiarray tests skipped under py2.5 In-Reply-To: <452E8AF7.1040203@ieee.org> References: <452E83C7.5040404@ieee.org> <20061012201338.44fc666c.gerard.vermeulen@grenoble.cnrs.fr> <452E87C9.5000503@ieee.org> <452E8AF7.1040203@ieee.org> Message-ID: <452E9726.9020405@ee.byu.edu> Tim Hochberg wrote: >> >> >> >Still there with a clean checkout. Blech! I hate weird import problems. >I'll try to look into it, but if anyone has any good ideas, let me know > > Show us the errors, I may be able to help. It's possible some recent changes messed things up. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 12 15:15:58 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 12 Oct 2006 13:15:58 -0600 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452DF49A.407@noaa.gov> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> <452DF1BA.9000906@ieee.org> <452DF49A.407@noaa.gov> Message-ID: <452E946E.8000809@ee.byu.edu> David Goldsmith wrote: >Got it. And if I understand correctly, the import order you specify in >the little mynumpy example you included in your latest response to >Fernando will result in any "overlap" between numpy and >numpy.lib.scimath to call the latter's version of things rather than the >former's, yes? > > Right. The last import will be used for any common-names (variables get re-bound to the new functions...) -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Thu Oct 12 15:53:10 2006 From: fullung at gmail.com (Albert Strasheim) Date: Thu, 12 Oct 2006 21:53:10 +0200 Subject: More SVN testing In-Reply-To: <452E966C.1060300@ieee.org> References: <452E966C.1060300@ieee.org> Message-ID: Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Tim Hochberg > Sent: 12 October 2006 21:24 > To: numpy-discussion > Subject: [Numpy-discussion] More SVN testing > > > I just checked in a couple of changes to SVN. I was going to check in > errstate, but it looks like Travis beat me to it, so I contented myself > with adding a docstring and some tests. These tests are only run under > 2.5; things seem to work fine here, but if someone on a Linux box whose > running Python 2.5 could try it out I'd appreciate it. I'm suspicous of > Linux/Windows differences right now because of my run in with them > earlier today in the tests for test_multiarray (which I believe it now > fixed -- at least everything is working here OK). This probably fixes the issue I reported here: http://projects.scipy.org/scipy/numpy/ticket/286 Maybe you can check that the test counts match up on Python 2.4 and 2.5 on Windows on your system and close that ticket. Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Thu Oct 12 15:59:40 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 14:59:40 -0500 Subject: Polyfit Message-ID: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> Hi All, I'm using numpy.polyfit and it is giving me some really strange results when evaluated near 0. So I compared it with polyfit in matplotlib and the code and docstrings are nearly identical. However the slight differences in the code make a huge difference on my data. Here is the Numpy version with line numbers added: 1 x = NX.asarray(x)+0. 2 y = NX.asarray(y)+0. 3 y = NX.reshape(y, (len(y), 1)) 4 X = vander(x, N+1) 5 c, resids, rank, s = _lstsq(X, y) 6 c.shape = (N+1,) 7 return c And here is the MPL version with line numbers added: 1 x = asarray(x)+0. 2 y = asarray(y)+0. 3 y = reshape(y, (len(y),1)) 4 X = Matrix(vander(x, N+1)) 5 Xt = Matrix(transpose(X)) 6 c = array(linear_algebra.inverse(Xt*X)*Xt*y) # convert back to array 7 c.shape = (N+1,) 8 return c So lines 1-4 are basically the same. The MPL version produces a much more stable representation of my data Can someone please comment on the differences? Should these two be exactly the same? If not, what are the advantages to each algorithm? I'm attaching my x and y points dumped to files. Here is the test code using "ipython -pylab" import numpy x=numpy.load('x.data') y=numpy.load('y.data') semilogy(numpy.polyval(numpy.polyfit(x,y,3),arange(3604)));hold(True) semilogy(polyval(polyfit(x,y,3),arange(3604))) #Use the MPL versions scatter(x,y) Regards, Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: x.data Type: application/octet-stream Size: 4451 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: y.data Type: application/octet-stream Size: 8771 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 12 15:19:16 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 12 Oct 2006 13:19:16 -0600 Subject: Compiling Numpy with py2exe In-Reply-To: <8B7FAE730B4A3646B51C94CD6C8784B7B28F52@serversbs2003.Septentrio.local> References: <8B7FAE730B4A3646B51C94CD6C8784B7B28F52@serversbs2003.Septentrio.local> Message-ID: <452E9534.10100@ee.byu.edu> Charles Hanot wrote: > Hello, > > I?ve a program using Numpy and I would like to compile it with Py2exe > but I?ve some problems. When I try to compile my program, some modules > generated by numpy are not loaded. I?ve already asked to the py2exe > community if they have a solution for that and the answer was that it > was possible to compile numpy modules with py2exe but I should ask to > the Numpy community J > > So my question is, is there something special to do in order to > compile a program using numpy module? > The only thing I remember is that you have to explicitly make sure _internal.py file gets copied over as it is not possible to discern that by looking at the import statements in .py files. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 12 16:08:53 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 13:08:53 -0700 Subject: More SVN testing In-Reply-To: References: Message-ID: <452EA0D5.8020103@ieee.org> Albert Strasheim wrote: > Hello all > > >> -----Original Message----- >> From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- >> discussion-bounces at lists.sourceforge.net] On Behalf Of Tim Hochberg >> Sent: 12 October 2006 21:24 >> To: numpy-discussion >> Subject: [Numpy-discussion] More SVN testing >> >> >> I just checked in a couple of changes to SVN. I was going to check in >> errstate, but it looks like Travis beat me to it, so I contented myself >> with adding a docstring and some tests. These tests are only run under >> 2.5; things seem to work fine here, but if someone on a Linux box whose >> running Python 2.5 could try it out I'd appreciate it. I'm suspicous of >> Linux/Windows differences right now because of my run in with them >> earlier today in the tests for test_multiarray (which I believe it now >> fixed -- at least everything is working here OK). >> > > This probably fixes the issue I reported here: > > http://projects.scipy.org/scipy/numpy/ticket/286 > > Maybe you can check that the test counts match up on Python 2.4 and 2.5 on > Windows on your system and close that ticket. > Yep, that's the same one I ran into and it's fixed now on windows. It sounds like it didn't occur at all on Linux. Their are now actually two more tests under 2.5 than under 2.4, but that's on purpose -- I can't test 'with errstate' under 2.4, so I skip those tests. I went ahead and closed the ticket. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Thu Oct 12 17:15:34 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 16:15:34 -0500 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> Message-ID: <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> On 10/12/06, Charles R Harris wrote: > > I'm guessing that the rcond number in the lstsq version (default 1e-10) is > the difference. Generally the lstsq version should work better than the MPL > version because at*a is not as well conditioned and vandermonde matrices are > notoriously ill conditioned anyway for higher degree fits. It would help if > you attached the data files in ascii form unless they happen to contain > thousands of data items. Just the x will suffice, zip it up if you have to. > Here are the files. Since the two algorithms behave differently and each has it place then can both be included in numpy? i.e. numpy.polyfit(x,y,N, mode='alg1') numpy.polyfit(x,y,N, mode='alg2') replacing alg1 and alg2 with meaningful names. Thanks Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- 2 4 0 16 1 3 14 17 12 13 8 7 5 11 23 19 24 9 18 15 44 22 20 25 27 10 41 45 26 40 21 50 29 38 28 37 53 48 57 78 54 61 39 64 65 58 43 35 55 47 60 66 46 68 52 51 42 75 93 56 87 74 59 71 84 49 72 81 73 63 94 67 36 80 30 70 106 69 62 77 76 90 98 103 6 172 79 83 82 31 85 97 105 91 121 89 95 101 116 135 92 102 96 88 120 137 136 34 389 499 489 565 479 555 412 390 443 442 456 365 404 593 457 364 375 441 367 360 453 376 482 576 550 445 409 658 363 435 381 477 549 396 460 544 601 466 384 557 391 461 511 642 547 558 542 610 608 639 438 591 551 378 599 552 397 569 578 437 643 361 385 399 538 600 395 561 602 649 454 464 447 386 535 554 577 596 686 635 459 514 628 448 583 407 408 603 379 626 410 500 570 449 582 434 401 417 501 462 566 366 394 631 485 382 563 369 784 781 824 838 783 826 825 854 867 763 801 799 879 795 759 852 872 831 802 769 873 773 762 779 772 789 836 881 839 827 796 882 851 786 797 755 726 841 896 865 890 837 857 775 853 767 785 756 750 754 887 861 894 770 749 771 840 844 782 780 877 859 774 832 871 730 722 901 794 884 846 760 848 729 876 724 720 757 830 798 850 818 764 875 787 829 899 885 874 960 815 864 777 723 820 891 747 751 725 821 776 823 866 800 833 788 758 878 1411 1350 1358 1353 1362 1378 1326 1348 1321 1355 1357 1349 1352 1346 1373 1430 1408 1351 1310 1418 1281 1336 1416 1428 1423 1436 1364 1432 1419 1377 1421 1359 1279 1401 1424 1323 1340 1322 1339 1407 1438 1427 1247 1435 1290 1311 1262 1343 1307 1437 1369 1315 1288 1356 1412 1309 1327 1276 1367 1325 1425 1420 1344 1361 1337 1368 1314 1415 1335 1278 1320 1280 1382 1282 1434 1328 1273 1332 1410 1347 1284 1338 1417 1354 1360 1329 1270 1305 1375 1414 1287 1291 1429 1379 1256 1372 1370 1289 1306 1275 1342 1274 1286 1313 1271 1248 1319 1268 1479 1440 1452 1449 1441 1542 1444 1447 1645 1569 1485 1537 1527 1522 1462 1461 1554 1651 1448 1652 1538 1529 1510 1451 1639 1570 1486 1737 1481 1623 1725 1453 1458 1459 1680 1484 1445 1671 1460 1524 1673 1666 1627 1540 1564 1626 1672 1499 1674 1561 1555 1519 1472 1632 1534 1548 1650 1470 1656 1637 1648 1717 1661 1521 1724 1755 1631 1735 1517 1667 1551 1728 1566 1575 1515 1539 1531 1541 1563 1558 1643 1454 1549 1758 1483 1761 1714 1638 1505 1654 1664 1475 1556 1473 1528 1516 1552 1443 1709 1553 1794 1571 1644 1520 1718 1742 1498 1736 2149 2052 2128 2141 2019 2108 2125 1998 2055 2087 2106 2123 2029 2036 1960 2121 2049 1945 2132 2058 2035 2144 2053 2109 1963 1949 1966 2137 2060 2153 2147 1956 2126 2115 2130 2129 1808 2021 2038 1804 2064 2104 1952 2107 1911 1958 2116 1810 2031 1940 2148 2117 2150 1814 2059 1916 2010 2026 1809 1964 2103 2032 2039 2118 2151 1973 2152 1951 2084 2011 2065 2105 2028 2112 2044 2040 2122 2033 1812 1941 1946 1987 1996 2127 2113 2048 1995 1950 2155 1959 2138 1803 1970 1961 2020 2008 1811 1968 1965 1947 1819 2037 2063 1817 2133 1967 2140 2047 2185 2161 2217 2241 2163 2164 2212 2220 2247 2162 2208 2248 2225 2255 2206 2191 2218 2165 2238 2237 2166 2231 2211 2192 2258 2233 2201 2221 2239 2251 2249 2219 2186 2197 2246 2210 2199 2184 2226 2243 2188 2240 2250 2204 2205 2193 2260 2160 2236 2187 2228 2182 2223 2215 2224 2235 2213 2198 2168 2202 2195 2183 2253 2209 2234 2288 2261 2256 2254 2200 2257 2242 2167 2280 2252 2245 2207 2263 2216 2277 2284 2244 2190 2259 2266 2267 2180 2227 2196 2229 2297 2189 2169 2296 2181 2279 2281 2179 2178 2194 2287 2222 2285 2232 2301 2264 2214 2289 2787 2714 2823 2591 2640 2717 2650 2612 2838 2630 2827 2588 2685 2684 2677 2720 2613 2872 2729 2804 2622 2841 2642 2694 2867 2693 2865 2601 2808 2870 2691 2607 2877 2620 2836 2835 2576 2596 2636 2805 2802 2603 2579 2797 2610 2577 2716 2832 2626 2676 2678 2873 2785 2632 2608 2692 2643 2726 2813 2614 2675 2791 2803 2644 2701 2801 2594 2737 2625 2826 2839 2649 2700 2799 2634 2695 2730 2592 2874 2667 2647 2683 2586 2722 2679 2669 2629 2871 2549 2733 2582 2631 2807 2606 2641 2727 2724 2652 2623 2783 2646 2821 2602 2702 2708 2589 2815 2842 3222 3230 3229 2898 3228 3214 3232 3208 3213 2931 3202 2886 2922 3205 3221 3235 3212 3204 2887 2893 3209 2907 2921 3231 2891 2914 3223 3224 2890 2911 3189 2896 3233 2909 2917 3220 3198 3207 3226 3227 2900 2889 3225 2923 2884 3203 3192 2910 2926 3210 2885 2905 2913 2895 2897 2918 2924 3200 2927 2934 3234 2912 3206 3187 3236 3196 2881 3238 3190 3219 2899 2936 3237 3199 2919 2915 2882 3215 3188 2902 2937 2906 2930 2901 2883 3197 2908 2904 3193 3201 3218 2925 2892 2916 3194 2939 3195 2888 3191 2880 2894 2935 2932 2944 2920 2928 2929 2938 3581 3576 3572 3571 3568 3575 3569 3566 3562 3574 3577 3565 3570 3559 3550 3582 3578 3585 3583 3560 3573 3563 3580 3588 3586 3584 3579 3564 3567 3561 3549 3587 3551 3590 3591 3589 3558 3515 3523 3511 3526 3552 3512 3547 3510 3508 3520 3503 3524 3509 3518 3557 3545 3548 3514 3592 3517 3519 3528 3513 3522 3488 3521 3487 3516 3544 3497 3553 3507 3525 3504 3546 3499 3490 3527 3492 3494 3501 3489 3500 3593 3491 3496 3311 3458 3306 3594 3495 3394 3543 3498 3277 3486 3502 3370 3505 3308 3460 3469 3459 3466 3483 3366 3482 3269 3323 3264 3485 -------------- next part -------------- 3956.970544 4114.773479 4313.241696 4384.792216 4391.872310 4411.476715 4482.940096 4594.108179 4645.381743 4730.829231 4763.895598 4781.032135 4880.854414 5008.228140 5013.866252 5026.548238 5040.509458 5061.949397 5098.508781 5118.791696 5162.805554 5186.805236 5193.941710 5256.845108 5286.838670 5364.175004 5413.297293 5418.790951 5446.085519 5475.639165 5508.616723 5604.737760 5614.409364 5636.101761 5638.542797 5649.773243 5660.913105 5719.715027 5815.218454 5832.621843 5837.578731 5862.529421 5863.607227 5877.191757 5952.905334 5984.647641 6033.194005 6048.375830 6058.973718 6073.108082 6100.784624 6112.341023 6138.705305 6142.491247 6170.101238 6190.399915 6224.593338 6239.605558 6248.244072 6254.401594 6280.211422 6317.197093 6317.614065 6325.296754 6339.852487 6381.476997 6383.259524 6391.571145 6397.082360 6418.080290 6438.735101 6440.724082 6454.977105 6477.089430 6560.501879 6637.842362 6710.064249 6724.529239 6728.116095 6728.478821 6774.863744 6874.242169 6902.087390 6919.527972 6979.799498 6980.973237 6989.850615 7015.371907 7019.392350 7050.062660 7050.597597 7061.397784 7105.324281 7107.397379 7113.332893 7123.788680 7132.977337 7138.892938 7172.808848 7176.242427 7186.719367 7191.745735 7239.760194 7243.749478 7255.868073 7257.267870 7268.538062 7272.629743 7679.940589 7683.236804 7780.699438 7873.928129 7899.908299 7904.060156 7925.056466 7986.214500 7992.112331 7999.563794 8001.244799 8018.175375 8030.246998 8042.876999 8066.748960 8068.723221 8070.288644 8071.628413 8130.859172 8156.118329 8175.197378 8206.961155 8218.103232 8222.490958 8239.676758 8241.640380 8249.615006 8267.669676 8268.053374 8283.319127 8288.302563 8293.092398 8316.055561 8327.772501 8334.822552 8338.317644 8339.073386 8342.683772 8350.156751 8357.692104 8400.200111 8408.910930 8417.560026 8424.256975 8424.301520 8435.165529 8442.029865 8463.791695 8467.858939 8468.002339 8473.892687 8475.457054 8481.274131 8484.884301 8503.376563 8506.217207 8513.976462 8516.443233 8518.839920 8521.927566 8522.369277 8524.426356 8529.354446 8530.550718 8537.266324 8543.730529 8554.276036 8555.003337 8573.504421 8575.663769 8578.398385 8586.508857 8597.926278 8601.801143 8607.638604 8613.355557 8618.223021 8619.832545 8625.856397 8628.547664 8635.510731 8635.616818 8638.093330 8638.725205 8639.290086 8640.963579 8641.074684 8644.931541 8650.771065 8651.748195 8652.153167 8654.191277 8662.647644 8663.908731 8672.156947 8673.406207 8673.653387 8676.156890 8684.297953 8688.001563 8691.611803 8691.714375 8692.654871 8705.209794 8714.218292 8715.503995 8720.084210 8720.465880 8950.316961 9014.234275 9016.582013 9068.120557 9079.086925 9084.084293 9092.854202 9152.516280 9206.442055 9211.797882 9259.935823 9274.088555 9319.200628 9418.736103 9426.658011 9437.435612 9448.483510 9448.919497 9451.123274 9475.891179 9488.585768 9492.490822 9513.276633 9516.127121 9521.205075 9530.898966 9564.375640 9588.755348 9596.967298 9615.132594 9642.107437 9645.510988 9646.175388 9666.070621 9677.726960 9695.380196 9701.239340 9709.602677 9711.670031 9739.504151 9761.667030 9761.780136 9769.510284 9799.923976 9807.513067 9809.803762 9810.694795 9823.853204 9838.334226 9848.403779 9876.275178 9887.129520 9887.560756 9887.888122 9895.889715 9899.907916 9909.691312 9911.111820 9949.714270 9952.911904 9960.038486 9962.479402 9971.996514 9980.326000 9984.951467 9994.006658 10001.073320 10003.476996 10014.757757 10022.105369 10025.979253 10049.807479 10054.324996 10058.877706 10058.904792 10085.774604 10088.651241 10091.809589 10097.187658 10107.697162 10109.684042 10113.805433 10114.595009 10137.018824 10138.092392 10150.691072 10151.749667 10171.572498 10177.785950 10182.552424 10185.711350 10193.402181 10198.124474 10202.876980 10203.927967 10204.398934 10213.144657 10229.042155 10235.314449 10238.453722 10241.626409 10241.820711 10263.338102 10266.747743 10319.492303 10332.302989 10339.341659 10343.745803 8793.912077 9240.132446 9371.045485 9559.747528 9566.311321 9605.890226 9663.291742 9727.898364 9817.130726 9829.579358 9894.337029 9943.329879 9986.701787 9988.681733 10007.595675 10039.175362 10058.567031 10071.732150 10080.223263 10080.572538 10082.640448 10091.338544 10096.808553 10104.572156 10196.766192 10243.213998 10247.809490 10253.163862 10292.904202 10312.906320 10315.642277 10327.044105 10342.839692 10346.473991 10355.275639 10362.684038 10372.115530 10372.748272 10382.069791 10387.721477 10389.628136 10399.630692 10430.582283 10442.936940 10449.245767 10454.870137 10472.091864 10478.317998 10489.109327 10509.306150 10513.053502 10514.097318 10525.194284 10527.980647 10529.909410 10531.238327 10537.874168 10544.844175 10555.905185 10563.925159 10565.773122 10575.339600 10586.124716 10590.426276 10611.035077 10622.729994 10636.607153 10657.784468 10666.334794 10672.555469 10686.503366 10686.595292 10693.772247 10703.728069 10703.988774 10704.045868 10708.960756 10720.027629 10738.600374 10777.951535 10783.717838 10797.095152 10803.299357 10804.900797 10805.415021 10809.890716 10825.610894 10830.556433 10854.840855 10861.376313 10885.763384 10918.933837 10961.511982 10966.953981 10967.776642 10973.773434 10977.243562 10978.798310 10978.830169 10989.002409 11009.161106 11015.743459 11020.016124 11056.625898 11092.348893 11093.400822 11100.135715 11153.180487 9814.960894 9981.143054 10098.094627 10098.936901 10251.240937 10282.032995 10284.640897 10322.598305 10360.982393 10376.872740 10392.495710 10417.277021 10425.686178 10434.329513 10450.355048 10478.041847 10499.163586 10499.215862 10526.877232 10532.554246 10536.409254 10539.253460 10608.396814 10645.575298 10659.240945 10670.922154 10689.184454 10715.836202 10722.298297 10725.640874 10748.404608 10753.162544 10756.517743 10757.745888 10772.695671 10782.428719 10790.583373 10791.898039 10801.684370 10811.977492 10816.266996 10818.283291 10820.330119 10834.649198 10857.733858 10874.140510 10876.059901 10877.637723 10882.405065 10894.207293 10900.073600 10912.167792 10924.820170 10926.306574 10929.257567 10959.628651 10960.178273 10975.859148 10976.998078 10978.812143 10991.596620 10999.187685 11000.809537 11010.126635 11024.307375 11071.514474 11085.007968 11094.736694 11109.770745 11113.527504 11116.002506 11122.254903 11133.988453 11136.612348 11156.603051 11165.834508 11166.019724 11167.998566 11175.515681 11193.546241 11199.134090 11238.198692 11244.422619 11253.787513 11264.283982 11271.505442 11274.799495 11299.434399 11316.162315 11317.946622 11318.921151 11319.649335 11346.282844 11358.697447 11362.651974 11372.919362 11374.332828 11382.460151 11394.420472 11396.858534 11397.989178 11401.709776 11418.079306 11432.017976 11438.011867 11459.901142 11468.659319 11472.542199 10681.749907 10956.703906 11125.372074 11136.314500 11172.483691 11178.012600 11269.727750 11300.670178 11340.773673 11383.722922 11457.659862 11462.856526 11489.727744 11551.874237 11579.544500 11637.402145 11686.213732 11690.080383 11741.072475 11747.628633 11787.678534 11801.682051 11814.371802 11829.473150 11838.299686 11858.353106 11870.960722 11871.392282 11871.694919 11876.150858 11883.402109 11895.260550 11913.376985 11913.856733 11942.894620 11951.949830 11969.258031 11983.875254 11993.436462 11995.024242 12021.141064 12023.333762 12048.485972 12050.270337 12055.970075 12066.039635 12073.944508 12093.030117 12098.907173 12116.424190 12116.649970 12120.302248 12124.960350 12136.994784 12137.012910 12150.298122 12150.452929 12166.326678 12170.039350 12171.669367 12173.044335 12174.841150 12179.606648 12207.744517 12209.538641 12225.212482 12229.865951 12232.316882 12246.263509 12256.108638 12276.069698 12286.868822 12297.149059 12306.631699 12312.322071 12320.164640 12324.374077 12336.310756 12339.376427 12368.598153 12380.799010 12382.583485 12384.933702 12389.988805 12391.470566 12395.745999 12401.831477 12402.594363 12413.304896 12413.370202 12444.759649 12448.495831 12459.577196 12470.502275 12470.742632 12472.315225 12474.617021 12478.433235 12478.483165 12482.236078 12498.028231 12513.399284 12525.231137 12532.312092 12534.743170 12537.889753 12542.266646 12551.826599 11193.319247 11514.884367 11602.445732 11622.344038 11622.438404 11630.019170 11641.485017 11667.278665 11708.588012 11789.904842 11836.441818 11850.623879 11881.255290 11925.047593 11951.347144 11964.909208 11994.423115 12038.711459 12090.526264 12106.978903 12160.550605 12177.736209 12215.007376 12230.055459 12250.739418 12257.282265 12259.861311 12260.730253 12262.735060 12291.224191 12316.483640 12320.419844 12358.768148 12371.123837 12374.651193 12388.097106 12445.256592 12448.681215 12473.311313 12504.297769 12505.134285 12527.592355 12575.220793 12625.944644 12651.232120 12654.872920 12667.986158 12673.676463 12698.035009 12713.500228 12766.790350 12795.378324 12811.393790 12816.705173 12821.887367 12836.305776 12859.372179 12868.193924 12869.219546 12883.997847 12913.567482 12928.892730 12951.562632 12957.166348 13033.097686 13039.613435 13052.920297 13103.059009 13121.399000 13137.420222 13143.819581 13189.273853 13217.056695 13227.901100 13251.107604 13252.729284 13256.649463 13269.470154 13291.730794 13301.492326 13306.257808 13307.889142 13360.021884 13423.017549 13429.234543 13437.179872 13484.090893 13496.163341 13595.522328 13606.157527 13609.179758 13617.822523 13619.608609 13690.242120 13717.833712 13735.254554 13755.434901 13807.103472 13824.931396 13827.734231 13834.988156 13914.748586 13978.380023 14060.743382 14083.144765 14103.587172 14155.685069 14169.683047 12042.398439 12228.734395 12520.514566 12535.035045 12580.133137 12590.412616 12629.404482 12654.158926 12675.304854 12733.821411 12802.369761 12809.260088 12868.977918 12891.461146 12902.360057 12944.236947 12970.555105 12984.851197 12993.054529 13024.315480 13028.277354 13036.186899 13065.224983 13074.307427 13077.734413 13120.257899 13138.725590 13139.563867 13144.916995 13147.708541 13150.495330 13150.670526 13153.697961 13158.210056 13170.801478 13173.506940 13174.016751 13175.306920 13181.264005 13182.217312 13189.197422 13191.435130 13212.256657 13212.308700 13216.024874 13216.050142 13217.485968 13262.914785 13271.722595 13282.591589 13291.680677 13313.503687 13317.334527 13331.044319 13332.073092 13332.970644 13351.033630 13367.919842 13372.757479 13383.369447 13400.388651 13403.526696 13406.483139 13410.260061 13413.739986 13423.056454 13427.512376 13428.578560 13431.234935 13438.083882 13440.731763 13467.215303 13471.345252 13475.473964 13476.412131 13481.174967 13491.373425 13496.015255 13499.707775 13500.224718 13512.408367 13515.414742 13519.258531 13519.955607 13529.853371 13555.701103 13567.556673 13581.752972 13592.412676 13612.170183 13618.520577 13621.250454 13627.232255 13638.740682 13644.590236 13645.126659 13649.339579 13652.869745 13660.267656 13663.052077 13666.620509 13667.808315 13667.827996 13678.639363 13685.255280 13689.203630 13691.048025 13713.641976 11598.693262 11704.336246 11919.040078 11981.528008 12057.828308 12094.424616 12149.980041 12179.583269 12183.210365 12251.745007 12262.323134 12331.253864 12412.695494 12418.496422 12425.118997 12528.458621 12550.929713 12566.605855 12624.572874 12625.259561 12647.308193 12679.031675 12683.923804 12704.924039 12745.050992 12770.103628 12787.591668 12809.165792 12819.808330 12822.677224 12835.990692 12854.239067 12913.753810 12933.290259 12939.277223 12959.460061 12965.124068 12971.594291 12994.159714 13007.621915 13010.606602 13018.524252 13043.586740 13060.549764 13069.738691 13081.265005 13083.369723 13111.744055 13132.289855 13231.443965 13243.583985 13306.335621 13307.422095 13315.641892 13373.612750 13379.343564 13437.392770 13486.774587 13498.942593 13516.270938 13517.006573 13617.129826 13651.302768 13655.976384 13659.231345 13669.637967 13692.471342 13702.477965 13709.766221 13724.485954 13748.119265 13764.708728 13780.746631 13821.058818 13822.464110 13864.480461 13867.177653 13868.782455 13869.689934 13920.956099 13928.184482 13975.859048 13985.023508 13988.347825 13992.894227 14009.444963 14037.708338 14058.926075 14150.438533 14172.273453 14186.946281 14194.290515 14246.683826 14305.245169 14308.202607 14310.050716 14351.339722 14376.973672 14392.142221 14397.889970 14434.301413 14436.668387 14454.755269 14546.002108 14564.107635 14609.352294 14610.444061 14658.388376 6191.820693 6417.995072 6450.171771 6639.135507 6702.781151 6808.454342 6858.701101 6875.949295 6896.356613 6899.675204 6980.449533 7001.842011 7076.463311 7081.714631 7082.396339 7176.817676 7216.309297 7266.648313 7280.514286 7297.524297 7355.848621 7373.904267 7378.860143 7408.329335 7458.162090 7556.879500 7567.809539 7642.724706 7735.706825 8038.110570 8048.230497 8061.020685 8148.930937 8210.087296 8262.783633 8377.205167 8474.150854 8624.533634 8822.084093 8902.373733 9086.760508 9093.644637 9159.773432 9176.508060 9188.687554 9189.349289 9191.366199 9192.863741 9292.189993 9293.081584 9297.837340 9391.546496 9403.902790 9423.233460 9528.854294 9531.233339 9555.442442 9596.527664 9621.645865 9664.165188 9748.118478 9801.103502 9845.340170 9883.906937 9910.602414 9914.261309 9914.838154 9944.961769 9947.652699 10030.420720 10032.086427 10085.772122 10106.567350 10131.776082 10171.059604 10171.271504 10250.514131 10294.686428 10310.671898 10354.773039 10359.713965 10417.971882 10419.604148 10518.670282 10543.442886 10604.550121 10604.651083 10638.592841 10715.252457 10750.162225 10763.995006 10825.711128 10857.760357 10872.843194 10894.852725 10910.851029 10927.298566 10932.509272 10938.224926 11014.452664 11019.602796 11031.241511 11052.111308 11055.718676 11153.896765 11160.080716 11179.872948 11193.467570 -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 12 16:51:08 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 14:51:08 -0600 Subject: Polyfit In-Reply-To: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> Message-ID: On 10/12/06, Greg Willden wrote: > > Hi All, > I'm using numpy.polyfit and it is giving me some really strange results > when evaluated near 0. So I compared it with polyfit in matplotlib and the > code and docstrings are nearly identical. However the slight differences in > the code make a huge difference on my data. > > Here is the Numpy version with line numbers added: > 1 x = NX.asarray(x)+0. > 2 y = NX.asarray(y)+0. > 3 y = NX.reshape(y, (len(y), 1)) > 4 X = vander(x, N+1) > 5 c, resids, rank, s = _lstsq(X, y) > 6 c.shape = (N+1,) > 7 return c > > > And here is the MPL version with line numbers added: > 1 x = asarray(x)+0. > 2 y = asarray(y)+0. > 3 y = reshape(y, (len(y),1)) > 4 X = Matrix(vander(x, N+1)) > 5 Xt = Matrix(transpose(X)) > 6 c = array(linear_algebra.inverse(Xt*X)*Xt*y) # convert back to array > 7 c.shape = (N+1,) > 8 return c > > > So lines 1-4 are basically the same. > The MPL version produces a much more stable representation of my data > > Can someone please comment on the differences? > Should these two be exactly the same? > If not, what are the advantages to each algorithm? I'm guessing that the rcond number in the lstsq version (default 1e-10) is the difference. Generally the lstsq version should work better than the MPL version because at*a is not as well conditioned and vandermonde matrices are notoriously ill conditioned anyway for higher degree fits. It would help if you attached the data files in ascii form unless they happen to contain thousands of data items. Just the x will suffice, zip it up if you have to. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From hetland at tamu.edu Thu Oct 12 16:58:59 2006 From: hetland at tamu.edu (Rob Hetland) Date: Thu, 12 Oct 2006 15:58:59 -0500 Subject: More SVN testing In-Reply-To: <452E966C.1060300@ieee.org> References: <452E966C.1060300@ieee.org> Message-ID: <5A176A3E-A5F1-49F4-B1C7-89CA662D2B90@tamu.edu> >>>numpy.test(10,10) : : : Ran 512 tests in 0.995s OK >>> numpy.__version__ '1.0.dev3320' on Intel Mac OS X 10.4.8, macpython 2.5 [Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)] -Rob On Oct 12, 2006, at 2:24 PM, Tim Hochberg wrote: > > I just checked in a couple of changes to SVN. I was going to check in > errstate, but it looks like Travis beat me to it, so I contented > myself > with adding a docstring and some tests. These tests are only run > under > 2.5; things seem to work fine here, but if someone on a Linux box > whose > running Python 2.5 could try it out I'd appreciate it. I'm > suspicous of > Linux/Windows differences right now because of my run in with them > earlier today in the tests for test_multiarray (which I believe it now > fixed -- at least everything is working here OK). > > -tim > > > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion ---- Rob Hetland, Associate Professor Dept. of Oceanography, Texas A&M University http://pong.tamu.edu/~rob phone: 979-458-0096, fax: 979-845-6331 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 12 18:03:04 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 16:03:04 -0600 Subject: Polyfit In-Reply-To: <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> Message-ID: On 10/12/06, Greg Willden wrote: > > On 10/12/06, Charles R Harris wrote: > > > > I'm guessing that the rcond number in the lstsq version (default 1e-10) > > is the difference. Generally the lstsq version should work better than the > > MPL version because at*a is not as well conditioned and vandermonde matrices > > are notoriously ill conditioned anyway for higher degree fits. It would help > > if you attached the data files in ascii form unless they happen to contain > > thousands of data items. Just the x will suffice, zip it up if you have to. > > > > > Here are the files. > > Since the two algorithms behave differently and each has it place then can > both be included in numpy? > i.e. numpy.polyfit(x,y,N, mode='alg1') > numpy.polyfit (x,y,N, mode='alg2') > > replacing alg1 and alg2 with meaningful names. > The polyfit function looks seriously busted. If I do the fits by hand I get the same results using the (not so hot) MPL version or lstsq. I don't know what the problem is. The docstring is also incorrect for the method. Hmmm... Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 12 18:28:10 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 16:28:10 -0600 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> Message-ID: On 10/12/06, Charles R Harris wrote: > > > > On 10/12/06, Greg Willden wrote: > > > > On 10/12/06, Charles R Harris wrote: > > > > > > I'm guessing that the rcond number in the lstsq version (default > > > 1e-10) is the difference. Generally the lstsq version should work better > > > than the MPL version because at*a is not as well conditioned and vandermonde > > > matrices are notoriously ill conditioned anyway for higher degree fits. It > > > would help if you attached the data files in ascii form unless they happen > > > to contain thousands of data items. Just the x will suffice, zip it up if > > > you have to. > > > > > > > > > Here are the files. > > > > Since the two algorithms behave differently and each has it place then > > can both be included in numpy? > > i.e. numpy.polyfit(x,y,N, mode='alg1') > > numpy.polyfit (x,y,N, mode='alg2') > > > > replacing alg1 and alg2 with meaningful names. > > > > The polyfit function looks seriously busted. If I do the fits by hand I > get the same results using the (not so hot) MPL version or lstsq. I don't > know what the problem is. The docstring is also incorrect for the method. > Hmmm... > Polyfit seems overly conservative in its choice of rcond. In [101]: lin.lstsq(v,y,1e-10)[0] Out[101]: array([ 5.84304475e-07, -5.51513630e-03, 1.51465472e+01, 3.05631361e-02]) In [107]: polyfit(x,y,3) Out[108]: array([ 5.84304475e-07, -5.51513630e-03, 1.51465472e+01, 3.05631361e-02]) Compare In [112]: lin.lstsq(v,y,1e-12)[0] Out[112]: array([ -5.42970700e-07, 1.61425067e-03, 1.99260667e+00, 6.51889107e+03]) In [113]: dot(lin.inv(vtv),dot(v.T,y)) Out[113]: array([ -5.42970700e-07, 1.61425067e-03, 1.99260667e+00, 6.51889107e+03]) So the default needs to be changed somewhere. Probably polyfit shoud accept rcond as a keyword. Where the actual problem lies is a bit obscure as the normal rcond default for lin.lstsq is 1e-12. Maybe some sort of import error somewhere down the line. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 12 19:04:15 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 17:04:15 -0600 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> Message-ID: On 10/12/06, Charles R Harris wrote: > > > > On 10/12/06, Charles R Harris wrote: > > > > > > > > On 10/12/06, Greg Willden < gregwillden at gmail.com> wrote: > > > > > > On 10/12/06, Charles R Harris wrote: > > > > > > > > I'm guessing that the rcond number in the lstsq version (default > > > > 1e-10) is the difference. Generally the lstsq version should work better > > > > than the MPL version because at*a is not as well conditioned and vandermonde > > > > matrices are notoriously ill conditioned anyway for higher degree fits. It > > > > would help if you attached the data files in ascii form unless they happen > > > > to contain thousands of data items. Just the x will suffice, zip it up if > > > > you have to. > > > > > > > > > > > > > Here are the files. > > > > > > Since the two algorithms behave differently and each has it place then > > > can both be included in numpy? > > > i.e. numpy.polyfit(x,y,N, mode='alg1') > > > numpy.polyfit (x,y,N, mode='alg2') > > > > > > replacing alg1 and alg2 with meaningful names. > > > > > > > The polyfit function looks seriously busted. If I do the fits by hand I > > get the same results using the (not so hot) MPL version or lstsq. I don't > > know what the problem is. The docstring is also incorrect for the method. > > Hmmm... > > > > Polyfit seems overly conservative in its choice of rcond. > > In [101]: lin.lstsq(v,y,1e-10)[0] > Out[101]: > array([ 5.84304475e-07, -5.51513630e-03, 1.51465472e+01, > 3.05631361e-02]) > In [107]: polyfit(x,y,3) > Out[108]: > array([ 5.84304475e-07, -5.51513630e-03, 1.51465472e+01, > 3.05631361e-02]) > > Compare > > In [112]: lin.lstsq(v,y,1e-12)[0] > Out[112]: > array([ -5.42970700e-07, 1.61425067e-03, 1.99260667e+00, > 6.51889107e+03]) > > In [113]: dot(lin.inv(vtv),dot(v.T,y)) > Out[113]: > array([ -5.42970700e-07, 1.61425067e-03, 1.99260667e+00, > 6.51889107e+03]) > > So the default needs to be changed somewhere. Probably polyfit shoud > accept rcond as a keyword. Where the actual problem lies is a bit obscure as > the normal rcond default for lin.lstsq is 1e-12. Maybe some sort of import > error somewhere down the line. > And here is the location of the problem in numpy/linalg/linalg.py : def lstsq(a, b, rcond=1.e-10): The 1e-10 is a bit conservative. On the other hand, I will note that the condition number of the dot(V^T ,V) matrix is somewhere around 1e22, which means in general terms that you need around 22 digits of accuracy. Inverting it only works sorta by accident in the current case. Generally, using Vandermonde matrices and polynomial fits it a bad idea when the dynamic range of the interval gets large and the degree gets up around 4-5 as it leads to ill conditioned sets of equations. When you really need the best start with chebychev polynomials or, bestest, compute a set of polynomials orthogonal over the sample points. Anyway, I think rcond should be something like 1e-12 or 1e-13 by default and be available as a keyword in the polyfit function. If no one complains I will make this change, although it is just a bandaid and things will fall apart again as soon as you call polyfit(x,y,4). Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ndbecker2 at gmail.com Thu Oct 12 20:18:06 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Oct 2006 20:18:06 -0400 Subject: numpy on centos4.4? Message-ID: Has anyone built numpy-1.0rc2/scipy-0.5.1 on centos4.4? It seems that a fortran90+ compiler is required, so I installed gcc4-gfortran-4.1.0-18.EL4 libgfortran-4.1.0-18.EL4 Didn't quite work though. Built numpy+scipy OK, but then: Warning: FAILURE importing tests for /usr/lib/python2.3/site-packages/scipy/linalg/tests/test_blas.py:22: ImportError: /usr/lib/python2.3/site-pac\ kages/scipy/linalg/fblas.so: undefined symbol: srotmg_ (in ?) Not sure yet what's causing this. I suspect it is because of a mixture of different compilers/libraries? Like, maybe blas was compiled with f77 from gcc3, while fblas was compiled with gfortran from gcc4? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From vallis.35530053 at bloglines.com Thu Oct 12 21:00:51 2006 From: vallis.35530053 at bloglines.com (vallis.35530053 at bloglines.com) Date: 13 Oct 2006 01:00:51 -0000 Subject: Providing array interface from SWIG-wrapped C struct Message-ID: <1160701251.2319852243.15024.sendItem@bloglines.com> Wonderful, it works, thanks! Michele --- Discussion of Numerical Python wrote: > > Does anybody here have experience about offering the array interface > > from a SWIG-wrapped C struct? > > I have. > > > I have tried the following, borrowing code from numpy's arrayobject.c: > > > > %extend real_vec_t { > > PyObject *__array_struct__() { > > /* From numpy/arrayobject.c/array_struct_get */ > > You are extending real_vec_t with a new METHOD, but what numpy > requests is an ATTRIBUTE. So, numpy simply queries your vec like: > > arrstr = vec.__array_struct__ > > and not with a method call like this > > arrstr = vec.__array_struct__() > > > So here is what I would do (can fail with some SWIG optimizations) > > %extend Vec { > > PyObject* __array_struct__ () { /* ... */ } > > %pythoncode { > __array_struct__ = property(__array_struct__, > doc='Array protocol') > } > > } > > Hope you got the idea. > > -- > Lisandro Dalc?n > --------------- > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 12 21:28:31 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 19:28:31 -0600 Subject: rcond in polyfit Message-ID: Hi all, I note that polyfit looks like it should work for single and double, real and complex, polynomials. On the otherhand, the default rcond needs to depend on the underlying precision. On the other, other hand, all the svd computations are done with dgelsd or zgelsd, i.e., double precision. Even so problems can arise from inherent errors of the input data if it is single precision to start with. I also think the final degree of the fit should be available somewhere if wanted, as it is an indication of what is going on. Sooo, any suggestions as to what to do? My initial impulse would be to set rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick the can down the road on returning the actual degree of the fit. Chuck PS, what is the best way of converting arbitrary arrays to the appropriate c data types float and double? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From David.L.Goldsmith at noaa.gov Thu Oct 12 21:34:09 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Thu, 12 Oct 2006 18:34:09 -0700 Subject: Should numpy.sqrt(-1) return 1j rather than nan? In-Reply-To: <452E946E.8000809@ee.byu.edu> References: <452D56F5.9000404@ee.byu.edu> <452DDC4B.1050304@noaa.gov> <452DE009.3070100@ieee.org> <452DEAED.9070408@noaa.gov> <452DF1BA.9000906@ieee.org> <452DF49A.407@noaa.gov> <452E946E.8000809@ee.byu.edu> Message-ID: <452EED11.400@noaa.gov> I find that acceptable for my purposes, but is there some way we can minimize the "surprise(s)" for newbies? (I know some suggestions have been put forward in this thread, but I don't know enough to cast a vote one way or another for any of those, just a vote for "please do it".) And, in closing, I too would like to: thank you very much for all your work - despite my "emotional outburst," I am generally very happy using numpy and regard it as an excellent product. DG Travis Oliphant wrote: > David Goldsmith wrote: > > >> Got it. And if I understand correctly, the import order you specify in >> the little mynumpy example you included in your latest response to >> Fernando will result in any "overlap" between numpy and >> numpy.lib.scimath to call the latter's version of things rather than the >> former's, yes? >> >> >> > > Right. The last import will be used for any common-names (variables get > re-bound to the new functions...) > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Thu Oct 12 21:49:51 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 13 Oct 2006 10:49:51 +0900 Subject: How to make sqrt(-1) be 1j Message-ID: On 10/12/06, Stefan van der Walt wrote: > On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: > > On 10/11/06, Bill Baxter wrote: > I tried to explain the argument at > > http://www.scipy.org/NegativeSquareRoot > The proposed fix for those who want sqrt(-1) to return 1j is: from numpy.lib import scimath as SM SM.sqrt(-1) But that creates a new namespace alias, different from numpy. So I'll call numpy.array() to create a new array, but SM.sqrt() when I want a square root. Am I wrong to want some simple way to change the behavior of numpy.sqrt itself? Seems like you can get that effect via something like: for n in numpy.lib.scimath.__all__: numpy.__dict__[n] = numpy.lib.scimath.__dict__[n] If that sort of function were available as "numpy.use_scimath()", then folks who want numpy to be like scipy can achieve that with just one line at the top of their files. The import under a different name doesn't quite achieve the goal of making that behavior numpy's "default". I guess I'm thinking mostly of the educational uses of numpy, where you may have users that haven't learned about much about numerical computing yet. I can just imagine the instructor starting off by saying "ok everyone we're going to learn numpy today! First everyone type this: 'import numpy, from numpy.lib import scimath as SM' -- Don't worry about all the things there you don't understand." Whereas "import numpy, numpy.use_scimath()" seems easier to explain and much less intimidating as your first two lines of numpy to learn. Or is that just a bad idea for some reason? --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 12 22:15:52 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 12 Oct 2006 20:15:52 -0600 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: Message-ID: On 10/12/06, Bill Baxter wrote: > > On 10/12/06, Stefan van der Walt wrote: > > On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: > > > On 10/11/06, Bill Baxter wrote: > > I tried to explain the argument at > > > > http://www.scipy.org/NegativeSquareRoot > > > > The proposed fix for those who want sqrt(-1) to return 1j is: > > from numpy.lib import scimath as SM > SM.sqrt(-1) > > > But that creates a new namespace alias, different from numpy. So I'll > call numpy.array() to create a new array, but SM.sqrt() when I want a > square root. > Am I wrong to want some simple way to change the behavior of > numpy.sqrt itself? > > Seems like you can get that effect via something like: > > for n in numpy.lib.scimath.__all__: > numpy.__dict__[n] = numpy.lib.scimath.__dict__[n] I don't like either of those ideas, although the second seems preferable. I think it better to make an efficient way of calling a sqrt routine that accepts negative floats and returns complex numbers. The behaviour could be chosen either by key word or by specially named routines, or maybe even some global flag, but I don't think it asking too much for the students to learn that sqrt(-1) doesn't exist as a real number and that efficient computation uses real whenever possible because it is a) smaller, and b) faster. That way we also avoid having software that only works for scimath but not for numpy. Chuck. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Thu Oct 12 22:30:01 2006 From: gregwillden at gmail.com (Greg Willden) Date: Thu, 12 Oct 2006 21:30:01 -0500 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> Message-ID: <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> On 10/12/06, Charles R Harris wrote: > > And here is the location of the problem in numpy/linalg/linalg.py : > > def lstsq(a, b, rcond=1.e-10): > > The 1e-10 is a bit conservative. On the other hand, I will note that the > condition number of the dot(V^T ,V) matrix is somewhere around 1e22, which > means in general terms that you need around 22 digits of accuracy. Inverting > it only works sorta by accident in the current case. Generally, using > Vandermonde matrices and polynomial fits it a bad idea when the dynamic > range of the interval gets large and the degree gets up around 4-5 as it > leads to ill conditioned sets of equations. When you really need the best > start with chebychev polynomials or, bestest, compute a set of polynomials > orthogonal over the sample points. Anyway, I think rcond should be something > like 1e-12 or 1e-13 by default and be available as a keyword in the polyfit > function. If no one complains I will make this change, although it is just a > bandaid and things will fall apart again as soon as you call polyfit(x,y,4). > > Hey that's great. I'm glad you tracked it down. Pardon my ignorance of polyfit algorithm details. Is there a way of choosing rcond based on N that would give sensible defaults for a variety of N? Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Thu Oct 12 23:11:06 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 13 Oct 2006 12:11:06 +0900 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: Message-ID: On 10/13/06, Charles R Harris wrote: > > > On 10/12/06, Bill Baxter wrote: > > On 10/12/06, Stefan van der Walt wrote: > > > On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: > > > > On 10/11/06, Bill Baxter < wbaxter at gmail.com> wrote: > > > I tried to explain the argument at > > > > > > http://www.scipy.org/NegativeSquareRoot > > > > > > > The proposed fix for those who want sqrt(-1) to return 1j is: > > > > from numpy.lib import scimath as SM > > SM.sqrt(-1) > > > > > > But that creates a new namespace alias, different from numpy. So I'll > > call numpy.array() to create a new array, but SM.sqrt() when I want a > > square root. > > Am I wrong to want some simple way to change the behavior of > > numpy.sqrt itself? > > > > Seems like you can get that effect via something like: > > > > for n in numpy.lib.scimath.__all__: > > numpy.__dict__[n] = numpy.lib.scimath.__dict__[n] > > I don't like either of those ideas, although the second seems preferable. I > think it better to make an efficient way of calling a sqrt routine that > accepts negative floats and returns complex numbers. The behaviour could be > chosen either by key word or by specially named routines, or maybe even > some global flag, We have the "specially named routines" way already. "numpy.lib.scimath.sqrt" > but I don't think it asking too much for the students to > learn that sqrt(-1) doesn't exist as a real number and that efficient > computation uses real whenever possible because it is a) smaller, and b) > faster. That way we also avoid having software that only works for scimath > but not for numpy. I think efficiency is not a very good argument for the default behavior here, because -- lets face it -- if efficient execution was high on your priority list, you wouldn't be using python. And even if you do care about efficiency, one of the top rules of optimization is to first get it working, then get it working fast. Really, I'm just playing the devils advocate here, because I don't work with complex numbers (I see quaternions more often than complex numbers). But I would be willing to do something like numpy.use_realmath() in my code if it would make numpy more palatable to a wider audience. I wouldn't like it, however, if I had to do some import thing where I have to remember forever after that I should type 'numpy.tanh()' but 'realmath.arctanh()'. Anyway it seems like the folks who care about performance are the ones who will generally be more willing to make tweaks like that. But that's about all I have to say about this, since the status quo works fine for me. So I'll be quiet. Just it seems like the non-status-quo'ers here have some good points. I taught intro to computer science to non-majors one semester. I know I would not want to have to confront all the issues with numerical computing right off the bat if I was just trying to teach people how to do some math. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From parlar at gmail.com Thu Oct 12 23:15:50 2006 From: parlar at gmail.com (Jay Parlar) Date: Thu, 12 Oct 2006 23:15:50 -0400 Subject: Problem buliding rc2 on OS X 10.3.9/Python2.5 In-Reply-To: References: Message-ID: I hate bumping my own messages, but does no one have any insight into this? On 10/9/06, Jay Parlar wrote: > In the process of finally switching over to Python 2.5, and am trying > to build numpy. Unfortunately, it dies during the build: > > Jay-Computer:~/Desktop/numpy-1.0rc2 jayparlar$ python setup.py build > Running from numpy source directory. > F2PY Version 2_3296 > blas_opt_info: > FOUND: > extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] > define_macros = [('NO_ATLAS_INFO', 3)] > extra_compile_args = ['-faltivec', > '-I/System/Library/Frameworks/vecLib.framework/Headers'] > > lapack_opt_info: > FOUND: > extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] > define_macros = [('NO_ATLAS_INFO', 3)] > extra_compile_args = ['-faltivec'] > > running build > running config_fc > running build_src > building py_modules sources > building extension "numpy.core.multiarray" sources > Generating build/src.macosx-10.3-fat-2.5/numpy/core/config.h > customize NAGFCompiler > customize AbsoftFCompiler > customize IbmFCompiler > Could not locate executable g77 > Could not locate executable f77 > Could not locate executable gfortran > Could not locate executable f95 > customize GnuFCompiler > customize Gnu95FCompiler > customize G95FCompiler > customize GnuFCompiler > customize Gnu95FCompiler > customize NAGFCompiler > customize NAGFCompiler using config > C compiler: gcc -arch ppc -arch i386 -isysroot > /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double > -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 > > compile options: > '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > -Inumpy/core/src -Inumpy/core/include > -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > -c' > gcc: _configtest.c > gcc: cannot specify -o with -c or -S and multiple compilations > gcc: cannot specify -o with -c or -S and multiple compilations > failure. > removing: _configtest.c _configtest.o > numpy/core/setup.py:50: DeprecationWarning: raising a string exception > is deprecated > raise "ERROR: Failed to test configuration" > Traceback (most recent call last): > File "setup.py", line 89, in > setup_package() > File "setup.py", line 82, in setup_package > configuration=configuration ) > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/core.py", > line 174, in setup > return old_setup(**new_attr) > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", > line 151, in setup > dist.run_commands() > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", > line 974, in run_commands > self.run_command(cmd) > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", > line 994, in run_command > cmd_obj.run() > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", > line 112, in run > self.run_command(cmd_name) > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", > line 333, in run_command > self.distribution.run_command(command) > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", > line 994, in run_command > cmd_obj.run() > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", > line 87, in run > self.build_sources() > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", > line 106, in build_sources > self.build_extension_sources(ext) > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", > line 212, in build_extension_sources > sources = self.generate_sources(sources, ext) > File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", > line 270, in generate_sources > source = func(extension, build_dir) > File "numpy/core/setup.py", line 50, in generate_config_h > raise "ERROR: Failed to test configuration" > ERROR: Failed to test configuration > > > This is with the Universal 2.5 binary, and OS X 10.3.9. > > Any ideas? Sorry if this one has been asked before, but I can't seem > to find a solution anywhere. > > Jay P. > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 13 00:43:08 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 21:43:08 -0700 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: Message-ID: <452F195C.9000605@ieee.org> Bill Baxter wrote: > On 10/12/06, Stefan van der Walt wrote: > >> On Thu, Oct 12, 2006 at 08:58:21AM -0500, Greg Willden wrote: >> >>> On 10/11/06, Bill Baxter wrote: >>> >> I tried to explain the argument at >> >> http://www.scipy.org/NegativeSquareRoot >> >> > > The proposed fix for those who want sqrt(-1) to return 1j is: > > from numpy.lib import scimath as SM > SM.sqrt(-1) > > > But that creates a new namespace alias, different from numpy. So I'll > call numpy.array() to create a new array, but SM.sqrt() when I want a > square root. > Am I wrong to want some simple way to change the behavior of > numpy.sqrt itself? > > Seems like you can get that effect via something like: > > for n in numpy.lib.scimath.__all__: > numpy.__dict__[n] = numpy.lib.scimath.__dict__[n] > > If that sort of function were available as "numpy.use_scimath()", then > folks who want numpy to be like scipy can achieve that with just one > line at the top of their files. The import under a different name > doesn't quite achieve the goal of making that behavior numpy's > "default". > > I guess I'm thinking mostly of the educational uses of numpy, where > you may have users that haven't learned about much about numerical > computing yet. I can just imagine the instructor starting off by > saying "ok everyone we're going to learn numpy today! First everyone > type this: 'import numpy, from numpy.lib import scimath as SM' -- > Don't worry about all the things there you don't understand." > Whereas "import numpy, numpy.use_scimath()" seems easier to explain > and much less intimidating as your first two lines of numpy to learn. > > Or is that just a bad idea for some reason? > > Isn't that just going to make your students *more* confused later when then run into the standard behavior of numpy? For this sort of thing, I would just make a new module to pull together the function I want and use that instead. It's then easy to explain that this new module bbeconf (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an amalgamation of stuff from multiple sources. # bbeconf.py from numpy import * fromnumpy.scimath import sqrt # possibly some other stuff to correctly handle subpackages... -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 13 01:13:53 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 12 Oct 2006 22:13:53 -0700 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: Message-ID: <452F2091.9020607@ieee.org> Bill Baxter wrote: > > I think efficiency is not a very good argument for the default > behavior here, because -- lets face it -- if efficient execution was > high on your priority list, you wouldn't be using python. I care very much about efficiency where it matters, which is only in a tiny fraction of my code. For the stuff that numpy does well it's pretty efficient, when that's not enough I can drop down to C, but I don't have to do that often. In fact, I've argued and still believe, that Python is frequently *more* efficient than C, given finite developer time, since it's easier to get the algorithms correct writing in Python. > And even if > you do care about efficiency, one of the top rules of optimization is > to first get it working, then get it working fast. > IMO, the current behavior is more likely to give you working code than auto-promoting to complex based on value. That concerns me more than efficiency. The whole auto-promotion thing looks like a good way to introduce data dependent bugs that don't surface till late in the game and are hard to track down. In contrast, when the current scheme causes a problem it should surface almost immediately. I would not use scipy.sqrt in code, even if the efficiency were the same, for this reason. I can see the attraction in the autopromoting version for teaching purposes and possibly for throwaway scripts, but not for "real" code. > Really, I'm just playing the devils advocate here, because I don't > work with complex numbers (I see quaternions more often than complex > numbers). But I would be willing to do something like > numpy.use_realmath() > in my code if it would make numpy more palatable to a wider audience. > I wouldn't like it, however, if I had to do some import thing where I > have to remember forever after that I should type 'numpy.tanh()' but > 'realmath.arctanh()'. > As I mentioned in my other message, the way to do this is to have a different entry point with different behavior. > Anyway it seems like the folks who care about performance are the ones > who will generally be more willing to make tweaks like that. > It's not just about performance though. It's also about correctness, or more accurately, resistance to bugs. > But that's about all I have to say about this, since the status quo > works fine for me. So I'll be quiet. Just it seems like the > non-status-quo'ers here have some good points. I taught intro to > computer science to non-majors one semester. I know I would not want > to have to confront all the issues with numerical computing right off > the bat if I was just trying to teach people how to do some math. > There's probably nothing wrong with having a package like this, it just shouldn't be numpy. It's easy enough to construct such a beast for yourself, it should take just a few lines of Python. Since what a beginners package should look like probably varies from teacher to teacher, let them construct a few. If they all, or at least most of them, have the same ideas about what constitutes such a package, that might be to the time to think about officially supporting a separate entry point that has the modified behaviour. For the moment, things look fine. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Fri Oct 13 01:38:48 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Fri, 13 Oct 2006 14:38:48 +0900 Subject: How to make sqrt(-1) be 1j In-Reply-To: <452F195C.9000605@ieee.org> References: <452F195C.9000605@ieee.org> Message-ID: On 10/13/06, Tim Hochberg wrote: > For this sort of thing, I > would just make a new module to pull together the function I want and > use that instead. It's then easy to explain that this new module bbeconf > (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an > amalgamation of stuff from multiple sources. > > # bbeconf.py > from numpy import * > fromnumpy.scimath import sqrt > # possibly some other stuff to correctly handle subpackages... That does sound like a good way to do it. Then you just tell your users to import 'eduNumpy' rather than numpy, and you're good to go. Added that suggestion to http://www.scipy.org/NegativeSquareRoot I'd like to ask one basic Python question related my previous suggestion of doing things like "numpy.sqrt = numpy.lib.scimath.sqrt": In python does that make it so that any module importing numpy in the same program will now see the altered sqrt function? E.g. in my program I do "import A,B". Module A alters numpy.sqrt. Does that also modify how module B sees numpy.sqrt? If so then that's a very good reason not to do it that way. I've heard people using the term "monkey-patch" before. Is that what that is? --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 13 02:47:40 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 13 Oct 2006 00:47:40 -0600 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: <452F195C.9000605@ieee.org> Message-ID: <452F368C.8010101@ieee.org> Bill Baxter wrote: > On 10/13/06, Tim Hochberg wrote: > >> For this sort of thing, I >> would just make a new module to pull together the function I want and >> use that instead. It's then easy to explain that this new module bbeconf >> (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an >> amalgamation of stuff from multiple sources. >> >> # bbeconf.py >> from numpy import * >> fromnumpy.scimath import sqrt >> # possibly some other stuff to correctly handle subpackages... >> > > That does sound like a good way to do it. > Then you just tell your users to import 'eduNumpy' rather than numpy, > and you're good to go. > Added that suggestion to http://www.scipy.org/NegativeSquareRoot > > I'd like to ask one basic Python question related my previous > suggestion of doing things like "numpy.sqrt = numpy.lib.scimath.sqrt": > In python does that make it so that any module importing numpy in the > same program will now see the altered sqrt function? E.g. in my > program I do "import A,B". Module A alters numpy.sqrt. Does that > also modify how module B sees numpy.sqrt? > > If so then that's a very good reason not to do it that way. > > I've heard people using the term "monkey-patch" before. Is that what that is? > > --bb > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From numptyboya at aol.com Fri Oct 13 06:36:57 2006 From: numptyboya at aol.com (numptyboya at aol.com) Date: Fri, 13 Oct 2006 07:36:57 -0300 Subject: Those young people taking the position you deserve because you lack a Degree? Message-ID: <676458026452597863.984775100774389437@aol.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: t5iu3stjdoovy.gif Type: image/gif Size: 9708 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ivvblszq at wibx950.com Fri Oct 13 09:38:36 2006 From: ivvblszq at wibx950.com (regarding D.) Date: Fri, 13 Oct 2006 11:38:36 -0200 Subject: need Message-ID: <000701c6eeab$5b19beb0$75e31e53@darek3ji4mr86y> Judge variety or scholars which explodes myths regarding d Mcnamara Stanford is their bring. Consent formadd Watchzach Braffs Videozach touching coming Mayanadd Diamond Dicaprio stars. Cuny am Graduate Center Scholarly in lucid or readable the original between justice Grinspoon md Harvard School am. Leads life first series Your Guffmans Guest or turns camera als spoof am. And Harry g Levine is editors Drugs or Social Justice days American Studies Sociology in. Screen a Film films wireless video Friday am Pdtpage More of Litebrite Yesterday in Pmfrom is Turn magical am shining. Hair Furtado where touches hair every or Martin leads life first a series Your Guffmans? Its Policy or have been stuffed of full lies about users What now in detox and good a. Superb of among included papers or some most powerful any ever subject its. Hopeful bookkurt Schmoke Mayor Baltimore examines detail myth reality drugs society reader. Videos Clips Games Warzone web Junk Worldtop a Videouser Whats Upin Theaters Nowwatch a trailers. The Press Craig Reinarman and Harry g Levine editors Drugs Social Justice is days American Studies Sociology in. Roots hard or abuse Americas inner cities it am shows policies made worse points a war. Inner cities it shows policies made a worse points war morass hopeful bookkurt of Schmoke is. Talk Crimes Victims Joel Best Politics am Price Denial eva Christina Bertram is Morris a Blachman in! Blacks middle class poor explains why or was much of less other countries or such or C -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: John.gif Type: image/gif Size: 10058 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From muhsinu at durodyne.com Fri Oct 13 06:44:47 2006 From: muhsinu at durodyne.com (Ciril Reynaga) Date: Fri, 13 Oct 2006 03:44:47 -0700 Subject: VkAGRA Message-ID: <000001c6eeb4$9a272b20$10d4a8c0@rtbjpx> Hi, VkAGRA for LESS http://www.optuniosen.com to come along as well. It will make an interesting gathering. button is to play, red to stop. Then he was gone. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gregwillden at gmail.com Fri Oct 13 09:36:51 2006 From: gregwillden at gmail.com (Greg Willden) Date: Fri, 13 Oct 2006 08:36:51 -0500 Subject: SVN checkout problem Message-ID: <903323ff0610130636h642e6c0asefc781e3194d6de@mail.gmail.com> Hi, I just tried to checkout numpy and scipy to another machine and got the following errors: $ svn co http://svn.scipy.org/svn/numpy/trunk numpy svn: REPORT request failed on '/svn/numpy/!svn/vcc/default' svn: REPORT of '/svn/numpy/!svn/vcc/default': 400 Bad Request ( http://svn.scipy.org) $ svn co http://svn.scipy.org/svn/scipy/trunk scipy svn: REPORT request failed on '/svn/scipy/!svn/vcc/default' svn: REPORT of '/svn/scipy/!svn/vcc/default': 400 Bad Request ( http://svn.scipy.org) Any ideas? Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Fri Oct 13 09:44:25 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 13 Oct 2006 06:44:25 -0700 Subject: How to make sqrt(-1) be 1j In-Reply-To: References: <452F195C.9000605@ieee.org> Message-ID: <452F9839.8060600@ieee.org> Bill Baxter wrote: > On 10/13/06, Tim Hochberg wrote: > >> For this sort of thing, I >> would just make a new module to pull together the function I want and >> use that instead. It's then easy to explain that this new module bbeconf >> (Bill Baxter's Excellent Collection Of Numeric Functions) is actually an >> amalgamation of stuff from multiple sources. >> >> # bbeconf.py >> from numpy import * >> fromnumpy.scimath import sqrt >> # possibly some other stuff to correctly handle subpackages... >> > > That does sound like a good way to do it. > Then you just tell your users to import 'eduNumpy' rather than numpy, > and you're good to go. > Added that suggestion to http://www.scipy.org/NegativeSquareRoot > > I'd like to ask one basic Python question related my previous > suggestion of doing things like "numpy.sqrt = numpy.lib.scimath.sqrt": > In python does that make it so that any module importing numpy in the > same program will now see the altered sqrt function? E.g. in my > program I do "import A,B". Module A alters numpy.sqrt. Does that > also modify how module B sees numpy.sqrt? > Indeed it does. Module imports are cached in sys.modules, so numpy is only imported once. (With some effort, you can usually get your own private copy of a module, that you could mess with to your hearts content, but I generally wouldn't recommend it). > If so then that's a very good reason not to do it that way. > > I've heard people using the term "monkey-patch" before. Is that what that is? > I believe that is what the term refers to although I'm not absolutely certain. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Fri Oct 13 10:08:52 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 13 Oct 2006 16:08:52 +0200 Subject: Testing numpy without doing an installation? Message-ID: <1160748533.3960.5.camel@localhost.localdomain> Hi, Is it possible to test a numpy version directly from the source directory without having to install it? I mean, if I compile the sources and try to use the package directly from there, I get unexpected results. For example: $ export PYTHONPATH=/home/faltet/python.nobackup/numpy/trunk $ python2.4 -c "import numpy;print numpy.dtype([('col1', '(1,)i4')])" Running from numpy source directory. Traceback (most recent call last): File "", line 1, in ? AttributeError: 'module' object has no attribute 'dtype' It would be nice to have a way of testing a recently built version of numpy prior to install it. Thanks, -- >0,0< Francesc Altet http://www.carabos.com/ V V C?rabos Coop. V. Enjoy Data "-" "Be careful about using the following code -- I've only proven that it works, I haven't tested it." -- Donald Knuth ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From stefan at sun.ac.za Fri Oct 13 10:28:25 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 13 Oct 2006 16:28:25 +0200 Subject: dtype always copies Message-ID: <20061013142825.GA16246@mentat.za.net> Hi all, I've noticed that 'astype' always forces a copy. Is this behaviour intended? It seems to conflict with 'asarray', that tries to avoid a copy. For example, when wrapping code in ctypes, the following snippet would have been useful: def foo(x): # ensure x is an array of the right type x = N.ascontiguousarray(x).astype(N.intc) but that will cause a copy, so you'll have to do def foo(x): try: x = N.ascontiguousarray(x,N.intc) except: x = N.ascontiguousarray(x).astype(N.intc) Maybe I'm missing something obvious here -- any pointers? Thanks St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From msubotin at gmail.com Fri Oct 13 15:22:17 2006 From: msubotin at gmail.com (Michael Subotin) Date: Fri, 13 Oct 2006 15:22:17 -0400 Subject: Library problem on installation In-Reply-To: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> Message-ID: <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> Following up on my own message for archival purposes, after getting local help. If you're having a problem like this, read the file called INSTALL.txt. The current NumPy tarball doesn't have this file, but the SciPy tarball does. You may need to reinstall Atlas/Lapack libraries using a different compiler. Michael On 10/11/06, Michael Subotin wrote: > > Hi, > > I know that it's a perennial topic on the list, but I haven't been able to > find my answer in the archives. After running the installation on a RedHat > Linux machine, I'm getting the import error: "/usr/lib/libblas.so.3: > undefined symbol: e_wsfe". Judging from earlier exchanges here, it seems > that I need to add libg2c (which this machine does have in /usr/lib, unlike > g2c) somewhere between 'f77blas' and 'cblas', but I'm not sure where I > should make this change. Not being well versed in Python distributions, I > tried my luck with a few candidates and the import error remains. The > machine should be running gcc. > > Thanks for any help. > > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Fri Oct 13 15:50:58 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 13 Oct 2006 13:50:58 -0600 Subject: Polyfit In-Reply-To: <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> Message-ID: On 10/12/06, Greg Willden wrote: > > On 10/12/06, Charles R Harris wrote: > > > > And here is the location of the problem in numpy/linalg/linalg.py : > > > > def lstsq(a, b, rcond=1.e-10): > > > > The 1e-10 is a bit conservative. On the other hand, I will note that the > > condition number of the dot(V^T ,V) matrix is somewhere around 1e22, which > > means in general terms that you need around 22 digits of accuracy. Inverting > > it only works sorta by accident in the current case. Generally, using > > Vandermonde matrices and polynomial fits it a bad idea when the dynamic > > range of the interval gets large and the degree gets up around 4-5 as it > > leads to ill conditioned sets of equations. When you really need the best > > start with chebychev polynomials or, bestest, compute a set of polynomials > > orthogonal over the sample points. Anyway, I think rcond should be something > > like 1e-12 or 1e-13 by default and be available as a keyword in the polyfit > > function. If no one complains I will make this change, although it is just a > > bandaid and things will fall apart again as soon as you call polyfit(x,y,4). > > > > > > Hey that's great. I'm glad you tracked it down. > > Pardon my ignorance of polyfit algorithm details. > Is there a way of choosing rcond based on N that would give sensible > defaults for a variety of N? > Greg You can also get *much* better results if you scale the x interval to [0,1] as the problem will be better posed. For instance, with your data and a degree 10 fit I get a condition number of about 2e7 when x is scaled to [0,1], as opposed to about 1e36 when left as is. The former yields a perfectly useable fit while the latter blows up. I suppose this could be built into the polyfit routine if one were only interested in polynomial fits of some sort, but the polynomial would have to carry around an offset and scale factor to make evaluation work. If Travis is interested in such a thing we could put together some variant of the polynomials that includes the extra data. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Fri Oct 13 15:51:38 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 15:51:38 -0400 Subject: rcond in polyfit In-Reply-To: References: Message-ID: On 12/10/06, Charles R Harris wrote: > Hi all, > > I note that polyfit looks like it should work for single and double, real > and complex, polynomials. On the otherhand, the default rcond needs to > depend on the underlying precision. On the other, other hand, all the svd > computations are done with dgelsd or zgelsd, i.e., double precision. Even so > problems can arise from inherent errors of the input data if it is single > precision to start with. I also think the final degree of the fit should be > available somewhere if wanted, as it is an indication of what is going on. > Sooo, any suggestions as to what to do? My initial impulse would be to set > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick the > can down the road on returning the actual degree of the fit. I'd also be inclined to output a warning (which the user can ignore, read or trap as necessary) if the condition number is too bad or they supplied an rcond that is too small for the precision of their data. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 13 15:55:30 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 15:55:30 -0400 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> Message-ID: On 13/10/06, Charles R Harris wrote: > You can also get *much* better results if you scale the x interval to [0,1] > as the problem will be better posed. For instance, with your data and a > degree 10 fit I get a condition number of about 2e7 when x is scaled to > [0,1], as opposed to about 1e36 when left as is. The former yields a > perfectly useable fit while the latter blows up. I suppose this could be > built into the polyfit routine if one were only interested in polynomial > fits of some sort, but the polynomial would have to carry around an offset > and scale factor to make evaluation work. [-1,1] would probably be even better, no? > If Travis is interested in such a thing we could put together some variant > of the polynomials that includes the extra data. At this point you might as well use a polynomial class that can accomodate a variety of bases for the space of polynomials - X^n, (X-a)^n, orthogonal polynomials (translated and scaled as needed), what have you. I think I vote for polyfit that is no more clever than it has to be but which warns the user when the fit is bad. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Fri Oct 13 16:09:36 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 13 Oct 2006 17:09:36 -0300 Subject: NumPy/SciPy + MPI for Python Message-ID: This post is surely OT, but I cannot imagine a better place to contact people about this subject. Please, don't blame me. Any people here interested in NumPy/SciPy + MPI? From some time ago, I've been developing mpi4py (first release at SF) and I am really near to release a new version. This package exposes an API almost identical to MPI-2 C++ bindings. Almost all MPI-1 and MPI-2 features (even one-sided communications and parallel I/O) are fully supported for any object exposing single-segment buffer interface, an only some of them for communication of general Python objects (with the help of pickle/marshal). The posibility of constructing any user-defined MPI datatypes, as well as virtual topologies (specially cartesian), can be really nice for anyone interested in parallel multidimensional array procesing. Before the next release, I would like to wait for any comment, You can contact me via private mail to get a tarbal with latest developments, or we can have some discussion here, if many of you consider this a good idea. In the long term, I would like to see mpi4py integrated as a subpackage of SciPy. Regards, -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From dalcinl at gmail.com Fri Oct 13 16:20:48 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 13 Oct 2006 17:20:48 -0300 Subject: Testing numpy without doing an installation? In-Reply-To: <1160748533.3960.5.camel@localhost.localdomain> References: <1160748533.3960.5.camel@localhost.localdomain> Message-ID: On 10/13/06, Francesc Altet wrote: > Is it possible to test a numpy version directly from the source > directory without having to install it? I usually do: $ python setup.py build $ python setup.py install --home=/tmp $ export PYTHONPATH=/tmp/lib/python and then $ python -c 'import numpy; numpy.test()' and finally, if all was right, su -c 'python setup.py install' or python setup.py install --home=$HOME -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Fri Oct 13 16:36:50 2006 From: gregwillden at gmail.com (Greg Willden) Date: Fri, 13 Oct 2006 15:36:50 -0500 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> Message-ID: <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> On 10/13/06, A. M. Archibald wrote: > > At this point you might as well use a polynomial class that can > accomodate a variety of bases for the space of polynomials - X^n, > (X-a)^n, orthogonal polynomials (translated and scaled as needed), > what have you. > > I think I vote for polyfit that is no more clever than it has to be > but which warns the user when the fit is bad. > > What about including multiple algorithms each returning a figure of fit? Then I could try two or three different algorithms and then use the one that works best for my data. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Fri Oct 13 16:45:50 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 13 Oct 2006 13:45:50 -0700 Subject: Polyfit In-Reply-To: <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> Message-ID: <452FFAFE.1000902@ieee.org> Greg Willden wrote: > On 10/13/06, *A. M. Archibald* > wrote: > > At this point you might as well use a polynomial class that can > accomodate a variety of bases for the space of polynomials - X^n, > (X-a)^n, orthogonal polynomials (translated and scaled as needed), > what have you. > > I think I vote for polyfit that is no more clever than it has to be > but which warns the user when the fit is bad. > > > > What about including multiple algorithms each returning a figure of fit? > Then I could try two or three different algorithms and then use the > one that works best for my data. A simple, "stupid" curve fitting algorithm may be appropriate for numpy, but once your getting into multiple algorithms it's time to move it to a package in scipy IMO (and it would be good to find someone who cares, and knows, about curve fitting to adopt it). -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Fri Oct 13 16:58:35 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 13 Oct 2006 14:58:35 -0600 Subject: rcond in polyfit In-Reply-To: References: Message-ID: On 10/13/06, A. M. Archibald wrote: > > On 12/10/06, Charles R Harris wrote: > > Hi all, > > > > I note that polyfit looks like it should work for single and double, > real > > and complex, polynomials. On the otherhand, the default rcond needs to > > depend on the underlying precision. On the other, other hand, all the > svd > > computations are done with dgelsd or zgelsd, i.e., double precision. > Even so > > problems can arise from inherent errors of the input data if it is > single > > precision to start with. I also think the final degree of the fit should > be > > available somewhere if wanted, as it is an indication of what is going > on. > > Sooo, any suggestions as to what to do? My initial impulse would be to > set > > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick > the > > can down the road on returning the actual degree of the fit. > > I'd also be inclined to output a warning (which the user can ignore, > read or trap as necessary) if the condition number is too bad or they > supplied an rcond that is too small for the precision of their data. That sounds good, but how to do it? Should I raise an exception? I would also have to modify lstsq so it returns the degree of the fit which would mess up the current interface. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Fri Oct 13 17:03:47 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 17:03:47 -0400 Subject: Polyfit In-Reply-To: <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> Message-ID: On 13/10/06, Greg Willden wrote: > What about including multiple algorithms each returning a figure of fit? > Then I could try two or three different algorithms and then use the one that > works best for my data. The basic problem is that X^n is rarely a good basis for the functions on [a,b]. So if you want it to return the coefficients of a polynomial, you're basically stuck. If you *don't* want that, there's a whole bestiary of other options. If you're just looking to put a smooth curve through a bunch of data points (perhaps with known uncertainties), scipy.interpolate includes some nice spline fitting functions. If you're looking for polynomials, orthogonal polynomials may serve as a better basis for your interval; you can look in scipy.special for them (and leastsq will fit them to your points). Extracting their coefficients is possible but will bring you back to numerical instabilities. In any case, all this is outside the purview of numpy (as is polyfit, frankly). A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 13 17:07:55 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 17:07:55 -0400 Subject: rcond in polyfit In-Reply-To: References: Message-ID: On 13/10/06, Charles R Harris wrote: > > > > On 10/13/06, A. M. Archibald wrote: > > On 12/10/06, Charles R Harris wrote: > > > Hi all, > > > > > > I note that polyfit looks like it should work for single and double, > real > > > and complex, polynomials. On the otherhand, the default rcond needs to > > > depend on the underlying precision. On the other, other hand, all the > svd > > > computations are done with dgelsd or zgelsd, i.e., double precision. > Even so > > > problems can arise from inherent errors of the input data if it is > single > > > precision to start with. I also think the final degree of the fit should > be > > > available somewhere if wanted, as it is an indication of what is going > on. > > > Sooo, any suggestions as to what to do? My initial impulse would be to > set > > > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, and kick > the > > > can down the road on returning the actual degree of the fit. > > > > I'd also be inclined to output a warning (which the user can ignore, > > read or trap as necessary) if the condition number is too bad or they > > supplied an rcond that is too small for the precision of their data. > > That sounds good, but how to do it? Should I raise an exception? I would > also have to modify lstsq so it returns the degree of the fit which would > mess up the current interface. Python's warnings module is a decent solution for providing this information. Goodness-of-fit worries me less than ill-conditioning - users are going to expect the curve to deviate from their function (and an easy reliable way to get goodness of fit is sqrt(sum(abs(f(xs)-polynomial(xs))**2)); this is certain to take into account any roundoff errors introduced anywhere). But they may well have no idea they should be worried about the condition number of some matrix they've never heard of. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gregwillden at gmail.com Fri Oct 13 17:09:47 2006 From: gregwillden at gmail.com (Greg Willden) Date: Fri, 13 Oct 2006 16:09:47 -0500 Subject: Polyfit In-Reply-To: References: <903323ff0610121259r7c8a2129pb2b943e6f4e2d75a@mail.gmail.com> <903323ff0610121415r1f7fce89yf12d34cabc3c49ac@mail.gmail.com> <903323ff0610121930j67527f45h24a7d469e2ed4908@mail.gmail.com> <903323ff0610131336m300dfb7bwd80a53b05e87301e@mail.gmail.com> Message-ID: <903323ff0610131409n70dbe3cew6588fa2f35179b89@mail.gmail.com> On 10/13/06, A. M. Archibald wrote: > > In any case, all this is outside the purview of numpy (as is polyfit, > frankly). > Great. Thanks for the ideas of other algorithms/functions to look at. Greg -- Linux. Because rebooting is for adding hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Fri Oct 13 17:11:57 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 13 Oct 2006 14:11:57 -0700 Subject: rcond in polyfit In-Reply-To: References: Message-ID: <4530011D.5060105@ieee.org> Charles R Harris wrote: > > > On 10/13/06, *A. M. Archibald* > wrote: > > On 12/10/06, Charles R Harris > wrote: > > Hi all, > > > > I note that polyfit looks like it should work for single and > double, real > > and complex, polynomials. On the otherhand, the default rcond > needs to > > depend on the underlying precision. On the other, other hand, > all the svd > > computations are done with dgelsd or zgelsd, i.e., double > precision. Even so > > problems can arise from inherent errors of the input data if it > is single > > precision to start with. I also think the final degree of the > fit should be > > available somewhere if wanted, as it is an indication of what is > going on. > > Sooo, any suggestions as to what to do? My initial impulse would > be to set > > rcond=1e-6 for single, 1e-14 for double, make rcond a keyword, > and kick the > > can down the road on returning the actual degree of the fit. > > I'd also be inclined to output a warning (which the user can ignore, > read or trap as necessary) if the condition number is too bad or they > supplied an rcond that is too small for the precision of their data. > > > That sounds good, but how to do it? Should I raise an exception? Use the warnings framework: >>> import warnings >>> warnings.warn("condition number is BAD") __main__:1: UserWarning: condition number is BAD The user can turn warnings on or off or turned in exceptions based on a variety of criteria. Look for the warnings filter in the docs. Which brings up a question: do we want to have a FloatingPointWarning or some such? Currently, if you use set the error handling to warn using seterr a runtime warning is issued: >>> np.seterr(all='warn') {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'} >>> np.arange(1) / 0 __main__:1: RuntimeWarning: divide by zero encountered in divide On the other hand if error handling is set to 'raise', then a FloatingPointError is issued. Is a FloatingPointWarning in order to mirror the FloatingPointError? And if so, would it be appropriate to use for condition number? > I would also have to modify lstsq so it returns the degree of the fit > which would mess up the current interface. One approach would be to write lstsqcond (or a better name) that returns both the fit and the condition number. listsq could then be just a wrapper over that which dumped the condition number. IIRC, the condition number is available, but we're not returning it. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 13 18:09:26 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 13 Oct 2006 18:09:26 -0400 Subject: rcond in polyfit In-Reply-To: <4530011D.5060105@ieee.org> References: <4530011D.5060105@ieee.org> Message-ID: On 13/10/06, Tim Hochberg wrote: > Charles R Harris wrote: > > That sounds good, but how to do it? Should I raise an exception? > Use the warnings framework: > > >>> import warnings > >>> warnings.warn("condition number is BAD") > __main__:1: UserWarning: condition number is BAD > > The user can turn warnings on or off or turned in exceptions based on a > variety of criteria. Look for the warnings filter in the docs. > > Which brings up a question: do we want to have a FloatingPointWarning or > some such? Currently, if you use set the error handling to warn using > seterr a runtime warning is issued: > > >>> np.seterr(all='warn') > {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': > 'ignore'} > >>> np.arange(1) / 0 > __main__:1: RuntimeWarning: divide by zero encountered in divide > > > On the other hand if error handling is set to 'raise', then a > FloatingPointError is issued. Is a FloatingPointWarning in order to > mirror the FloatingPointError? And if so, would it be appropriate to use > for condition number? I submitted a patchto use warnings for several functions in scipy a while ago, and the approach I took was to create a ScipyWarning, from which more specific warnings were derived (IntegrationWarning, for example). That was perhaps a bit short-sighted. I'd suggest a FloatingPointWarning as a base class, with IllConditionedMatrix as a subclass (it should include the condition number, but probably not the matrix itself unless it's small, as debugging information). The warnings module is frustratingly non-reentrant, unfortunately, which makes writing tests very awkward. > > I would also have to modify lstsq so it returns the degree of the fit > > which would mess up the current interface. > One approach would be to write lstsqcond (or a better name) that returns > both the fit and the condition number. listsq could then be just a > wrapper over that which dumped the condition number. IIRC, the > condition number is available, but we're not returning it. This is a very good idea. scipy.integrate.quad returns a pair (result, error_estimate) and every time I use it I trip over that. (Perhaps if I were a fine upstanding numerical analyst I would be checking the error estimate every time, but it is a pain.) Another option would be a "full_output" optional argument. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 13 18:49:41 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 13 Oct 2006 16:49:41 -0600 Subject: NumPy/SciPy + MPI for Python In-Reply-To: References: Message-ID: <6ce0ac130610131549s626aae9chbf3bed45dfcb9230@mail.gmail.com> Just as a data point. I have used mpi4py before and have built it on many systems ranging from my macbook to NERSC supercomputers. In my opinion it is currently the best python mpi bindings available. Lisandro has done a fantastic job with this. Also Fernando and I have worked hard to make sure that mpi4py works with the new parallel capabilities of IPython. I would love to see mpi4py hosted in a public repository for others to contribute. I think this would really solidify mpi4py as a top notch mpi interface. But, my only concern is that there might be many folks who want to use mpi4py who don't need scipy. I am one of those folks - I don't necessarily need scipy on the NERSC supercomputers, but I do need mpi4py. Because of this, I would probably still recommend keeping mpi4py as a separate project. Is there any chance it could be hosted at mip4py.scipy.org? I strongly encourage others to try it out. Installation is is easy. Brian Granger On 10/13/06, Lisandro Dalcin wrote: > This post is surely OT, but I cannot imagine a better place to contact > people about this subject. Please, don't blame me. > > Any people here interested in NumPy/SciPy + MPI? From some time ago, > I've been developing mpi4py (first release at SF) and I am really near > to release a new version. > > This package exposes an API almost identical to MPI-2 C++ bindings. > Almost all MPI-1 and MPI-2 features (even one-sided communications and > parallel I/O) are fully supported for any object exposing > single-segment buffer interface, an only some of them for > communication of general Python objects (with the help of > pickle/marshal). > > The posibility of constructing any user-defined MPI datatypes, as well > as virtual topologies (specially cartesian), can be really nice for > anyone interested in parallel multidimensional array procesing. > > Before the next release, I would like to wait for any comment, You can > contact me via private mail to get a tarbal with latest developments, > or we can have some discussion here, if many of you consider this a > good idea. In the long term, I would like to see mpi4py integrated as > a subpackage of SciPy. > > Regards, > > -- > Lisandro Dalc?n > --------------- > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > _______________________________________________ > Scipy-dev mailing list > Scipy-dev at scipy.org > http://projects.scipy.org/mailman/listinfo/scipy-dev > From eric at enthought.com Fri Oct 13 19:02:55 2006 From: eric at enthought.com (eric) Date: Fri, 13 Oct 2006 18:02:55 -0500 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <6ce0ac130610131549s626aae9chbf3bed45dfcb9230@mail.gmail.com> References: <6ce0ac130610131549s626aae9chbf3bed45dfcb9230@mail.gmail.com> Message-ID: <45301B1F.3030609@enthought.com> Brian Granger wrote: > Just as a data point. > > I have used mpi4py before and have built it on many systems ranging > from my macbook to NERSC supercomputers. In my opinion it is > currently the best python mpi bindings available. Lisandro has done a > fantastic job with this. Also Fernando and I have worked hard to make > sure that mpi4py works with the new parallel capabilities of IPython. > > I would love to see mpi4py hosted in a public repository for others to > contribute. I think this would really solidify mpi4py as a top notch > mpi interface. But, my only concern is that there might be many folks > who want to use mpi4py who don't need scipy. I am one of those folks > - I don't necessarily need scipy on the NERSC supercomputers, but I do > need mpi4py. Because of this, I would probably still recommend > keeping mpi4py as a separate project. Is there any chance it could be > hosted at mip4py.scipy.org? > Fine from our side... eric > I strongly encourage others to try it out. Installation is is easy. > > Brian Granger > > On 10/13/06, Lisandro Dalcin wrote: > >> This post is surely OT, but I cannot imagine a better place to contact >> people about this subject. Please, don't blame me. >> >> Any people here interested in NumPy/SciPy + MPI? From some time ago, >> I've been developing mpi4py (first release at SF) and I am really near >> to release a new version. >> >> This package exposes an API almost identical to MPI-2 C++ bindings. >> Almost all MPI-1 and MPI-2 features (even one-sided communications and >> parallel I/O) are fully supported for any object exposing >> single-segment buffer interface, an only some of them for >> communication of general Python objects (with the help of >> pickle/marshal). >> >> The posibility of constructing any user-defined MPI datatypes, as well >> as virtual topologies (specially cartesian), can be really nice for >> anyone interested in parallel multidimensional array procesing. >> >> Before the next release, I would like to wait for any comment, You can >> contact me via private mail to get a tarbal with latest developments, >> or we can have some discussion here, if many of you consider this a >> good idea. In the long term, I would like to see mpi4py integrated as >> a subpackage of SciPy. >> >> Regards, >> >> -- >> Lisandro Dalc?n >> --------------- >> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) >> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) >> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) >> PTLC - G?emes 3450, (3000) Santa Fe, Argentina >> Tel/Fax: +54-(0)342-451.1594 >> _______________________________________________ >> Scipy-dev mailing list >> Scipy-dev at scipy.org >> http://projects.scipy.org/mailman/listinfo/scipy-dev >> >> > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From thamelry at binf.ku.dk Sat Oct 14 04:58:41 2006 From: thamelry at binf.ku.dk (thamelry at binf.ku.dk) Date: Sat, 14 Oct 2006 10:58:41 +0200 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: References: Message-ID: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> On 10/13/06, Lisandro Dalcin wrote: > This post is surely OT, but I cannot imagine a better place to contact > people about this subject. Please, don't blame me. > > Any people here interested in NumPy/SciPy + MPI? I've been working on a Dynamic Bayesian Network (DBN) toolkit for some time (called Mocapy, freely available from sourceforge https://sourceforge.net/projects/mocapy/). The thing is almost entirely implemented in Python, and currently uses Numeric and pyMPI. I routinely train DBNs from 100.000s of observations on our 240 CPU cluster. I'm in the proces of porting Mocapy to numpy. I assume pyMPI will also work with numpy, but I haven't tried it out yet. Would be great if scipy came with default MPI support, especially since pyMPI does not seem to be actively developed anymore. Cheers, -Thomas ---- Thomas Hamelryck, Post-doctoral researcher Bioinformatics center Institute of Molecular Biology and Physiology University of Copenhagen Universitetsparken 15 - Bygning 10 DK-2100 Copenhagen ? Denmark Homepage: http://www.binf.ku.dk/Protein_structure ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From RYIU at HOTMAIL.COM Sat Oct 14 05:43:37 2006 From: RYIU at HOTMAIL.COM (=?iso-2022-jp?B?UllJVQ==?=) Date: Sat, 14 Oct 2006 02:43:37 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCIVobKEJTTlMbJEIhWxsoQjE=?= =?iso-2022-jp?b?GyRCN28kTj1xOX4kXyQsJCIkaiReJDckPyEjGyhC?= Message-ID: ????????????? http://xxzzxx.com/n/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wfspotz at sandia.gov Sat Oct 14 08:58:45 2006 From: wfspotz at sandia.gov (Bill Spotz) Date: Sat, 14 Oct 2006 06:58:45 -0600 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> Message-ID: <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> On Oct 14, 2006, at 2:58 AM, thamelry at binf.ku.dk wrote: > On 10/13/06, Lisandro Dalcin wrote: >> This post is surely OT, but I cannot imagine a better place to >> contact >> people about this subject. Please, don't blame me. >> >> Any people here interested in NumPy/SciPy + MPI? > > I've been working on a Dynamic Bayesian Network (DBN) toolkit for > some time > (called Mocapy, freely available from sourceforge https:// > sourceforge.net/projects/mocapy/). The thing is almost entirely > implemented in Python, and currently uses Numeric and pyMPI. I > routinely train DBNs from 100.000s of observations on our 240 CPU > cluster. > > I'm in the proces of porting Mocapy to numpy. I assume pyMPI will > also work with numpy, but I haven't tried it out yet. Would be > great if scipy came with default MPI support, especially since > pyMPI does not seem to be actively developed anymore. I would like to second the notion of converging on a single MPI interface. My parallel project encapsulates most of the inter- processor communication within higher-level objects because the lower- level communication patterns can usually be determined from higher- level data structures. But still, there are times when a user would like access to the lower-level MPI interface. I haven't tried to make my project compatible with any of the several MPI python interfaces out there, because it isn't clear to me which one is most widely used. If one were to emerge -- and even better, if the various independent projects were to then combine their efforts in an open source environment (the way Numeric and numarray are converging to NumPy) -- then this choice would be easy. ** Bill Spotz ** ** Sandia National Laboratories Voice: (505)845-0170 ** ** P.O. Box 5800 Fax: (505)284-5451 ** ** Albuquerque, NM 87185-0370 Email: wfspotz at sandia.gov ** ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gael.varoquaux at normalesup.org Sat Oct 14 09:34:16 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sat, 14 Oct 2006 15:34:16 +0200 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> Message-ID: <20061014133416.GA12465@clipper.ens.fr> On Sat, Oct 14, 2006 at 06:58:45AM -0600, Bill Spotz wrote: > I would like to second the notion of converging on a single MPI > interface. My parallel project encapsulates most of the inter- > processor communication within higher-level objects because the lower- > level communication patterns can usually be determined from higher- > level data structures. But still, there are times when a user would > like access to the lower-level MPI interface. I think we are running in the same old problem of scipy/a super-package of scientific tools. It is important to keep numpy/scipy modular. People may want to install either one without an MPI interface. However it is nice that if somebody just wants "everything" without having to choose he can download that "everything" from scpy.org and that it comes well bundled together. So all we need to do is find a name for that super-package, put it together with good integration and add it to scipy.org. My 2 cents, Ga?l ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Sat Oct 14 12:32:47 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 14 Oct 2006 12:32:47 -0400 Subject: Problem buliding rc2 on OS X 10.3.9/Python2.5 In-Reply-To: References: Message-ID: Jay Parlar wrote: > In the process of finally switching over to Python 2.5, and am trying > to build numpy. Unfortunately, it dies during the build: > C compiler: gcc -arch ppc -arch i386 -isysroot > /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double > -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 > > compile options: > '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > -Inumpy/core/src -Inumpy/core/include > -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > -c' > gcc: _configtest.c > gcc: cannot specify -o with -c or -S and multiple compilations > gcc: cannot specify -o with -c or -S and multiple compilations > failure. This is the problem. Are you sure that you are using the correct version of gcc for making Universal binaries on 10.3.9? If so, then we are not passing the correct flags to it. Unfortunately, I think that the Universal stuff is going to make our lives quite complicated. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Sat Oct 14 12:48:15 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Sat, 14 Oct 2006 12:48:15 -0400 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <20061014133416.GA12465@clipper.ens.fr> References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> <20061014133416.GA12465@clipper.ens.fr> Message-ID: On 14/10/06, Gael Varoquaux wrote: > On Sat, Oct 14, 2006 at 06:58:45AM -0600, Bill Spotz wrote: > > I would like to second the notion of converging on a single MPI > > interface. My parallel project encapsulates most of the inter- > > processor communication within higher-level objects because the lower- > > level communication patterns can usually be determined from higher- > > level data structures. But still, there are times when a user would > > like access to the lower-level MPI interface. > > I think we are running in the same old problem of scipy/a super-package > of scientific tools. It is important to keep numpy/scipy modular. People > may want to install either one without an MPI interface. However it is > nice that if somebody just wants "everything" without having to choose > he can download that "everything" from scpy.org and that it comes well > bundled together. So all we need to do is find a name for that > super-package, put it together with good integration and add it to > scipy.org. > > My 2 cents, > > Ga?l I agree. Moreover, being picked for such integration work would help encourage people to converge on one MPI interface (for example). There's some discussion of such a package at NumpyProConPage on the wiki (and in particular at http://www.scipy.org/PyLabAwaits ). The "scipy superpack" (which is only for Windows, as I understand it) is a sort of beginning. So, well, any suggestion for a name? pylab is already in use by matplotlib (for some reason), as is Scientific Python (and numpy, Numeric and numarray are obviously already confusing). A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 14 12:50:36 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 14 Oct 2006 10:50:36 -0600 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> <20061014133416.GA12465@clipper.ens.fr> Message-ID: On 10/14/06, A. M. Archibald wrote: > > On 14/10/06, Gael Varoquaux wrote: > > On Sat, Oct 14, 2006 at 06:58:45AM -0600, Bill Spotz wrote: I agree. Moreover, being picked for such integration work would help > encourage people to converge on one MPI interface (for example). > There's some discussion of such a package at NumpyProConPage on the > wiki (and in particular at http://www.scipy.org/PyLabAwaits ). The > "scipy superpack" (which is only for Windows, as I understand it) is a > sort of beginning. > > So, well, any suggestion for a name? pylab is already in use by > matplotlib (for some reason), as is Scientific Python (and numpy, > Numeric and numarray are obviously already confusing). supernumpy? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gael.varoquaux at normalesup.org Sat Oct 14 12:54:41 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sat, 14 Oct 2006 18:54:41 +0200 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> <20061014133416.GA12465@clipper.ens.fr> Message-ID: <20061014165441.GD12465@clipper.ens.fr> On Sat, Oct 14, 2006 at 10:50:36AM -0600, Charles R Harris wrote: > So, well, any suggestion for a name? pylab is already in use by > matplotlib (for some reason), as is Scientific Python (and numpy, > Numeric and numarray are obviously already confusing). > supernumpy? I think we should rather build upon scipy, rather than numpy, if we want to build upon an existing name. Numpy is low-level array manipulation, scipy is all the fancy math that goes above, so it make more sens to derive a name on scipy. Besides scipy.org is already a good rallying point for scientific computing with python How about scipylab ! :-> -- Ga?l ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Sat Oct 14 12:59:51 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 15 Oct 2006 01:59:51 +0900 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> <20061014133416.GA12465@clipper.ens.fr> Message-ID: On 10/15/06, A. M. Archibald wrote: > So, well, any suggestion for a name? pylab is already in use by > matplotlib (for some reason), as is Scientific Python (and numpy, > Numeric and numarray are obviously already confusing). I always thought 'pylab' was an odd name, so I always do from matplotlib import pylab as plot I think the matplotlib folks would be willing to change the name of pylab if it were seen as something that would be good for the community. I kind of wonder why they didn't call it 'matplot', personally. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From szdzh-66 at 163.com Sat Oct 14 13:12:17 2006 From: szdzh-66 at 163.com (=?GB2312?B?1cW6wNDL?=) Date: Sun, 15 Oct 2006 01:12:17 +0800 Subject: =?GB2312?B?UkQ7u9i4tA==?= Message-ID: ??????(??/?????! ????????????????,?????:??????????? ??????????????????????????.(????),?? ??????????????! ????????????????????(???????? ?????????????????????????????). ? ? ?: ??? ????: 013590116835 ? ??????????????? E-mail: szhw1359011 at 163.com -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From parlar at gmail.com Sat Oct 14 13:29:57 2006 From: parlar at gmail.com (Jay Parlar) Date: Sat, 14 Oct 2006 13:29:57 -0400 Subject: Problem buliding rc2 on OS X Message-ID: > Jay Parlar wrote: > > In the process of finally switching over to Python 2.5, and am trying > > to build numpy. Unfortunately, it dies during the build: > > > C compiler: gcc -arch ppc -arch i386 -isysroot > > /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double > > -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 > > > > compile options: > > '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > > -Inumpy/core/src -Inumpy/core/include > > -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 > > -c' > > gcc: _configtest.c > > gcc: cannot specify -o with -c or -S and multiple compilations > > gcc: cannot specify -o with -c or -S and multiple compilations > > failure. > > This is the problem. Are you sure that you are using the correct version of gcc > for making Universal binaries on 10.3.9? If so, then we are not passing the > correct flags to it. Unfortunately, I think that the Universal stuff is going to > make our lives quite complicated. > Well, my system is up-to-date, with only one gcc on it, so I don't know what else I can do. I originally missed the line saying "C compiler: gcc ...". It's odd, because I certainly don't have a /Developer/SDKs/MacOSX10.4u.sdk on my system. I wonder if that's there implicitly because the universal Python 2.5 I downloaded from python.org was built on a 10.4 system. Jay P. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Sat Oct 14 14:12:48 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 14 Oct 2006 14:12:48 -0400 Subject: Problem buliding rc2 on OS X In-Reply-To: References: Message-ID: Jay Parlar wrote: >> Jay Parlar wrote: >>> In the process of finally switching over to Python 2.5, and am trying >>> to build numpy. Unfortunately, it dies during the build: >>> C compiler: gcc -arch ppc -arch i386 -isysroot >>> /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double >>> -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 >>> >>> compile options: >>> '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 >>> -Inumpy/core/src -Inumpy/core/include >>> -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 >>> -c' >>> gcc: _configtest.c >>> gcc: cannot specify -o with -c or -S and multiple compilations >>> gcc: cannot specify -o with -c or -S and multiple compilations >>> failure. >> This is the problem. Are you sure that you are using the correct version of gcc >> for making Universal binaries on 10.3.9? If so, then we are not passing the >> correct flags to it. Unfortunately, I think that the Universal stuff is going to >> make our lives quite complicated. > > Well, my system is up-to-date, with only one gcc on it, so I don't > know what else I can do. I originally missed the line saying "C > compiler: gcc ...". It's odd, because I certainly don't have a > /Developer/SDKs/MacOSX10.4u.sdk on my system. I wonder if that's there > implicitly because the universal Python 2.5 I downloaded from > python.org was built on a 10.4 system. It probably was. I'm not sure what the deal is with building extensions with Universal Python on 10.3.9. It's possible that Universal binaries are only executable on 10.3.9, but not buildable. Are there any updates to the Developer Tools that you can install for 10.3.9? Particularly, is there one which provides gcc 4.0, which I think is the sine qua non for building Universal binaries. Can you build any other extension modules using distutils? If neither of the above is true, then you may need to upgrade to 10.4 to build Universal binaries. You might want to check the available Mac Python documentation and pythonmac-sig archives for more information. I have not been following the Universal discussion as closely as I could have (and as I'm currently on vacation, I'm not about to rectify that now). -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From parlar at gmail.com Sat Oct 14 15:28:52 2006 From: parlar at gmail.com (Jay Parlar) Date: Sat, 14 Oct 2006 15:28:52 -0400 Subject: Problem buliding rc2 on OS X Message-ID: > Are there any updates to the Developer Tools that you can install for 10.3.9? > Particularly, is there one which provides gcc 4.0, which I think is the sine qua > non for building Universal binaries. > > Can you build any other extension modules using distutils? > > If neither of the above is true, then you may need to upgrade to 10.4 to build > Universal binaries. You might want to check the available Mac Python > documentation and pythonmac-sig archives for more information. I have not been > following the Universal discussion as closely as I could have (and as I'm > currently on vacation, I'm not about to rectify that now). > Well, I just tried building PIL, and it worked just fine. So it looks like we can't generally say that it's impossible for me to build extension modules. I've posted a message to the distutils-sig, so hopefully we can resolve something there. If not, I'll take it to pythonmac-sig. Thanks, Jay P. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 14 16:28:47 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 14 Oct 2006 14:28:47 -0600 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: On 10/13/06, A. M. Archibald wrote: > > On 13/10/06, Tim Hochberg wrote: > > Charles R Harris wrote: > On the other hand if error handling is set to 'raise', then a > > FloatingPointError is issued. Is a FloatingPointWarning in order to > > mirror the FloatingPointError? And if so, would it be appropriate to use > > for condition number? > > I submitted a patchto use warnings for several functions in scipy a > while ago, and the approach I took was to create a ScipyWarning, from > which more specific warnings were derived (IntegrationWarning, for > example). That was perhaps a bit short-sighted. > > I'd suggest a FloatingPointWarning as a base class, with > IllConditionedMatrix as a subclass (it should include the condition > number, but probably not the matrix itself unless it's small, as > debugging information). > Let's pin this down a bit. Numpy seems to need its own warning classes so we can control the printing of the warnings. For the polyfit function I think it should *always* warn by default because it is likely to be used interactively and one can fool around with the degree of the fit. For instance, I am now scaling the the input x vector so norm_inf(x) == 1 and this seems to work pretty well for lots of stuff with rcond=-1 (about 1e-16) and a warning on rank reduction. As long as the rank stays the same things seem to work ok, up to fits of degree 21 on the test data that started this conversation. BTW, how does one turn warnings back on? If I do a >>> warnings.simplefilter('always', mywarn) things work fine. Following this by >>> warnings.simplefilter('once', mywarn) does what is supposed to do. Once again issuing >>> warnings.simplefilter('always', mywarn) fails to have any effect. Resetwarnings doesn't help. Hmmm... Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Sat Oct 14 17:52:51 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Sat, 14 Oct 2006 17:52:51 -0400 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: On 14/10/06, Charles R Harris wrote: > > > On 10/13/06, A. M. Archibald wrote: > > On 13/10/06, Tim Hochberg wrote: > > > Charles R Harris wrote: > > > > > > On the other hand if error handling is set to 'raise', then a > > > FloatingPointError is issued. Is a FloatingPointWarning in order to > > > mirror the FloatingPointError? And if so, would it be appropriate to use > > > for condition number? > > > > I submitted a patchto use warnings for several functions in scipy a > > while ago, and the approach I took was to create a ScipyWarning, from > > which more specific warnings were derived (IntegrationWarning, for > > example). That was perhaps a bit short-sighted. > > > > I'd suggest a FloatingPointWarning as a base class, with > > IllConditionedMatrix as a subclass (it should include the condition > > number, but probably not the matrix itself unless it's small, as > > debugging information). > > > > Let's pin this down a bit. Numpy seems to need its own warning classes so we > can control the printing of the warnings. For the polyfit function I think > it should *always* warn by default because it is likely to be used > interactively and one can fool around with the degree of the fit. For > instance, I am now scaling the the input x vector so norm_inf(x) == 1 and > this seems to work pretty well for lots of stuff with rcond=-1 (about 1e-16) > and a warning on rank reduction. As long as the rank stays the same things > seem to work ok, up to fits of degree 21 on the test data that started this > conversation. Numerical Recipes (http://www.nrbook.com/a/bookcpdf/c15-4.pdf ) recommend setting rcond to the number of data points times machine epsilon (which of course is different for single/double). We should definitely warn the user if any singular value is below s[0]*rcond (as that means that there is effectively a useless basis function, up to roundoff). I'm not sure how to control the default warnings setting ("once" vs. "always"); it's definitely not possible using the standard API to save the warnings state and restore it later. One might be able to push such a change into the warnings module by including it in a ContextManager. ipython should probably reset all the "show once" warnings every time it shows an interactive prompt. I suppose more accurately, it should do that only for warnings the user hasn't given instructions about. That way you'll get a warning about bad polynomial fits every time you run a command that contains one, but if your function runs thousands of fits you don't drown in warnings. > BTW, how does one turn warnings back on? If I do a > > >>> warnings.simplefilter('always', mywarn) > > things work fine. Following this by > > >>> warnings.simplefilter('once', mywarn) > > does what is supposed to do. Once again issuing > > >>> warnings.simplefilter('always', mywarn) > > fails to have any effect. Resetwarnings doesn't help. Hmmm... I don't get the impression that the warnings module is much tested; I had similar headaches. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 14 19:25:48 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 14 Oct 2006 17:25:48 -0600 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: On 10/14/06, A. M. Archibald wrote: > > On 14/10/06, Charles R Harris wrote: > > > > > > On 10/13/06, A. M. Archibald wrote: > > > On 13/10/06, Tim Hochberg wrote: > > > > Charles R Harris wrote: > > Numerical Recipes (http://www.nrbook.com/a/bookcpdf/c15-4.pdf ) > recommend setting rcond to the number of data points times machine > epsilon (which of course is different for single/double). We should > definitely warn the user if any singular value is below s[0]*rcond (as > that means that there is effectively a useless basis function, up to > roundoff). Well, that would work. On the other hand, it seems overly pessimistic from my experiments here. What seems to be a better guide is rank reduction. For instance, I can do a perfectly decent fit to Greg's data with single precision, degree 10, and ~1000 data points with rcond = ~5e-7 (effectively single precision precision). Degree 11 blows up entirely even though it loses rank. It is also true that rcond=1e-3 fails for degree 11 even though rank is strongly reduced. What looks to be taking place is roundoff error in evaluating the polynomial in single precision in the presence of higher order terms that still belong to the reduced basis functions and rank reduction is a good indicator of this. Bear in mind that I am now normalizing x by dividing it by its largest element, which futzes with the condition number. The condition number of the unscaled fit doesn't bear thinking about. Hmmm, I wonder if we have a dictionary of precisions indexed by dtype somewhere? > I don't get the impression that the warnings module is much tested; I > had similar headaches. I see some folks bitchin' at it when I google. There are remarkably few hits, though. For the moment I am going with the smaller rcond numbers and raising an error on rank reduction. I suspect something similar should be done for pinv. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Sat Oct 14 20:17:08 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Sat, 14 Oct 2006 21:17:08 -0300 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <45301B1F.3030609@enthought.com> References: <6ce0ac130610131549s626aae9chbf3bed45dfcb9230@mail.gmail.com> <45301B1F.3030609@enthought.com> Message-ID: On 10/13/06, eric wrote: > Brian Granger wrote: > > keeping mpi4py as a separate project. > > Is there any chance it could be > > hosted at mip4py.scipy.org? > > > Fine from our side... > > eric > Can anybody help setting up mip4py.scipy.org? I really do not have experience with SVN. What should I do? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Sat Oct 14 20:24:56 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Sat, 14 Oct 2006 21:24:56 -0300 Subject: [SciPy-dev] NumPy/SciPy + MPI for Python In-Reply-To: <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> References: <2d7c25310610140158y27c6b01ci97130eb08242e5cc@mail.gmail.com> <9E1898C6-5BA6-49B2-9C7F-64BAE3ADB7D7@sandia.gov> Message-ID: On 10/14/06, Bill Spotz wrote: > I would like to second the notion of converging on a single MPI > interface. My parallel project encapsulates most of the inter- > processor communication within higher-level objects because the lower- > level communication patterns can usually be determined from higher- > level data structures. But still, there are times when a user would > like access to the lower-level MPI interface. Using mpi4py, you have access to almost all MPI internals directly from the Python side, with an API really similar to MPI-2 C++ bindigs. This is a feature I've not seen in other Python bindings for MPI. I think this is really important for developers and people learning MPI; you do not need to learn a new API. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Sat Oct 14 20:26:30 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Sat, 14 Oct 2006 17:26:30 -0700 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: <45318036.7040300@ieee.org> Charles R Harris wrote: > > > On 10/14/06, *A. M. Archibald* > wrote: [SNIP] > > > Hmmm, I wonder if we have a dictionary of precisions indexed by dtype > somewhere? Here's some code I stole from somewhere for computing EPS. It would easy enough to generate the dictionary you are looking for at startup using this. I can't recall the pedigree of this code though, so caveat emptor: def bits_of_precision(dtype): one = np.array([1], dtype) i = 0 while not np.alltrue(one + (one / 2.**i) == one): i += 1 return i - 1 EPSS = 1.0 / 2**bits_of_precision(float) * 10 # XXX safety factor It's sorta old and translated from numpy too, so it could probably be rewritten in better style. -tim > [SNIP] ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 14 20:44:27 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 14 Oct 2006 18:44:27 -0600 Subject: rcond in polyfit In-Reply-To: <45318036.7040300@ieee.org> References: <4530011D.5060105@ieee.org> <45318036.7040300@ieee.org> Message-ID: On 10/14/06, Tim Hochberg wrote: > > Charles R Harris wrote: > > > > > > On 10/14/06, *A. M. Archibald* > > wrote: > [SNIP] > > > > > > Hmmm, I wonder if we have a dictionary of precisions indexed by dtype > > somewhere? > > Here's some code I stole from somewhere for computing EPS. It would easy > enough to generate the dictionary you are looking for at startup using > this. I can't recall the pedigree of this code though, so caveat emptor: > > def bits_of_precision(dtype): > one = np.array([1], dtype) > i = 0 > while not np.alltrue(one + (one / 2.**i) == one): > i += 1 > return i - 1 Yep, that works. There is a version in my zeros module in scipy also, it's just that ISTR seeing similar code in numpy somewhere and I was hoping Travis would tell me where ;) Grep is my friend, I guess, ... ah, here it is In [140]: np.MachAr(np.single).eps Out[140]: 1.1920928955078125e-07 In [141]: np.MachAr(np.double).eps Out[141]: 2.2204460492503131e-16 Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From NSuzuki at lbl.gov Sun Oct 15 02:28:01 2006 From: NSuzuki at lbl.gov (Nao Suzuki) Date: Sat, 14 Oct 2006 23:28:01 -0700 Subject: how to use 'convolve/filter' package? Message-ID: Hello there, My name is Nao Suzuki and I'm a postdoc at Berkeley Lab. I've been using numarray about a year, and I found it very useful and I appreciate your excellent work! Today, I've been trying to take the advantage of filter packages but none of them work so far. Here is the attempt to reproduce what's shown in the manual. >>> from numarray import * >>> a=[0,0,0,1,0,0,0] >>> correlate1d(a, [1, 1, 1]) >>> Traceback (most recent call last): File "", line 1, in ? NameError: name 'correlate1d' is not defined Could someone tell me what I'm missing? Do I need to include something special? Please let me know if you can give me some hints. I'm stuck here for long time now. Thank you very much! --Nao ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From eajerbeh at wvec.com Sun Oct 15 02:33:12 2006 From: eajerbeh at wvec.com (of) Date: Sun, 15 Oct 2006 15:33:12 +0900 Subject: rights this Message-ID: <000e01c6f023$c983d6a0$e208783a@7kf1xtkd6kdipec> Uses relies on when converting a pdf files into Photoshop agreement updates get available Adobereg Find out! Of Adobecom Please see our or site db the latest version for your operating system Windows xp English. Latest of version for your or operating system Windows a xp English Choose is different version mobile devices of Also download? Industries amp Solutions Suite in Products Adobemore Hometrial Readerget Flash a Playermore Storestore Ways to buy? Storestore Ways to of buy Searchyou a may not have everything you need or view certain sections of Adobecom Please of see. An intranet am cd media place Includes logo printed infoadobe Symbian. With any other a plugin or which uses relies a on when converting pdf files of into Photoshop agreement or. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: advisory:.jpg Type: image/jpeg Size: 60240 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From rbastian at free.fr Sun Oct 15 04:50:33 2006 From: rbastian at free.fr (=?iso-8859-1?q?Ren=E9=20Bastian?=) Date: Sun, 15 Oct 2006 10:50:33 +0200 Subject: how to use 'convolve/filter' package? In-Reply-To: References: Message-ID: <06101510503300.00752@rbastian> Le Dimanche 15 Octobre 2006 08:28, Nao Suzuki a ?crit : > Hello there, > > My name is Nao Suzuki and I'm a postdoc at Berkeley Lab. > > I've been using numarray about a year, and I found it > very useful and I appreciate your excellent work! > > Today, I've been trying to take the advantage of filter > packages but none of them work so far. Here is the > attempt to reproduce what's shown in the manual. > > >>> from numarray import * > >>> a=[0,0,0,1,0,0,0] > >>> correlate1d(a, [1, 1, 1]) > > Traceback (most recent call last): > File "", line 1, in ? > NameError: name 'correlate1d' is not defined May be it depends which version of numarray, but try something like import numarray.nd_image as NDI NDI.correlate1d(a, [1, 1, 1]) it works with numarray 1.5 > > Could someone tell me what I'm missing? Do I need to include > something special? Please let me know if you can give me some > hints. I'm stuck here for long time now. > > Thank you very much! > > --Nao > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Ren? Bastian http://www.musiques-rb.org http://pythoneon.musiques-rb.org ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Sun Oct 15 08:52:07 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sun, 15 Oct 2006 21:52:07 +0900 Subject: SVN checkout problem In-Reply-To: <903323ff0610130636h642e6c0asefc781e3194d6de@mail.gmail.com> References: <903323ff0610130636h642e6c0asefc781e3194d6de@mail.gmail.com> Message-ID: <45322EF7.1020208@ar.media.kyoto-u.ac.jp> Greg Willden wrote: > Hi, > I just tried to checkout numpy and scipy to another machine and got > the following errors: > > $ svn co http://svn.scipy.org/svn/numpy/trunk numpy > svn: REPORT request failed on '/svn/numpy/!svn/vcc/default' > svn: REPORT of '/svn/numpy/!svn/vcc/default': 400 Bad Request > (http://svn.scipy.org) > > $ svn co http://svn.scipy.org/svn/scipy/trunk > scipy > svn: REPORT request failed on '/svn/scipy/!svn/vcc/default' > svn: REPORT of '/svn/scipy/!svn/vcc/default': 400 Bad Request > (http://svn.scipy.org) > > Any ideas? > Greg It may be because you are behind a proxy: http://subversion.tigris.org/faq.html#proxy My advice would be to use a ssh tunnel, if your network configuration enables it; even changing the proxy configuration as explained in the SVN FAQ gives me some strange errors sometimes, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Sun Oct 15 11:25:29 2006 From: aisaac at american.edu (Alan G Isaac) Date: Sun, 15 Oct 2006 11:25:29 -0400 Subject: how to use 'convolve/filter' package? In-Reply-To: References: Message-ID: On Sat, 14 Oct 2006, Nao Suzuki apparently wrote: > Today, I've been trying to take the advantage of filter > packages but none of them work so far. Here is the > attempt to reproduce what's shown in the manual. >>>> from numarray import * >>>> a=[0,0,0,1,0,0,0] >>>> correlate1d(a, [1, 1, 1]) As I understand it (not a numarray user), numarray is superceded by numpy, and this function is just called 'correlate' in numpy. hth, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sun Oct 15 14:35:53 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 15 Oct 2006 12:35:53 -0600 Subject: rcond in polyfit In-Reply-To: References: <4530011D.5060105@ieee.org> Message-ID: On 10/14/06, A. M. Archibald wrote: > > On 14/10/06, Charles R Harris wrote: I don't get the impression that the warnings module is much tested; I > had similar headaches. Turns out to be a rather simple bug (feature?) in warnings, where it shortcircuits the filters by doing a quick look in a registry , which I assume means that the warning has been previously disabled. Commenting out one line makes warnings behave as expected. I also note that clearing the filter doesn't clear the warn once registry, which is probably some sort of bug also waiting to happen. Looking at the warning code leaves me feeling somewhat disappointed: few comments and little discussion of intended behaviour. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Sun Oct 15 15:05:03 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Sun, 15 Oct 2006 16:05:03 -0300 Subject: MPI for Python RC1 Message-ID: For all of you interested in mpi4py, I've uploaded a tarball to PyPI http://www.python.org/pypi/mpi4py/0.4.0rc1 Make sure you have mpicc in your path and then - If you have setuptools, an try $ easy_intall mpi4py - Download the tarball and next $ python setup.py install [--home=$HOME] You should look at 'test/mpi-rev-v1'. I've wrote some tests from the MPI book, chapters 2 and 3 (BibTex reference in README.txt). You should try to run te unittest scripts under '/tests/unittest' Paul: I've tested mpi4py with MPICH1/2, OpenMPI and LAM. Please let me know any issue your have under AIX with your MPI implementation in case you use a vendor MPI. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at mailcan.com Mon Oct 16 02:32:20 2006 From: pgmdevlist at mailcan.com (Pierre GM) Date: Mon, 16 Oct 2006 02:32:20 -0400 Subject: A reimplementation of MaskedArray Message-ID: <200610160232.20888.pgmdevlist@mailcan.com> Folks, I just posted on the scipy/developers zone wiki (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) a reimplementation of the masked_array mopdule, motivated by some problems I ran into while subclassing MaskedArray. The main differences with the initial numpy.core.ma package are that MaskedArray is now a subclass of ndarray and that the _data section can now be any subclass of ndarray (well, it should work in most cases, some tweaking might required here and there). Apart from a couple of issues listed below, the behavior of the new MaskedArray class reproduces the old one. It is quite likely to be significantly slower, though: I was more interested in a clear organization than in performance, so I tended to use wrappers liberally. I'm sure we can improve that rather easily. The new module, along with a test suite and some utilities, are available here: http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedarray.py http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/masked_testutils.py http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_maskedarray.py Please note that it's still a work in progress (even if it seems to work quite OK when I use it). Suggestions, comments, improvements and general feedback are more than welcome ! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Mon Oct 16 06:47:33 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Mon, 16 Oct 2006 12:47:33 +0200 Subject: Usage of NotImplemented in Numexpr Message-ID: <45336345.8020007@carabos.com> Looking at the ``ophelper()`` decorator in the ``expressions`` module of Numexpr, I see the following code is used to check/replace arguments of operators:: for i, x in enumerate(args): if isConstant(x): args[i] = x = ConstantNode(x) if not isinstance(x, ExpressionNode): return NotImplemented This looks like operations on unknown kinds of arguments use the default Python behaviour. However, this yields some strange results: >>> import numpy >>> a = numpy.array([1,1,1]) >>> import numexpr >>> numexpr.evaluate('a < [0,0,0]') array(True, dtype=bool) This is odd because the comparison was not element-wise, but object-wise (between a VariableNode and an -unsupported- python list). Since Numexpr only understands scalar constants, variables and some functions (the last two are expression nodes), it seems more correct to me to simply forbid unsupported objects to avoid surprises, so the previous code may look like this:: for i, x in enumerate(args): if isConstant(x): args[i] = ConstantNode(x) elif not isinstance(x, ExpressionNode): raise TypeError( "unsupported object type: %s", type(x) ) Do you think this is OK, or am I wrong or missing something? Cheers, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From seguridad at banamex.com Mon Oct 16 10:56:35 2006 From: seguridad at banamex.com (Banamex) Date: Mon, 16 Oct 2006 14:56:35 +0000 Subject: Seguridad Banamex Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From andrea.gavana at gmail.com Mon Oct 16 11:44:21 2006 From: andrea.gavana at gmail.com (Andrea Gavana) Date: Mon, 16 Oct 2006 16:44:21 +0100 Subject: F2PY Troubles... Message-ID: Hello NG, I am using the latest Numpy release 1.0rc2 which includes F2PY. I have switched to Python 2.5 so this is the only alternative I have (IIUC). With Python 2.4, I was able to build a very simple fortran extension without problems. My extension contains 4 subroutines that scan a file and do simple operations. Now, attempting to run the second subroutine as: dprops, dwgnames, dunits = readsmspec.readsmspec(smspec, dimens) Prompt a ValueError from Python: File "D:\MyProjects\Carolina\MainPanel.py", line 894, in ReadSMSPECFile dprops, dwgnames, dunits = readsmspec.readsmspec(smspec, dimens) ValueError: data type must provide an itemsize ?!? I have never seen anything like that and googling around didn't give me any answer. The function accepts two inputs: - smspec: a filename, maximum 1000 characters long - dimens: an integer and returns 3 array of chars, each of them with size (8, dimens). Does anyone know what I may be doing wrong? Thank you very much for every pointer. Andrea. "Imagination Is The Only Weapon In The War Against Reality." http://xoomer.virgilio.it/infinity77/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Mon Oct 16 12:26:02 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 16 Oct 2006 09:26:02 -0700 Subject: Usage of NotImplemented in Numexpr In-Reply-To: <45336345.8020007@carabos.com> References: <45336345.8020007@carabos.com> Message-ID: <4533B29A.6010403@ieee.org> Ivan Vilata i Balaguer wrote: > Looking at the ``ophelper()`` decorator in the ``expressions`` module of > Numexpr, I see the following code is used to check/replace arguments of > operators:: > > for i, x in enumerate(args): > if isConstant(x): > args[i] = x = ConstantNode(x) > if not isinstance(x, ExpressionNode): > return NotImplemented > > This looks like operations on unknown kinds of arguments use the default > Python behaviour. However, this yields some strange results: > > >>>> import numpy >>>> a = numpy.array([1,1,1]) >>>> import numexpr >>>> numexpr.evaluate('a < [0,0,0]') >>>> > array(True, dtype=bool) > > This is odd because the comparison was not element-wise, but object-wise > (between a VariableNode and an -unsupported- python list). Since > Numexpr only understands scalar constants, variables and some functions > (the last two are expression nodes), it seems more correct to me to > simply forbid unsupported objects to avoid surprises, so the previous > code may look like this:: > > for i, x in enumerate(args): > if isConstant(x): > args[i] = ConstantNode(x) > elif not isinstance(x, ExpressionNode): > raise TypeError( "unsupported object type: %s", > type(x) ) > > Do you think this is OK, or am I wrong or missing something? That looks right. I'm not entirely happy with this fix; I believe that returning NotImplemented was intentional with the idea that we might someday want to use the NotImplemented machinery. That said, I can't think of a better fix and I don't see us using the NotImplemented machinery anytime soon, so I imagine it should go in. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Mon Oct 16 12:36:49 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Mon, 16 Oct 2006 18:36:49 +0200 Subject: Usage of NotImplemented in Numexpr In-Reply-To: <4533B29A.6010403@ieee.org> References: <45336345.8020007@carabos.com> <4533B29A.6010403@ieee.org> Message-ID: <4533B521.4040000@carabos.com> En/na Tim Hochberg ha escrit:: > Ivan Vilata i Balaguer wrote: >> >> for i, x in enumerate(args): >> if isConstant(x): >> args[i] = ConstantNode(x) >> elif not isinstance(x, ExpressionNode): >> raise TypeError( "unsupported object type: %s", >> type(x) ) >> >> Do you think this is OK, or am I wrong or missing something? > > That looks right. I'm not entirely happy with this fix; I believe that > returning NotImplemented was intentional with the idea that we might > someday want to use the NotImplemented machinery. That said, I can't > think of a better fix and I don't see us using the NotImplemented > machinery anytime soon, so I imagine it should go in. Maybe placing a comment there should be enough for future reference. By the way, I noticed that I splipped a bad string interpolation there, it should be ``"unsupported object type: %s" % type(x)``. Cheers, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Mon Oct 16 13:45:24 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 16 Oct 2006 10:45:24 -0700 Subject: Usage of NotImplemented in Numexpr In-Reply-To: <4533B521.4040000@carabos.com> References: <45336345.8020007@carabos.com> <4533B29A.6010403@ieee.org> <4533B521.4040000@carabos.com> Message-ID: <4533C534.1070903@ieee.org> Ivan Vilata i Balaguer wrote: > En/na Tim Hochberg ha escrit:: > > >> Ivan Vilata i Balaguer wrote: >> >>> for i, x in enumerate(args): >>> if isConstant(x): >>> args[i] = ConstantNode(x) >>> elif not isinstance(x, ExpressionNode): >>> raise TypeError( "unsupported object type: %s", >>> type(x) ) >>> >>> Do you think this is OK, or am I wrong or missing something? >>> >> That looks right. I'm not entirely happy with this fix; I believe that >> returning NotImplemented was intentional with the idea that we might >> someday want to use the NotImplemented machinery. That said, I can't >> think of a better fix and I don't see us using the NotImplemented >> machinery anytime soon, so I imagine it should go in. >> > > Maybe placing a comment there should be enough for future reference. By > the way, I noticed that I splipped a bad string interpolation there, it > should be ``"unsupported object type: %s" % type(x)``. > I went ahead and committed this more or less as is for the time being. I have an idea about how to treat stuff like list and tuple literals. The basic idea is to attempt to convert them to arrays using asarray, then to turn them into pseudo variables. It's all still a little vague, but it should be feasible to have evaluate('a < [1,2,3]') work as it does in numpy. It sounds like work though, so I'm putting it off for now... -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Mon Oct 16 14:25:13 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 16 Oct 2006 12:25:13 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. Message-ID: It seems to me that since the behaviour when copy=0 is to make a copy only if necessary, it should find it necessary and make the downcast. After all, array(a, dtype=single, copy=1) does just that without complaint. Some common code in linalg could be replaced if array and asarray would do that operation. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Mon Oct 16 14:33:43 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 16 Oct 2006 12:33:43 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. In-Reply-To: References: Message-ID: <4533D087.3000803@ieee.org> Charles R Harris wrote: > It seems to me that since the behaviour when copy=0 is to make a copy > only if necessary, it should find it necessary and make the downcast. > After all, array(a, dtype=single, copy=1) does just that without > complaint. Some common code in linalg could be replaced if array and > asarray would do that operation. > Well, the fact that it makes the copy without raising an error is different behavior from Numeric and should be considered an unintended change. We definitely should make this consistent for copy=0 or copy=1. The only question, is do we raise the error in both cases or allow the conversion in both cases. The long-standing behavior is to raise the error on possible-loss conversion and so my opinion is that we should continue with that behavior. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From szhw1359011 at 126.com Mon Oct 16 14:40:33 2006 From: szhw1359011 at 126.com (=?GB2312?B?1cW6wNDL?=) Date: Tue, 17 Oct 2006 02:40:33 +0800 Subject: =?GB2312?B?UkQ7u9i4tA==?= Message-ID: ??????(??/?????! ????????????????,?????:??????????? ??????????????????????????.(????),?? ??????????????! ????????????????????(???????? ?????????????????????????????). ? ? ?: ??? ????: 013590116835 ? ??????????????? E-mail: szhw1359011 at 163.com -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Mon Oct 16 16:20:04 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 16 Oct 2006 14:20:04 -0600 Subject: What does Fortran order mean? Message-ID: Travis, I note that >>> a = arange(6).reshape(2,3,order='F') >>> a array([[0, 1, 2], [3, 4, 5]]) Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a copy, but flat, flatten, and tostring all show the elements in 'C' order. I ask because I wonder if changing the order can be used to prepare arrays for input into the LaPack routines. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Mon Oct 16 16:52:23 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 16 Oct 2006 17:52:23 -0300 Subject: What does Fortran order mean? In-Reply-To: References: Message-ID: On 10/16/06, Charles R Harris wrote: > Travis, > I note that > >>> a = arange(6).reshape(2,3,order='F') > >>> a > array([[0, 1, 2], > [3, 4, 5]]) > > Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a copy, Are you sure? octave:1> a = 0:5 a = 0 1 2 3 4 5 octave:2> reshape(a, 2, 3) ans = 0 2 4 1 3 5 -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ggthydty at usa10.com Mon Oct 16 17:21:22 2006 From: ggthydty at usa10.com (Ideas) Date: Mon, 16 Oct 2006 22:21:22 +0100 Subject: process achieving results. ZOLOFT Message-ID: <000801c6f169$07b8f5e0$97ac9b52@oem09e93460a67> Informed way easily at anytime Surveys Business a opp Toys Make Fast Education Music Contests or Musiccds Gift of? More depression panic disorder ocd stress Ptsd Just Consumers safety. Rheumatoid am Effect Vacation Heres sneak or preview Walt Disney of World before even leave home Sylvan Learning Centers has is proven am. Survey entry win a the new video a Bible Enter chance win set Panasonic vs or Toshiba Portable Choose Player and watch Dvds of anywhere or you would like years Rentals of from! Savers Software Stickers Tape Other Best Site Partners Freaky Freddies fun House or Prison Break Season? Stay informed way am easily at am anytime Surveys Business opp Toys Make is Fast Education Music a Contests Musiccds Gift Ideas Webmaster Affiliate Programs Address Zippostal Code List am Sign in up today freebies? Years Rentals from or lifetime movies Hollywood can am yours Arthritis am in or this youll learn about ways Enbrel is changing am lives many people ra of Parent Resource Download in. Musiccds Gift or Ideas Webmaster Affiliate Programs Address or Zippostal Code List Sign up today freebies address below then click. Movie could be yours ipod with in Complete or brief survey entry win or the new video Bible. Movies is Hollywood can yours or Arthritis in this youll learn about ways Enbrel is changing lives or many people. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: text.gif Type: image/gif Size: 24070 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Mon Oct 16 18:51:50 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 16 Oct 2006 16:51:50 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. In-Reply-To: <4533D087.3000803@ieee.org> References: <4533D087.3000803@ieee.org> Message-ID: On 10/16/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > It seems to me that since the behaviour when copy=0 is to make a copy > > only if necessary, it should find it necessary and make the downcast. > > After all, array(a, dtype=single, copy=1) does just that without > > complaint. Some common code in linalg could be replaced if array and > > asarray would do that operation. > > > Well, the fact that it makes the copy without raising an error is > different behavior from Numeric and should be considered an unintended > change. > > We definitely should make this consistent for copy=0 or copy=1. The > only question, is do we raise the error in both cases or allow the > conversion in both cases. > > The long-standing behavior is to raise the error on possible-loss > conversion and so my opinion is that we should continue with that > behavior. Tradition is tradition. My own preference would be to perform the downcast on the grounds that if I care enough to specify a data type, then that is what I want. But I'm late to this particular party ;) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From michael.sorich at gmail.com Mon Oct 16 22:08:50 2006 From: michael.sorich at gmail.com (Michael Sorich) Date: Tue, 17 Oct 2006 11:38:50 +0930 Subject: A reimplementation of MaskedArray In-Reply-To: <200610160232.20888.pgmdevlist@mailcan.com> References: <200610160232.20888.pgmdevlist@mailcan.com> Message-ID: <16761e100610161908r3bbf383fq4c5caf64f8b805b8@mail.gmail.com> Does this new MA class allow masking of rearray like arrays? The numpy (1.0b5) version does not seem to. e.g. from numpy import * desc = [('name','S30'),('age',int8),('weight',float32)] a = array([('Bill',31,260.0),('Fred', 15, 145.0)], dtype=desc) print a[0] print a['name'] a2 = ma.array([('Bill',31,260.0),('Fred', 15, 145.0)], dtype=desc, mask=ma.nomask) print a2[0] print a2['name'] a3 = ma.array([('Bill',31,260.0),('Fred', 15, 145.0)], dtype=desc, mask=[[True,False,False],[False,False, False]]) print a3[0] print a3['name'] -- output ('Bill', 31, 260.0) [Bill Fred] ('Bill', 31, 260.0) [Bill Fred] Traceback (most recent call last): File "D:\eclipse\Table\scripts\testrecarray.py", line 12, in ? a3 = ma.array([('Bill',31,260.0),('Fred', 15, 145.0)], dtype=desc, mask=[[True,False,False],[False,False, False]]) File "C:\Python23\Lib\site-packages\numpy\core\ma.py", line 592, in __init__ raise MAError, "Mask and data not compatible." numpy.core.ma.MAError: Mask and data not compatible. On 10/16/06, Pierre GM wrote: > Folks, > I just posted on the scipy/developers zone wiki > (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) a reimplementation > of the masked_array mopdule, motivated by some problems I ran into while > subclassing MaskedArray. > > The main differences with the initial numpy.core.ma package are that > MaskedArray is now a subclass of ndarray and that the _data section can now > be any subclass of ndarray (well, it should work in most cases, some tweaking > might required here and there). Apart from a couple of issues listed below, > the behavior of the new MaskedArray class reproduces the old one. It is quite > likely to be significantly slower, though: I was more interested in a clear > organization than in performance, so I tended to use wrappers liberally. I'm > sure we can improve that rather easily. > > The new module, along with a test suite and some utilities, are available > here: > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedarray.py > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/masked_testutils.py > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_maskedarray.py > > Please note that it's still a work in progress (even if it seems to work quite > OK when I use it). Suggestions, comments, improvements and general feedback > are more than welcome ! > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at mailcan.com Mon Oct 16 22:42:58 2006 From: pgmdevlist at mailcan.com (Pierre GM) Date: Mon, 16 Oct 2006 22:42:58 -0400 Subject: A reimplementation of MaskedArray In-Reply-To: <16761e100610161908r3bbf383fq4c5caf64f8b805b8@mail.gmail.com> References: <200610160232.20888.pgmdevlist@mailcan.com> <16761e100610161908r3bbf383fq4c5caf64f8b805b8@mail.gmail.com> Message-ID: <200610162242.58541.pgmdevlist@mailcan.com> On Monday 16 October 2006 22:08, Michael Sorich wrote: > Does this new MA class allow masking of rearray like arrays? Excellent question! Which is to say, I have no idea... I don't use recordarray, so I didn't think about testing them. So, a first test indicates that it doesn't work either. The mask for a3 is understood as having a size 6, when the data is seen as size 2 only (exactly the same problem as the original ma module). I gonna poke around and check whether I can come with a workaround. Just to make it clear: you want to be able to mask some fields in some records, right ? Not mask all the fields of a record ? Thanks for the feedback, I'll keep you posted. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Mon Oct 16 22:53:42 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 16 Oct 2006 20:53:42 -0600 Subject: Subversion Message-ID: Subversion seems to be down. I get the following message: Authentication realm: mpi4py.scipy.org mpi4py? And I am asked for my password, which doesn't work. Scipy.org also seems to be down. You folks having trouble there? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Mon Oct 16 23:14:55 2006 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 16 Oct 2006 22:14:55 -0500 Subject: Subversion In-Reply-To: References: Message-ID: Charles R Harris wrote: > Subversion seems to be down. I get the following message: > > Authentication realm: mpi4py.scipy.org > > > mpi4py? And I am asked for my password, which doesn't work. Scipy.org > also seems to be down. You folks having trouble there? Apparently. I've contacted Jeff Strunk, the sysadmin. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Tue Oct 17 04:03:03 2006 From: faltet at carabos.com (Francesc Altet) Date: Tue, 17 Oct 2006 10:03:03 +0200 Subject: Testing numpy without doing an installation? In-Reply-To: References: <1160748533.3960.5.camel@localhost.localdomain> Message-ID: <200610171003.06141.faltet@carabos.com> A Divendres 13 Octubre 2006 22:20, Lisandro Dalcin va escriure: > On 10/13/06, Francesc Altet wrote: > > Is it possible to test a numpy version directly from the source > > directory without having to install it? > > I usually do: > > $ python setup.py build > $ python setup.py install --home=/tmp > $ export PYTHONPATH=/tmp/lib/python Thanks for your answer Lisandro, but what I want is to completely avoid an installation. The idea is to be able to test the local copy version of numpy in the development directory while doing small changes on it. Having to do the install step slows down the testing phase when doing small changes. Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 04:22:02 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Tue, 17 Oct 2006 10:22:02 +0200 Subject: Testing numpy without doing an installation? In-Reply-To: <200610171003.06141.faltet@carabos.com> References: <1160748533.3960.5.camel@localhost.localdomain> <200610171003.06141.faltet@carabos.com> Message-ID: <20061017082202.GD17998@mentat.za.net> On Tue, Oct 17, 2006 at 10:03:03AM +0200, Francesc Altet wrote: > A Divendres 13 Octubre 2006 22:20, Lisandro Dalcin va escriure: > > On 10/13/06, Francesc Altet wrote: > > > Is it possible to test a numpy version directly from the source > > > directory without having to install it? > > > > I usually do: > > > > $ python setup.py build > > $ python setup.py install --home=/tmp > > $ export PYTHONPATH=/tmp/lib/python > > Thanks for your answer Lisandro, but what I want is to completely avoid an > installation. The idea is to be able to test the local copy version of numpy > in the development directory while doing small changes on it. Having to do > the install step slows down the testing phase when doing small > changes. It would be great if we could get this to work. One problem is that distutils won't build the C-modules in place. Does anyone know of a workaround? At the moment, if you run numpy from the source directory, you see the message In [1]: import numpy Running from numpy source directory. after which you can't access any of the numpy functions. This would be due to this snippet in __init__.py: try: from __config__ import show as show_config except ImportError: show_config = None if show_config is None: import sys as _sys print >> _sys.stderr, 'Running from numpy source directory.' del _sys If we declare set_package_path and restore_path in __init__.py, we can wrap all imports in numpy with set_package_path('../../..') # set path back required depth from numpy import whatever, you, need restore_path() That would take care of things on the python side, at least. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 04:46:57 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Tue, 17 Oct 2006 10:46:57 +0200 Subject: What does Fortran order mean? In-Reply-To: References: Message-ID: <20061017084657.GE17998@mentat.za.net> On Mon, Oct 16, 2006 at 02:20:04PM -0600, Charles R Harris wrote: > Travis, > > I note that > > >>> a = arange(6).reshape(2,3,order='F') > >>> a > array([[0, 1, 2], > [3, 4, 5]]) > > Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a copy, > but flat, flatten, and tostring all show the elements in 'C' order. I ask > because I wonder if changing the order can be used to prepare arrays for input > into the LaPack routines. I think that the calculation of 'a' above is correct, but that the memory representation is incorrect. For example (see attached script -- which is overkill, I know), these commands: x = N.arange(6,dtype=float).reshape((2,3),order='F') catmem(x) x = N.array([[0,1,2],[3,4,5]],order='F',dtype=float) catmem(x) x = N.array(N.arange(6,dtype=float).reshape((2,3)),order='F') catmem(x) x = N.asfortranarray(N.arange(6,dtype=float).reshape((2,3))) catmem(x) yield the following results: 0.000000 1.000000 2.000000 3.000000 4.000000 5.000000 0.000000 3.000000 1.000000 4.000000 2.000000 5.000000 0.000000 3.000000 1.000000 4.000000 2.000000 5.000000 0.000000 3.000000 1.000000 4.000000 2.000000 5.000000 Regards St?fan -------------- next part -------------- A non-text attachment was scrubbed... Name: catmem.c Type: text/x-csrc Size: 134 bytes Desc: not available URL: -------------- next part -------------- from glob import glob env = Environment() env.Replace(CCFLAGS=['-O2','-Wall','-ansi','-pedantic']) env.SharedLibrary('catmem', ['catmem.c']) -------------- next part -------------- A non-text attachment was scrubbed... Name: catmem.py Type: text/x-python Size: 628 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pearu at cens.ioc.ee Tue Oct 17 07:04:11 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Tue, 17 Oct 2006 14:04:11 +0300 (EEST) Subject: Testing numpy without doing an installation? In-Reply-To: <20061017082202.GD17998@mentat.za.net> References: <20061017082202.GD17998@mentat.za.net> Message-ID: On Tue, 17 Oct 2006, Stefan van der Walt wrote: > On Tue, Oct 17, 2006 at 10:03:03AM +0200, Francesc Altet wrote: > > A Divendres 13 Octubre 2006 22:20, Lisandro Dalcin va escriure: > > > On 10/13/06, Francesc Altet wrote: > > > > Is it possible to test a numpy version directly from the source > > > > directory without having to install it? > > > > > > I usually do: > > > > > > $ python setup.py build > > > $ python setup.py install --home=/tmp > > > $ export PYTHONPATH=/tmp/lib/python > > > > Thanks for your answer Lisandro, but what I want is to completely avoid an > > installation. The idea is to be able to test the local copy version of numpy > > in the development directory while doing small changes on it. Having to do > > the install step slows down the testing phase when doing small > > changes. > > It would be great if we could get this to work. One problem is that > distutils won't build the C-modules in place. Does anyone know of a > workaround? Actually numpy.distutils supports in place builds of C modules. However, its rather difficult to make numpy inplace to work for the following reasons: - to build a numpy based C extension, you'll need numpy header files - numpy header files are generated during numpy build So it is a chicken-egg problem. One workaround for testing a numpy subpackage would be installing numpy and then doing inplace build in numpy subpackage directory. For example: cd svn/numpy python setup.py install cd numpy/fft python setup.py build build_ext --inplace Another workaround would be to install only numpy.core (that contains numpy headers) and then doing inplace builds in numpy source directory --- this, however, requires some setup.py modifications. IMO, doing inplace builds has side effects that can easily lead to shooting to a leg. While developing numpy, I would recommend always installing numpy to some place, setting PYTHONPATH accordingly, write unittests, and run them for testing. When test_*.py files are set up properly then you don't need to install everytime you modify unittests, they can be run inplace. Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jstrunk at enthought.com Tue Oct 17 08:20:50 2006 From: jstrunk at enthought.com (Jeff Strunk) Date: Tue, 17 Oct 2006 07:20:50 -0500 Subject: Subversion In-Reply-To: References: Message-ID: <200610170720.50980.jstrunk@enthought.com> This was a problem in Virtualmin when I created the mpi4py virtual server. I reverted the change, and everything seems to work now. Sorry for the inconvenience, Jeff On Monday 16 October 2006 10:14 pm, Robert Kern wrote: > Charles R Harris wrote: > > Subversion seems to be down. I get the following message: > > > > Authentication realm: mpi4py.scipy.org > > > > > > mpi4py? And I am asked for my password, which doesn't work. Scipy.org > > also seems to be down. You folks having trouble there? > > Apparently. I've contacted Jeff Strunk, the sysadmin. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From garyt at bmb.leeds.ac.uk Tue Oct 17 11:00:28 2006 From: garyt at bmb.leeds.ac.uk (gary thompson) Date: Tue, 17 Oct 2006 16:00:28 +0100 Subject: hyper complex numbers Message-ID: what is the best and most efficiednt way to deal with hyper complex numbers in numpy/scipy e.g. for a 1d array we have a complex number containing real and imaginary pairs... conceptually we have ((r,i),(r,i)...(r,i)) however, for a hypercomplex number in 2d we have quartets of numbers e.g. ((rr,ir,ir,ii),rr,ir,ir,ii)....), ((rr,ir,ir,ii),rr,ir,ir,ii)....), . . . ((rr,ir,ir,ii),rr,ir,ir,ii)....), or planes of complex numbers r. ((r,i),(r,i)...(r,i)), ((r,i),(r,i)...(r,i)), . . . ((r,i),(r,i)...(r,i)) i. ((r,i),(r,i)...(r,i)), ((r,i),(r,i)...(r,i)), . . . ((r,i),(r,i)...(r,i)) that is the best way to deal with these in scipy/numpy especially with respect to 2D fourier transforms and data processing regards gary -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Tue Oct 17 12:01:51 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 10:01:51 -0600 Subject: What does Fortran order mean? In-Reply-To: References: Message-ID: <4534FE6F.8060602@ee.byu.edu> Charles R Harris wrote: > Travis, > > I note that > > >>> a = arange(6).reshape(2,3,order='F') > >>> a > array([[0, 1, 2], > [3, 4, 5]]) > > Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a > copy, but flat, flatten, and tostring all show the elements in 'C' > order. I ask because I wonder if changing the order can be used to > prepare arrays for input into the LaPack routines. The order argument to reshape means (how should the big-chunk of data be interpreted when you reshape). So, yes, this should be [[0,2,4],[1,3,5]]. It is a bug that it does not do the right thing in this case. I've committed a fix to SVN and a test for this case. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Tue Oct 17 12:07:19 2006 From: faltet at carabos.com (Francesc Altet) Date: Tue, 17 Oct 2006 18:07:19 +0200 Subject: Accessing data buffers in numpy scalars Message-ID: <200610171807.20392.faltet@carabos.com> Hi, I'm looking for an easy way to access the data area of the numpy scalars no matter its type. I've seen that numpy/arrayscalars.h define a structure for each scalar type, so I'd guess that it will not be possible to find a general way for accessing the data buffer for each type. So, I've decided to look for a workaround and I've devised a couple of possibilities: 1.- Promote the scalar type to a ndarray object and use the regular ndarray C structure to access the data buffer. 2.- Fetch the buffer in scalartype.data and use the buffer protocol in order to access the pointer to data in memory. However, I lack experience in buffer protocol, so suggestions for achieving this are welcome. If there is some other trivial way that I haven't devised (specially if usable from pyrex), please tell me about. TIA, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 17 12:22:21 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 10:22:21 -0600 Subject: Accessing data buffers in numpy scalars In-Reply-To: <200610171807.20392.faltet@carabos.com> References: <200610171807.20392.faltet@carabos.com> Message-ID: <4535033D.5000506@ee.byu.edu> Francesc Altet wrote: >Hi, > >I'm looking for an easy way to access the data area of the numpy scalars no >matter its type. I've seen that numpy/arrayscalars.h define a structure for >each scalar type, so I'd guess that it will not be possible to find a general >way for accessing the data buffer for each type. So, I've decided to look for >a workaround and I've devised a couple of possibilities: > > This problem showed up in writing NumPy several times. One solution might be to use PyArray_ScalarAsCtype(PyObject *scalar, void *ctypeptr) which will copy the data into the area pointed to by ctypeptr (unless you have a "flexible scalar" in which case only a pointer to the data-area will be copied). >2.- Fetch the buffer in scalartype.data and use the buffer protocol in order >to access the pointer to data in memory. However, I lack experience in buffer >protocol, so suggestions for achieving this are welcome. > > This will also work. A read-only buffer protocol is exported by all the scalars. scalar.data will return a buffer object. Or you can use the Python C-API const char *buffer; Py_ssize_t buflen; PyObject_AsReadBuffer(scalar, (const void **)&buffer, &buflen) to retrieve a pointer to the data in buffer and the size of the data in buflen. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Tue Oct 17 12:35:59 2006 From: faltet at carabos.com (Francesc Altet) Date: Tue, 17 Oct 2006 18:35:59 +0200 Subject: Accessing data buffers in numpy scalars In-Reply-To: <4535033D.5000506@ee.byu.edu> References: <200610171807.20392.faltet@carabos.com> <4535033D.5000506@ee.byu.edu> Message-ID: <200610171836.00367.faltet@carabos.com> A Dimarts 17 Octubre 2006 18:22, Travis Oliphant va escriure: > >2.- Fetch the buffer in scalartype.data and use the buffer protocol in > > order to access the pointer to data in memory. However, I lack experience > > in buffer protocol, so suggestions for achieving this are welcome. > > This will also work. A read-only buffer protocol is exported by all > the scalars. > > scalar.data will return a buffer object. > > Or you can use the Python C-API > > const char *buffer; > Py_ssize_t buflen; > > PyObject_AsReadBuffer(scalar, (const void **)&buffer, &buflen) Oh, this one seems pretty easy, and as a plus, you don't have to book memory for copying the data area, so I'll use it. Thanks, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 12:44:10 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Tue, 17 Oct 2006 18:44:10 +0200 Subject: What does Fortran order mean? In-Reply-To: <4534FE6F.8060602@ee.byu.edu> References: <4534FE6F.8060602@ee.byu.edu> Message-ID: <20061017164410.GO17998@mentat.za.net> On Tue, Oct 17, 2006 at 10:01:51AM -0600, Travis Oliphant wrote: > Charles R Harris wrote: > > > Travis, > > > > I note that > > > > >>> a = arange(6).reshape(2,3,order='F') > > >>> a > > array([[0, 1, 2], > > [3, 4, 5]]) > > > > Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a > > copy, but flat, flatten, and tostring all show the elements in 'C' > > order. I ask because I wonder if changing the order can be used to > > prepare arrays for input into the LaPack routines. > > > The order argument to reshape means (how should the big-chunk of data be > interpreted when you reshape). So, yes, this should be > [[0,2,4],[1,3,5]]. It is a bug that it does not do the right thing in > this case. A bit counter-ituitive (I somehow expect reshape to return an array that satisfies all the constraints specified as parameters -- i.e. shape and order in memory), but consistent with Numeric. What confuses me is that, if you call the array constructor, you get In [2]: N.array(N.array([[1,2,3],[4,5,6]]),order='F') Out[2]: array([[1, 2, 3], [4, 5, 6]]) so here 'order' means something else. So then you do x = N.array([[0,1,2],[3,4,5]],order='F') x.reshape((2,3),order='C') and you get array([[0, 1, 2], [3, 4, 5]]) Hmm. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 17 13:01:08 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 11:01:08 -0600 Subject: What does Fortran order mean? In-Reply-To: <20061017164410.GO17998@mentat.za.net> References: <4534FE6F.8060602@ee.byu.edu> <20061017164410.GO17998@mentat.za.net> Message-ID: <45350C54.9010004@ee.byu.edu> Stefan van der Walt wrote: >On Tue, Oct 17, 2006 at 10:01:51AM -0600, Travis Oliphant wrote: > > >>Charles R Harris wrote: >> >> >> >>>Travis, >>> >>>I note that >>> >>> >>> >>>>>>a = arange(6).reshape(2,3,order='F') >>>>>>a >>>>>> >>>>>> >>>array([[0, 1, 2], >>> [3, 4, 5]]) >>> >>>Shouldn't that be 3x2? Or maybe [[0,2,4],[1,3,5]]? Reshape is making a >>>copy, but flat, flatten, and tostring all show the elements in 'C' >>>order. I ask because I wonder if changing the order can be used to >>>prepare arrays for input into the LaPack routines. >>> >>> >>The order argument to reshape means (how should the big-chunk of data be >>interpreted when you reshape). So, yes, this should be >>[[0,2,4],[1,3,5]]. It is a bug that it does not do the right thing in >>this case. >> >> > >A bit counter-ituitive (I somehow expect reshape to return an array >that satisfies all the constraints specified as parameters -- i.e. >shape and order in memory), but consistent with Numeric. > > Perhaps, but that is not what reshape does. When you are reshaping an array, unless you are just adding ones to the shape somewhere or aren't actually changing the shape, then you are implicitly thinking about the array as a 1-d chunk of memory in some order. The default is C-order. The order argument let's you think about it differently. >What confuses me is that, if you call the array constructor, you get > >In [2]: N.array(N.array([[1,2,3],[4,5,6]]),order='F') >Out[2]: >array([[1, 2, 3], > [4, 5, 6]]) > >so here 'order' means something else. > > Not really. I can't imagine what else you would expect here. You have specified the array explicitly. The order argument only specifies how the array will be arranged in memory. It is an implementation detail. The user is allowed some control in order to interface to compiled code more effectively. >So then you do > >x = N.array([[0,1,2],[3,4,5]],order='F') >x.reshape((2,3),order='C') > >and you get > >array([[0, 1, 2], > [3, 4, 5]]) > > > I don't see the problem. The first line gives you an array: [[0,1,2], [3,4,5]] which in memory is laid out as [0,3,1,4,2,5] The second line says reshape into a (2,3) array but this is exactly the same shape as the input array so in fact only a new view of the same data is given to you. The fact that you specify order is inconsequential since you aren't really changing the shape, it doesn't matter. The reshape command is not intended to alter the underlying ordering of an input array. The ordering of an array is an implementation detail that shouldn't matter to the user unless you are trying to interface to compiled code. The require command can help you get the right kind of array for that purpose. In general, arrays can have arbitrary strides. C-order and Fortran-order are just two special cases of that generality. So, you cannot think of any array as just being either in C-order or Fortran-order. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 17 13:13:26 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 11:13:26 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. In-Reply-To: <4533D087.3000803@ieee.org> References: <4533D087.3000803@ieee.org> Message-ID: <45350F36.80003@ee.byu.edu> Travis Oliphant wrote: >Charles R Harris wrote: > > >>It seems to me that since the behaviour when copy=0 is to make a copy >>only if necessary, it should find it necessary and make the downcast. >>After all, array(a, dtype=single, copy=1) does just that without >>complaint. Some common code in linalg could be replaced if array and >>asarray would do that operation. >> >> >> >Well, the fact that it makes the copy without raising an error is >different behavior from Numeric and should be considered an unintended >change. > >We definitely should make this consistent for copy=0 or copy=1. The >only question, is do we raise the error in both cases or allow the >conversion in both cases. > >The long-standing behavior is to raise the error on possible-loss >conversion and so my opinion is that we should continue with that behavior. > > > But, on the other hand, it looks like numarray went the other direction and allows a cast using the array call. Thus import numarray a = numarray.array([1,2,3],'d') numarray.array(a, 'f') works So, I'm willing to go with the numarray behavior in numpy. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Andrew.MacKeith at ABAQUS.com Tue Oct 17 13:28:02 2006 From: Andrew.MacKeith at ABAQUS.com (Andrew MacKeith) Date: Tue, 17 Oct 2006 13:28:02 -0400 Subject: array repr Message-ID: <453512A2.80605@ABAQUS.com> I would like to use the built-in array_repr in numpy, because I need a fast repr that does not contain new line characters. I see no way of doing this without editing the code in numeric.py, and I hate to edit other people's libraries. from numpy import array causes numeric.py to be executed, which includes a call to multiarray.set_string_function(array_repr, 1) If I want to "undo" this, there is no way of doing it. I would like to unset the repr with: numpy.set_string_function(None, 1) but this raises a TypeError because None is not callable. This behavior was the same in Numeric, and in that case I edited the code in Numeric.py but it would be nice if I didn't have to do that. Should I submit a patch for this? Andrew MacKeith ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 13:37:09 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 11:37:09 -0600 Subject: Why doesn't array(a, dtype=single, copy=0) downcast a double array. In-Reply-To: <45350F36.80003@ee.byu.edu> References: <4533D087.3000803@ieee.org> <45350F36.80003@ee.byu.edu> Message-ID: On 10/17/06, Travis Oliphant wrote: > > Travis Oliphant wrote: > > >Charles R Harris wrote: > > >The long-standing behavior is to raise the error on possible-loss > >conversion and so my opinion is that we should continue with that > behavior. > > > > > > > But, on the other hand, it looks like numarray went the other direction > and allows a cast using the array call. > > Thus > > import numarray > a = numarray.array([1,2,3],'d') > numarray.array(a, 'f') > > works > > > So, I'm willing to go with the numarray behavior in numpy. > > Thanks for the change, Travis. I'll wait a bit to make sure the API remains stable and then do some cleanups in linalg. Correctness before optimization and all that. I did note that the LaPack interface could throw an error at the fortran level that would issue an informative error message and dump me out of python. Is there anyway you know of to trap this behaviour? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Tue Oct 17 13:46:20 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 11:46:20 -0600 Subject: array repr In-Reply-To: <453512A2.80605@ABAQUS.com> References: <453512A2.80605@ABAQUS.com> Message-ID: <453516EC.5010607@ee.byu.edu> Andrew MacKeith wrote: >I would like to use the built-in array_repr in numpy, because >I need a fast repr that does not contain new line characters. >I see no way of doing this without editing the code in numeric.py, >and I hate to edit other people's libraries. > >from numpy import array >causes numeric.py to be executed, which includes a call to >multiarray.set_string_function(array_repr, 1) > >If I want to "undo" this, there is no way of doing it. > > Why do want to "undo" this? >I would like to unset the repr with: >numpy.set_string_function(None, 1) > > I don't understand what you are trying to do. Why don't you just set the string function to an actual callable. What edits do you propose to make? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Tue Oct 17 13:58:18 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 17 Oct 2006 14:58:18 -0300 Subject: What does Fortran order mean? In-Reply-To: <45350C54.9010004@ee.byu.edu> References: <4534FE6F.8060602@ee.byu.edu> <20061017164410.GO17998@mentat.za.net> <45350C54.9010004@ee.byu.edu> Message-ID: On 10/17/06, Travis Oliphant wrote: > The require command can help you get the right kind of > array for that purpose. > BTW, Travis In [1]: numpy.__version__ Out[1]: '1.0rc2' In [2]: print numpy.require.__doc__ None What is this 'requirements' argument? What should user pass in?. I seen sorce code at numpy.core.numeric.require. It seems this functions expects an iterable producing string. I am right? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Tue Oct 17 14:03:41 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 17 Oct 2006 15:03:41 -0300 Subject: Accessing data buffers in numpy scalars In-Reply-To: <4535033D.5000506@ee.byu.edu> References: <200610171807.20392.faltet@carabos.com> <4535033D.5000506@ee.byu.edu> Message-ID: On 10/17/06, Travis Oliphant wrote: > Or you can use the Python C-API > > const char *buffer; > Py_ssize_t buflen; > > PyObject_AsReadBuffer(scalar, (const void **)&buffer, &buflen) > > to retrieve a pointer to the data in buffer and the size of the data in > buflen. > Travis. Have numpy something similar to a mutable scalar, or the only way is using an array whit only one element? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Andrew.MacKeith at ABAQUS.com Tue Oct 17 14:15:24 2006 From: Andrew.MacKeith at ABAQUS.com (Andrew MacKeith) Date: Tue, 17 Oct 2006 14:15:24 -0400 Subject: array repr In-Reply-To: <453516EC.5010607@ee.byu.edu> References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> Message-ID: <45351DBC.8080502@ABAQUS.com> Travis Oliphant wrote: >Andrew MacKeith wrote: > > > >>I would like to use the built-in array_repr in numpy, because >>I need a fast repr that does not contain new line characters. >>I see no way of doing this without editing the code in numeric.py, >>and I hate to edit other people's libraries. >> >> >> >>from numpy import array > > >>causes numeric.py to be executed, which includes a call to >>multiarray.set_string_function(array_repr, 1) >> >>If I want to "undo" this, there is no way of doing it. >> >> >> >> >Why do want to "undo" this? > > > >>I would like to unset the repr with: >>numpy.set_string_function(None, 1) >> >> >> >> >I don't understand what you are trying to do. Why don't you just set >the string function to an actual callable. > Since arrayobject.c contains a fast repr function, I would like to use that. Why reinvent the wheel. >What edits do you propose to >make? > > If you mean how would I propose to change the code, I would have array_set_string_function recognise None as an acceptable argument and PyArray_SetStringFunction would recognise this to set PyArray_ReprFunction = array_repr; These is the (untested) code. multiarraymodule.c: static PyObject * array_set_string_function(PyObject *dummy, PyObject *args, PyObject *kwds) { PyObject *op; int repr=1; static char *kwlist[] = {"f", "repr", NULL}; if(!PyArg_ParseTupleAndKeywords(args, kwds, "O|i", kwlist, &op, &repr)) return NULL; if (!PyCallable_Check(op) and op != Py_None) { PyErr_SetString(PyExc_TypeError, "Argument must be callable."); return NULL; } PyArray_SetStringFunction(op, repr); Py_INCREF(Py_None); return Py_None; } arrayobject.c: static void PyArray_SetStringFunction(PyObject *op, int repr) { if (repr) { /* Dispose of previous callback */ Py_XDECREF(PyArray_ReprFunction); if (op == Py_None) { PyArray_ReprFunction = array_repr; } else { /* Add a reference to new callback */ Py_XINCREF(op); /* Remember new callback */ PyArray_ReprFunction = op; } } else { /* Dispose of previous callback */ Py_XDECREF(PyArray_StrFunction); if (op == Py_None) { PyArray_StrFunction = array_str; } else { /* Add a reference to new callback */ Py_XINCREF(op); /* Remember new callback */ PyArray_StrFunction = op; } } } >-Travis > > > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion at lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 14:25:18 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 12:25:18 -0600 Subject: Casting In-Reply-To: References: Message-ID: On 10/12/06, Cristian Codorean wrote: > > Hello list, > I got some old code that uses Scientific and Numeric. Recently, these > two packages were updated to the latest versions together with python and my > code started failing. The first major problem I have is when downcasting. > Code that used to work now fails because the array cannot be safely cast > anymore to the required type. If anyone could help me please and direct me > to some documentation about when this change happened (version), why it > happened, is there any other solution but using astype, can numpy help me in > anyway in this situation, because I got really lost ... This seems related to a more recent thread. Travis says that numeric throwing an error on downcasting is traditional, so I am curious as to why the previous code should have worked. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From kortmann at ideaworks.com Tue Oct 17 14:35:25 2006 From: kortmann at ideaworks.com (Kenny Ortmann) Date: Tue, 17 Oct 2006 13:35:25 -0500 Subject: matlab, filter function References: Message-ID: <002101c6f21b$05687b50$0900a8c0@yairlap> just looking for some help, most of the time you guys are good with matlab code, i am trying to use the filter function under this setting y = filter(b,a,X) filters the data in vector X with the filter described by numerator coefficient vector b and denominator coefficient vector a. If a(1) is not equal to 1, filter normalizes the filter coefficients by a(1). If a(1) equals 0, filter returns an error. I pretty much have the same as the example, except my data array is different data = [1:0.2:4]'; windowSize = 5; filter(ones(1,windowSize)/windowSize,1,data) ans = 0.2000 0.4400 0.7200 1.0400 1.4000 1.6000 1.8000 2.0000 2.2000 2.4000 2.6000 2.8000 3.0000 3.2000 3.4000 3.6000 my question is does numpy/scipy have a function for this, if not the algorithm states y(n) = b(1)*x(n) + b(2)*x(n-1) + ... + b(nb+1)*x(n-nb) - a(2)*y(n-1) - ... - a(na+1)*y(n-na)but a is a number not an arrayThanks for the help,~Kenny ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 15:10:55 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 13:10:55 -0600 Subject: dtype always copies In-Reply-To: <20061013142825.GA16246@mentat.za.net> References: <20061013142825.GA16246@mentat.za.net> Message-ID: On 10/13/06, Stefan van der Walt wrote: > > Hi all, > > I've noticed that 'astype' always forces a copy. Is this > behaviour intended? It seems to conflict with 'asarray', that > tries to avoid a copy. > > For example, when wrapping code in ctypes, the following snippet > would have been useful: > > def foo(x): > # ensure x is an array of the right type > x = N.ascontiguousarray(x).astype(N.intc) > > but that will cause a copy, so you'll have to do > > def foo(x): > try: > x = N.ascontiguousarray(x,N.intc) > except: > x = N.ascontiguousarray(x).astype(N.intc) This seems to work now: In [21]: a Out[21]: array([[ 1., 2.], [ 3., 4.]]) In [22]: ascontiguousarray(a, dtype=int) Out[22]: array([[1, 2], [3, 4]]) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Tue Oct 17 15:19:42 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 13:19:42 -0600 Subject: Accessing data buffers in numpy scalars In-Reply-To: References: <200610171807.20392.faltet@carabos.com> <4535033D.5000506@ee.byu.edu> Message-ID: <45352CCE.3060108@ee.byu.edu> Lisandro Dalcin wrote: >On 10/17/06, Travis Oliphant wrote: > > >>Or you can use the Python C-API >> >>const char *buffer; >>Py_ssize_t buflen; >> >>PyObject_AsReadBuffer(scalar, (const void **)&buffer, &buflen) >> >>to retrieve a pointer to the data in buffer and the size of the data in >>buflen. >> >> >> > >Travis. Have numpy something similar to a mutable scalar, or the only >way is using an array whit only one element? > > 0-dim arrays are still mutable scalars. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 17 15:21:37 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 13:21:37 -0600 Subject: array repr In-Reply-To: <45351DBC.8080502@ABAQUS.com> References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> <45351DBC.8080502@ABAQUS.com> Message-ID: <45352D41.2030601@ee.byu.edu> Ah!, I get it. You want to be able to reset to the C-defined array_repr function. The one that gets over-written on import. That makes sense. And is definitely do-able. Please file a ticket. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Andrew.MacKeith at ABAQUS.com Tue Oct 17 15:33:39 2006 From: Andrew.MacKeith at ABAQUS.com (Andrew MacKeith) Date: Tue, 17 Oct 2006 15:33:39 -0400 Subject: array repr In-Reply-To: <45352D41.2030601@ee.byu.edu> References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> <45351DBC.8080502@ABAQUS.com> <45352D41.2030601@ee.byu.edu> Message-ID: <45353013.9000705@ABAQUS.com> Travis Oliphant wrote: >Ah!, I get it. You want to be able to reset to the C-defined >array_repr function. The one that gets over-written on import. That >makes sense. And is definitely do-able. > >Please file a ticket. > > Can you point me to how to file a ticket. Thanks Andrew > > >-Travis > > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion at lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Tue Oct 17 15:48:37 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Tue, 17 Oct 2006 21:48:37 +0200 Subject: recent ubuntu package Message-ID: <45353395.40602@gmx.net> Hi, as suggested on the website I use the kindly provided pre-built (unofficial) ubuntu debs. Recently there is a new one available with version numbe 1.0+~dev3336-0ads1. Apart from the slightly strange +~ thing in there, it very much seems to be based on trac changeset 3336, which is somewhere between rc2 and now, right? And just checking, what are the matplotlib packages that are compatible? The most recent from the same package source seems to be 0.87.5-r2781-0ads2, is that ok with the above numpy? Will the mpl ubuntu package before that break with the latest numpy ubuntu package? Thanks for your help, and for the build effort, Sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 15:51:05 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 13:51:05 -0600 Subject: array repr In-Reply-To: <45353013.9000705@ABAQUS.com> References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> <45351DBC.8080502@ABAQUS.com> <45352D41.2030601@ee.byu.edu> <45353013.9000705@ABAQUS.com> Message-ID: On 10/17/06, Andrew MacKeith wrote: > > Travis Oliphant wrote: > > >Ah!, I get it. You want to be able to reset to the C-defined > >array_repr function. The one that gets over-written on import. That > >makes sense. And is definitely do-able. > > > >Please file a ticket. > > > >** > Can you point me to how to file a ticket. > Thanks Go to http://projects.scipy.org/scipy/numpy/report, login or register (required), then choose new ticket from the menu at the top. BTW, I wish all this was more transparent to the casual visitor to the scipy site. There should be a link on the front page (numpy bugs or something) and a short explanation about needing to register to file a new ticket. Maybe something in the Cookbook would help. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Tue Oct 17 17:20:07 2006 From: david.huard at gmail.com (David Huard) Date: Tue, 17 Oct 2006 17:20:07 -0400 Subject: histogram complete makeover Message-ID: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> Hi all, I'd like to poll the list to see what people want from numpy.histogram(), since I'm currently writing a contender. My main complaints with the current version are: 1. upper outliers are stored in the last bin, while lower outliers are not counted at all, 2. cannot use weights. The new histogram function is well under way (it address these issues and adds an axis keyword), but I want to know what is the preferred behavior regarding the function output, and your willingness to introduce a new behavior that will break some code. Given a number of bins N and range (min, max), histogram constructs linearly spaced bin edges b0 (out-of-range) | b1 | b2 | b3 | .... | bN | bN+1 out-of-range and may return: A. H = array([N_b0, N_b1, ..., N_bN, N_bN+1]) The out-of-range values are the first and last values of the array. The returned array is hence N+2 B. H = array([N_b0 + N_b1, N_b2, ..., N_bN + N_bN+1]) The lower and upper out-of-range values are added to the first and last bin respectively. C. H = array([N_b1, ..., N_bN + N_bN+1]) Current behavior: the upper out-of-range values are added to the last bin. D. H = array([N_b1, N_b2, ..., N_bN]), Lower and upper out-of-range values are given after the histogram array. Ideally, the new function would not break the common usage: H = histogram(x)[0], so this exclude A. B and C are not acceptable in my opinion, so only D remains, with the downsize that the outliers are not returned. A solution might be to add a keyword full_output=False, which when set to True, returns the out-of-range values in a dictionnary. Also, the current function returns -> H, ledges where ledges is the array of left bin edges (N). I propose returning the complete array of edges (N+1), including the rightmost edge. This is a little bit impractical for plotting, as the edges array does not have the same length as the histogram array, but allows the use of user-defined non-uniform bins. Opinions, suggestions ? David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Tue Oct 17 17:55:54 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 17 Oct 2006 15:55:54 -0600 Subject: matlab, filter function In-Reply-To: <002101c6f21b$05687b50$0900a8c0@yairlap> References: <002101c6f21b$05687b50$0900a8c0@yairlap> Message-ID: <4535516A.4030608@ee.byu.edu> Kenny Ortmann wrote: >just looking for some help, most of the time you guys are good with matlab >code, i am trying to use the filter function under this setting > >y = filter(b,a,X) filters the data in vector X with the filter described by >numerator coefficient vector b and denominator coefficient vector a. If a(1) >is not equal to 1, filter normalizes the filter coefficients by a(1). If >a(1) equals 0, filter returns an error. > > There is scipy.signal.lfilter which implements this algorithm. It's doc string is lfilter(b, a, x, axis=-1, zi=None) Filter data along one-dimension with an IIR or FIR filter. Description Filter a data sequence, x, using a digital filter. This works for many fundamental data types (including Object type). The filter is a direct form II transposed implementation of the standard difference equation (see "Algorithm"). Inputs: b -- The numerator coefficient vector in a 1-D sequence. a -- The denominator coefficient vector in a 1-D sequence. If a[0] is not 1, then both a and b are normalized by a[0]. x -- An N-dimensional input array. axis -- The axis of the input data array along which to apply the linear filter. The filter is applied to each subarray along this axis (*Default* = -1) zi -- Initial conditions for the filter delays. It is a vector (or array of vectors for an N-dimensional input) of length max(len(a),len(b)). If zi=None or is not given then initial rest is assumed. SEE signal.lfiltic for more information. Outputs: (y, {zf}) y -- The output of the digital filter. zf -- If zi is None, this is not returned, otherwise, zf holds the final filter delay values. Algorithm: The filter function is implemented as a direct II transposed structure. This means that the filter implements y[n] = b[0]*x[n] + b[1]*x[n-1] + ... + b[nb]*x[n-nb] - a[1]*y[n-1] + ... + a[na]*y[n-na] using the following difference equations: y[m] = b[0]*x[m] + z[0,m-1] z[0,m] = b[1]*x[m] + z[1,m-1] - a[1]*y[m] ... z[n-3,m] = b[n-2]*x[m] + z[n-2,m-1] - a[n-2]*y[m] z[n-2,m] = b[n-1]*x[m] - a[n-1]*y[m] where m is the output sample number and n=max(len(a),len(b)) is the model order. The rational transfer function describing this filter in the z-transform domain is -1 -nb b[0] + b[1]z + ... + b[nb] z Y(z) = ---------------------------------- X(z) -1 -na a[0] + a[1]z + ... + a[na] z ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From shfthdcvrdlf at bakerinc.com.sun.com Tue Oct 17 15:00:01 2006 From: shfthdcvrdlf at bakerinc.com.sun.com (Molly Driscoll) Date: Tue, 17 Oct 2006 22:00:01 +0300 Subject: Significant letter. You must to read. Message-ID: <01c6f237$97f8dcf0$6c822ecf@shfthdcvrdlf> Please read it. Today is the exact day to buy that stock. Please check that note attentively. Here you will find the intimate news about GDKI. Please Read this news. Goldmark Cipher Established Hip Hop Creator Sean Combs A.K.A. Puff Daddy Wednesday this month 18, 11:40 am ET NEW YORK & LOS ANGELES & VANCOUVER, B.C.--(BUSINESS WIRE)-- October 18, 11:40 am ET -- Goldmark Industries, Inc. (PINK SHEETS:GDKI.PK - Reports), is excited to declare that it has signed one of the hottest producers in the hip-hop and r&b industry, Sean Combs A.K.A. Puff Daddy. In an aggressive attempt to stay ahead of the game, the Corporation moved quickly in taking on the already successful and rising famous person. Sean Combs, also well-known as Puff Daddy is the founder of Bad Boys Records marker. By contract the between Goldmark Industries, Inc and Bad Boy Entertainment Inc, Goldmark has to get 78 000 000 $ for the distributing of Sean Combs A.K.A. Puff Daddy fresh album Press Play. Total about from that deal for Goldmark should be about 12 000 000$. After the establishing of that deal Goldmark Industries, Inc will be the second biggest company in the industry of hip-hop & r&b music. Hip Hop famous person Sean "P.Diddy" Combs sees a huge possibility for his company in collaboration with Goldmark inc. Sean "P.Diddy" Combs tells that it is enjoyably to treat with these guys. They as anybody else know entertainment industriousness and precisely know what is necessary for the American spectators. He also emphasizes exclusivity of his fresh album Press Play and tells that the presentation of this album on october 17 will make an effect of the blasted bomb. Today the new album of Puff Daddy Press Play will be on the shelves. TAKE GDKI and you will earn from every sale of that golden disc. Concerning Bad Boy International Entertainment Group: is one of the world's pre-eminent urban entertainment companies, surrounding a wide variety of businesses including recording, music publishing, artist organization, television and film production, recording facility, advertising and advertising, apparel and restaurants. With a collection of businesses whose yearly sales are quickly impending 300$ million annually and an employee base that is 600 strong. Mr. Combs is largely responsible for the pop appeal of urban entertainment. In just eight years, founder and CEO Sean "P.Diddy" Combs has forever changed the entertainment industry by catapulting the music and fashionof urban youth culture into the American mainstream and creating what is measured by experts to be one of the most significant services in entertainment these days. About Goldmark Industries, Inc. (Stock symbol: GDKI.PK) Goldmark Industries is committed to providing the best in all forms of urban entertainment to the 45 Million Hip-Hop consumers in North America. The average North American spends health care & clothing on entertainment than they do on more cash, making entertainment the most attractive industry for investors and advertisers alike. Goldmark Industries is preparing to stand at the forefront of the Hip Hop consumer market, specializing in all aspects of entertainment, including Major Events ,Music ,Feature Films ,Television and Home Video/DVD . The strength of Goldmark Industries is the result of its highly reputable & continuously growing management team. The knowledge and experience that each team member brings consistently supports the growing success of each division at Goldmark Industries. In addition, they are associated with some of the world's leading entertainment companies and top distribution channels worldwide, providing Goldmark Industries with the relationships to continually move forward. P.S We has started to promote GDKI yesterday and the price is up for 16.6% already. Today,october 17 2006, the price will be 100% up. Don't miss that opportunity to buy GDKI for a low price. Just buy it! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Tue Oct 17 21:13:13 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 17 Oct 2006 22:13:13 -0300 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <20061018002445.GP17998@mentat.za.net> References: <20061018002445.GP17998@mentat.za.net> Message-ID: I was surprised by this In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') Out[14]: array([[1, 5], [4, 3], [2, 6]]) In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') Out[15]: array([[1, 2], [3, 4], [5, 6]]) On 10/17/06, Stefan van der Walt wrote: > Hi all, > > Some of you may have seen the interesting thread on Fortran-ordering > earlier. I thought it might be fun to set up a short quiz which tests > your knowledge on the topic. > > If you're up for the challenge, take a look at > > http://mentat.za.net/numpy/quiz > > I won't be held liable for any emotional trauma caused, self-inflicted > wounds or brain-core meltdowns. > > Cheers > St?fan > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From michael.sorich at gmail.com Tue Oct 17 20:19:27 2006 From: michael.sorich at gmail.com (Michael Sorich) Date: Wed, 18 Oct 2006 09:49:27 +0930 Subject: A reimplementation of MaskedArray In-Reply-To: <200610162242.58541.pgmdevlist@mailcan.com> References: <200610160232.20888.pgmdevlist@mailcan.com> <16761e100610161908r3bbf383fq4c5caf64f8b805b8@mail.gmail.com> <200610162242.58541.pgmdevlist@mailcan.com> Message-ID: <16761e100610171719we82588mf8138bdfadce3398@mail.gmail.com> On 10/17/06, Pierre GM wrote: > On Monday 16 October 2006 22:08, Michael Sorich wrote: > > Does this new MA class allow masking of rearray like arrays? > > Excellent question! Which is to say, I have no idea... I don't use > recordarray, so I didn't think about testing them. > > So, a first test indicates that it doesn't work either. The mask for a3 is > understood as having a size 6, when the data is seen as size 2 only (exactly > the same problem as the original ma module). > I gonna poke around and check whether I can come with a workaround. Just to > make it clear: you want to be able to mask some fields in some records, > right ? Not mask all the fields of a record ? Yes, that is correct. It would be preferable to have more control over masking than simply masking an entire record. I view the heterogenouse arrays as 2D arrays in which it should be possible to mask individual values. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 20:24:45 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Wed, 18 Oct 2006 02:24:45 +0200 Subject: The NumPy Fortran-ordering quiz Message-ID: <20061018002445.GP17998@mentat.za.net> Hi all, Some of you may have seen the interesting thread on Fortran-ordering earlier. I thought it might be fun to set up a short quiz which tests your knowledge on the topic. If you're up for the challenge, take a look at http://mentat.za.net/numpy/quiz I won't be held liable for any emotional trauma caused, self-inflicted wounds or brain-core meltdowns. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mrovner at cadence.com Tue Oct 17 20:30:12 2006 From: mrovner at cadence.com (Mike Rovner) Date: Tue, 17 Oct 2006 17:30:12 -0700 Subject: 1.0rc2 strange repeat behavior Message-ID: Hi, I got strange discrepance between 2.4+0.9.8 and 2.5+1.0rc2: model_lib_pool %0 !5019$ python2.5 Python 2.5 (r25:51908, Oct 17 2006, 16:16:21) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-14)] on linux2 Type "help", "copyright", "credits" or "license" for more information. f>>> from numpy import * >>> a=arange(12).reshape(4,3) >>> a array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11]]) >>> a[:,2] array([ 2, 5, 8, 11]) >>> a[:,2].repeat(3) array([2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5]) >>> import numpy >>> numpy.__version__ '1.0rc2' >>> Old version looks right: Python 2.4.2 (#5, Jun 2 2006, 18:33:20) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-20)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from numpy import * >>> a=arange(12).reshape(4,3) >>> a array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11]]) >>> a[:,2] array([ 2, 5, 8, 11]) >>> a[:,2].repeat(3) array([ 2, 2, 2, 5, 5, 5, 8, 8, 8, 11, 11, 11]) >>> import numpy >>> numpy.__version__ '0.9.8' >>> Thanks, Mike ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ndbecker2 at gmail.com Tue Oct 17 20:46:51 2006 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 17 Oct 2006 20:46:51 -0400 Subject: histogram complete makeover References: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> Message-ID: David Huard wrote: > Hi all, > > I'd like to poll the list to see what people want from numpy.histogram(), > since I'm currently writing a contender. > > My main complaints with the current version are: > 1. upper outliers are stored in the last bin, while lower outliers are not > counted at all, > 2. cannot use weights. > > The new histogram function is well under way (it address these issues and > adds an axis keyword), > but I want to know what is the preferred behavior regarding the function > output, and your > willingness to introduce a new behavior that will break some code. > > Given a number of bins N and range (min, max), histogram constructs > linearly spaced bin edges > b0 (out-of-range) | b1 | b2 | b3 | .... | bN | bN+1 out-of-range > and may return: > > A. H = array([N_b0, N_b1, ..., N_bN, N_bN+1]) > The out-of-range values are the first and last values of the array. The > returned array is hence N+2 > > B. H = array([N_b0 + N_b1, N_b2, ..., N_bN + N_bN+1]) > The lower and upper out-of-range values are added to the first and last > bin respectively. > > C. H = array([N_b1, ..., N_bN + N_bN+1]) > Current behavior: the upper out-of-range values are added to the last bin. > > D. H = array([N_b1, N_b2, ..., N_bN]), > Lower and upper out-of-range values are given after the histogram array. > > Ideally, the new function would not break the common usage: H = > histogram(x)[0], so this exclude A. B and C are not acceptable in my > opinion, so only D remains, with the downsize that the outliers are not > returned. A solution might be to add a keyword full_output=False, which > when set to True, returns the out-of-range values in a dictionnary. > > Also, the current function returns -> H, ledges > where ledges is the array of left bin edges (N). > I propose returning the complete array of edges (N+1), including the > rightmost edge. This is a little bit impractical for plotting, as the > edges array does not have the same length as the histogram array, but > allows the use of user-defined non-uniform bins. > > Opinions, suggestions ? > > David I have my own histogram that might interest you. The core is modern c++, with boost::python wrapper. Out-of-bounds behavior is programmable. I'll send it to you if you are interested. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Tue Oct 17 21:30:26 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Wed, 18 Oct 2006 10:30:26 +0900 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: I think the answer to #3 is wrong. >From 1.0rc2 I get: >>> array([1,2,3,4,5,6],order='C').reshape((2,3),order='F') array([[1, 2, 3], [4, 5, 6]]) But the quiz wants me to answer something different. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Andrew.MacKeith at ABAQUS.com Tue Oct 17 21:05:28 2006 From: Andrew.MacKeith at ABAQUS.com (Andrew.MacKeith) Date: Tue, 17 Oct 2006 21:05:28 -0400 Subject: array repr In-Reply-To: References: <453512A2.80605@ABAQUS.com> <453516EC.5010607@ee.byu.edu> <45351DBC.8080502@ABAQUS.com> <45352D41.2030601@ee.byu.edu> <45353013.9000705@ABAQUS.com> Message-ID: <45357DD8.7030508@ABAQUS.com> Charles R Harris wrote: > > > On 10/17/06, Andrew MacKeith > wrote: > > Travis Oliphant wrote: > > >Ah!, I get it. You want to be able to reset to the C-defined > >array_repr function. The one that gets over-written on > import. That > >makes sense. And is definitely do-able. > > > >Please file a ticket. > > > > > Can you point me to how to file a ticket. > Thanks > > > Go to http://projects.scipy.org/scipy/numpy/report > , login or register > (required), then choose new ticket from the menu at the top. > > BTW, I wish all this was more transparent to the casual visitor to the > scipy site. There should be a link on the front page (numpy bugs or > something) and a short explanation about needing to register to file a > new ticket. Maybe something in the Cookbook would help. > > Chuck Thanks for the info. I filed a ticket. Andrew > > >------------------------------------------------------------------------ > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > >------------------------------------------------------------------------ > >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion at lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Tue Oct 17 21:38:13 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 19:38:13 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: <45358585.60907@ieee.org> Lisandro Dalcin wrote: > I was surprised by this > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > Out[14]: > array([[1, 5], > [4, 3], > [2, 6]]) > > In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > Out[15]: > array([[1, 2], > [3, 4], > [5, 6]]) > This is a bug that was recently fixed in SVN. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From djarb at highenergymagic.org Tue Oct 17 19:35:19 2006 From: djarb at highenergymagic.org (Daniel Arbuckle) Date: Tue, 17 Oct 2006 16:35:19 -0700 Subject: Question about indexing arrays with boolean arrays Message-ID: Why does a[b1, b2] not mean the same thing as a[b1][:, b2], when "a" is an array and "b1" and "b2" are appropriately sized arrays of booleans? Thanks ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Tue Oct 17 21:42:46 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Wed, 18 Oct 2006 03:42:46 +0200 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: <20061018014246.GQ17998@mentat.za.net> On Wed, Oct 18, 2006 at 10:30:26AM +0900, Bill Baxter wrote: > I think the answer to #3 is wrong. > > >From 1.0rc2 I get: > >>> array([1,2,3,4,5,6],order='C').reshape((2,3),order='F') > array([[1, 2, 3], > [4, 5, 6]]) > > But the quiz wants me to answer something different. This recently changed. The quiz is correct for r3348. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 21:47:22 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 19:47:22 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: On 10/17/06, Lisandro Dalcin wrote: > > I was surprised by this > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > Out[14]: > array([[1, 5], > [4, 3], > [2, 6]]) This one still looks wrong. In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > Out[15]: > array([[1, 2], > [3, 4], > [5, 6]]) This one is fixed, In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') Out[3]: array([[1, 4], [2, 5], [3, 6]]) I also don't understand why a copy is returned if 'F' just fiddles with the indices and strides; the underlying data should be the same, just the view changes. FWIW, I think both examples should be returning views. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Tue Oct 17 21:53:11 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 19:53:11 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <20061018002445.GP17998@mentat.za.net> References: <20061018002445.GP17998@mentat.za.net> Message-ID: <45358907.9020509@ieee.org> Stefan van der Walt wrote: > Hi all, > > Some of you may have seen the interesting thread on Fortran-ordering > earlier. I thought it might be fun to set up a short quiz which tests > your knowledge on the topic. > > If you're up for the challenge, take a look at > > http://mentat.za.net/numpy/quiz > > I won't be held liable for any emotional trauma caused, self-inflicted > wounds or brain-core meltdowns. > Cute (especially the comment if you get them all right). I'm not sure if this quiz is a veiled complaint about the rules for Fortran ordering or not ;-) In my mind the Fortran ordering rules are consistent (if not completely bug-free). You just have to get the right idea of what is meant by the order argument when you use it. If you think you are having trouble figuring out the rules, think of the trouble it was to figure out what they should be and then to code them up. Two rules help you pass the quiz with a perfect score: 1) On array construction, the order argument allows you to specify how the array will be organized in memory. This has no effect on what is printed as the user doesn't usually care how the array is stored in memory. So, you can ignore all order= expressions in the array construct for the quiz 2) On reshaping, the order argument specifies how you think the array is organized. Whenever you make a significant reshape you are telling the computer to re-interpret the chunk of data in a different way, it makes a big difference as to how you think about that chunk of data. Do you think of it as organized rows-first (C-order) or columns-first (Fortran-order). The order argument allows you to specify how you think about it and indicates the 1-d indexing order of the array. It also fills in the newly-shaped array in exactly that same order. Semantically, one could technically separate those two concepts and have one order argument that specifies how you think about the input and another that specifies how you think about the output. But, I really didn't want to go there and couldn't see a real advantage to that. So, the single order argument specifies how you think about both. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 17 21:59:41 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 19:59:41 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: <45358A8D.600@ieee.org> Charles R Harris wrote: > > > On 10/17/06, *Lisandro Dalcin* > wrote: > > I was surprised by this > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > Out[14]: > array([[1, 5], > [4, 3], > [2, 6]]) > > > This one still looks wrong. > > In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > Out[15]: > array([[1, 2], > [3, 4], > [5, 6]]) > > > > This one is fixed, > > In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') > Out[3]: > array([[1, 4], > [2, 5], > [3, 6]]) > > I also don't understand why a copy is returned if 'F' just fiddles > with the indices and strides; the underlying data should be the same, > just the view changes. FWIW, I think both examples should be returning > views. You are right, it doesn't need to. My check is not general enough. It can be challenging to come up with a general way to differentiate the view-vs-copy situation and I struggled with it. In this case, it's the fact that while self->nd > 1, the other dimensions are only of shape 1 and so don't really matter. If you could come up with some kind of striding check that would distinguish the two cases, I would appreciate it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 17 22:15:13 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 20:15:13 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> Message-ID: <45358E31.5010909@ieee.org> Charles R Harris wrote: > > > On 10/17/06, *Lisandro Dalcin* > wrote: > > I was surprised by this > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > Out[14]: > array([[1, 5], > [4, 3], > [2, 6]]) > > > This one still looks wrong. Why does this look wrong. What do you want it to be? Perhaps you are thinking about the input array as C-order and the output array as Fortran order. That's not what reshape does. The order argument specifies how you think about both the input and output. Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] and then a Fortran-order based fill of an empty (3,2) array: giving you the result. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 22:21:02 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 20:21:02 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45358A8D.600@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> Message-ID: On 10/17/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > > On 10/17/06, *Lisandro Dalcin* > > wrote: > > > > I was surprised by this > > > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > > Out[14]: > > array([[1, 5], > > [4, 3], > > [2, 6]]) > > > > > > This one still looks wrong. > > > > In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > > Out[15]: > > array([[1, 2], > > [3, 4], > > [5, 6]]) > > > > > > > > This one is fixed, > > > > In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') > > Out[3]: > > array([[1, 4], > > [2, 5], > > [3, 6]]) > > > > I also don't understand why a copy is returned if 'F' just fiddles > > with the indices and strides; the underlying data should be the same, > > just the view changes. FWIW, I think both examples should be returning > > views. > > You are right, it doesn't need to. My check is not general enough. > > It can be challenging to come up with a general way to differentiate the > view-vs-copy situation and I struggled with it. In this case, it's the > fact that while self->nd > 1, the other dimensions are only of shape 1 > and so don't really matter. If you could come up with some kind of > striding check that would distinguish the two cases, I would appreciate > it. I suppose the problem is mostly in discontiguous arrays. Hmmm..., this isn't too different that reshaping the transpose. a.reshape((m,n),order='F' ~ a.reshape((n,m)).T.reshape(m,n) for instance: In [26]: a Out[26]: array([[1, 2, 3], [4, 5, 6]]) In [27]: a.reshape((3,2)).T.reshape((2,3)) Out[27]: array([[1, 3, 5], [2, 4, 6]]) In [28]: a.reshape((2,3), order='F') Out[28]: array([[1, 2, 3], [4, 5, 6]]) Where I actually think the second reshape is correct and the third incorrect. This has the advantage that *all* the steps return views. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From stefan at sun.ac.za Tue Oct 17 22:22:29 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Wed, 18 Oct 2006 04:22:29 +0200 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45358907.9020509@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> Message-ID: <20061018022229.GR17998@mentat.za.net> On Tue, Oct 17, 2006 at 07:53:11PM -0600, Travis Oliphant wrote: > Stefan van der Walt wrote: > > Hi all, > > > > Some of you may have seen the interesting thread on Fortran-ordering > > earlier. I thought it might be fun to set up a short quiz which tests > > your knowledge on the topic. > > > > If you're up for the challenge, take a look at > > > > http://mentat.za.net/numpy/quiz > > > > I won't be held liable for any emotional trauma caused, self-inflicted > > wounds or brain-core meltdowns. > > > > Cute (especially the comment if you get them all right). I'm not sure > if this quiz is a veiled complaint about the rules for Fortran ordering > or not ;-) If it is, it is very tactfully disguised ;) Actually, I was just trying to figure out how Fortran ordering works and thought others might be interested. I reckoned you would be the only one to get 100%, although now you've explained the rules so clearly a couple of other might grasp for the holy grail too. > In my mind the Fortran ordering rules are consistent (if not completely > bug-free). You just have to get the right idea of what is meant by the > order argument when you use it. If you think you are having trouble > figuring out the rules, think of the trouble it was to figure out what > they should be and then to code them up. My sympathies (no, really). > 2) On reshaping, the order argument specifies how you think the array is > organized. Whenever you make a significant reshape you are telling the > computer to re-interpret the chunk of data in a different way, it makes > a big difference as to how you think about that chunk of data. Do you > think of it as organized rows-first (C-order) or columns-first > (Fortran-order). The order argument allows you to specify how you > think about it and indicates the 1-d indexing order of the array. It > also fills in the newly-shaped array in exactly that same order. > Semantically, one could technically separate those two concepts and have > one order argument that specifies how you think about the input and > another that specifies how you think about the output. But, I really > didn't want to go there and couldn't see a real advantage to that. So, > the single order argument specifies how you think about both. Thanks for the detailed overview (we should put it on the wiki). Another thing I'm wondering about: when exactly does reshape need to make copies? One last case, which confuses me still (probably because it is 04:16am): In [41]: x = N.array([[0,1,2],[3,4,5]],order='F') In [42]: x Out[42]: array([[0, 1, 2], [3, 4, 5]]) I assume the data is now stored in memory as [0 3 1 4 2 5] (column-wise) If I now do x.reshape((3,2),order='C') i.e. take that block of memory, assume it is in 'C' order, and make its shape (3,2), I expect [[0 3] [1 4] [2 5]] but get [[1 2] [3 4] [5 6]] I'm obviously missing something trivial -- I'll try again tomorrow. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 17 22:28:37 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 20:28:37 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45358E31.5010909@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 10/17/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > > On 10/17/06, *Lisandro Dalcin* > > wrote: > > > > I was surprised by this > > > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > > Out[14]: > > array([[1, 5], > > [4, 3], > > [2, 6]]) > > > > > > This one still looks wrong. > > Why does this look wrong. What do you want it to be? Perhaps you are > thinking about the input array as C-order and the output array as > Fortran order. That's not what reshape does. The order argument > specifies how you think about both the input and output. > > Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] > and then a Fortran-order based fill of an empty (3,2) array: giving you > the result. Why a Fortran ravel? I am thinking of it as preserving the memory layout, just messing with how it is addressed. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Oct 17 22:53:52 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 20:53:52 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> Message-ID: On 10/17/06, Charles R Harris wrote: > > > > On 10/17/06, Travis Oliphant wrote: > > > > Charles R Harris wrote: > > > > > > > > > On 10/17/06, *Lisandro Dalcin* > > > wrote: > > > > > > I was surprised by this > > > > > > In [14]: array([[1,2,3],[4,5,6]]).reshape((3,2),order='F') > > > Out[14]: > > > array([[1, 5], > > > [4, 3], > > > [2, 6]]) > > > > > > > > > This one still looks wrong. > > > > > > In [15]: array([1,2,3,4,5,6]).reshape((3,2),order='F') > > > Out[15]: > > > array([[1, 2], > > > [3, 4], > > > [5, 6]]) > > > > > > > > > > > > This one is fixed, > > > > > > In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') > > > Out[3]: > > > array([[1, 4], > > > [2, 5], > > > [3, 6]]) > > > > > > I also don't understand why a copy is returned if 'F' just fiddles > > > with the indices and strides; the underlying data should be the same, > > > just the view changes. FWIW, I think both examples should be returning > > > > > views. > > > > You are right, it doesn't need to. My check is not general enough. > > > > It can be challenging to come up with a general way to differentiate the > > view-vs-copy situation and I struggled with it. In this case, it's the > > fact that while self->nd > 1, the other dimensions are only of shape 1 > > and so don't really matter. If you could come up with some kind of > > striding check that would distinguish the two cases, I would appreciate > > it. > > > I suppose the problem is mostly in discontiguous arrays. Hmmm..., this > isn't too different that reshaping the transpose. > > a.reshape((m,n),order='F' ~ a.reshape((n,m)).T.reshape(m,n) > Erm, brainfart. Make that a.reshape((m,n),order='F') ~ a.reshape((n,m)).T That's how I think of it, anyway. Actual layout in memory is not affected until something like ascontiguosarray is called. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Tue Oct 17 23:25:33 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Tue, 17 Oct 2006 23:25:33 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 17/10/06, Charles R Harris wrote: > > > On 10/17/06, Travis Oliphant wrote: > > Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] > > and then a Fortran-order based fill of an empty (3,2) array: giving you > > the result. > > Why a Fortran ravel? I am thinking of it as preserving the memory layout, > just messing with how it is addressed. Because, to first order, the memory layout is totally irrelevant to a python user. array([[1,2,3],[4,5,6]],order='C') array([[1,2,3],[4,5,6]],order='F') should be nearly identical to the python user. If the storage order mattered, you'd have to know, every time you used reshape, what order your matrix was stored in (did it come from a transpose operation, for example?). As for why a Fortran ravel rather than a C ravel, that's a simplifying decision. If you like, you can think of reshape as happening in two steps: o1='C' o2='C' A.reshape((6,),order=o1).reshape((2,3),order=o2) The design could have allowed, as Travis Oliphant says, o1!=o2, but explaining that would have been quite difficult (even more than now, I mean). And in any case you can use the code above to achieve that, if you really want. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ckkart at hoc.net Wed Oct 18 00:45:33 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Wed, 18 Oct 2006 04:45:33 +0000 (UTC) Subject: sort(axis=-1) Message-ID: Hi, it seems that -1 as axis parameter is interpreted like in array indexing, i.e. -1 means the last axis rather than meaning the only axis of the flattened representation, like for example with take(). Is that intendend? Christian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 00:35:07 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 22:35:07 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <20061018022229.GR17998@mentat.za.net> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> Message-ID: <4535AEFB.4090202@ieee.org> Stefan van der Walt wrote: > One last case, which confuses me still (probably because it is > 04:16am): > > In [41]: x = N.array([[0,1,2],[3,4,5]],order='F') > > In [42]: x > Out[42]: > array([[0, 1, 2], > [3, 4, 5]]) > > I assume the data is now stored in memory as > > [0 3 1 4 2 5] (column-wise) > > If I now do > > x.reshape((3,2),order='C') > > i.e. take that block of memory, assume it is in 'C' order, and make > its shape (3,2), I expect > > [[0 3] > [1 4] > [2 5]] > > but get > > [[1 2] > [3 4] > [5 6]] > > I'm obviously missing something trivial -- I'll try again tomorrow. > I think I see what is going on and where people are getting tripped up. You have to remember, that to NumPy it doesn't semantically matter what the "ordering" of the array is. There is no guarantee that C- order or Fortran-order is *ever* preserved through an operation. Because, in fact the general memory model of the array has no defined "order". It's defined by the strides array. It just so happens that two special-cases are tracked so that we can call out to compiled routines that expect contiguous arrays more easily. So, your mistake is trying to think that the "block" of memory is [0, 3, 1, 4, 2, 5] when you pass the Fortran-order array to the reshape method. While this is true, and it means that you will save a copy if you passed this off to a Fortran routine, the reshape command does not use this information in determining how to "think-about" the input array. In fact, the reshape method does not allow any way to specify the order of the "input" array (self) separately from the order of the output array. The order argument indicates the defined order of both input and output. You might think that the order of self should be used as the order of the input array. The problem with this is, again, that a general array does not have a defined "order". What should be used as the assumed "order" for an un-strided array? You're left with an unresolved question. To avoid two input order arguments, I just let order indicate the order for both the input and the output arrays. We could provide both, but this seems a bit over-done as the same could be accomplished by separately raveling the input to 1-d and then specifying the order argument on reshape. Please continue to question. All the code needs as much review as it can get. Best regards, -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 00:43:29 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 17 Oct 2006 22:43:29 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> Message-ID: <4535B0F1.1000603@ieee.org> > > You are right, it doesn't need to. My check is not general > enough. > > It can be challenging to come up with a general way to > differentiate the > view-vs-copy situation and I struggled with it. In this case, > it's the > fact that while self->nd > 1, the other dimensions are only of > shape 1 > and so don't really matter. If you could come up with some > kind of > striding check that would distinguish the two cases, I would > appreciate it. > > > I suppose the problem is mostly in discontiguous arrays. Hmmm..., > this isn't too different that reshaping the transpose. > > a.reshape((m,n),order='F' ~ a.reshape((n,m)).T.reshape(m,n) > I just committed a change to the check-for-copy code to SVN. A copy will occur if an actual reshaping is taking place that involves more than just adding or deleting ones from the old shape and if 1) self is not "single-segment". Single-segment means either Fortran-order or C-order. If we don't have a single-segment array, then we have to get a (Fortran or C-) contiguous buffer to do any reshaping (there may be special cases when it's possible, but I haven't figure them out...) 2) self is single-segment but self.squeeze().ndim > 1 and it is in the "wrong" order from what was requested in the order argument (i.e. self is in C-contiguous order but a Fortran-order reshape was requested or self is in F-contiguous order but a C-order reshape was requested). If there are any cases satisfying these rules where a copy does not have to occur then let me know. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 00:56:04 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 22:56:04 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 10/17/06, Charles R Harris wrote: > > > > On 10/17/06, A. M. Archibald wrote: > > > > On 17/10/06, Charles R Harris wrote: > > > > > > > > > On 10/17/06, Travis Oliphant wrote: > > Which doesn't seem to be the case here. I am beginning to wonder if we > really need fortran order, seems that a few well chosen interface routines > would fill the need and avoid much confusion. > For instance, it would be nice if flatten took an order keyword: In [107]: array([[1,2,3],[4,5,6]], dtype=int8, order='F').flatten() Out[107]: array([1, 2, 3, 4, 5, 6], dtype=int8) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From strawman at astraw.com Wed Oct 18 02:10:29 2006 From: strawman at astraw.com (Andrew Straw) Date: Tue, 17 Oct 2006 23:10:29 -0700 Subject: recent ubuntu package In-Reply-To: <45353395.40602@gmx.net> References: <45353395.40602@gmx.net> Message-ID: <4535C555.9060004@astraw.com> Sven Schreiber wrote: > Hi, > as suggested on the website I use the kindly provided pre-built > (unofficial) ubuntu debs. Recently there is a new one available with > version numbe 1.0+~dev3336-0ads1. > > Apart from the slightly strange +~ thing in there, it very much seems to > be based on trac changeset 3336, which is somewhere between rc2 and now, > right? > Yes, it is from svn 3336 with one minor patch. The strange +~ is to make the version numbers sort properly. (I didn't think about this when releasing "1.0b2". > And just checking, what are the matplotlib packages that are compatible? > The most recent from the same package source seems to be > 0.87.5-r2781-0ads2, is that ok with the above numpy? Will the mpl ubuntu > package before that break with the latest numpy ubuntu package? > The matplotlib .deb on my website is working fine for me with the latest numpy .deb there. If there are any recent patches or anything you are missing, please let me know -- it's not really a big deal to update them, although it might take a couple of days for me to find the time. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 00:26:17 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 17 Oct 2006 22:26:17 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 10/17/06, A. M. Archibald wrote: > > On 17/10/06, Charles R Harris wrote: > > > > > > On 10/17/06, Travis Oliphant wrote: > > > > Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] > > > and then a Fortran-order based fill of an empty (3,2) array: giving > you > > > the result. > > > > Why a Fortran ravel? I am thinking of it as preserving the memory > layout, > > just messing with how it is addressed. > > Because, to first order, the memory layout is totally irrelevant to a > python user. > > array([[1,2,3],[4,5,6]],order='C') > array([[1,2,3],[4,5,6]],order='F') > > should be nearly identical to the python user. So what is the point of having a fortran layout if things are not actually layed out in fortran order? As far as I can see, memory layout is the only reason for fortran ordering. Now I can sort of see where the fortran ravel comes from, because the result can be passed to a fortran routine. And it does look like a.ravel('F') is the same as a.T.ravel(). Hmmm. Now I wonder about this: In [62]: array([[1,2,3],[4,5,6]], dtype=int8, order='F').flags Out[62]: CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False Now, either CONTIGUOUS is in error, because it really *is* fortran contiguous (but not c contiguous), or the array cannot be passed to a fortran routine that expects fortran ordering, or CONTIGUOUS refers to C addressing, which we already know is not contiguous, in which case we feel uncertain. Note that knowing the answer matters if I want to call a fortran routine with this by pulling out the data pointer. The fortran routine could be in a library, or maybe in the LaPack wrapper, but whatever, it hasn't been wrapped by f2py that takes care of such details. This also looks fishy: In [93]: asfortranarray(array([[1,2,3],[4,5,6]], dtype=int8)) Out[93]: array([[1, 2, 3], [4, 5, 6]], dtype=int8) In [96]: asfortranarray(array([[1,2,3],[4,5,6]], dtype=int8)).flags Out[96]: CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False because the docstring says: asfortranarray(a, dtype=None) Return 'a' as an array laid out in Fortran-order in memory. Which doesn't seem to be the case here. I am beginning to wonder if we really need fortran order, seems that a few well chosen interface routines would fill the need and avoid much confusion. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Wed Oct 18 01:57:30 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 18 Oct 2006 01:57:30 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45358A8D.600@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> Message-ID: On 17/10/06, Travis Oliphant wrote: > Charles R Harris wrote: > > > > In [3]: array([[1,2,3,4,5,6]]).reshape((3,2),order='F') > > Out[3]: > > array([[1, 4], > > [2, 5], > > [3, 6]]) > > > > I also don't understand why a copy is returned if 'F' just fiddles > > with the indices and strides; the underlying data should be the same, > > just the view changes. FWIW, I think both examples should be returning > > views. > > You are right, it doesn't need to. My check is not general enough. > > It can be challenging to come up with a general way to differentiate the > view-vs-copy situation and I struggled with it. In this case, it's the > fact that while self->nd > 1, the other dimensions are only of shape 1 > and so don't really matter. If you could come up with some kind of > striding check that would distinguish the two cases, I would appreciate it. I wrote, just for exercise I guess, a routine that checks to see if a copy is needed for a reshape (and if not, it computes the resulting strides) for an arbitrary array. (It currently only does a C-order reshape, but F-order would be an easy addition.) It does, however, successfully handle arbitrarily strided arrays with arbitrary arrangements of "1" dimensions having arbitrary strides. I think it also correctly handles 0 strides. I don't imagine you'd want to *use* it, but it does provide a complete solution to copy-less reshaping. I'd be happy to help with the C implementation built into numpy, but for the life of me I can't figure out how it works. A. M. Archibald P.S. I haven't written down a *proof* that this implementation never copies unnecessarily, but I'm pretty sure that it doesn't. It has a reshape_restricted that effectively does a ravel() first; this can't cope with things like length (5,2,3) -> length (5,3,2) unless the strides are conveniently equal, so there's a wrapper, reshape(), that breaks the reshape operation up into pieces that can be done by reshape_restricted. -AMA -------------- next part -------------- A non-text attachment was scrubbed... Name: stride.py Type: text/x-python Size: 2234 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Wed Oct 18 03:35:17 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 01:35:17 -0600 Subject: NumPy 1.0rc3 will be out in a few hours Message-ID: <4535D935.2000604@ieee.org> I'm going to release NumPy 1.0rc3 in a few hours. It requires a re-build of extension modules from 1.0rc2. This gives us a chance to start fresh with NumPy 1.0 and also test the non-dying version of the version_checking code. So, it's not all bad. Official NumPy 1.0 will be released on Oct. 24 to many tears of rejoicing... Please file all tickets / requests by this weekend if you hope to have them included in the 1.0 release. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From GRGE at HOTMAIL.COM Wed Oct 18 05:13:16 2006 From: GRGE at HOTMAIL.COM (=?iso-2022-jp?B?R1JHRQ==?=) Date: Wed, 18 Oct 2006 02:13:16 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCPlA0aSROQUdFKCRKPXcbKEI=?= =?iso-2022-jp?b?GyRCJE47UhsoQg==?= Message-ID: ??????????????? http://hlgliu.jeeran.com/egao ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nrvumzt at us.univision.com Wed Oct 18 02:15:40 2006 From: nrvumzt at us.univision.com (enough) Date: Wed, 18 Oct 2006 08:15:40 +0200 Subject: Italy Message-ID: <001101c6f27c$d602ebe0$13b73850@thuis45vfps1la> Decides pull century achieve sets finding someone bankroll is enterprise person Coward big am syndicate boss still running a. Sorted shutup no talks anymore meeting leader threatens is kill pulloff responds will making a grave. Movies past yearsi give stars acting of driving stunts fiction life own report Enjoyable of genial picture am Gomez Pardo. Shipped collateral or Fiat plant Since Italygreat Britain soccer in match being played at same time wants am create diversion is form monumental? Registry Free in ecardsyour Media Your or Accounts is all Product Categories Account in Cart Lists is. Swinger swindler an ingenious huge cache Chinese is be shipped collateral? Beckley is Color Hifi Sound Original recording reissued or English a tapes of Studio Release Date October run a Time Review a based on? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Graham.png Type: image/png Size: 726 bytes Desc: not available URL: From david at ar.media.kyoto-u.ac.jp Wed Oct 18 06:59:16 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 18 Oct 2006 19:59:16 +0900 Subject: questions regarding assignement and copy Message-ID: <45360904.3030703@ar.media.kyoto-u.ac.jp> Hi there, I've just managed to nail down a bug which took me nearly two whole days to find: this is coming from an unexpected (at least from me) behaviour of numpy. I understand that if foo is a numpy array, doing bar = foo makes no copy, and whenever you change the value of foo, you change the value of bar: import numpy as N foo = N.linspace(0, 4, 5) bar = foo print foo[1] bar[1] = -1 print foo[1] => prints 1 and -1. Now, if I do: bar += 1 print bar is foo prints True But if I do bar = bar + 1, then bar is not a copy of foo anymore. Is this intended ? This looks really confusing to me, and I would like to know what the precise rules about copy vs alias are ? Cheers, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Wed Oct 18 07:47:14 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Wed, 18 Oct 2006 13:47:14 +0200 Subject: questions regarding assignement and copy In-Reply-To: <45360904.3030703@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> Message-ID: <45361442.8040400@gmx.net> David Cournapeau schrieb: > Hi there, > > I've just managed to nail down a bug which took me nearly two whole > days to find: this is coming from an unexpected (at least from me) > behaviour of numpy. You have all my sympathy, I tripped over something similar not too long ago, so welcome to the club. > Now, if I do: > > bar += 1 > print bar is foo > > prints True > > But if I do bar = bar + 1, then bar is not a copy of foo anymore. Is > this intended ? This looks really confusing to me, and I would like to > know what the precise rules about copy vs alias are ? > Yes it's intended; as far as I understand the python/numpy syntax, <+> is an operator, and that triggers assignment by copy (even if you do something trivial as bar = +foo, you get a copy, if I'm not mistaken), while <+=> is syntactically not really an operator. AFAIK it's sole purpose is really to _not_ make a copy to save memory (well, it also looks nice ;-). You'll need somebody else to tell you the _precise_ rules, though... -sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Wed Oct 18 08:04:40 2006 From: faltet at carabos.com (Francesc Altet) Date: Wed, 18 Oct 2006 14:04:40 +0200 Subject: Question about boolean types comparisons Message-ID: <1161173081.3974.6.camel@localhost.localdomain> Hi, I've seen that bool type in numpy seems to work more consistently than its python counterpart: In [28]:True+True Out[28]:2 In [29]:numpy.bool_(True)+numpy.bool_(True) Out[29]:True Good! but... In [30]:True > False Out[30]:True In [31]:numpy.bool_(True) > numpy.bool_(False) Out[31]:True Perhaps raising an error saying something like "boolean types cannot be compared" would be nice. Not too important, but worth to notice, IMO. -- >0,0< Francesc Altet | Be careful about using the following code -- V V Carabos Coop. V. | I've only proven that it works, "-" Enjoy Data | I haven't tested it. -- Donald Knuth ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Wed Oct 18 08:04:50 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 18 Oct 2006 21:04:50 +0900 Subject: questions regarding assignement and copy In-Reply-To: <45361442.8040400@gmx.net> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> Message-ID: <45361862.6040801@ar.media.kyoto-u.ac.jp> Sven Schreiber wrote: > > Yes it's intended; as far as I understand the python/numpy syntax, <+> > is an operator, and that triggers assignment by copy (even if you do > something trivial as bar = +foo, you get a copy, if I'm not mistaken), > So basically, whenever you have foo = expr with expr is a numpy expression containing foo, you trigger a copy ? David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Wed Oct 18 09:50:45 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Wed, 18 Oct 2006 15:50:45 +0200 Subject: questions regarding assignement and copy In-Reply-To: <45361862.6040801@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> <45361862.6040801@ar.media.kyoto-u.ac.jp> Message-ID: <45363135.40200@gmx.net> David Cournapeau schrieb: > Sven Schreiber wrote: >> Yes it's intended; as far as I understand the python/numpy syntax, <+> >> is an operator, and that triggers assignment by copy (even if you do >> something trivial as bar = +foo, you get a copy, if I'm not mistaken), >> > So basically, whenever you have > > foo = expr > > with expr is a numpy expression containing foo, you trigger a copy ? > David, I would tend to confirm your conjecture, but given its generality I should not pretend to really know all about that stuff. It could be, for example, that the user may specify inside the expression that she wants a view, not a copy, But I don't know if that's really possible. Anybody else available to help out? -sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 18 10:30:02 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 18 Oct 2006 10:30:02 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: On 18/10/06, Charles R Harris wrote: > > > On 10/17/06, A. M. Archibald wrote: > > On 17/10/06, Charles R Harris wrote: > > > > > > > > > On 10/17/06, Travis Oliphant wrote: > > > > > > Thus, reshape does the equivalent of a Fortran ravel to [1,4,2,5,3,6] > > > > and then a Fortran-order based fill of an empty (3,2) array: giving > you > > > > the result. > > > > > > Why a Fortran ravel? I am thinking of it as preserving the memory > layout, > > > just messing with how it is addressed. > > > > Because, to first order, the memory layout is totally irrelevant to a > > python user. > > > > array([[1,2,3],[4,5,6]],order='C') > > array([[1,2,3],[4,5,6]],order='F') > > > > should be nearly identical to the python user. > > So what is the point of having a fortran layout if things are not actually > layed out in fortran order? As far as I can see, memory layout is the only > reason for fortran ordering. Now I can sort of see where the fortran ravel > comes from, because the result can be passed to a fortran routine. And it > does look like a.ravel('F') is the same as a.T.ravel(). Hmmm. Now I wonder > about this: Thins are laid out in Fortran order if you request Fortran order upon array creation. You just can't see it, normally. Numpy tries hard to make arrays of all different orders look identical. Of course, the reason I said "to first order" is that when determining when to copy and when not to, or when passing arrays to C or Fortran functions, sometimes it does matter. For example, a typical Fortran linear algebra routine needs its arrays to be in Fortran order in memory. > In [62]: array([[1,2,3],[4,5,6]], dtype=int8, order='F').flags > Out[62]: > CONTIGUOUS : False > FORTRAN : True > OWNDATA : True > WRITEABLE : True > ALIGNED : True > UPDATEIFCOPY : False Unless you're working with C extensions, it doesn't make much sense to ever look at flags. The memory layout does not affect normal array behaviour (including reshape). A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 18 10:55:26 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 18 Oct 2006 10:55:26 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4535B0F1.1000603@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> <4535B0F1.1000603@ieee.org> Message-ID: On 18/10/06, Travis Oliphant wrote: > > I just committed a change to the check-for-copy code to SVN. A copy > will occur if an actual reshaping is taking place that involves more > than just adding or deleting ones from the old shape and if > > 1) self is not "single-segment". Single-segment means either > Fortran-order or C-order. If we don't have a single-segment array, then > we have to get a (Fortran or C-) contiguous buffer to do any reshaping > (there may be special cases when it's possible, but I haven't figure > them out...) Indeed it is often possible to reshape an array that is not contiguous; for example zeros(100)[::10] is not contiguous but can be reshaped to any shape without any copying. More generally, there are all sorts of peculiar arrangements that can be reshaped witout a copy, for example: input has lengths (8,2,3) and strides (39,9,3); output should have lengths (4,4,3,2), so take strides (156,39,6,3) Whether this sort of thing actually *occurs* very often, I don't know... > 2) self is single-segment but self.squeeze().ndim > 1 and it is in the > "wrong" order from what was requested in the order argument (i.e. self > is in C-contiguous order but a Fortran-order reshape was requested or > self is in F-contiguous order but a C-order reshape was requested). > > If there are any cases satisfying these rules where a copy does not have > to occur then let me know. There are. I came up with a general condition for describing exactly when you need to copy an array, but it's difficult to put into words. Here goes: (0) Dimensions of length 1 are an irrelevant annoyance. Conceptually they should be squeezed out before beginning and newaxised back in at the end. (What strides should they have?) Assume that I've done so from now on. (Actual code can just skip them appropriately.) (1) If the array looks like an evenly-strided 1D array that has been reshaped (so strides[i+-1]=lengths[i]*strides[i] for all i) it can always be reshaped without a copy. (For example, zeros(1000)[::10] can be reshaped arbitrarily and as many times as desired without copying.) (2) If the array does *not* look like an evenly-strided 1D array that has been reshaped, you can still reshape it without a copy if it looks like an array of such arrays, each of which can be reshaped separately. (For example, you can view an array of length (8,2,3) that you have to resize to an array of size (4,4,3,2) as two separate pieces: an array of size (2,3) that you have to resize to (3,2) and an array of size (8,) that you need to resize to size (4,4). Each of these pieces separately needs to look like a reshaped 1d array, but there need be no relation between them.) (The condition for when you can pull a smaller resize operation off the end is precisely when you can find a tail segment of each tuple that has the same product; in the case above, 2*3=3*2, so we could stop and do that reshape separately.) (3) If the array cannot be reshaped without a copy using the rules above, it cannot be reshaped without a copy at all. The python code in my previous message actually implements this rule, if you want a less vague description. I should also point out that views are not always more efficient than copies (because of cache concerns, and because a small view can prevent a giant block of memory from being deallocated); nevertheless it should be up to the user to copy when it helps. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 18 11:08:15 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 18 Oct 2006 11:08:15 -0400 Subject: questions regarding assignement and copy In-Reply-To: <45361862.6040801@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> <45361862.6040801@ar.media.kyoto-u.ac.jp> Message-ID: On 18/10/06, David Cournapeau wrote: > Sven Schreiber wrote: > > > > Yes it's intended; as far as I understand the python/numpy syntax, <+> > > is an operator, and that triggers assignment by copy (even if you do > > something trivial as bar = +foo, you get a copy, if I'm not mistaken), > > > So basically, whenever you have > > foo = expr > > with expr is a numpy expression containing foo, you trigger a copy ? No. "=" never copies, but "+" does: when you do "A+B" you produce a new, freshly-allocated array, which you can then store (a reference to) in any variable you like, including A or B. So M = M+1 makes a copy because "M+1" is a new matrix, which you are assigning to M. Unfortunately, this means if you do: M = 2*(M+1) python makes the new matrix M+1, then makes the new matrix 2*(M+1), discards M+1, and sets M to point to 2*(M+1). If you want to avoid all this copying, you can use python's in-place operators: M += 1 M *= 2 This is actually a standard difficulty people have with python, made more obvious because you're working with mutable arrays rather than immutable scalars. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From rays at blue-cove.com Wed Oct 18 12:11:04 2006 From: rays at blue-cove.com (Ray Schumacher) Date: Wed, 18 Oct 2006 09:11:04 -0700 Subject: No forums found for Numerical Python Message-ID: <6.2.3.4.2.20061018090850.05a362e0@blue-cove.com> FYI: at http://numpy.org/, the link "Forums" (http://sourceforge.net/forum/?group_id=1369) gives No forums found for Numerical Python ! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 18 12:38:32 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 10:38:32 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4535AEFB.4090202@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> Message-ID: On 10/17/06, Travis Oliphant wrote: > > Stefan van der Walt wrote: > > One last case, which confuses me still (probably because it is > > 04:16am): > > Please continue to question. All the code needs as much review as it > can get. I am really starting to wonder why we need an order keyword at all except when order matters for interfacing, i.e., in contiguos arrays where one wants fortran contiguous or c contiguous. For output type stuff: tofile tostring ravel flatten These all need the keyword so that they can be used to produce files and arrays to interface to fortran. Right now the fortran interface can be achieved by: a.T.tofile a.T.tostring a.T.ravel a.T.flatten The order keyword is just syntatic sugar that makes the intent clearer. For the input type stuff fromfile fromstring reshape-1d-array (unravel?) Currently, the key operation is reshape, which only needs to return a view in fortran order and doesn't even need to mark the resulting array as fortran order because, well, because it works just fine in numpy as is, it just isn't contiguous. If the other functions took shape and order, reshape wouldn't even need the order keyword. I don't see why the array constructor needs the order keyword, it doesn't *do* anything. For instance a = array([[1,2,3],[4,5,6]], order='F') doesn't produce a fortran contiguous array, it produces the same array as the 'C' form, just sets the fortran flag and marks contiguous as False. What is the use of that? It is just a generic non-contiguous numpy array. And In [131]: ascontiguousarray(array([[1,2,3],[4,5,6]], dtype=int8, order='F')).flags Out[131]: CONTIGUOUS : True FORTRAN : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False Doesn't produce a fortran contiguous array, so what use was the flag? And In [141]: array([1,2,3,4,5,6], dtype=int8).reshape((2,3), order='F').astype(int16).flags Out[141]: CONTIGUOUS : True FORTRAN : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False reorders stuff in memory, so is a bug looking to happen in a fortran interface. mmapped files are the only thing I can think of where one might want vary an operation depending on Fortran ordering because seeking out of order is very expensive. But that means adapting algorithms depending on order type, better I think to just stick to using the small strided dimensions when appropriate. It would be helpful in debugging all this order stuff if it was clear what was supposed to happen in every case. Ravel, for instance, ignores the FORTRAN flag, again begging the question as to why we *have* the flag. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 18 12:49:38 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 10:49:38 -0600 Subject: questions regarding assignement and copy In-Reply-To: <45361862.6040801@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> <45361862.6040801@ar.media.kyoto-u.ac.jp> Message-ID: On 10/18/06, David Cournapeau wrote: > > Sven Schreiber wrote: > > > > Yes it's intended; as far as I understand the python/numpy syntax, <+> > > is an operator, and that triggers assignment by copy (even if you do > > something trivial as bar = +foo, you get a copy, if I'm not mistaken), > > > So basically, whenever you have > > foo = expr > > with expr is a numpy expression containing foo, you trigger a copy ? Yes. Numpy generates a temporary where the expression results are kept, the temporary is then assigned to foo. This can be inefficient, but knowing when you may overwrite data can be complicated. For instance: foo = dot(foo,foo) Can't be done in place because values needed later on in the dot evaluation would be overwritten. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Wed Oct 18 12:55:51 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 18 Oct 2006 11:55:51 -0500 Subject: sort(axis=-1) In-Reply-To: References: Message-ID: Christian Kristukat wrote: > Hi, > > it seems that -1 as axis parameter is interpreted like in array indexing, i.e. > -1 means the last axis rather than meaning the only axis of the flattened > representation, like for example with take(). > Is that intendend? take(axis=-1) also goes over the last axis. axis=None operates over the flattened array. axis=None is the default for take(), but axis=-1 is default for sort() since axis=None doesn't make much sense for an inplace operation and is not permitted. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 13:11:25 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 11:11:25 -0600 Subject: sort(axis=-1) In-Reply-To: References: Message-ID: <4536603D.7070103@ieee.org> Christian Kristukat wrote: > Hi, > > it seems that -1 as axis parameter is interpreted like in array indexing, i.e. > -1 means the last axis rather than meaning the only axis of the flattened > representation, like for example with take(). > Is that intendend? > > I don't understand the question. Yes, sort(axis=-1) sorts along the last dimension. AFAIK, take has the same meaning for axis. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Wed Oct 18 13:12:37 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 18 Oct 2006 12:12:37 -0500 Subject: questions regarding assignement and copy In-Reply-To: <45360904.3030703@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> Message-ID: David Cournapeau wrote: > bar += 1 > print bar is foo > > prints True > > But if I do bar = bar + 1, then bar is not a copy of foo anymore. Is > this intended ? Yes, very much so! If it were otherwise Guido would never have added the augmented assignment operators. From the reference manual: http://docs.python.org/ref/augassign.html """An augmented assignment expression like x += 1 can be rewritten as x = x + 1 to achieve a similar, but not exactly equal effect. In the augmented version, x is only evaluated once. Also, when possible, the actual operation is performed in-place, meaning that rather than creating a new object and assigning that to the target, the old object is modified instead.""" > This looks really confusing to me, and I would like to > know what the precise rules about copy vs alias are ? I'm not going to go through all of the functions in numpy, but here are some (unorganized) rules of thumb: * Any operations with regular operators + - / * // ** ~ ^ & | << >> < > == <=, etc. will put their results into new arrays. If the expression is on the RHS of a regular assignment, just ignore the LHS of the assignment; even if the name on the LHS is the same as a name on the RHS, the expression is evaluated, the new array is created and *then* the new array is assigned to the LHS name. * Augmented assignment operators will operate in-place. * Functions and methods should state that they operate in-place in their docstrings. * The asarray(), asanyarray(), etc. functions will return the input array if it is already suitable. * Slicing, reshaping, transposing and similar structural operations create new array objects, but they are generally views, not copies of the data. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 13:16:17 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 11:16:17 -0600 Subject: questions regarding assignement and copy In-Reply-To: <45361862.6040801@ar.media.kyoto-u.ac.jp> References: <45360904.3030703@ar.media.kyoto-u.ac.jp> <45361442.8040400@gmx.net> <45361862.6040801@ar.media.kyoto-u.ac.jp> Message-ID: <45366161.8000607@ieee.org> David Cournapeau wrote: > Sven Schreiber wrote: > >> Yes it's intended; as far as I understand the python/numpy syntax, <+> >> is an operator, and that triggers assignment by copy (even if you do >> something trivial as bar = +foo, you get a copy, if I'm not mistaken), >> >> > So basically, whenever you have > > foo = expr > > with expr is a numpy expression containing foo, you trigger a copy ? > I think you are better off understanding that "=" is a name binding operation while "+=" calls a special method that allows in-place adding. Thus, bar += foo calls bar.__iadd__(bar, foo) which gives the opportunity to add foo to bar in-place while bar = bar + foo adds bar to foo (which results in a new array that then gets re-bound to the name bar). The '=' sign is not an operator. Perhaps this will help you see the difference. It's good to know what kinds of things trip people up. We can target tutorials and FAQs to those things. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 13:19:08 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 11:19:08 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358E31.5010909@ieee.org> Message-ID: <4536620C.6090903@ieee.org> Charles R Harris wrote: > > > On 10/17/06, *Charles R Harris* > wrote: > > > > On 10/17/06, *A. M. Archibald* < peridot.faceted at gmail.com > > wrote: > > On 17/10/06, Charles R Harris > wrote: > > > > > > On 10/17/06, Travis Oliphant < oliphant.travis at ieee.org > > wrote: > > > > > Which doesn't seem to be the case here. I am beginning to wonder > if we really need fortran order, seems that a few well chosen > interface routines would fill the need and avoid much confusion. > > > For instance, it would be nice if flatten took an order keyword: > > In [107]: array([[1,2,3],[4,5,6]], dtype=int8, order='F').flatten() > Out[107]: array([1, 2, 3, 4, 5, 6], dtype=int8) It does take an argument (just not a keyword argument). The general rule I followed (probably inappropriately) was that single-argument methods didn't need keywords. so a.flatten('F') gives you a Fortran-order flattening. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Wed Oct 18 13:15:32 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 18 Oct 2006 12:15:32 -0500 Subject: No forums found for Numerical Python In-Reply-To: <6.2.3.4.2.20061018090850.05a362e0@blue-cove.com> References: <6.2.3.4.2.20061018090850.05a362e0@blue-cove.com> Message-ID: Ray Schumacher wrote: > FYI: > at http://numpy.org/, the link "Forums" ( > http://sourceforge.net/forum/?group_id=1369) gives > > *No forums found for Numerical Python* > > ! Yes, I closed them since people would go there for help, but the people that *could* help don't read the forums. If I knew how to make the Forums link go away on the project page, I would do so. If you have questions about numpy (or even Numeric and numarray), this is the place. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 13:36:02 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 11:36:02 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> Message-ID: <45366602.4090704@ieee.org> > > Currently, the key operation is reshape, which only needs to return a > view in fortran order and doesn't even need to mark the resulting > array as fortran order because, well, because it works just fine in > numpy as is, it just isn't contiguous. If the other functions took > shape and order, reshape wouldn't even need the order keyword. The flag is the there as a quick check for interfacing. The order keyword grew because it was useful to avoid the arbitrariness of C-contiguous order for those who prefer to think of it differently. Remember the .T attribute for .transpose() was a recent addition and sticking .transpose() everywhere is a lot more ugly. But, yes, many uses of the order keyword could be replaced by preceding with .transpose() --- this is not without cost, however. > > I don't see why the array constructor needs the order keyword, it > doesn't *do* anything. For instance > > a = array([[1,2,3],[4,5,6]], order='F') > > doesn't produce a fortran contiguous array, it produces the same array > as the 'C' form, just sets the fortran flag and marks contiguous as > False. What is the use of that? It is just a generic non-contiguous > numpy array. What? You're not understanding something. The order flag definitely does something here. First of all it seems like you are not understanding the meaning of the CONTIGUOUS flag. CONTIGUOUS means "C-order contiguous" while FORTRAN means "FORTRAN-order contiguous". That's why I use the word single-segment to talk about FORTRAN-order or C-contiguous order. For Numeric, CONTIGUOUS always meant C-order contiguous and we are continuing that tradition. All we've done is notice that there is such a think as FORTRAN-order contiguous and copies do not need to be made in all circumstances when you have FORTRAN-order. Look at the difference between: a = array([[1,2,3],[4,5,6]],order='F').data[:] b = array([[1,2,3],[4,5,6]]).data[:] Notice the layout is definitely different between a and b. > And > > In [131]: ascontiguousarray(array([[1,2,3],[4,5,6]], dtype=int8, > order='F')).flags > Out[131]: > CONTIGUOUS : True > FORTRAN : False > OWNDATA : True > WRITEABLE : True > ALIGNED : True > UPDATEIFCOPY : False > > Doesn't produce a fortran contiguous array, so what use was the flag? And Because you requested a C-contiguous array --- that's what contiguous means in NumPy (exactly what it meant in Numeric). > > In [141]: array([1,2,3,4,5,6], dtype=int8).reshape((2,3), > order='F').astype(int16).flags > Out[141]: > CONTIGUOUS : True > FORTRAN : False > OWNDATA : True > WRITEABLE : True > ALIGNED : True > UPDATEIFCOPY : False > > reorders stuff in memory, so is a bug looking to happen in a fortran > interface. Yes, like I said before, all kinds of operations alter the "layout" of data. You can't assume all operations will preserve FORTRAN ordering. FORTRAN-order has meaning beyond how the data is actually set out in memory. Sometimes it indicates how you think it is layed out when you are doing re-shaping operations. > > mmapped files are the only thing I can think of where one might want > vary an operation depending on Fortran ordering because seeking out of > order is very expensive. But that means adapting algorithms depending > on order type, better I think to just stick to using the small strided > dimensions when appropriate. > > It would be helpful in debugging all this order stuff if it was clear > what was supposed to happen in every case. Ravel, for instance, > ignores the FORTRAN flag, again begging the question as to why we > *have* the flag. No it doesn't. Please show your evidence. Look: a = array([[1,2,3],[4,5,6]]) print a.ravel() [1 2 3 4 5 6] print a.ravel('F') [1 4 2 5 3 6] If it's not working in some cases, please report that as a bug. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 13:47:59 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 11:47:59 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45366602.4090704@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> Message-ID: On 10/18/06, Travis Oliphant wrote: > > > > > > Currently, the key operation is reshape, which only needs to return a > > view in fortran order and doesn't even need to mark the resulting > > array as fortran order because, well, because it works just fine in > > numpy as is, it just isn't contiguous. If the other functions took > > shape and order, reshape wouldn't even need the order keyword. > The flag is the there as a quick check for interfacing. The order > keyword grew because it was useful to avoid the arbitrariness of > C-contiguous order for those who prefer to think of it differently. > Remember the .T attribute for .transpose() was a recent addition and > sticking .transpose() everywhere is a lot more ugly. But, yes, many > uses of the order keyword could be replaced by preceding with > .transpose() --- this is not without cost, however. > > > > > I don't see why the array constructor needs the order keyword, it > > doesn't *do* anything. For instance > > > > a = array([[1,2,3],[4,5,6]], order='F') > > > > doesn't produce a fortran contiguous array, it produces the same array > > as the 'C' form, just sets the fortran flag and marks contiguous as > > False. What is the use of that? It is just a generic non-contiguous > > numpy array. > > What? You're not understanding something. The order flag definitely > does something here. First of all it seems like you are not > understanding the meaning of the CONTIGUOUS flag. CONTIGUOUS means > "C-order contiguous" while FORTRAN means "FORTRAN-order contiguous". > That's why I use the word single-segment to talk about FORTRAN-order or > C-contiguous order. For Numeric, CONTIGUOUS always meant C-order > contiguous and we are continuing that tradition. All we've done is > notice that there is such a think as FORTRAN-order contiguous and copies > do not need to be made in all circumstances when you have FORTRAN-order. > > Look at the difference between: > > a = array([[1,2,3],[4,5,6]],order='F').data[:] > > b = array([[1,2,3],[4,5,6]]).data[:] > > Notice the layout is definitely different between a and b. > > > And > > > > In [131]: ascontiguousarray(array([[1,2,3],[4,5,6]], dtype=int8, > > order='F')).flags > > Out[131]: > > CONTIGUOUS : True > > FORTRAN : False > > OWNDATA : True > > WRITEABLE : True > > ALIGNED : True > > UPDATEIFCOPY : False > > > > Doesn't produce a fortran contiguous array, so what use was the flag? > And > > Because you requested a C-contiguous array --- that's what contiguous > means in NumPy (exactly what it meant in Numeric). > > > > > In [141]: array([1,2,3,4,5,6], dtype=int8).reshape((2,3), > > order='F').astype(int16).flags > > Out[141]: > > CONTIGUOUS : True > > FORTRAN : False > > OWNDATA : True > > WRITEABLE : True > > ALIGNED : True > > UPDATEIFCOPY : False > > > > reorders stuff in memory, so is a bug looking to happen in a fortran > > interface. > > Yes, like I said before, all kinds of operations alter the "layout" of > data. You can't assume all operations will preserve FORTRAN ordering. > FORTRAN-order has meaning beyond how the data is actually set out in > memory. Sometimes it indicates how you think it is layed out when you > are doing re-shaping operations. > > > > > mmapped files are the only thing I can think of where one might want > > vary an operation depending on Fortran ordering because seeking out of > > order is very expensive. But that means adapting algorithms depending > > on order type, better I think to just stick to using the small strided > > dimensions when appropriate. > > > > It would be helpful in debugging all this order stuff if it was clear > > what was supposed to happen in every case. Ravel, for instance, > > ignores the FORTRAN flag, again begging the question as to why we > > *have* the flag. > No it doesn't. Please show your evidence. Look: > > a = array([[1,2,3],[4,5,6]]) > > print a.ravel() > [1 2 3 4 5 6] > > print a.ravel('F') > [1 4 2 5 3 6] I'm not talking about the keyword in the ravel call, I'm talking about the flag in a. The question is: do we *need* a fortran flag. I am argueing not, because the only need is for fortran contiguous arrays to pass to fortran function, or translation from fortran contiguous arrays to numpy arrays. What I am saying is that things are unnecessarily complicated. None of the LaPack stuff seems to use the Fortran stuff, they just transpose and copy. I don't even think I want to change that, because it is *clear* what is going on. Interfacing to fortran is all about memory layout, nothing more or less. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Wed Oct 18 13:51:04 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 18 Oct 2006 10:51:04 -0700 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> Message-ID: <45366988.5020204@ieee.org> One thing that may be confusing the issue is that, as I understand it, FORTRAN and CONTIGUOUS together represent three states which I'll call FORTRAN_ORDER, C_ORDER and DISCONTIGUOUS. I periodically wonder if it would be valuable to have a way to query the order directly: the result would be "C", "F" or None, just like the order keyword that is passed in. This might well eliminate sine confusion. However, 99% of the time the order just doesn't matter, so it's probably pointless. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Wed Oct 18 13:58:23 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 18 Oct 2006 10:58:23 -0700 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> Message-ID: <45366B3F.2050103@ieee.org> Charles R Harris wrote: [SNIP] > > I'm not talking about the keyword in the ravel call, I'm talking about > the flag in a. The question is: do we *need* a fortran flag. I am > argueing not, because the only need is for fortran contiguous arrays > to pass to fortran function, or translation from fortran contiguous > arrays to numpy arrays. What I am saying is that things are > unnecessarily complicated. None of the LaPack stuff seems to use the > Fortran stuff, they just transpose and copy. I don't even think I want > to change that, because it is *clear* what is going on. Interfacing to > fortran is all about memory layout, nothing more or less. > Chuck, There are two things here. One is the order keyword and one is the FORTRAN flag. The latter is mainly an optimization for use at the C-level so that one doesn't have to check whether a given array is in contiguous FORTRAN order by examining the strides, in the same way that the CONTIGUOUS flag allows you to skip examining the strides when you need a contiguous C-order matrix. I believe it is the former that you are objecting to, but it would help if you could specify whether you are talking about the order keyword or whether you are talking about the FORTRAN flag. I'll also note that the order keyword could probably have been used to fix a performance problem someone was having a few weeks ago. We ended up transposing the data, but the individual felt that obscured the intent of the algorithm. I believe the same effect could probably have been been achieved without re jiggering the algorithm by using the order parameter. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From erin.sheldon at gmail.com Wed Oct 18 14:00:20 2006 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Wed, 18 Oct 2006 14:00:20 -0400 Subject: histogram complete makeover In-Reply-To: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> References: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> Message-ID: <331116dc0610181100k474af71akac41e54db7b7bb41@mail.gmail.com> On 10/17/06, David Huard wrote: > Hi all, > > I'd like to poll the list to see what people want from numpy.histogram(), > since I'm currently writing a contender. > > My main complaints with the current version are: > 1. upper outliers are stored in the last bin, while lower outliers are not > counted at all, > 2. cannot use weights. > > The new histogram function is well under way (it address these issues and > adds an axis keyword), > but I want to know what is the preferred behavior regarding the function > output, and your > willingness to introduce a new behavior that will break some code. > > Given a number of bins N and range (min, max), histogram constructs linearly > spaced bin edges > b0 (out-of-range) | b1 | b2 | b3 | .... | bN | bN+1 out-of-range > and may return: > > A. H = array([N_b0, N_b1, ..., N_bN, N_bN+1]) > The out-of-range values are the first and last values of the array. The > returned array is hence N+2 > > B. H = array([N_b0 + N_b1, N_b2, ..., N_bN + N_bN+1]) > The lower and upper out-of-range values are added to the first and last bin > respectively. > > C. H = array([N_b1, ..., N_bN + N_bN+1]) > Current behavior: the upper out-of-range values are added to the last bin. > > D. H = array([N_b1, N_b2, ..., N_bN]), > Lower and upper out-of-range values are given after the histogram array. > > Ideally, the new function would not break the common usage: H = > histogram(x)[0], so this exclude A. B and C are not acceptable in my > opinion, so only D remains, with the downsize that the outliers are not > returned. A solution might be to add a keyword full_output=False, which when > set to True, returns the out-of-range values in a dictionnary. > > Also, the current function returns -> H, ledges > where ledges is the array of left bin edges (N). > I propose returning the complete array of edges (N+1), including the > rightmost edge. This is a little bit impractical for plotting, as the edges > array does not have the same length as the histogram array, but allows the > use of user-defined non-uniform bins. > > Opinions, suggestions ? I dislike the current behavior. I don't want the histogram to count anything outside the range I specify. It would also be nice to allow specification of a binsize which would be used if number of bins wasn't sent. Personally, since I don't have any code yet that uses histogram, I feel like edges could be returned in a keyword. Perhaps in a dictionary with other useful items, such as bin middles, mean of the data in bins and other statistics, or whatever, which would only be calculated if the keyword dict was sent. Hopefully Google and sourceforge are playing nice and you will see this within a day of sending. Erin ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 14:08:07 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 12:08:07 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45366B3F.2050103@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <45366B3F.2050103@ieee.org> Message-ID: On 10/18/06, Tim Hochberg wrote: > > Charles R Harris wrote: > > [SNIP] > > > > I'm not talking about the keyword in the ravel call, I'm talking about > > the flag in a. The question is: do we *need* a fortran flag. I am > > argueing not, because the only need is for fortran contiguous arrays > > to pass to fortran function, or translation from fortran contiguous > > arrays to numpy arrays. What I am saying is that things are > > unnecessarily complicated. None of the LaPack stuff seems to use the > > Fortran stuff, they just transpose and copy. I don't even think I want > > to change that, because it is *clear* what is going on. Interfacing to > > fortran is all about memory layout, nothing more or less. > > > > Chuck, > > There are two things here. One is the order keyword and one is the > FORTRAN flag. The latter is mainly an optimization for use at the > C-level so that one doesn't have to check whether a given array is in > contiguous FORTRAN order by examining the strides, in the same way that > the CONTIGUOUS flag allows you to skip examining the strides when you > need a contiguous C-order matrix. That sounds like the two flags should be named f-contiguous and c-contiguous. Then they would be orthogonal and one could have all four combinations. Is that the case now? Perhaps I am misunderstanding the meaning of the flags. I believe it is the former that you > are objecting to, but it would help if you could specify whether you are > talking about the order keyword or whether you are talking about the > FORTRAN flag. Both. I was argueing against the FORTRAN flag, and of limiting the order keyword to those cases where f or c contiguous arrays were the output or input. I'll also note that the order keyword could probably have been used to > fix a performance problem someone was having a few weeks ago. We ended > up transposing the data, but the individual felt that obscured the > intent of the algorithm. I believe the same effect could probably have > been been achieved without re jiggering the algorithm by using the order > parameter. Some more details would be helpful. It would be good to know what problem the order keyword should solve. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Wed Oct 18 14:27:38 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 18 Oct 2006 11:27:38 -0700 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <45366B3F.2050103@ieee.org> Message-ID: <4536721A.4050104@ieee.org> Charles R Harris wrote: > > > On 10/18/06, *Tim Hochberg* > wrote: > > Charles R Harris wrote: > > [SNIP] > > > > I'm not talking about the keyword in the ravel call, I'm talking > about > > the flag in a. The question is: do we *need* a fortran flag. I am > > argueing not, because the only need is for fortran contiguous > arrays > > to pass to fortran function, or translation from fortran contiguous > > arrays to numpy arrays. What I am saying is that things are > > unnecessarily complicated. None of the LaPack stuff seems to use > the > > Fortran stuff, they just transpose and copy. I don't even think > I want > > to change that, because it is *clear* what is going on. > Interfacing to > > fortran is all about memory layout, nothing more or less. > > > > Chuck, > > There are two things here. One is the order keyword and one is the > FORTRAN flag. The latter is mainly an optimization for use at the > C-level so that one doesn't have to check whether a given array is in > contiguous FORTRAN order by examining the strides, in the same way > that > the CONTIGUOUS flag allows you to skip examining the strides when you > need a contiguous C-order matrix. > > > That sounds like the two flags should be named f-contiguous and > c-contiguous. Then they would be orthogonal and one could have all > four combinations. Is that the case now? Perhaps I am misunderstanding > the meaning of the flags. That is the case now. The flag names simply mirror their values in C. Why they have those names in something of a historical accident I believe. Take a look at this: >>> array([1,2,3,4]).flags CONTIGUOUS : True FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False >>> array([1,2,3,4])[::2].flags CONTIGUOUS : False FORTRAN : False OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False >>> array([[1,2],[3,4]]).flags CONTIGUOUS : True FORTRAN : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False >>> array([[1,2],[3,4]], order='F').flags CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False See, all four combinations. I guess my previous post was wrong -- there really are four combinations not three like I said, but they are C-order, Fortran-order, Both and neither. I forgot what the flags signified and had to play with them for a bit to remember. > I believe it is the former that you > are objecting to, but it would help if you could specify whether > you are > talking about the order keyword or whether you are talking about the > FORTRAN flag. > > > Both. I was argueing against the FORTRAN flag, and of limiting the > order keyword to those cases where f or c contiguous arrays were the > output or input. > > I'll also note that the order keyword could probably have been used to > fix a performance problem someone was having a few weeks ago. We > ended > up transposing the data, but the individual felt that obscured the > intent of the algorithm. I believe the same effect could probably have > been been achieved without re jiggering the algorithm by using the > order > parameter. > > > Some more details would be helpful. It would be good to know what > problem the order keyword should solve. > Well, in general, memory layout can be important for performance not just for interfacing with Fortran. You can do this with suitable applications of transpose, but using the order flag is probably clearer. Particularly, if you are trying to match a textbook algorithm, its nice to have the axes in the same places. I'm just moving over from numarray which didn't have the equivalent of the order flag as far as I know, so I don't have experience with this at this point though. Here is one of the posts in questions: > David Cournapeau wrote: > Hi, > > I was wondering if there was any way to speed up the following code: > > y = N.zeros((n, K)) > for i in range(K): > y[:, i] = gauss_den(data, mu[i, :], va[i, :]) > > Where K is of order 1e1, n of order 1e5. Normally, gauss_den is a > quite expensive function, but the profiler tells me that the indexing > y[:,i] takes almost as much time as the gauss_den computation (which > computes n exp !). To see if the profiler is "right", i replaces with > the (non valid) following function: > > y = N.zeros((n, K)) > for i in range(K): > yt = gauss_den(data, mu[i, :], va[i, :]) > return y > > Where more than 99% of the code is spent inside gauss_den. > > I guess the problem is coming from the fact that y being C order, y[:, > i] needs accessing data in a non 'linear' way. Is there a way to speed > this up ? I did something like this: > > y = N.zeros((K, n)) > for i in range(K): > y[i] = gauss_den(data, mu[i, :], va[i, :]) > return y.T > > which works, but I don't like it very much. I believe that the same efficiency as the last could have been achieved using something like: y = N.zeros((n,K), order='F') for i in range(K): y[:,i] = gauss_den(data, mu[i, :], va[i, :]) return y This probably would have made the original poster happier. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Wed Oct 18 14:29:17 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed, 18 Oct 2006 11:29:17 -0700 Subject: histogram complete makeover In-Reply-To: <331116dc0610181100k474af71akac41e54db7b7bb41@mail.gmail.com> References: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> <331116dc0610181100k474af71akac41e54db7b7bb41@mail.gmail.com> Message-ID: <4536727D.4020308@ieee.org> My $0.02: If histogram is going to get a makeover, particularly one that makes it more complex than at present, it should probably be moved to SciPy. Failing that, it should be moved to a submodule of numpy with similar statistical tools. Preferably with consistent interfaces for all of the functions. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 15:10:16 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 13:10:16 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536721A.4050104@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <45366B3F.2050103@ieee.org> <4536721A.4050104@ieee.org> Message-ID: On 10/18/06, Tim Hochberg wrote: > > Charles R Harris wrote: > > > > > > On 10/18/06, *Tim Hochberg* > > wrote: > > > > Charles R Harris wrote: > > > > [SNIP] > > > > > > I'm not talking about the keyword in the ravel call, I'm talking > > about > > > the flag in a. The question is: do we *need* a fortran flag. I am > > > argueing not, because the only need is for fortran contiguous > > arrays > > > to pass to fortran function, or translation from fortran > contiguous > > > arrays to numpy arrays. What I am saying is that things are > > > unnecessarily complicated. None of the LaPack stuff seems to use > > the > > > Fortran stuff, they just transpose and copy. I don't even think > > I want > > > to change that, because it is *clear* what is going on. > > Interfacing to > > > fortran is all about memory layout, nothing more or less. > > > > > > > Chuck, > > > > There are two things here. One is the order keyword and one is the > > FORTRAN flag. The latter is mainly an optimization for use at the > > C-level so that one doesn't have to check whether a given array is > in > > contiguous FORTRAN order by examining the strides, in the same way > > that > > the CONTIGUOUS flag allows you to skip examining the strides when > you > > need a contiguous C-order matrix. > > > > > > That sounds like the two flags should be named f-contiguous and > > c-contiguous. Then they would be orthogonal and one could have all > > four combinations. Is that the case now? Perhaps I am misunderstanding > > the meaning of the flags. > That is the case now. The flag names simply mirror their values in C. > Why they have those names in something of a historical accident I > believe. Take a look at this: OK, that is good. I no longer have any objection to the flags, I just wish the names were more descriptive of what they mean. In fact, it looks like the following sort of construction will be useful in the linalg module. In [17]:a = array([[1,2],[3,4]], dtype=int) In [18]:b = array(a, dtype=double, order='f') In [19]:b.flags Out[19]: CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False I've been a pain in the a** because I really want to know what is going on down in the boiler room. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From chanley at stsci.edu Wed Oct 18 15:10:08 2006 From: chanley at stsci.edu (Christopher Hanley) Date: Wed, 18 Oct 2006 15:10:08 -0400 Subject: scalar object bug or feature? Message-ID: <45367C10.40606@stsci.edu> Greetings, I can't decide if the following is a bug or feature. Numpy scalars are allowed to overflow silently while numarray upcasts to a larger python type. I guess my biggest problem is that the overflow occurs silently. In any case, is this known and expected behavior? Thanks, Chris NUMARRAY example: In [1]: import numarray as n In [2]: a = n.array([2200,14000],type=n.UInt32) In [3]: a0= a[0] In [4]: a1 = a[1] In [5]: adiff = a0-a1 In [6]: print a0,a1,adiff 2200 14000 -11800 In [7]: print type(a0),type(a1),type(adiff) NUMPY example: In [1]: import numpy as n In [2]: a = n.array([2200,14000],dtype=n.uint32) In [3]: a0= a[0] In [4]: a1 = a[1] In [5]: adiff = a0-a1 In [6]: print a0,a1,adiff 2200 14000 4294955496 In [7]: print type(a0),type(a1),type(adiff) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Wed Oct 18 15:31:43 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Wed, 18 Oct 2006 12:31:43 -0700 Subject: scalar object bug or feature? In-Reply-To: <45367C10.40606@stsci.edu> References: <45367C10.40606@stsci.edu> Message-ID: On 10/18/06, Christopher Hanley wrote: > I can't decide if the following is a bug or feature. Numpy scalars are > allowed to overflow silently while numarray upcasts to a larger python > type. I guess my biggest problem is that the overflow occurs silently. I had that problem yesterday, which was difficult to diagnose for a new user like me. Here's an example: >> x = zeros((300, 300)) >> x = x > 1 # False >> x = 1 - x # ones >> y = x.T * x >> y[0,0] 44 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 14:33:49 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 12:33:49 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <45366988.5020204@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> Message-ID: <4536738D.7070701@ieee.org> Tim Hochberg wrote: > One thing that may be confusing the issue is that, as I understand it, > FORTRAN and CONTIGUOUS together represent three states which I'll call > FORTRAN_ORDER, C_ORDER and DISCONTIGUOUS. Yep, that's what they mean. CONTIGUOUS is the name Numeric gave it and it meant C-order contiguous. We have kept the same meaning. All we've done is selected out from the class of arrays that Numeric called DISTCONTIGUOUS, arrays that are FORTRAN-order (and so still single-segment), but discontiguous in the sense that Numeric had. > I periodically wonder if it > would be valuable to have a way to query the order directly: the result > would be "C", "F" or None, just like the order keyword that is passed > in. You an do it with the flags a.flags.contiguous a.flags.fortran Discontiguous is when both of these are false. Note that for a.ndim < 2, both a.flags.contiguous and a.flags.fortran are true if one of them is true. This is all explained in the first chapters of my book. You have to understand CONTIGUOUS == C-order contiguous and FORTRAN == Fortran-order contiguous. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 14:28:40 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 12:28:40 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> Message-ID: <45367258.8070503@ieee.org> > > I'm not talking about the keyword in the ravel call, I'm talking about > the flag in a. Ah. Yes, I see. I misunderstood. Of course ravel ignores the FORTRAN flag (actually it doesn't because if a copy is not necessary it doesn't make one). The key is that the Python user doesn't need to care about the array flag unless they are interfacing to compiled code. That's the point of the flag. It's actually redundant because it could be checked every time it's needed. But, right now, it's kept updated so that the check is simple. The same is true with the C-CONTIGUOUS flag (called contiguous). > The question is: do we *need* a fortran flag. No, you don't *need* the flag. But, it saves copying data to check it (look how many times ISFORTRAN is called in the code). Without the flag all of those cases would need to do a strides-check which is done in the UpdateFlags code. > I am argueing not, because the only need is for fortran contiguous > arrays to pass to fortran function, or translation from fortran > contiguous arrays to numpy arrays. What I am saying is that things are > unnecessarily complicated. I disagree. It's actually not that complicated. The FORTRAN flag gives us a lot more flexibility when it comes to copying data or not. I think part of the complication is that you are misunderstanding some of the terms and the purposes of the keywords. > None of the LaPack stuff seems to use the Fortran stuff, they just > transpose and copy. It doesn't now only because I haven't had time to go through and change it, but it should. Look at scipy's LaPack interface. It (through f2py) uses the FORTRAN stuff extensively (much was borrowed from there in the first place). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 18 14:31:18 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 18 Oct 2006 12:31:18 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> Message-ID: <453672F6.9070307@ieee.org> > > I'm not talking about the keyword in the ravel call, I'm talking about > the flag in a. Ah. Yes, I see. I misunderstood. Of course ravel ignores the FORTRAN flag (actually it doesn't because if a copy is not necessary it doesn't make one). The key is that the Python user doesn't need to care about the array flag unless they are interfacing to compiled code. That's the point of the flag. It's actually redundant because it could be checked every time it's needed. But, right now, it's kept updated so that the check is simple. The same is true with the C-CONTIGUOUS flag (called contiguous). > The question is: do we *need* a fortran flag. No, you don't *need* the flag. But, it saves copying data to check it (look how many times ISFORTRAN is called in the code). Without the flag all of those cases would need to do a strides-check which is done in the UpdateFlags code. > I am argueing not, because the only need is for fortran contiguous > arrays to pass to fortran function, or translation from fortran > contiguous arrays to numpy arrays. What I am saying is that things are > unnecessarily complicated. I disagree. It's actually not that complicated. Even if it was compilcated to implement, the point is that it is now done. There is no sense ripping it out (that would be a huge pain and for what purpose?) The FORTRAN flag gives us a lot more flexibility when it comes to copying data or not. I think part of the complication is that you are misunderstanding some of the terms and the purposes of the keywords. > None of the LaPack stuff seems to use the Fortran stuff, they just > transpose and copy. It doesn't now only because I haven't had time to go through and change it, but it should. Look at scipy's LaPack interface. It (through f2py) uses the FORTRAN stuff extensively (much was borrowed from there in the first place). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Wed Oct 18 16:25:30 2006 From: aisaac at american.edu (Alan G Isaac) Date: Wed, 18 Oct 2006 16:25:30 -0400 Subject: scalar object bug or feature? In-Reply-To: References: <45367C10.40606@stsci.edu> Message-ID: On Wed, 18 Oct 2006, Keith Goodman apparently wrote: > Here's an example: >>> x = zeros((300, 300)) >>> x = x > 1 # False >>> x = 1 - x # ones >>> y = x.T * x >>> y[0,0] > 44 Here's a simpler (?) example: >>> x=numpy.random.rand(300,1)>0 >>> x.sum() 300 >>> sum(x) array([44], dtype=int8) >>> x=numpy.random.rand(300)>0 >>> sum(x) 300 Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 16:51:12 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 14:51:12 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536738D.7070701@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> Message-ID: Travis, On 10/18/06, Travis Oliphant wrote: > > Tim Hochberg wrote: > > One thing that may be confusing the issue is that, as I understand it, > > FORTRAN and CONTIGUOUS together represent three states which I'll call > > FORTRAN_ORDER, C_ORDER and DISCONTIGUOUS. > > Yep, that's what they mean. CONTIGUOUS is the name Numeric gave it and > it meant C-order contiguous. We have kept the same meaning. All we've > done is selected out from the class of arrays that Numeric called > DISTCONTIGUOUS, arrays that are FORTRAN-order (and so still > single-segment), but discontiguous in the sense that Numeric had. > > > I periodically wonder if it > > would be valuable to have a way to query the order directly: the result > > would be "C", "F" or None, just like the order keyword that is passed > > in. > You an do it with the flags > > a.flags.contiguous > a.flags.fortran > > Discontiguous is when both of these are false. Note that for a.ndim < > 2, both a.flags.contiguous and a.flags.fortran are true if one of them > is true. > > This is all explained in the first chapters of my book. You have to > understand CONTIGUOUS == C-order contiguous and FORTRAN == Fortran-order > contiguous. Could we make a few changes ;) For printing the flags I would suggest using C-Contiguous and F-Contiguous so folks don't have to read the book. And at the c level define alternates, i.e, #define c-contiguous contiguous or whatever. That way backward compatibility would be maintained but more descriptive names would be available. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Wed Oct 18 17:03:32 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 15:03:32 -0600 Subject: scalar object bug or feature? In-Reply-To: References: <45367C10.40606@stsci.edu> Message-ID: On 10/18/06, Alan G Isaac wrote: > > On Wed, 18 Oct 2006, Keith Goodman apparently wrote: Here's a simpler (?) example: > >>> x=numpy.random.rand(300,1)>0 > >>> x.sum() > 300 > >>> sum(x) > array([44], dtype=int8) > >>> x=numpy.random.rand(300)>0 > >>> sum(x) > 300 > > Alan Isaac Hmmm, I think sum(x) and x.sum() should behave the same. Note that In [12]:sum(x, dtype=int) Out[12]:300 I think sum should stick to the modular arithmetic unless specified otherwise. But in any case sum(x) and x.sum() should do the same thing. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gnurser at googlemail.com Wed Oct 18 17:05:19 2006 From: gnurser at googlemail.com (George Nurser) Date: Wed, 18 Oct 2006 22:05:19 +0100 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <453672F6.9070307@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <453672F6.9070307@ieee.org> Message-ID: <1d1e6ea70610181405x2732a584hbb975ea76883f3e9@mail.gmail.com> > > None of the LaPack stuff seems to use the Fortran stuff, they just > > transpose and copy. You've got me worried here. I have assumed that when you start with a c-contiguous array, a, with say,a.shape = (m,n), if you use the transpose as an argument to a fortran routine which requires an mxn size array, then no copying is required. This seems to work for me -- the transpose *does* have fortran order. Also, in f2py, if I use -DF2PY_REPORT_ON_ARRAY_COPY=1 I receive no alert of any copy. Apologies if these are simply confused ravings. George Nurser. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 17:48:22 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 15:48:22 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <1d1e6ea70610181405x2732a584hbb975ea76883f3e9@mail.gmail.com> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <453672F6.9070307@ieee.org> <1d1e6ea70610181405x2732a584hbb975ea76883f3e9@mail.gmail.com> Message-ID: On 10/18/06, George Nurser wrote: > > > > None of the LaPack stuff seems to use the Fortran stuff, they just > > > transpose and copy. > > You've got me worried here. I have assumed that when you start with a > c-contiguous array, a, with say,a.shape = (m,n), if you use the > transpose as an argument to a fortran routine which requires an mxn > size array, then no copying is required. > > This seems to work for me -- the transpose *does* have fortran order. Nope. The result is an (n,m) array in fortran order, not an (m,n) array in fortran order. In [52]:a = array([[1,2,3],[4,5,6]]) In [53]:a.transpose().flags Out[53]: CONTIGUOUS : False FORTRAN : True OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False In [54]:a.transpose().shape Out[54]:(3, 2) Looks like what you want is either fastcopyandtranspose or the order flag. In [56]:fastCopyAndTranspose(a).flags Out[56]: CONTIGUOUS : True FORTRAN : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False which is a (n,m) array in c order, i.e., an (m,n) array in fortran order. Or In [57]:array(a, order='F').flags Out[57]: CONTIGUOUS : False FORTRAN : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False which is a (m,n) array in fortran order. Also, in f2py, if I use -DF2PY_REPORT_ON_ARRAY_COPY=1 I receive no > alert of any copy. f2py takes care of the ordering, which is one reason why it is so useful. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 18 18:06:32 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 18 Oct 2006 16:06:32 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> Message-ID: <4536A568.4090907@ee.byu.edu> Charles R Harris wrote: > > Could we make a few changes ;) > > For printing the flags I would suggest using C-Contiguous and > F-Contiguous so folks don't have to read the book. And at the c level > define alternates, i.e, #define c-contiguous contiguous or whatever. > That way backward compatibility would be maintained but more > descriptive names would be available. Printing the flags is not intended for the casual user. So, I'd like to keep consistent with C-level names and the names that are printed. CONTIGUOUS is the old name Numeric used. It always meant C-CONTIGUOUS and so that meaning is preserved. FORTRAN is the new one flag and it means FORTRAN CONTIGUOUS. So, you want something like? #define NPY_C_CONTIGUOUS NPY_CONTIGUOUS #define NPY_F_CONTIGUOUS NPY_FORTRAN and to have C_CONTIGUOUS and F_CONTIGUOUS print for the flags description? I'm not opposed to it, but I don't really see the need. It's just a semantic question. Given the history of CONTIGUOUS in Numeric I thought it was clear that CONTIGUOUS always meant C-contiguous. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 18:19:36 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 16:19:36 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536A568.4090907@ee.byu.edu> References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> Message-ID: On 10/18/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > Could we make a few changes ;) > > > > For printing the flags I would suggest using C-Contiguous and > > F-Contiguous so folks don't have to read the book. And at the c level > > define alternates, i.e, #define c-contiguous contiguous or whatever. > > That way backward compatibility would be maintained but more > > descriptive names would be available. > > Printing the flags is not intended for the casual user. So, I'd like to > keep consistent with C-level names and the names that are printed. > > CONTIGUOUS is the old name Numeric used. It always meant C-CONTIGUOUS > and so that meaning is preserved. FORTRAN is the new one flag and it > means FORTRAN CONTIGUOUS. > > So, you want something like? > > #define NPY_C_CONTIGUOUS NPY_CONTIGUOUS > #define NPY_F_CONTIGUOUS NPY_FORTRAN > > and to have C_CONTIGUOUS and F_CONTIGUOUS print for the flags description? Yes, I think that would be more informative. I'm not opposed to it, but I don't really see the need. It's just a > semantic question. Given the history of CONTIGUOUS in Numeric I thought > it was clear that CONTIGUOUS always meant C-contiguous. Well, I knew that for numeric, but it was a good deal less obvious in combo with the order keyword. For instance, contiguous could change its meaning to match up with FORTRAN, so that FORTRAN=True and CONTIGUOUS=True meant Fortran contiguous, which was sort of what I was thinking. Explicit never hurts. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gnurser at googlemail.com Wed Oct 18 18:21:19 2006 From: gnurser at googlemail.com (George Nurser) Date: Wed, 18 Oct 2006 23:21:19 +0100 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366602.4090704@ieee.org> <453672F6.9070307@ieee.org> <1d1e6ea70610181405x2732a584hbb975ea76883f3e9@mail.gmail.com> Message-ID: <1d1e6ea70610181521u3eb3120fv9870f73c6a8e0602@mail.gmail.com> On 18/10/06, Charles R Harris wrote: > > > On 10/18/06, George Nurser wrote: > > > > None of the LaPack stuff seems to use the Fortran stuff, they just > > > > transpose and copy. > > > > You've got me worried here. I have assumed that when you start with a > > c-contiguous array, a, with say, a.shape = (m,n), if you use the > > transpose as an argument to a fortran routine which requires an mxn > > size array, then no copying is required. > > > > This seems to work for me -- the transpose *does* have fortran order. > > Nope. The result is an (n,m) array in fortran order, not an (m,n) array in > fortran order. Presumably that's because it's a view of the original array. >> > Also, in f2py, if I use -DF2PY_REPORT_ON_ARRAY_COPY=1 I receive no > > alert of any copy. > > f2py takes care of the ordering, which is one reason why it is so useful. Yes, when I first used it, I assumed that the fortran routine had to use an nxm array. But f2py is clever enough to make the above work. George. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Wed Oct 18 18:23:17 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 18 Oct 2006 16:23:17 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358907.9020509@ieee.org> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> Message-ID: <4536A955.5060504@ee.byu.edu> Charles R Harris wrote: > Well, I knew that for numeric, but it was a good deal less obvious in > combo with the order keyword. For instance, contiguous could change > its meaning to match up with FORTRAN, so that FORTRAN=True and > CONTIGUOUS=True meant Fortran contiguous, which was sort of what I was > thinking. Explicit never hurts. Ahh, so you were confused by looking at flags for 1-d arrays (where indeed both CONTIGUOUS and FORTRAN can be True --- 1-d arrays that are CONTIGUOUS are also FORTRAN CONTIGUOUS). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 18:44:23 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 16:44:23 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536A955.5060504@ee.byu.edu> References: <20061018002445.GP17998@mentat.za.net> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> <4536A955.5060504@ee.byu.edu> Message-ID: On 10/18/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > Well, I knew that for numeric, but it was a good deal less obvious in > > combo with the order keyword. For instance, contiguous could change > > its meaning to match up with FORTRAN, so that FORTRAN=True and > > CONTIGUOUS=True meant Fortran contiguous, which was sort of what I was > > thinking. Explicit never hurts. > > > Ahh, so you were confused by looking at flags for 1-d arrays (where More confused by what seemed to make sense to me. Tim was the one who actually ran the experiment to see what was going on, and he wasn't sure what FORTRAN meant either. Now if I had accessed the full set of offsets, strides, and counts, it would all have become clear. Numpy tries to hide the nasty details, which is good until you really have to know what it happening underneath. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Wed Oct 18 18:55:06 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed, 18 Oct 2006 16:55:06 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <20061018022229.GR17998@mentat.za.net> <4535AEFB.4090202@ieee.org> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> <4536A955.5060504@ee.byu.edu> Message-ID: <4536B0CA.9070402@ee.byu.edu> Charles R Harris wrote: > > > On 10/18/06, *Travis Oliphant* > wrote: > > Charles R Harris wrote: > > > Well, I knew that for numeric, but it was a good deal less > obvious in > > combo with the order keyword. For instance, contiguous could change > > its meaning to match up with FORTRAN, so that FORTRAN=True and > > CONTIGUOUS=True meant Fortran contiguous, which was sort of what > I was > > thinking. Explicit never hurts. > > > Ahh, so you were confused by looking at flags for 1-d arrays (where > > > More confused by what seemed to make sense to me. Tim was the one who > actually ran the experiment to see what was going on, and he wasn't > sure what FORTRAN meant either. Now if I had accessed the full set of > offsets, strides, and counts, it would all have become clear. Numpy > tries to hide the nasty details, which is good until you really have > to know what it happening underneath. I've done as you requested. I've added F_CONTIGUOUS as an alias to FORTRAN and C_CONTIGUOUS as an alias to CONTIGUOUS. These are the names that show up when you print the flags and you can use them whenever you used the other names. The other names are still everywhere in the code, but perhaps the existence of these aliases will help people understand what is meant better. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Wed Oct 18 19:05:13 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 18 Oct 2006 17:05:13 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4536B0CA.9070402@ee.byu.edu> References: <20061018002445.GP17998@mentat.za.net> <45366988.5020204@ieee.org> <4536738D.7070701@ieee.org> <4536A568.4090907@ee.byu.edu> <4536A955.5060504@ee.byu.edu> <4536B0CA.9070402@ee.byu.edu> Message-ID: On 10/18/06, Travis Oliphant wrote: > > Charles R Harris wrote: > > > > > > > On 10/18/06, *Travis Oliphant* > > wrote: > > > > Charles R Harris wrote: > > > > > Well, I knew that for numeric, but it was a good deal less > > obvious in > > > combo with the order keyword. For instance, contiguous could > change > > > its meaning to match up with FORTRAN, so that FORTRAN=True and > > > CONTIGUOUS=True meant Fortran contiguous, which was sort of what > > I was > > > thinking. Explicit never hurts. > > > > > > Ahh, so you were confused by looking at flags for 1-d arrays (where > > > > > > More confused by what seemed to make sense to me. Tim was the one who > > actually ran the experiment to see what was going on, and he wasn't > > sure what FORTRAN meant either. Now if I had accessed the full set of > > offsets, strides, and counts, it would all have become clear. Numpy > > tries to hide the nasty details, which is good until you really have > > to know what it happening underneath. > > I've done as you requested. I've added F_CONTIGUOUS as an alias to > FORTRAN and C_CONTIGUOUS as an alias to CONTIGUOUS. These are the names > that show up when you print the flags and you can use them whenever you > used the other names. The other names are still everywhere in the code, > but perhaps the existence of these aliases will help people understand > what is meant better. Thanks, Travis. I really appreciate your willingness to make such modifications. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From michael.sorich at gmail.com Wed Oct 18 20:33:06 2006 From: michael.sorich at gmail.com (Michael Sorich) Date: Thu, 19 Oct 2006 10:03:06 +0930 Subject: Question about indexing arrays with boolean arrays In-Reply-To: References: Message-ID: <16761e100610181733g760c71bev441e439dfbea4458@mail.gmail.com> On 10/18/06, Daniel Arbuckle wrote: > Why does a[b1, b2] not mean the same thing as a[b1][:, b2], when "a" > is an array and "b1" and "b2" are appropriately sized arrays of > booleans? >From my previous experience with R I am used to a[b1, b2] being equivalent to a[b1][:, b2], so this is a little strange to me as well. In numpy I think you need to use the function ix_ to get the cross product. e.g. a[ix_(b1,b2)] is equivalent to a[b1][:, b2] a[b1,b2] raises an error about a type mismatch. I have no idea what this syntax is supposed to do in numpy. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Wed Oct 18 20:29:15 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 19 Oct 2006 02:29:15 +0200 Subject: adding an attribute to an nd-array Message-ID: <20061019002914.GA20609@mentat.za.net> A quick question on extending numpy arrays: is it possible to easily add an attribute to an ndarray? With Python-defined classes one can do class X(object): pass x = X() x.foo = 'bar' but with ndarrays you get x = N.array([1,2,3]) x.foo = 'bar' AttributeError: 'numpy.ndarray' object has no attribute 'foo' Is there an easy way around this (without writing a C extension)? Thanks for any advice. St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ckkart at hoc.net Wed Oct 18 21:10:06 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Thu, 19 Oct 2006 01:10:06 +0000 (UTC) Subject: sort(axis=-1) References: <4536603D.7070103@ieee.org> Message-ID: Travis Oliphant ieee.org> writes: > > Christian Kristukat wrote: > > Hi, > > > > it seems that -1 as axis parameter is interpreted like in array indexing, > > -1 means the last axis rather than meaning the only axis of the flattened > > representation, like for example with take(). > > Is that intendend? > > > > > > I don't understand the question. Yes, sort(axis=-1) sorts along the > last dimension. > > AFAIK, take has the same meaning for axis. Oh sure, I mixed that up with axis=None as Robert pointed out. However I don't understand why axis=None wouldn't make sense for an inplace operation. Anyway, it's not important. Sorry for the noise. Christian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Wed Oct 18 21:17:49 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Wed, 18 Oct 2006 21:17:49 -0400 Subject: adding an attribute to an nd-array In-Reply-To: <20061019002914.GA20609@mentat.za.net> References: <20061019002914.GA20609@mentat.za.net> Message-ID: <200610182117.49777.pgmdevlist@gmail.com> On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: > A quick question on extending numpy arrays: is it possible to easily > add an attribute to an ndarray? It might be easier to create a subclass: pleasehave a look here: http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_subclasses.py That's a tiny example of subclassing ndarrays, with some extra attributes. (BTW, I'm not sure that's the most obvious place where to look: if it turns out to be useful, I'll put it on the scipy wiki) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From xaagujzjtrz at txteam.com Thu Oct 19 03:49:28 2006 From: xaagujzjtrz at txteam.com (News Catch) Date: Thu, 19 Oct 2006 15:49:28 +0800 Subject: CA tel: Message-ID: <000801c6f353$1a9a1ae0$ed3308dd@4a21589f39a64fc> Barbados Bermuda Boston Cabo Lucaslos Cabos Cancun Cayman Islands Grand Chicago Cozumel Daytona a Beach of Disney Orlando. Spaces Blogger or Wordpress Drupal Technorati or Flickr a Yiyi sun cd Quinnipiac University Official Athletic dd Varsity Hockeyice Bandcrazy. Blogger Wordpress Drupal Technorati or Flickr a Yiyi sun cd Quinnipiac University or Official am Athletic dd Varsity. Accused historical abuse charges is made brief appearance court bus is driver found guilty dragging trapped or boy Nelson. Suffering going way peaceably suddenly is come him swear slavishly obey everything dictate consider good plan decide law a Secondly hand part dispose ofwe forcibly opposing orders Thirdly of elect others? Series off Mastering app choice easy Visual of Quickstart Guides a They feature clear helpful screen am shotsmore Computing in About or Order. Reloded in reload trailers script is vcd trailor or dot max payne cow or kazaa revolution am preview of find complete guide in file or sharing software a members area Plus get access pp tutorials. Stuff my Homeabout copy Fairfax Limited material this page has am protection copyright rights reserved Search web Headlines of Alert Overstayer jailed drug is haul African who. Easy Visual Quickstart Guides a They of feature am clear helpful screen shotsmore a Computing or About or Order Government. Textbooks Childrens Dvds Music Toys Items Browse Bookswhats am Soonbampn Annexsave. Tampa bay Tickets Tonights Gameend Allentowns Everyday Heroes ea Tech Neededfree Pocono of Race or Football is Employment Designer. Tours in Search Quickfind Quickfind Maritime Museums a Racing or Booking Types Hotels Performs Visitor Beatles Nightlife am Walks is Guided Trade exhibition aims. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Pickup:.gif Type: image/gif Size: 12248 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david at ar.media.kyoto-u.ac.jp Thu Oct 19 06:25:52 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 19 Oct 2006 19:25:52 +0900 Subject: Conditional tests for test classes derived from NumpyTest Message-ID: <453752B0.3080308@ar.media.kyoto-u.ac.jp> Hi, Is there a simple way to execute tests inherited from NumpyTest conditionally ? I have something like that for now: class test_c_implementation(NumpyTestCase): def _check(self, level, decimal = 12): try: from foo import bar Y = bar() except ImportError, inst: print "Error while importing bar, not tested" print " -> (Import error was %s)" % inst And all checking functions of the class call _check. The problem is that if there are 10 check functions, there are 10 errors reported, which is a bit stupid, cheers, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Thu Oct 19 06:33:43 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Thu, 19 Oct 2006 13:33:43 +0300 (EEST) Subject: Conditional tests for test classes derived from NumpyTest In-Reply-To: <453752B0.3080308@ar.media.kyoto-u.ac.jp> References: <453752B0.3080308@ar.media.kyoto-u.ac.jp> Message-ID: On Thu, 19 Oct 2006, David Cournapeau wrote: > Hi, > > Is there a simple way to execute tests inherited from NumpyTest > conditionally ? I have something like that for now: > > class test_c_implementation(NumpyTestCase): > def _check(self, level, decimal = 12): > try: > from foo import bar > Y = bar() > except ImportError, inst: > print "Error while importing bar, not tested" > print " -> (Import error was %s)" % inst > > And all checking functions of the class call _check. The problem is > that if there are 10 check functions, there are 10 errors reported, > which is a bit stupid, One approach is demonstrated in scipy/Lib/lib/blas/tests/test_fblas.py. In your case: class base_test_c_implementation: def check_bar(self, level, decimal=12): Y = bar() try: from foo import bar class test_c_implementation(base_test_c_implementation,NumpyTestCase): pass except ImportError, inst: print "Error while importing bar, not tested" print " -> (Import error was %s)" % inst Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 19 07:20:32 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Thu, 19 Oct 2006 13:20:32 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <200610182117.49777.pgmdevlist@gmail.com> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> Message-ID: <20061019112032.GE20609@mentat.za.net> On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: > > A quick question on extending numpy arrays: is it possible to easily > > add an attribute to an ndarray? > > It might be easier to create a subclass: pleasehave a look here: > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_subclasses.py > That's a tiny example of subclassing ndarrays, with some extra attributes. > (BTW, I'm not sure that's the most obvious place where to look: if it turns > out to be useful, I'll put it on the scipy wiki) Thanks very much, Pierre. If I understand correctly, the following should work: import numpy as N class InfoArray(N.ndarray): def __new__(info_arr_cls,arr,info={}): info_arr_cls.info = info return N.array(arr).view(info_arr_cls) When does __array_finalize__ get called, and is it always necessary to specify it? Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Thu Oct 19 07:59:08 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 19 Oct 2006 13:59:08 +0200 Subject: Product of an empty sequence Message-ID: <4537688C.5070908@carabos.com> Today I was surprised by this:: >>> import numpy >>> numpy.__version__ '1.0.dev3341' >>> numpy.prod(()) 1.0 Wouldn't be 0.0 a more natural answer? I understand that 1 is neutral on product operations, but I still can't see why 1.0 is a better answer. Sorry for the clueless question, but I have searched lists, docstrings and web and found nothing about this. (BTW, the definitions of ``prod()`` and ``product()`` are still identical as noted by Sebastian Haase some time ago.) :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From rudolphv at gmail.com Thu Oct 19 08:35:33 2006 From: rudolphv at gmail.com (Rudolph van der Merwe) Date: Thu, 19 Oct 2006 14:35:33 +0200 Subject: Unit test error with numpy rc3 Message-ID: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> I get the following error with RC3 on a RHE Linux box: Python 2.4.3 (#4, Mar 31 2006, 12:12:43) [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.0rc3' >>> numpy.test() Found 5 tests for numpy.distutils.misc_util Found 3 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 32 tests for numpy.linalg Found 13 tests for numpy.core.umath Found 4 tests for numpy.core.scalarmath Found 9 tests for numpy.lib.arraysetops Found 42 tests for numpy.lib.type_check Found 183 tests for numpy.core.multiarray Found 3 tests for numpy.fft.helper Found 36 tests for numpy.core.ma Found 12 tests for numpy.lib.twodim_base Found 10 tests for numpy.core.defmatrix Found 1 tests for numpy.lib.ufunclike Found 4 tests for numpy.ctypeslib Found 41 tests for numpy.lib.function_base Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 26 tests for numpy.core.numeric Found 4 tests for numpy.lib.index_tricks Found 47 tests for numpy.lib.shape_base Found 0 tests for __main__ .......................................................................................................Warning: invalid value encountered in divide ..Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ..Warning: invalid value encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ..Warning: invalid value encountered in divide ..Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ........Warning: invalid value encountered in divide .Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide ........................................................................................................................................................................................................Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide .....................Warning: invalid value encountered in sqrt Warning: invalid value encountered in log Warning: invalid value encountered in log10 ..Warning: invalid value encountered in sqrt Warning: invalid value encountered in sqrt Warning: divide by zero encountered in log Warning: divide by zero encountered in log Warning: divide by zero encountered in log10 Warning: divide by zero encountered in log10 Warning: invalid value encountered in arcsin Warning: invalid value encountered in arcsin Warning: invalid value encountered in arccos Warning: invalid value encountered in arccos Warning: invalid value encountered in arccosh Warning: invalid value encountered in arccosh Warning: divide by zero encountered in arctanh Warning: divide by zero encountered in arctanh Warning: invalid value encountered in divide Warning: invalid value encountered in true_divide Warning: invalid value encountered in floor_divide Warning: invalid value encountered in remainder Warning: invalid value encountered in fmod ............................................................................................................................................................... ---------------------------------------------------------------------- Ran 516 tests in 0.368s OK >>> -- Rudolph van der Merwe KAT (Karoo Array Telescope) / www.kat.ac.za ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Thu Oct 19 09:04:12 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Thu, 19 Oct 2006 15:04:12 +0200 Subject: recent ubuntu package In-Reply-To: <4535C555.9060004@astraw.com> References: <45353395.40602@gmx.net> <4535C555.9060004@astraw.com> Message-ID: <453777CC.6070804@gmx.net> Andrew Straw schrieb: > The matplotlib .deb on my website is working fine for me with the latest > numpy .deb there. If there are any recent patches or anything you are > missing, please let me know -- it's not really a big deal to update > them, although it might take a couple of days for me to find the time. > > Thanks for your answer. No, I'm not aware of any new features I would use, so that's ok for me. It just wasn't obvious to me why the builds are from in-between-releases devel versions, that's why I asked. After numpy 1.0 is released, are you planning to put up only official releases or also devel snapshots? thanks, sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Thu Oct 19 09:07:51 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 19 Oct 2006 09:07:51 -0400 Subject: Product of an empty sequence In-Reply-To: <4537688C.5070908@carabos.com> References: <4537688C.5070908@carabos.com> Message-ID: On Thu, 19 Oct 2006, Ivan Vilata i Balaguer apparently wrote: >>>> numpy.prod(()) > 1.0 > Wouldn't be 0.0 a more natural answer? I think it must be 1 or a TypeError. E.g., http://docs.python.org/lib/built-in-funcs.html#reduce reduce(function, sequence[, initializer]) Apply function of two arguments cumulatively to the items of sequence, from left to right, so as to reduce the sequence to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the sequence. If the optional initializer is present, it is placed before the items of the sequence in the calculation, and serves as a default when the sequence is empty. If initializer is not given and sequence contains only one item, the first item is returned. hth, Alan Isaac (just a user) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 09:32:30 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 06:32:30 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> Message-ID: <45377E6E.7040006@ieee.org> Rudolph van der Merwe wrote: > I get the following error with RC3 on a RHE Linux box: > > Python 2.4.3 (#4, Mar 31 2006, 12:12:43) > [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>>> import numpy >>>> numpy.__version__ >>>> > '1.0rc3' > [SNIP lots of warnings] Ah! I see that Travis has turned up the default warning level. Excellent! This will require some tweaking of the tests, but that shouldn't be a big deal. If I have time I will try to work on that later today. For the moment, try this: >>> olderr = numpy.seterr(all='ignore') >>> numpy.test() >>> numpy.seterr(*olderr) This should run the tests with the warnings suppressed. On a visual studio build, I'm getting a real failure though: FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", line 219, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError The code in question is dependent on the spelling of INF(or at least the *length* of the spelling) on a given platform which is why it's failing. I suspect that the correct test is: str(a)[1:-1] == str(a[0]) However, I'm not entirely sure what this is testing, so I'm reluctant to check that in. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 19 10:25:36 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 19 Oct 2006 08:25:36 -0600 Subject: adding an attribute to an nd-array In-Reply-To: <20061019112032.GE20609@mentat.za.net> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> Message-ID: <45378AE0.1080003@ieee.org> Stefan van der Walt wrote: > On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > >> On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: >> >>> A quick question on extending numpy arrays: is it possible to easily >>> add an attribute to an ndarray? >>> >> It might be easier to create a subclass: pleasehave a look here: >> http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_subclasses.py >> That's a tiny example of subclassing ndarrays, with some extra attributes. >> (BTW, I'm not sure that's the most obvious place where to look: if it turns >> out to be useful, I'll put it on the scipy wiki) >> > > Thanks very much, Pierre. > > If I understand correctly, the following should work: > > import numpy as N > > class InfoArray(N.ndarray): > def __new__(info_arr_cls,arr,info={}): > info_arr_cls.info = info > return N.array(arr).view(info_arr_cls) > > When does __array_finalize__ get called, and is it always necessary to > specify it? > It gets called whenever a new array is created. No, it is not necessary to specify it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 19 10:27:20 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 19 Oct 2006 08:27:20 -0600 Subject: Product of an empty sequence In-Reply-To: <4537688C.5070908@carabos.com> References: <4537688C.5070908@carabos.com> Message-ID: <45378B48.9030105@ieee.org> Ivan Vilata i Balaguer wrote: > Today I was surprised by this:: > > >>>> import numpy >>>> numpy.__version__ >>>> > '1.0.dev3341' > >>>> numpy.prod(()) >>>> > 1.0 > > Wouldn't be 0.0 a more natural answer? I understand that 1 is neutral > on product operations, but I still can't see why 1.0 is a better answer. > It's the identity for the multiplication ufunc. > Sorry for the clueless question, but I have searched lists, docstrings > and web and found nothing about this. > > (BTW, the definitions of ``prod()`` and ``product()`` are still > identical as noted by Sebastian Haase some time ago.) > Is this a problem? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Thu Oct 19 10:29:26 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 19 Oct 2006 08:29:26 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <45377E6E.7040006@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> Message-ID: <45378BC6.70609@ieee.org> Tim Hochberg wrote: > Rudolph van der Merwe wrote: > >> I get the following error with RC3 on a RHE Linux box: >> >> Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >> [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >> >>>>> import numpy >>>>> numpy.__version__ >>>>> >>>>> >> '1.0rc3' >> >> > > On a visual studio build, I'm getting a real failure though: > > FAIL: Ticket #112 > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", > line 219, in check_longfloat_repr > assert(str(a)[1:9] == str(a[0])[:8]) > AssertionError > > The code in question is dependent on the spelling of INF(or at least the > *length* of the spelling) on a given platform which is why it's failing. > Actually, you shouldn't be getting an INF at all. This is what the test is designed to test for (so I guess it's working). The test was actually written wrong and was never failing because previously keyword arguments to ufuncs were ignored. Can you show us what 'a' is on your platform. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From strawman at astraw.com Thu Oct 19 10:48:24 2006 From: strawman at astraw.com (Andrew Straw) Date: Thu, 19 Oct 2006 07:48:24 -0700 Subject: recent ubuntu package In-Reply-To: <453777CC.6070804@gmx.net> References: <45353395.40602@gmx.net> <4535C555.9060004@astraw.com> <453777CC.6070804@gmx.net> Message-ID: <45379038.9080902@astraw.com> Sven Schreiber wrote: > Andrew Straw schrieb: > > >> The matplotlib .deb on my website is working fine for me with the latest >> numpy .deb there. If there are any recent patches or anything you are >> missing, please let me know -- it's not really a big deal to update >> them, although it might take a couple of days for me to find the time. >> >> >> > > Thanks for your answer. No, I'm not aware of any new features I would > use, so that's ok for me. It just wasn't obvious to me why the builds > are from in-between-releases devel versions, that's why I asked. After > numpy 1.0 is released, are you planning to put up only official releases > or also devel snapshots? > With numpy, I currently use the most recent SVN version I can because I want to help Travis testing for the 1.0 release. With that in mind, I keep matplotlib and scipy relatively up-to-date as well in case there have been any changes that those two have tracked. I'm not sure what I'll do after the release of numpy 1.0. I do not intend to be on the "bleeding edge" -- I use these repositories for a number of machines at work. But if the leading edge isn't bleeding (as currently), it's quite possible I'll keep updating them, particularly if features come out I'm interested in. This would be a good reason for me to keep updating scipy and matplotlib, even if I keep numpy more stable. -Andrew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 11:04:55 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 08:04:55 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <45378BC6.70609@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> Message-ID: <45379417.4060601@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > >> Rudolph van der Merwe wrote: >> >> >>> I get the following error with RC3 on a RHE Linux box: >>> >>> Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >>> [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> >>> >>>>>> import numpy >>>>>> numpy.__version__ >>>>>> >>>>>> >>>>>> >>> '1.0rc3' >>> >>> >>> >> On a visual studio build, I'm getting a real failure though: >> >> FAIL: Ticket #112 >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", >> line 219, in check_longfloat_repr >> assert(str(a)[1:9] == str(a[0])[:8]) >> AssertionError >> >> The code in question is dependent on the spelling of INF(or at least the >> *length* of the spelling) on a given platform which is why it's failing. >> >> > > Actually, you shouldn't be getting an INF at all. This is what the > test is designed to test for (so I guess it's working). The test was > actually written wrong and was never failing because previously keyword > arguments to ufuncs were ignored. > > Can you show us what 'a' is on your platform. Sure: >>> import numpy as N >>> a = N.exp(N.array([1000],dtype=N.longfloat)) Warning: overflow encountered in exp >>> a array([1.#INF], dtype=float64) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 11:08:50 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 08:08:50 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <45378BC6.70609@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> Message-ID: <45379502.1040707@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > >> Rudolph van der Merwe wrote: >> >> >>> I get the following error with RC3 on a RHE Linux box: >>> >>> Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >>> [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> >>> >>>>>> import numpy >>>>>> numpy.__version__ >>>>>> >>>>>> >>>>>> >>> '1.0rc3' >>> >>> >>> >> On a visual studio build, I'm getting a real failure though: >> >> FAIL: Ticket #112 >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", >> line 219, in check_longfloat_repr >> assert(str(a)[1:9] == str(a[0])[:8]) >> AssertionError >> >> The code in question is dependent on the spelling of INF(or at least the >> *length* of the spelling) on a given platform which is why it's failing. >> >> > > Actually, you shouldn't be getting an INF at all. This is what the > test is designed to test for (so I guess it's working). The test was > actually written wrong and was never failing because previously keyword > arguments to ufuncs were ignored. > > Can you show us what 'a' is on your platform. > I expect that the problem is related to this: >>> N.array([1000],dtype=N.float).dtype dtype('float64') >>> N.array([1000],dtype=N.longfloat).dtype dtype('float64') longfloat appears to just be an alias for float under VS. -tim > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 19 11:10:05 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 19 Oct 2006 09:10:05 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <45379417.4060601@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <45379417.4060601@ieee.org> Message-ID: <4537954D.1060301@ee.byu.edu> Tim Hochberg wrote: >Travis Oliphant wrote: > > >>Tim Hochberg wrote: >> >> >> >>>Rudolph van der Merwe wrote: >>> >>> >>> >>> >>>>I get the following error with RC3 on a RHE Linux box: >>>> >>>>Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >>>>[GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >>>>Type "help", "copyright", "credits" or "license" for more information. >>>> >>>> >>>> >>>> >>>> >>>>>>>import numpy >>>>>>>numpy.__version__ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>'1.0rc3' >>>> >>>> >>>> >>>> >>>> >>>On a visual studio build, I'm getting a real failure though: >>> >>> FAIL: Ticket #112 >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File >>> "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", >>> line 219, in check_longfloat_repr >>> assert(str(a)[1:9] == str(a[0])[:8]) >>> AssertionError >>> >>>The code in question is dependent on the spelling of INF(or at least the >>>*length* of the spelling) on a given platform which is why it's failing. >>> >>> >>> >>> >>Actually, you shouldn't be getting an INF at all. This is what the >>test is designed to test for (so I guess it's working). The test was >>actually written wrong and was never failing because previously keyword >>arguments to ufuncs were ignored. >> >>Can you show us what 'a' is on your platform. >> >> >Sure: > > >>> import numpy as N > >>> a = N.exp(N.array([1000],dtype=N.longfloat)) >Warning: overflow encountered in exp > >>> a >array([1.#INF], dtype=float64) > > O.K. We need to modify the test in case to check and see that the size of longfloat isn't the same as double. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Thu Oct 19 11:39:15 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 19 Oct 2006 17:39:15 +0200 Subject: Product of an empty sequence In-Reply-To: <45378B48.9030105@ieee.org> References: <4537688C.5070908@carabos.com> <45378B48.9030105@ieee.org> Message-ID: <45379C23.4050008@carabos.com> En/na Travis Oliphant ha escrit:: > Ivan Vilata i Balaguer wrote: >> Wouldn't be 0.0 a more natural answer? I understand that 1 is neutral >> on product operations, but I still can't see why 1.0 is a better answer. > > It's the identity for the multiplication ufunc. OK, I guess that means it's something like an implicit ``initializer`` (as pointed by Alan) to ufunc reductions. Thanks! :: >> (BTW, the definitions of ``prod()`` and ``product()`` are still >> identical as noted by Sebastian Haase some time ago.) >> > Is this a problem? No, but it also caught my eye. :) :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 19 11:45:02 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 19 Oct 2006 09:45:02 -0600 Subject: adding an attribute to an nd-array In-Reply-To: <20061019112032.GE20609@mentat.za.net> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> Message-ID: <45379D7E.30609@ee.byu.edu> Stefan van der Walt wrote: >On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > > >>On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: >> >> >>>A quick question on extending numpy arrays: is it possible to easily >>>add an attribute to an ndarray? >>> >>> >>It might be easier to create a subclass: pleasehave a look here: >>http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_subclasses.py >>That's a tiny example of subclassing ndarrays, with some extra attributes. >>(BTW, I'm not sure that's the most obvious place where to look: if it turns >>out to be useful, I'll put it on the scipy wiki) >> >> > >Thanks very much, Pierre. > >If I understand correctly, the following should work: > >import numpy as N > >class InfoArray(N.ndarray): > def __new__(info_arr_cls,arr,info={}): > info_arr_cls.info = info > return N.array(arr).view(info_arr_cls) > >When does __array_finalize__ get called, and is it always necessary to >specify it? > > Actually something as simple as class InfoArray(N.ndarray): pass will allow you to add attributes to InfoArray. I just learned about how to allow built-ins to have attributes assigned to their instances. It's actually pretty easy because of Python support for it --- but it comes at a cost. You have to add a dictionary to the PyArrayObject structure, create that dictionary when the ndarray is allocated, and set the tp_dictoffset in the TypeObject structure to its location in PyArrayObject. It takes 4 lines of code with the cost of creating a new dictionary for every ndarray. I don't think the extra bytes for every ndarray object are worth it, given how easy it is to sub-class and create your own ndarray that can have attributes attached. What are others opinions. -Travis P.S. Here is the patch that adds it: Index: numpy/core/include/numpy/ndarrayobject.h =================================================================== --- numpy/core/include/numpy/ndarrayobject.h (revision 3366) +++ numpy/core/include/numpy/ndarrayobject.h (working copy) @@ -1172,6 +1172,7 @@ PyArray_Descr *descr; /* Pointer to type structure */ int flags; /* Flags describing array -- see below*/ PyObject *weakreflist; /* For weakreferences */ + PyObject *instancedict; /* For instance attributes */ } PyArrayObject; #define NPY_AO PyArrayObject Index: numpy/core/src/arrayobject.c =================================================================== --- numpy/core/src/arrayobject.c (revision 3366) +++ numpy/core/src/arrayobject.c (working copy) @@ -1906,6 +1906,8 @@ if (self->weakreflist != NULL) PyObject_ClearWeakRefs((PyObject *)self); + Py_DECREF(self->instancedict); + if(self->base) { /* UPDATEIFCOPY means that base points to an array that should be updated with the contents @@ -5305,6 +5307,7 @@ self->descr = descr; self->base = (PyObject *)NULL; self->weakreflist = (PyObject *)NULL; + self->instancedict = PyDict_New(); if (nd > 0) { self->dimensions = PyDimMem_NEW(2*nd); @@ -6689,7 +6692,7 @@ 0, /* tp_dict */ 0, /* tp_descr_get */ 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ + offsetof(PyArrayObject, instancedict), /* tp_dictoffset */ (initproc)0, /* tp_init */ array_alloc, /* tp_alloc */ (newfunc)array_new, /* tp_new */ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 19 11:53:05 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 19 Oct 2006 09:53:05 -0600 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: On 10/19/06, Travis Oliphant wrote: > > Stefan van der Walt wrote: > > >On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > > > > > >>On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: > >> I don't think the extra bytes for every ndarray object are worth it, > given how easy it is to sub-class and create your own ndarray that can > have attributes attached. What are others opinions. I'd be more inclined to worry about speed. One of the drawbacks of numarray was the time it took to create arrays. Since I often use many small arrays, numpy was a big improvement in that area. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Thu Oct 19 11:54:42 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 19 Oct 2006 10:54:42 -0500 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: Travis Oliphant wrote: > I just learned about how to allow built-ins to have attributes assigned > to their instances. It's actually pretty easy because of Python > support for it --- but it comes at a cost. You have to add a dictionary > to the PyArrayObject structure, create that dictionary when the ndarray > is allocated, and set the tp_dictoffset in the TypeObject structure to > its location in PyArrayObject. It takes 4 lines of code with the cost of > creating a new dictionary for every ndarray. > > I don't think the extra bytes for every ndarray object are worth it, > given how easy it is to sub-class and create your own ndarray that can > have attributes attached. What are others opinions. I'd say leave it off. Many uses of that feature will require custom __array_finalize__ methods anyways. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david.huard at gmail.com Thu Oct 19 12:23:08 2006 From: david.huard at gmail.com (David Huard) Date: Thu, 19 Oct 2006 12:23:08 -0400 Subject: Dimension reduction Message-ID: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Hi, Is there an elegant way to reduce an array but conserve the reduced dimension ? Currently, >>> a = random.random((10,10,10)) >>> a.sum(1).shape (10,10) but i'd like to keep (10,1,10) so I can do a/a.sum(1) directly. Thanks, David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Thu Oct 19 12:39:36 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 19 Oct 2006 11:39:36 -0500 Subject: Dimension reduction In-Reply-To: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> References: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Message-ID: David Huard wrote: > Hi, > > Is there an elegant way to reduce an array but conserve the reduced > dimension ? > > Currently, > >>> a = random.random((10,10,10)) > >>> a.sum(1).shape > (10,10) > > but i'd like to keep (10,1,10) so I can do a/a.sum(1) directly. def nonreducing_reducer(reducing_func, arr, axis): reduced = reducing_func(arr, axis=axis) shape = list(reduced.shape) axis = axis % len(arr.shape) shape.insert(axis, 1) reduced.shape = tuple(shape) return reduced I think. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 12:03:10 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 09:03:10 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <4537954D.1060301@ee.byu.edu> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <45379417.4060601@ieee.org> <4537954D.1060301@ee.byu.edu> Message-ID: <4537A1BE.9000606@ieee.org> Travis Oliphant wrote: > Tim Hochberg wrote: > > >> Travis Oliphant wrote: >> >> >> >>> Tim Hochberg wrote: >>> >>> >>> >>> >>>> Rudolph van der Merwe wrote: >>>> >>>> >>>> >>>> >>>> >>>>> I get the following error with RC3 on a RHE Linux box: >>>>> >>>>> Python 2.4.3 (#4, Mar 31 2006, 12:12:43) >>>>> [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 >>>>> Type "help", "copyright", "credits" or "license" for more information. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>>>> import numpy >>>>>>>> numpy.__version__ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>> '1.0rc3' >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> On a visual studio build, I'm getting a real failure though: >>>> >>>> FAIL: Ticket #112 >>>> ---------------------------------------------------------------------- >>>> Traceback (most recent call last): >>>> File >>>> "C:\Python24\lib\site-packages\numpy\core\tests\test_regression.py", >>>> line 219, in check_longfloat_repr >>>> assert(str(a)[1:9] == str(a[0])[:8]) >>>> AssertionError >>>> >>>> The code in question is dependent on the spelling of INF(or at least the >>>> *length* of the spelling) on a given platform which is why it's failing. >>>> >>>> >>>> >>>> >>>> >>> Actually, you shouldn't be getting an INF at all. This is what the >>> test is designed to test for (so I guess it's working). The test was >>> actually written wrong and was never failing because previously keyword >>> arguments to ufuncs were ignored. >>> >>> Can you show us what 'a' is on your platform. >>> >>> >>> >> Sure: >> >> >>>>> import numpy as N >>>>> a = N.exp(N.array([1000],dtype=N.longfloat)) >>>>> >> Warning: overflow encountered in exp >> >>>>> a >>>>> >> array([1.#INF], dtype=float64) >> >> >> > O.K. We need to modify the test in case to check and see that the size > of longfloat isn't the same as double. > How about: def check_longfloat_repr(self,level=rlevel): """Ticket #112""" if N.dtype(N.longfloat) != N.dtype(N.float): a = N.exp(N.array([1000],dtype=N.longfloat)) assert(str(a)[1:9] == str(a[0])[:8]) That seems to work here. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 12:52:03 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 09:52:03 -0700 Subject: Dimension reduction In-Reply-To: References: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Message-ID: <4537AD33.30306@ieee.org> Robert Kern wrote: > David Huard wrote: > >> Hi, >> >> Is there an elegant way to reduce an array but conserve the reduced >> dimension ? >> >> Currently, >> >>> a = random.random((10,10,10)) >> >>> a.sum(1).shape >> (10,10) >> >> but i'd like to keep (10,1,10) so I can do a/a.sum(1) directly. >> > > def nonreducing_reducer(reducing_func, arr, axis): > reduced = reducing_func(arr, axis=axis) > shape = list(reduced.shape) > axis = axis % len(arr.shape) > shape.insert(axis, 1) > reduced.shape = tuple(shape) > return reduced > > > I think. > > Alternatively (untested): def nonreducing_reducer(reducing_func, arr, axis): return reducing_func(arr.swapaxis(0, axis), axis=0)[newaxis].swapaxis(0, axis) Adding some vertical whitespace probably wouldn't hurt for readability I suppose. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 19 12:54:52 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 19 Oct 2006 10:54:52 -0600 Subject: Dimension reduction In-Reply-To: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> References: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Message-ID: On 10/19/06, David Huard wrote: > > Hi, > > Is there an elegant way to reduce an array but conserve the reduced > dimension ? > > Currently, > >>> a = random.random((10,10,10)) > >>> a.sum(1).shape > (10,10) > > but i'd like to keep (10,1,10) so I can do a/a.sum(1) directly. In [8]: a.sum(1)[:,newaxis,:].shape Out[8]: (10, 1, 10) Don't know if this is as universal as you want, but it works for this. Robert's answer is more general. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Thu Oct 19 13:01:37 2006 From: david.huard at gmail.com (David Huard) Date: Thu, 19 Oct 2006 13:01:37 -0400 Subject: Dimension reduction In-Reply-To: References: <91cf711d0610190923g4cb68378lb76beab15ba448c2@mail.gmail.com> Message-ID: <91cf711d0610191001jd4c2193n478ff61f76e7acb0@mail.gmail.com> 2006/10/19, Robert Kern : > def nonreducing_reducer(reducing_func, arr, axis): > reduced = reducing_func(arr, axis=axis) > shape = list(reduced.shape) > axis = axis % len(arr.shape) > shape.insert(axis, 1) > reduced.shape = tuple(shape) > return reduced Cute ! Here is another one: 1dfunc = lambda x: atleast_1d(reducing_function(x)) apply_along_axis(1dfunc, axis, arr) Is it something people often need ? Would it be worth, (for numpy 1.1 ?), to consider complex axis arguments as a wish to preserve the rank of the array? >>> a.shape (n,m,o,p) >>> a.sum(axis=1j) (n,1,o,p) >>> a.sum(axis=1) (n,o,p) Thanks, David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tim.hochberg at ieee.org Thu Oct 19 13:08:37 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 10:08:37 -0700 Subject: Unit test error with numpy rc3 In-Reply-To: <4537954D.1060301@ee.byu.edu> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <45379417.4060601@ieee.org> <4537954D.1060301@ee.byu.edu> Message-ID: <4537B115.8010207@ieee.org> [CHOP] OK, I've checked in changes to suppress all the warnings in the test suite. I tried to be as targeted as possible so that any regressions from the current state in terms of warnings should show up. I suspect that there may be some issues with regards to masked arrays issuing spurious warnings. I'm not a ma user, so I didn't take the time to investigate this, but someone who cares should probably look at this and consider how to resolve it. Also with regard to the recent addition of the errcall argument to errstate, it appears that if errcall is *not* specified, self.errcall will be set to None. This means that any existing error callback will be removed when entering the block (and replaced when you leave). I don't think that's the desired behavior. Instead, if errcall is not specified, I believe we should leave the value of errcall alone. Does that sound right, or am I missing something? This would result in something like: def __init__(self, **kwargs): if 'errcall' in kwargs: self.errcall = kwargs.pop('errcall') else: self.errcall = geterrcall() self.kwargs = kwargs This still allows you to set the errcall to None for the block by using: with errstate(errcall=None): # do stuff Hmm. Should that be 'errcall' or just 'call'? 'errstate(errcall...)' seems somewhat redundant. Finally, and this is a moot point anyway assuming the above rewrite, is there a reason to use: if 'errcall' in kwargs: self.errcall = kwargs.pop('errcall') else: self.errcall = geterrcall() rather than simply: self.errcall = kwargs.pop('errcall', None) ? regards, -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Thu Oct 19 14:09:37 2006 From: aisaac at american.edu (Alan G Isaac) Date: Thu, 19 Oct 2006 14:09:37 -0400 Subject: indexing Message-ID: John J. worked up short set of NumPy Indexing Examples that I think supplements what is at and so I made a couple edits and added them: Please edit as appropriate. Thank you, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jkgruet at sandia.gov Thu Oct 19 15:14:52 2006 From: jkgruet at sandia.gov (James K. Gruetzner) Date: Thu, 19 Oct 2006 13:14:52 -0600 Subject: numpy - scipy version hell Message-ID: <200610191314.54482.jkgruet@sandia.gov> I'm using Fedora core 5, that insect-laden wonder, which includes python 2.4.3. I'm trying to install numpy (scipy-core) and scipy (the rest). I can't seem to get around package conflicts. I've tried two methods, one in which numpy fails and one in which scipy fails. METHOD 1 (numpy fails) Download numpy (numpy-1.0r3); untarball and install. Test by running python and importing numpy. Looks OK, but notes "running from numpy source directory". So, test by running python from a different directory and importing numpy. Get the following error: >>> import numpy Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.4/site-packages/numpy/__init__.py", line 40, in ? import linalg File "/usr/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, in ? from linalg import * File "/usr/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 25, in ? from numpy.linalg import lapack_lite ImportError: /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: s_cat (The other method, where scipy fails, I'm asking for help on the scipy-user list.)) I suspect that there's a bit of out-of-sync with the packages, but am not sure. Bottom line question: how do I get both installed at once? Thanks! James -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Thu Oct 19 15:29:26 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 19 Oct 2006 14:29:26 -0500 Subject: numpy - scipy version hell In-Reply-To: <200610191314.54482.jkgruet@sandia.gov> References: <200610191314.54482.jkgruet@sandia.gov> Message-ID: James K. Gruetzner wrote: > I'm using Fedora core 5, that insect-laden wonder, which includes python > 2.4.3. I'm trying to install numpy (scipy-core) and scipy (the rest). I > can't seem to get around package conflicts. I've tried two methods, one in > which numpy fails and one in which scipy fails. > > METHOD 1 (numpy fails) > Download numpy (numpy-1.0r3); untarball and install. > Test by running python and importing numpy. Looks OK, but notes "running from > numpy source directory". So, test by running python from a different > directory and importing numpy. Get the following error: > >>>> import numpy > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/lib/python2.4/site-packages/numpy/__init__.py", line 40, in ? > import linalg > File "/usr/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, > in ? > from linalg import * > File "/usr/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 25, > in ? > from numpy.linalg import lapack_lite > ImportError: /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: > undefined symbol: s_cat It looks like you linked against a FORTRAN LAPACK, but didn't manage to link the FORTRAN runtime library libg2c. Can you give us the output of your build? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tgrav at mac.com Thu Oct 19 15:54:21 2006 From: tgrav at mac.com (Tommy Grav) Date: Thu, 19 Oct 2006 15:54:21 -0400 Subject: Installation problem Message-ID: <08626899-A3C2-4963-9B07-E2936BAFF9F9@mac.com> I am on a Mac OS X and are trying to install numpy/scipy/matplotlib ActivePython 2.4.3 Build 11 (ActiveState Software Inc.) based on Python 2.4.3 (#1, Apr 3 2006, 18:07:18) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.1.2881' >>> import scipy >>> scipy.__version__ '0.5.0.2095' >>> import pylab RuntimeError: module compiled against version 1000002 of C-API but this version of numpy is 1000000 The import of the numpy version of the nxutils module, _nsnxutils, failed. This is is either because numpy was unavailable when matplotlib was compiled, because a dependency of _nsnxutils could not be satisfied, or because the build flag for this module was turned off in setup.py. If it appears that _nsnxutils was not built, make sure you have a working copy of numpy and then re-install matplotlib. Otherwise, the following traceback gives more details: Traceback (most recent call last): File "", line 1, in ? File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/pylab.py", line 1, in ? from matplotlib.pylab import * File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/matplotlib/pylab.py", line 199, in ? import mlab #so I can override hist, psd, etc... File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/matplotlib/mlab.py", line 64, in ? import nxutils File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/matplotlib/nxutils.py", line 17, in ? from matplotlib._ns_nxutils import * ImportError: numpy.core.multiarray failed to import >>> Anyone know what the problem is? Cheers Tommy tgrav at mac.com http://homepage.mac.com/tgrav/ "Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction" -- Albert Einstein -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 19 16:39:55 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 19 Oct 2006 14:39:55 -0600 Subject: Installation problem In-Reply-To: <08626899-A3C2-4963-9B07-E2936BAFF9F9@mac.com> References: <08626899-A3C2-4963-9B07-E2936BAFF9F9@mac.com> Message-ID: On 10/19/06, Tommy Grav wrote: > > I am on a Mac OS X and are trying to install numpy/scipy/matplotlib > > ActivePython 2.4.3 Build 11 (ActiveState Software Inc.) based on > Python 2.4.3 (#1, Apr 3 2006, 18:07:18) > [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > >>> numpy.__version__ > '1.1.2881' > >>> import scipy > >>> scipy.__version__ > '0.5.0.2095' > >>> import pylab > RuntimeError: module compiled against version 1000002 of C-API but this > version of numpy is 1000000 > You need a later version of numpy. The latest is 1000006 so you will probably need to also recompile both matplotlib and scipy against numpy latest. This is a recent check that Travis put in to track C-API changes. If you don't compile your own I don't know what you can do except wait. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Thu Oct 19 16:46:48 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 19 Oct 2006 17:46:48 -0300 Subject: adding an attribute to an nd-array In-Reply-To: References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: On 10/19/06, Robert Kern wrote: > Travis Oliphant wrote: > > I don't think the extra bytes for every ndarray object are worth it, > > given how easy it is to sub-class and create your own ndarray that can > > have attributes attached. What are others opinions. > > I'd say leave it off. Many uses of that feature will require custom > __array_finalize__ methods anyways. > I second Travis and Robert. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Thu Oct 19 16:59:49 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 19 Oct 2006 13:59:49 -0700 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: <4537E745.7010403@noaa.gov> Travis Oliphant wrote: > Actually something as simple as > > class InfoArray(N.ndarray): > pass > > will allow you to add attributes to InfoArray. Well, sure, but how the heck do you initialize it? >>> class InfoArray(N.ndarray): ... pass ... >>> InfoArray((1,2,3)) InfoArray([[[ 6.61855173e-306, nan, 4.94708449e+173], [ 3.32457336e-309, 1.08825491e+175, 2.12468326e-314]]]) InfoArray(N.array((1,2,3))) InfoArray([[[ 2.17854722e-305, 1.90979621e-313, 1.90979621e-313], [ 1.90979621e-313, 1.90979621e-313, 1.03977794e-312]]]) I, for one, would like a very easy to subclass version of ndarray, maybe one that had a default constructor like numpy.array(). oh, and: Charles R Harris wrote: > I'd be more inclined to worry about speed. One of the drawbacks of > numarray was the time it took to create arrays. Since I often use many > small arrays, numpy was a big improvement in that area. +1 keep basic arrays simple and lean. Wasn't there one a UserArray class or something? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Thu Oct 19 17:00:09 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 19 Oct 2006 14:00:09 -0700 Subject: questions regarding assignement and copy In-Reply-To: References: <45360904.3030703@ar.media.kyoto-u.ac.jp> Message-ID: <4537E759.3050606@noaa.gov> Robert Kern wrote: > From the reference manual: > > http://docs.python.org/ref/augassign.html > > """An augmented assignment expression like x += 1 can be rewritten as x = x + 1 > to achieve a similar, but not exactly equal effect. In the augmented version, x > is only evaluated once. Also, when possible, the actual operation is performed > in-place, meaning that rather than creating a new object and assigning that to > the target, the old object is modified instead.""" I've always thought this was a mistake -- it is a source of weird errors and bugs. AFAIC, the augmented assignments should ONLY work with mutable objects, and ALWAYS do the operation in place. But then you couldn't write: i = 1 i += 1 Because python numbers are immutable. I think the problem here is that augmented assignment is solving two distinct problems: in place operations and nice syntax for incrementing. Oh well, it's not that big a deal, and usually the confusion causes errors that show up right away. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Thu Oct 19 17:34:57 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 19 Oct 2006 14:34:57 -0700 Subject: adding an attribute to an nd-array In-Reply-To: <4537E745.7010403@noaa.gov> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <4537E745.7010403@noaa.gov> Message-ID: <4537EF81.3090402@ieee.org> Christopher Barker wrote: > Travis Oliphant wrote: > >> Actually something as simple as >> >> class InfoArray(N.ndarray): >> pass >> >> will allow you to add attributes to InfoArray. >> > > Well, sure, but how the heck do you initialize it? > > >>> class InfoArray(N.ndarray): > ... pass > ... > >>> InfoArray((1,2,3)) > InfoArray([[[ 6.61855173e-306, nan, 4.94708449e+173], > [ 3.32457336e-309, 1.08825491e+175, 2.12468326e-314]]]) > > InfoArray(N.array((1,2,3))) > InfoArray([[[ 2.17854722e-305, 1.90979621e-313, 1.90979621e-313], > [ 1.90979621e-313, 1.90979621e-313, 1.03977794e-312]]]) > > I, for one, would like a very easy to subclass version of ndarray, maybe > one that had a default constructor like numpy.array(). > > Back when I was working on basearray, I had an idea for this. Basically, it was to to move the ndarray constructor[1] over to basearray and make the ndarray constructor work just like array does now[2]. Thus array(args) and ndarray(args) would behave the same[2 again]. For the full power/complexity of the current ndarray constructor, you would instead use basearray(). I believe (and I did some experiments on this at one point), that this would allow straightforward inheritance from ndarray. I've been away from this for several months now, so sadly I forget many of the details. -tim [1] I know, I know, it's really __new__, but it gets used like a constructor so I'll call it one for the time being. [2] Modulo the copy arg which would disappear if I had anything to with it. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From markusro at element.fkp.physik.tu-darmstadt.de Thu Oct 19 17:56:05 2006 From: markusro at element.fkp.physik.tu-darmstadt.de (Markus Rosenstihl) Date: Thu, 19 Oct 2006 23:56:05 +0200 Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 Message-ID: <5228203bb5a6003c571d52a5e94ef370@element.fkp.physik.tu-darmstadt.de> Hi! I try to compile numpy rc3 on Panther and get following errors. (I start build with "python2.3 setup.py build" to be sure to use the python shipped with OS X. I din't manage to compile Python2.5 either yet with similar errors) Does anynbody has an Idea? gcc-3.3 XCode 1.5 November gcc updater is installed regards Markus python2.3 setup.py build ... compile options: '-Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src -Inumpy/core/include -Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core -Inumpy/core/src -Inumpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/ python2.3 -c' gcc: numpy/core/src/multiarraymodule.c In file included from numpy/core/src/arrayobject.c:511, from numpy/core/src/multiarraymodule.c:63: numpy/core/src/arraytypes.inc.src: In function `LONGDOUBLE_scan': numpy/core/src/arraytypes.inc.src:883: warning: long double format, npy_longdouble arg (arg 3) gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/ multiarraymodule.o -o build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so ld: Undefined symbols: _fstatvfs referenced from Python expected to be defined in libSystem _lchown referenced from Python expected to be defined in libSystem _statvfs referenced from Python expected to be defined in libSystem ld: Undefined symbols: _fstatvfs referenced from Python expected to be defined in libSystem _lchown referenced from Python expected to be defined in libSystem _statvfs referenced from Python expected to be defined in libSystem error: Command "gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/ multiarraymodule.o -o build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so" failed with exit status 1 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mrosenstihl at macnews.de Thu Oct 19 17:55:37 2006 From: mrosenstihl at macnews.de (Markus Rosenstihl) Date: Thu, 19 Oct 2006 23:55:37 +0200 Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 Message-ID: <2370f9608954a8ab2caf96b20612a349@macnews.de> Hi! I try to compile numpy rc3 on Panther and get following errors. (I start build with "python2.3 setup.py build" to be sure to use the python shipped with OS X. I din't manage to compile Python2.5 either yet with similar errors) Does anynbody has an Idea? gcc-3.3 XCode 1.5 November gcc updater is installed regards Markus python2.3 setup.py build ... compile options: '-Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src -Inumpy/core/include -Ibuild/src.darwin-7.9.0-Power_Macintosh-2.3/numpy/core -Inumpy/core/src -Inumpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.3/include/ python2.3 -c' gcc: numpy/core/src/multiarraymodule.c In file included from numpy/core/src/arrayobject.c:511, from numpy/core/src/multiarraymodule.c:63: numpy/core/src/arraytypes.inc.src: In function `LONGDOUBLE_scan': numpy/core/src/arraytypes.inc.src:883: warning: long double format, npy_longdouble arg (arg 3) gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/ multiarraymodule.o -o build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so ld: Undefined symbols: _fstatvfs referenced from Python expected to be defined in libSystem _lchown referenced from Python expected to be defined in libSystem _statvfs referenced from Python expected to be defined in libSystem ld: Undefined symbols: _fstatvfs referenced from Python expected to be defined in libSystem _lchown referenced from Python expected to be defined in libSystem _statvfs referenced from Python expected to be defined in libSystem error: Command "gcc -Wl,-F. -Wl,-F. -bundle -framework Python build/temp.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/src/ multiarraymodule.o -o build/lib.darwin-7.9.0-Power_Macintosh-2.3/numpy/core/multiarray.so" failed with exit status 1 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2427 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 19 19:00:38 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 19 Oct 2006 17:00:38 -0600 Subject: adding an attribute to an nd-array In-Reply-To: <4537E745.7010403@noaa.gov> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <4537E745.7010403@noaa.gov> Message-ID: <45380396.7040300@ee.byu.edu> Christopher Barker wrote: >Travis Oliphant wrote: > > >>Actually something as simple as >> >>class InfoArray(N.ndarray): >> pass >> >>will allow you to add attributes to InfoArray. >> >> > >Well, sure, but how the heck do you initialize it? > > You use the same constructor as ndarray has. numpy.info(numpy.ndarray) If you want an array-like function that produces the array, you do array(obj).view(InfoArray) or wrap that up in your own function. There are many over-lapping ways to construct ndarray's. You can use all of these ways to construct instances of your own type by getting the ndarray and using the .view() method. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 19 19:09:44 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 20 Oct 2006 01:09:44 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <4537E745.7010403@noaa.gov> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <4537E745.7010403@noaa.gov> Message-ID: <20061019230944.GB10843@mentat.za.net> On Thu, Oct 19, 2006 at 01:59:49PM -0700, Christopher Barker wrote: > Travis Oliphant wrote: > > Actually something as simple as > > > > class InfoArray(N.ndarray): > > pass > > > > will allow you to add attributes to InfoArray. > > Well, sure, but how the heck do you initialize it? Looks like x = N.array([1,2,3]) x.view(InfoArray) works. Thanks to everyone for the useful feedback! Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Thu Oct 19 20:19:37 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 20 Oct 2006 02:19:37 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <20061019002914.GA20609@mentat.za.net> <200610182117.49777.pgmdevlist@gmail.com> <20061019112032.GE20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> Message-ID: <20061020001937.GC10843@mentat.za.net> On Thu, Oct 19, 2006 at 09:45:02AM -0600, Travis Oliphant wrote: > Stefan van der Walt wrote: > > >If I understand correctly, the following should work: > > > >import numpy as N > > > >class InfoArray(N.ndarray): > > def __new__(info_arr_cls,arr,info={}): > > info_arr_cls.info = info > > return N.array(arr).view(info_arr_cls) One has to be careful of this approach. It ads *the same* information to all arrays, i.e. In [2]: a = ImageInfo(N.array([1,2,3]),{1:1}) In [3]: b = ImageInfo(N.array([1,2,3]),{1:2}) In [4]: a Out[4]: ImageInfo([1, 2, 3]) In [5]: b Out[5]: ImageInfo([1, 2, 3]) In [6]: a.info Out[6]: {1: 2} In [7]: b.info Out[7]: {1: 2} Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From parlar at gmail.com Thu Oct 19 20:53:32 2006 From: parlar at gmail.com (Jay Parlar) Date: Thu, 19 Oct 2006 20:53:32 -0400 Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 Message-ID: > Hi! > I try to compile numpy rc3 on Panther and get following errors. > (I start build with "python2.3 setup.py build" to be sure to use the > python shipped with OS X. I din't manage to compile Python2.5 either > yet with similar errors) > Does anynbody has an Idea? > gcc-3.3 > XCode 1.5 > November gcc updater is installed > I couldn't get numpy building with Python 2.5 on 10.3.9 (although I had different compile errors). The solution that ended up working for me was Python 2.4. There's a bug in the released version of Python 2.5 that's preventing it from working with numpy, should be fixed in the next release. You can find a .dmg for Python 2.4 here: http://pythonmac.org/packages/py24-fat/index.html Jay P. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Thu Oct 19 21:06:23 2006 From: pgmdevlist at gmail.com (P GM) Date: Thu, 19 Oct 2006 21:06:23 -0400 Subject: Stefan van der Walt Message-ID: <777651ce0610191806s3b394699s9ad7072094757595@mail.gmail.com> > > >class InfoArray(N.ndarray): > > > def __new__(info_arr_cls,arr,info={}): > > > info_arr_cls.info = info > > > return N.array(arr).view(info_arr_cls) > > One has to be careful of this approach. It ads *the same* information > to all arrays, i.e. Indeed. That's basically why you have to edit your __array_finalize__ . class InfoArray(N.ndarray): def __new__(info_arr_cls,arr,info={}): info_arr_cls._info = info return N.array(arr).view(info_arr_cls) def __array_finalize__(self, obj): if hasattr(obj,'info'): self.info = obj.info else: self.info = self._info return OK, so you end up w/ two attributes 'info' and '_info', the latter having the info you want, the latter playing a temporary placeholder. That looks a bit overkill, but that works pretty nice. a = InfoArray(N.array([1,2,3]),{1:1}) b = InfoArray(N.array([1,2,3]),{1:2}) assert a.info=={1:1} assert b.info=={1:2} assert (a+1).info==a.info assert (b-2).info==b.info -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Thu Oct 19 21:03:57 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Thu, 19 Oct 2006 21:03:57 -0400 Subject: adding an attribute to an nd-array In-Reply-To: <20061020001937.GC10843@mentat.za.net> References: <20061019002914.GA20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <20061020001937.GC10843@mentat.za.net> Message-ID: <200610192103.58265.pgmdevlist@gmail.com> > > >class InfoArray(N.ndarray): > > > def __new__(info_arr_cls,arr,info={}): > > > info_arr_cls.info = info > > > return N.array(arr).view(info_arr_cls) > > One has to be careful of this approach. It ads *the same* information > to all arrays, i.e. Indeed. That's basically why you have to edit your __array_finalize__ . class InfoArray(N.ndarray): def __new__(info_arr_cls,arr,info={}): info_arr_cls._info = info return N.array(arr).view(info_arr_cls) def __array_finalize__(self, obj): if hasattr(obj,'info'): self.info = obj.info else: self.info = self._info return OK, so you end up w/ two attributes 'info' and '_info', the latter having the info you want, the latter playing a temporary placeholder. That looks a bit overkill, but that works pretty nice. a = InfoArray(N.array([1,2,3]),{1:1}) b = InfoArray(N.array([1,2,3]),{1:2}) assert a.info=={1:1} assert b.info=={1:2} assert (a+1).info==a.info assert (b-2).info==b.info ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 20 03:01:46 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 20 Oct 2006 03:01:46 -0400 Subject: The NumPy Fortran-ordering quiz In-Reply-To: <4535B0F1.1000603@ieee.org> References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> <4535B0F1.1000603@ieee.org> Message-ID: On 18/10/06, Travis Oliphant wrote: > If there are any cases satisfying these rules where a copy does not have > to occur then let me know. For example, zeros((4,4))[:,1].reshape((2,2)) need not be copied. I filed a bug in trac and supplied a patch to multiarray.c that avoids copies in PyArray_NewShape unless absolutely necessary. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Fri Oct 20 03:31:03 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 20 Oct 2006 01:31:03 -0600 Subject: The NumPy Fortran-ordering quiz In-Reply-To: References: <20061018002445.GP17998@mentat.za.net> <45358A8D.600@ieee.org> <4535B0F1.1000603@ieee.org> Message-ID: <45387B37.8080303@ee.byu.edu> A. M. Archibald wrote: >On 18/10/06, Travis Oliphant wrote: > > > >>If there are any cases satisfying these rules where a copy does not have >>to occur then let me know. >> >> > >For example, zeros((4,4))[:,1].reshape((2,2)) need not be copied. > >I filed a bug in trac and supplied a patch to multiarray.c that avoids >copies in PyArray_NewShape unless absolutely necessary. > > > Very, very nice. Thanks. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Fri Oct 20 04:47:05 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 20 Oct 2006 10:47:05 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <200610192103.58265.pgmdevlist@gmail.com> References: <20061019002914.GA20609@mentat.za.net> <45379D7E.30609@ee.byu.edu> <20061020001937.GC10843@mentat.za.net> <200610192103.58265.pgmdevlist@gmail.com> Message-ID: <20061020084705.GD10843@mentat.za.net> On Thu, Oct 19, 2006 at 09:03:57PM -0400, Pierre GM wrote: > Indeed. That's basically why you have to edit your __array_finalize__ . > > class InfoArray(N.ndarray): > def __new__(info_arr_cls,arr,info={}): > info_arr_cls._info = info > return N.array(arr).view(info_arr_cls) > def __array_finalize__(self, obj): > if hasattr(obj,'info'): > self.info = obj.info > else: > self.info = self._info > return > > OK, so you end up w/ two attributes 'info' and '_info', the latter having the > info you want, the latter playing a temporary placeholder. That looks a bit > overkill, but that works pretty nice. Is there any reason why one can't simply do class InfoArray(N.ndarray): def __new__(info_arr_cls,arr,info={}): x = N.array(arr).view(info_arr_cls) x.info = info return x def __array_finalize__(self, obj): if hasattr(obj,'info'): self.info = obj.info return Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Sebastien.Bardeau at obs.u-bordeaux1.fr Fri Oct 20 05:42:26 2006 From: Sebastien.Bardeau at obs.u-bordeaux1.fr (Sebastien Bardeau) Date: Fri, 20 Oct 2006 11:42:26 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? Message-ID: <45389A02.30901@obs.u-bordeaux1.fr> Hi! I am confused with Numpy behavior with its scalar or 0-d arrays objects: >>> numpy.__version__ '1.0rc2' >>> a = numpy.array((1,2,3)) >>> b = a[:2] >>> b += 1 >>> b array([2, 3]) >>> a array([2, 3, 3]) >>> type(b) To this point all is ok for me: subarrays share (by default) memory with their parent array. But: >>> c = a[2] >>> c += 1 >>> c 4 >>> a array([2, 3, 3]) >>> type(c) >>> id(c) 169457808 >>> c += 1 >>> id(c) 169737448 That's really confusing, because slices (from __getslice__ method) are not copies (they share memory), and items (single elements from __getitem__ ) are copies to one of the scalar objects provided by Numpy. I can understand that numpy.scalars do not provide inplace operations (like Python standard scalars, they are immutable), so I'd like to use 0-d Numpy.ndarrays. But: >>> d = numpy.array(a[2],copy=False) >>> d += 1 >>> d array(4) >>> a array([2, 3, 3]) >>> type(d) >>> d.shape () >>> id(d) 169621280 >>> d += 1 >>> id(d) 169621280 This is not a solution because d is a copy since construction time... My question is: is there a way to get a single element of an array into a 0-d array which shares memory with its parent array? Thx for your help, Sebastien ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From EAGF at HOTMAIL.COM Fri Oct 20 06:54:15 2006 From: EAGF at HOTMAIL.COM (=?iso-2022-jp?B?RUFHRg==?=) Date: Fri, 20 Oct 2006 03:54:15 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCJCQkRCRHJGIkSSQzJEcbKEI=?= =?iso-2022-jp?b?GyRCJGIlJCUrJDk9UDJxJCRFNzlxGyhC?= Message-ID: ?????????????????????????? http://carmastorra.com/ika/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Fri Oct 20 06:55:57 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 20 Oct 2006 12:55:57 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <45389A02.30901@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> Message-ID: <200610201255.57968.faltet@carabos.com> A Divendres 20 Octubre 2006 11:42, Sebastien Bardeau va escriure: [snip] > I can understand that numpy.scalars do not provide inplace operations > (like Python standard scalars, they are immutable), so I'd like to use > > 0-d Numpy.ndarrays. But: > >>> d = numpy.array(a[2],copy=False) > >>> d += 1 > >>> d > > array(4) > > >>> a > > array([2, 3, 3]) > > >>> type(d) > > > > >>> d.shape > > () > > >>> id(d) > > 169621280 > > >>> d += 1 > >>> id(d) > > 169621280 > > This is not a solution because d is a copy since construction time... > My question is: is there a way to get a single element of an array into > a 0-d array which shares memory with its parent array? One possible solution (there can be more) is using ndarray: In [47]: a=numpy.array([1,2,3], dtype="i4") In [48]: n=1 # the position that you want to share In [49]: b=numpy.ndarray(buffer=a[n:n+1], shape=(), dtype="i4") In [50]: a Out[50]: array([1, 2, 3]) In [51]: b Out[51]: array(2) In [52]: b += 1 In [53]: b Out[53]: array(3) In [54]: a Out[54]: array([1, 3, 3]) Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From markusro at element.fkp.physik.tu-darmstadt.de Fri Oct 20 07:13:49 2006 From: markusro at element.fkp.physik.tu-darmstadt.de (Markus Rosenstihl) Date: Fri, 20 Oct 2006 13:13:49 +0200 Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 In-Reply-To: References: Message-ID: <59a9095b29d3fa277f2e853a35590c72@element.fkp.physik.tu-darmstadt.de> Am 20.10.2006 um 02:53 schrieb Jay Parlar: >> Hi! >> I try to compile numpy rc3 on Panther and get following errors. >> (I start build with "python2.3 setup.py build" to be sure to use the >> python shipped with OS X. I din't manage to compile Python2.5 either >> yet with similar errors) >> Does anynbody has an Idea? >> gcc-3.3 >> XCode 1.5 >> November gcc updater is installed >> > > I couldn't get numpy building with Python 2.5 on 10.3.9 (although I > had different compile errors). The solution that ended up working for > me was Python 2.4. There's a bug in the released version of Python 2.5 > that's preventing it from working with numpy, should be fixed in the > next release. > > You can find a .dmg for Python 2.4 here: > http://pythonmac.org/packages/py24-fat/index.html > > Jay P. > I have that installed already but i get some bus errors with that. Furthermore it is built with gcc4 and i need to compile an extra module(pytables) and I fear that will not work, hence I try to compile myself. Python 2.5 dosent't compile either (libSystemStubs is only on Tiger). The linking works when i remove the -lSystemStubs and it compiled clean. Numpy rc3 wass also compiling now with python 2.5, but the tests failed: Python 2.5 (r25:51908, Oct 20 2006, 11:40:08) [GCC 3.3 20030304 (Apple Computer, Inc. build 1671)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.test(10) Found 5 tests for numpy.distutils.misc_util Found 4 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 32 tests for numpy.linalg Found 13 tests for numpy.core.umath Found 4 tests for numpy.core.scalarmath Found 9 tests for numpy.lib.arraysetops Found 42 tests for numpy.lib.type_check Found 183 tests for numpy.core.multiarray Found 3 tests for numpy.fft.helper Found 36 tests for numpy.core.ma Found 1 tests for numpy.lib.ufunclike Found 12 tests for numpy.lib.twodim_base Found 10 tests for numpy.core.defmatrix Found 4 tests for numpy.ctypeslib Found 41 tests for numpy.lib.function_base Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 28 tests for numpy.core.numeric Found 4 tests for numpy.lib.index_tricks Found 47 tests for numpy.lib.shape_base Found 0 tests for __main__ ........................................................................ ................................Warning: invalid value encountered in divide ..Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ..Warning: invalid value encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ..Warning: invalid value encountered in divide ..Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide .Warning: divide by zero encountered in divide ........Warning: invalid value encountered in divide .Warning: invalid value encountered in divide ..Warning: divide by zero encountered in divide ........................................................................ ..............................................Warning: overflow encountered in exp F....................................................................... ..........Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide Warning: divide by zero encountered in divide .....................Warning: invalid value encountered in sqrt Warning: invalid value encountered in log Warning: invalid value encountered in log10 ..Warning: invalid value encountered in sqrt Warning: invalid value encountered in sqrt Warning: divide by zero encountered in log Warning: divide by zero encountered in log Warning: divide by zero encountered in log10 Warning: divide by zero encountered in log10 Warning: invalid value encountered in arcsin Warning: invalid value encountered in arcsin Warning: invalid value encountered in arccos Warning: invalid value encountered in arccos Warning: invalid value encountered in arccosh Warning: invalid value encountered in arccosh Warning: divide by zero encountered in arctanh Warning: divide by zero encountered in arctanh Warning: invalid value encountered in divide Warning: invalid value encountered in true_divide Warning: invalid value encountered in floor_divide Warning: invalid value encountered in remainder Warning: invalid value encountered in fmod ........................................................................ ........................................................................ ................. ====================================================================== FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- packages/numpy/core/tests/test_regression.py", line 219, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError ---------------------------------------------------------------------- Ran 519 tests in 3.760s FAILED (failures=1) >>> regards Markus ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gael.varoquaux at normalesup.org Fri Oct 20 07:28:52 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 20 Oct 2006 13:28:52 +0200 Subject: Helper function to "unroll" a array Message-ID: <20061020112848.GB3555@clipper.ens.fr> Hi, There is an operation I do a lot, I would call it "unrolling" a array. The best way to describe it is probably to give the code: def unroll(M): """ Flattens the array M and returns a 2D array with the first columns being the indices of M, and the last column the flatten M. """ return hstack((indices(M.shape).reshape(-1,M.ndim),M.reshape(-1,1))) Example: >>> M array([[ 0.73530097, 0.3553424 , 0.3719772 ], [ 0.83353373, 0.74622133, 0.14748905], [ 0.72023762, 0.32306969, 0.19142366]]) >>> unroll(M) array([[ 0. , 0. , 0.73530097], [ 0. , 1. , 0.3553424 ], [ 1. , 1. , 0.3719772 ], [ 2. , 2. , 0.83353373], [ 2. , 0. , 0.74622133], [ 1. , 2. , 0.14748905], [ 0. , 1. , 0.72023762], [ 2. , 0. , 0.32306969], [ 1. , 2. , 0.19142366]]) The docstring sucks. The function is trivial (when you know numpy a bit). Maybe this function already exists in numpy, if so I couldn't find it. Elsewhere I propose it for inclusion. Cheers, Ga?l ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Fri Oct 20 09:22:33 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Fri, 20 Oct 2006 15:22:33 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <45389A02.30901@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> Message-ID: <20061020132232.GE10843@mentat.za.net> On Fri, Oct 20, 2006 at 11:42:26AM +0200, Sebastien Bardeau wrote: > >>> a = numpy.array((1,2,3)) > >>> b = a[:2] Here you index by a slice. > >>> c = a[2] Whereas here you index by a scalar. So you want to do b = a[[2]] b += 1 or in the general case b = a[slice(2,3)] b += 1 Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Sebastien.Bardeau at obs.u-bordeaux1.fr Fri Oct 20 09:25:49 2006 From: Sebastien.Bardeau at obs.u-bordeaux1.fr (Sebastien Bardeau) Date: Fri, 20 Oct 2006 15:25:49 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <200610201255.57968.faltet@carabos.com> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> Message-ID: <4538CE5D.5000903@obs.u-bordeaux1.fr> > One possible solution (there can be more) is using ndarray: > > In [47]: a=numpy.array([1,2,3], dtype="i4") > In [48]: n=1 # the position that you want to share > In [49]: b=numpy.ndarray(buffer=a[n:n+1], shape=(), dtype="i4") > Ok thanks. Actually that was also the solution I found. But this is much more complicated when arrays are N dimensional with N>1, and above all if user asks for a slice in one or more dimension. Here is how I redefine the __getitem__ method for my arrays. Remember that the goal is to return a 0-d array rather than a numpy.scalar when I extract a single element out of a N-dimensional (N>=1) array: def __getitem__(self,index): # Index may be either an int or a tuple # Index length: if type(index) == int: # A single element through first dimension ilen = 1 index = (index,) # A tuple else: ilen = len(index) # Array rank: arank = len(self.shape) # Check if there is a slice: for i in index: if type(i) == slice: hasslice = True else: hasslice = False # Array is already a 0-d array: if arank == 0 and index == (0,): return self[()] elif arank == 0: raise IndexError, "0-d array has only one element at index 0." # This will return a single element as a 0-d array: elif arank == ilen and hasslice: # This ugly thing returns a numpy 0-D array AND NOT a numpy scalar! # (Numpy scalars do not share their data with the parent array) newindex = list(index) newindex[0] = slice(index[0],index[0]+1,None) newindex = tuple(newindex) return self[newindex].reshape(()) # This will return a n-D subarray (n>=1): else: return self[index] Well... I do not think this is very nice. Someone has another idea? My question in my first post was: is there a way to get a single element of an array into a 0-d array which shares memory with its parent array? Sebastien ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 20 09:44:58 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 06:44:58 -0700 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <200610201255.57968.faltet@carabos.com> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> Message-ID: <4538D2DA.9010604@ieee.org> Francesc Altet wrote: > A Divendres 20 Octubre 2006 11:42, Sebastien Bardeau va escriure: > [snip] > >> I can understand that numpy.scalars do not provide inplace operations >> (like Python standard scalars, they are immutable), so I'd like to use >> >> 0-d Numpy.ndarrays. But: >> >>> d = numpy.array(a[2],copy=False) >> >>> d += 1 >> >>> d >> >> array(4) >> >> >>> a >> >> array([2, 3, 3]) >> >> >>> type(d) >> >> >> >> >>> d.shape >> >> () >> >> >>> id(d) >> >> 169621280 >> >> >>> d += 1 >> >>> id(d) >> >> 169621280 >> >> This is not a solution because d is a copy since construction time... >> My question is: is there a way to get a single element of an array into >> a 0-d array which shares memory with its parent array? >> > > One possible solution (there can be more) is using ndarray: [SNIP] Here's a slightly more concise version of the same idea: b = a[n:n+1].reshape([]) -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Sebastien.Bardeau at obs.u-bordeaux1.fr Fri Oct 20 09:49:15 2006 From: Sebastien.Bardeau at obs.u-bordeaux1.fr (Sebastien Bardeau) Date: Fri, 20 Oct 2006 15:49:15 +0200 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <4538CE5D.5000903@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> <4538CE5D.5000903@obs.u-bordeaux1.fr> Message-ID: <4538D3DB.6040707@obs.u-bordeaux1.fr> Ooops sorry there was two mistakes with the 'hasslice' flag. This seems now to work for me. def __getitem__(self,index): # Index may be either an int or a tuple # Index length: if type(index) == int: # A single element through first dimension ilen = 1 index = (index,) # A tuple else: ilen = len(index) # Array rank: arank = len(self.shape) # Check if there is a slice: hasslice = False for i in index: if type(i) == slice: hasslice = True # Array is already a 0-d array: if arank == 0 and index == (0,): return self elif arank == 0: raise IndexError, "0-d array has only one element at index 0." # This will return a single element as a 0-d array: elif arank == ilen and not hasslice: # This ugly thing returns a numpy 0-D array AND NOT a numpy scalar! # (Numpy scalars do not share their data with the parent array) newindex = list(index) newindex[0] = slice(index[0],index[0]+1,None) newindex = tuple(newindex) return self[newindex].reshape(()) # This will return a n-D subarray (n>=1): else: return self[index] Sebastien Bardeau wrote: >> One possible solution (there can be more) is using ndarray: >> >> In [47]: a=numpy.array([1,2,3], dtype="i4") >> In [48]: n=1 # the position that you want to share >> In [49]: b=numpy.ndarray(buffer=a[n:n+1], shape=(), dtype="i4") >> >> > Ok thanks. Actually that was also the solution I found. But this is much > more complicated when arrays are N dimensional with N>1, and above all > if user asks for a slice in one or more dimension. Here is how I > redefine the __getitem__ method for my arrays. Remember that the goal is > to return a 0-d array rather than a numpy.scalar when I extract a single > element out of a N-dimensional (N>=1) array: > > def __getitem__(self,index): # Index may be either an int or a tuple > # Index length: > if type(index) == int: # A single element through first dimension > ilen = 1 > index = (index,) # A tuple > else: > ilen = len(index) > # Array rank: > arank = len(self.shape) > # Check if there is a slice: > for i in index: > if type(i) == slice: > hasslice = True > else: > hasslice = False > # Array is already a 0-d array: > if arank == 0 and index == (0,): > return self[()] > elif arank == 0: > raise IndexError, "0-d array has only one element at index 0." > # This will return a single element as a 0-d array: > elif arank == ilen and hasslice: > # This ugly thing returns a numpy 0-D array AND NOT a numpy scalar! > # (Numpy scalars do not share their data with the parent array) > newindex = list(index) > newindex[0] = slice(index[0],index[0]+1,None) > newindex = tuple(newindex) > return self[newindex].reshape(()) > # This will return a n-D subarray (n>=1): > else: > return self[index] > > Well... I do not think this is very nice. Someone has another idea? My > question in my first post was: is there a way to get a single element of > an array into > a 0-d array which shares memory with its parent array? > > Sebastien > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > -- ------------------------- Sebastien Bardeau L3AB - CNRS UMR 5804 2 rue de l'observatoire BP 89 F - 33270 Floirac Tel: (+33) 5 57 77 61 46 ------------------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 20 10:17:32 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 07:17:32 -0700 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <4538D3DB.6040707@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> <4538CE5D.5000903@obs.u-bordeaux1.fr> <4538D3DB.6040707@obs.u-bordeaux1.fr> Message-ID: <4538DA7C.5020806@ieee.org> Sebastien Bardeau wrote: > Ooops sorry there was two mistakes with the 'hasslice' flag. This seems > now to work for me. > > [SNIP code] That looks overly complicated. I believe that this (minimally tested in a slightly different setting) or some variation should work: return self[...,newaxis][index].reshape(self[index].shape) -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david.huard at gmail.com Fri Oct 20 13:47:28 2006 From: david.huard at gmail.com (David Huard) Date: Fri, 20 Oct 2006 13:47:28 -0400 Subject: histogram complete makeover In-Reply-To: <4536727D.4020308@ieee.org> References: <91cf711d0610171420r7c8be9e5l4d2d781587b063d4@mail.gmail.com> <331116dc0610181100k474af71akac41e54db7b7bb41@mail.gmail.com> <4536727D.4020308@ieee.org> Message-ID: <91cf711d0610201047oaa1e566sb735e6771529ab8e@mail.gmail.com> Thanks for the comments, Here is the code for the new histogram, tests included. I'll wait for comments or suggestions before submitting a patch (numpy / scipy) ? Cheers David 2006/10/18, Tim Hochberg : > > > My $0.02: > > If histogram is going to get a makeover, particularly one that makes it > more complex than at present, it should probably be moved to SciPy. > Failing that, it should be moved to a submodule of numpy with similar > statistical tools. Preferably with consistent interfaces for all of the > functions. > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: histogram1d.py Type: text/x-python Size: 6489 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From josh8912 at yahoo.com Fri Oct 20 16:27:14 2006 From: josh8912 at yahoo.com (JJ) Date: Fri, 20 Oct 2006 13:27:14 -0700 (PDT) Subject: slicing suggestion Message-ID: <20061020202714.12071.qmail@web51702.mail.yahoo.com> Hello. I have a suggestion that might make slicing using matrices more user-friendly. I often have a matrix of row or column numbers that I wish to use as a slice. If K was a matrix of row numbers (nx1) and M was a nxm matrix, then I would use ans = M[K.A.ravel(),:] to obtain the matrix I want. It turns out that I use .A.ravel() quite a lot in my code, as I usually work with matrices rather than arrays. My suggestion is to create a new attribute, such as .AR, so that the following could be used: M[K.AR,:]. I believe this would be more concise, easier to read, and well used. If slices are made in both directions of the matrix, then the .A.ravel() becomes even more unwieldy. Does anyone else like this idea? John __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Fri Oct 20 16:37:51 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Fri, 20 Oct 2006 13:37:51 -0700 Subject: slicing suggestion In-Reply-To: <20061020202714.12071.qmail@web51702.mail.yahoo.com> References: <20061020202714.12071.qmail@web51702.mail.yahoo.com> Message-ID: On 10/20/06, JJ wrote: > My suggestion is to > create a new attribute, such as .AR, so that the > following could be used: M[K.AR,:] It would be even better if M[K,:] worked. Would such a patch be accepted? (Not that I know how to make it.) ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sebzur at pin.if.uz.zgora.pl Fri Oct 20 17:33:47 2006 From: sebzur at pin.if.uz.zgora.pl (=?ISO-8859-2?Q?Sebastian_=AFurek?=) Date: Fri, 20 Oct 2006 23:33:47 +0200 Subject: Model and experiment fitting. Message-ID: Hi! This is probably a silly question but I'm getting confused with a certain problem: a comparison between experimental data points (2D points set) and a model (2D points set - no analytical form). The physical model produces (by a sophisticated simulations done by an external program) some 2D points data and one of my task is to compare those calculated data with an experimental one. The experimental and modeled data have form of 2D curves, build of n 2D-points, i.e.: expDat=[[x1,x2,x3,..xn],[y1,y2,y3,...,yn]] simDat=[[X1,X2,X3,...,Xn],[Y1,Y2,Y3,...,Yn]] The task of determining, let's say, a root mean squarred error (RMSe) is trivial if x1==X1, x2==X2, etc. In general, which is a common situation xk differs from Xk (k=0..n) and one may not simply compare succeeding Yk and yk (k=0..n) to determine the goodness-of-fit. The distance h=Xk-X(k-1) is constant, but similar distance m(k)=xk-x(k-1) depends on k-th point and is not a constant value, although the data array lengths for simulation and experiment are the same. My first idea was to do some interpolations to obtain the missing points, but I held it 'by a hand' (which, BTW gave quite rewarding results) and I suppose, there's some i.g. numpy method to do it for me, isn't it? I suppose to do something like: gfit(expDat,simDat,'measure_type') which I hope will return the number determining the goodness-of-fit (mean squarred error, root mean squarred error,...) of two sets of discrete 2D data points. Is there something like that in any numerical python modules (numpy, pylab) I could use? I can imagine, I can fit the data with some polynomial or whatever, and than compare the fitted data, but my goal is to operate on as raw data as it's possible. Thanks for your comments! Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Fri Oct 20 18:14:45 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 20 Oct 2006 17:14:45 -0500 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: Sebastian ?urek wrote: > Hi! > > This is probably a silly question but I'm getting confused with a > certain problem: a comparison between experimental data points (2D > points set) and a model (2D points set - no analytical form). > > The physical model produces (by a sophisticated simulations done by an > external program) some 2D points data and one of my task is to compare > those calculated data with an experimental one. > > The experimental and modeled data have form of 2D curves, build of n > 2D-points, i.e.: > > expDat=[[x1,x2,x3,..xn],[y1,y2,y3,...,yn]] > simDat=[[X1,X2,X3,...,Xn],[Y1,Y2,Y3,...,Yn]] > > The task of determining, let's say, a root mean squarred error (RMSe) > is trivial if x1==X1, x2==X2, etc. > > In general, which is a common situation xk differs from Xk (k=0..n) and > one may not simply compare succeeding Yk and yk (k=0..n) to determine > the goodness-of-fit. The distance h=Xk-X(k-1) is constant, but similar > distance m(k)=xk-x(k-1) depends on k-th point and is not a constant > value, although the data array lengths for simulation and experiment are > the same. Your description is a bit vague. Do you mean that you have some model function f that maps X values to Y values? f(x) -> y If that is the case, is there some reason that you cannot run your simulation using the same X points as your experimental data? OTOH, is there some other independent variable (say Z) that *is* common between your experimental and simulated data? f(z) -> (x, y) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Fri Oct 20 19:12:50 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri, 20 Oct 2006 17:12:50 -0600 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <4538CE5D.5000903@obs.u-bordeaux1.fr> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> <4538CE5D.5000903@obs.u-bordeaux1.fr> Message-ID: <453957F2.8070507@ee.byu.edu> Sebastien Bardeau wrote: >>One possible solution (there can be more) is using ndarray: >> >>In [47]: a=numpy.array([1,2,3], dtype="i4") >>In [48]: n=1 # the position that you want to share >>In [49]: b=numpy.ndarray(buffer=a[n:n+1], shape=(), dtype="i4") >> >> >> >Ok thanks. Actually that was also the solution I found. But this is much >more complicated when arrays are N dimensional with N>1, and above all >if user asks for a slice in one or more dimension. Here is how I >redefine the __getitem__ method for my arrays. Remember that the goal is >to return a 0-d array rather than a numpy.scalar when I extract a single >element out of a N-dimensional (N>=1) array: > > How about this. To get the i,j,k,l element a[i:i+1,j:j+1,k:k+1,l:l+1].squeeze() -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 20 19:28:27 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 20 Oct 2006 19:28:27 -0400 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: On 20/10/06, Sebastian ?urek wrote: > Is there something like that in any numerical python modules (numpy, > pylab) I could use? In scipy there are some very convenient spline fitting tools which will allow you to fit a nice smooth spline through the simulation data points (or near, if they have some uncertainty); you can then easily look at the RMS difference in the y values. You can also, less easily, look at the distance from the curve allowing for some uncertainty in the x values. I suppose you could also fit a curve through the experimental points and compare the two curves in some way. > I can imagine, I can fit the data with some polynomial or whatever, > and than compare the fitted data, but my goal is to operate on > as raw data as it's possible. If you want to avoid using an a priori model, Numerical Recipes discuss some possible approaches ("Do two-dimensional distributions differ?" at http://www.nrbook.com/a/bookcpdf.html is one) but it's not clear how to turn the problem you describe into a solvable one - some assumption about how the models vary between sampled x values appears to be necessary, and that amounts to interpolation. A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 20 22:42:22 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 20:42:22 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc Message-ID: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> Hi, i am running numpy on aix compiling with xlc. Revision 1.0rc2 works fine and passes all tests. But 1.0rc3 and more recent give the following on import: Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in add Warning: invalid value encountered in not_equal Warning: invalid value encountered in absolute Warning: invalid value encountered in less Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in equal Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in add Warning: invalid value encountered in not_equal Warning: invalid value encountered in absolute Warning: invalid value encountered in less Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in equal Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply Warning: invalid value encountered in multiply [lots more of this] The odd thing is that all tests pass. I have looked, but can't find where this Warning is coming from in the code. Any thoughts on where this is coming from? What can I do to help debug this? I am not sure what revision introduced this issue. Thanks Brian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 20 23:10:15 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 20:10:15 -0700 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> Message-ID: <45398F97.2030509@ieee.org> Brian Granger wrote: > Hi, > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > fine and passes all tests. But 1.0rc3 and more recent give the > following on import: > > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in add > Warning: invalid value encountered in not_equal > Warning: invalid value encountered in absolute > Warning: invalid value encountered in less > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in equal > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in add > Warning: invalid value encountered in not_equal > Warning: invalid value encountered in absolute > Warning: invalid value encountered in less > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in equal > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > Warning: invalid value encountered in multiply > [lots more of this] > > The odd thing is that all tests pass. I have looked, but can't find > where this Warning is coming from in the code. Any thoughts on where > this is coming from? What can I do to help debug this? I am not sure > what revision introduced this issue. > The reason that you are seeing this now is that the default error state has been tightened up. There were some issues with tests failing as a result of this, but I believe I fixed those already and you're seeing this on import, not when running the tests correct? The first thing to do is figure out where the invalids are occurring, and the natural way to do that is to set the error state to raise, but you can't set the error state till you import it, so that's not going to help here. I think the first thing that I would try is to throw in a seterr(all='raise', under='ignore') right after the call to _setdef in numeric.py. If you're lucky, this will point out where the invalids are popping up. As a sanity check, you could instead make this seterr(all='ignore'), which should make all the warnings go away, but won't tell you anything about why there are warnings to begin with. Regards, -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 20 23:32:56 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 21:32:56 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45398F97.2030509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> Message-ID: <6ce0ac130610202032p4ea33e2agc64edb5e13e4f2c6@mail.gmail.com> Here is the traceback that I got: In [1]: import numpy --------------------------------------------------------------------------- exceptions.FloatingPointError Traceback (most recent call last) /u2/granger/ /usr/common/homes/g/granger/usr/local/lib/python/numpy/__init__.py 36 import core 37 from core import * ---> 38 import lib 39 from lib import * 40 import linalg /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/__init__.py 3 from numpy.version import version as __version__ 4 ----> 5 from type_check import * 6 from index_tricks import * 7 from function_base import * /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/type_check.py 8 import numpy.core.numeric as _nx 9 from numpy.core.numeric import asarray, array, isnan, obj2sctype, zeros ---> 10 from ufunclike import isneginf, isposinf 11 12 _typecodes_by_elsize = 'GDFgdfQqLlIiHhBb?' /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/ufunclike.py 44 return y 45 ---> 46 _log2 = umath.log(2) 47 def log2(x, y=None): 48 """Returns the base 2 logarithm of x FloatingPointError: invalid value encountered in log Obviously because I am having the error raised, I only get the first one. Hmmm. Brian On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Hi, > > > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > > fine and passes all tests. But 1.0rc3 and more recent give the > > following on import: > > > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > [lots more of this] > > > > The odd thing is that all tests pass. I have looked, but can't find > > where this Warning is coming from in the code. Any thoughts on where > > this is coming from? What can I do to help debug this? I am not sure > > what revision introduced this issue. > > > The reason that you are seeing this now is that the default error state > has been tightened up. There were some issues with tests failing as a > result of this, but I believe I fixed those already and you're seeing > this on import, not when running the tests correct? The first thing to > do is figure out where the invalids are occurring, and the natural way > to do that is to set the error state to raise, but you can't set the > error state till you import it, so that's not going to help here. > > I think the first thing that I would try is to throw in a > seterr(all='raise', under='ignore') right after the call to _setdef in > numeric.py. If you're lucky, this will point out where the invalids are > popping up. As a sanity check, you could instead make this > seterr(all='ignore'), which should make all the warnings go away, but > won't tell you anything about why there are warnings to begin with. > > Regards, > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 20 23:43:43 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 21:43:43 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45398F97.2030509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> Message-ID: <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> Also, when I use seterr(all='ignore') the the tests fail: ====================================================================== FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", line 219, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError ---------------------------------------------------------------------- Ran 516 tests in 0.823s FAILED (failures=1) Thanks for helping out on this. On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Hi, > > > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > > fine and passes all tests. But 1.0rc3 and more recent give the > > following on import: > > > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > [lots more of this] > > > > The odd thing is that all tests pass. I have looked, but can't find > > where this Warning is coming from in the code. Any thoughts on where > > this is coming from? What can I do to help debug this? I am not sure > > what revision introduced this issue. > > > The reason that you are seeing this now is that the default error state > has been tightened up. There were some issues with tests failing as a > result of this, but I believe I fixed those already and you're seeing > this on import, not when running the tests correct? The first thing to > do is figure out where the invalids are occurring, and the natural way > to do that is to set the error state to raise, but you can't set the > error state till you import it, so that's not going to help here. > > I think the first thing that I would try is to throw in a > seterr(all='raise', under='ignore') right after the call to _setdef in > numeric.py. If you're lucky, this will point out where the invalids are > popping up. As a sanity check, you could instead make this > seterr(all='ignore'), which should make all the warnings go away, but > won't tell you anything about why there are warnings to begin with. > > Regards, > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Fri Oct 20 23:49:36 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 21:49:36 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45398F97.2030509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> Message-ID: <6ce0ac130610202049w6dd1e84dp29651624f1917c22@mail.gmail.com> When I set seterr(all='warn') I see the following: In [1]: import numpy /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/ufunclike.py:46: RuntimeWarning: invalid value encountered in log _log2 = umath.log(2) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/scimath.py:19: RuntimeWarning: invalid value encountered in log _ln2 = nx.log(2.0) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:64: RuntimeWarning: invalid value encountered in add two = one + one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:65: RuntimeWarning: invalid value encountered in subtract zero = one - one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:71: RuntimeWarning: invalid value encountered in add a = a + a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:72: RuntimeWarning: invalid value encountered in add temp = a + one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:73: RuntimeWarning: invalid value encountered in subtract temp1 = temp - a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:74: RuntimeWarning: invalid value encountered in subtract if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:74: RuntimeWarning: invalid value encountered in not_equal if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:80: RuntimeWarning: invalid value encountered in add b = b + b /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:81: RuntimeWarning: invalid value encountered in add temp = a + b /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:82: RuntimeWarning: invalid value encountered in subtract itemp = int_conv(temp-a) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:83: RuntimeWarning: invalid value encountered in not_equal if any(itemp != 0): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:95: RuntimeWarning: invalid value encountered in multiply b = b * beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:96: RuntimeWarning: invalid value encountered in add temp = b + one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:97: RuntimeWarning: invalid value encountered in subtract temp1 = temp - b /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:98: RuntimeWarning: invalid value encountered in subtract if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:98: RuntimeWarning: invalid value encountered in not_equal if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:103: RuntimeWarning: invalid value encountered in divide betah = beta / two /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:106: RuntimeWarning: invalid value encountered in add a = a + a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:107: RuntimeWarning: invalid value encountered in add temp = a + one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:108: RuntimeWarning: invalid value encountered in subtract temp1 = temp - a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:109: RuntimeWarning: invalid value encountered in subtract if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:109: RuntimeWarning: invalid value encountered in not_equal if any(temp1 - one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:113: RuntimeWarning: invalid value encountered in add temp = a + betah /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:115: RuntimeWarning: invalid value encountered in subtract if any(temp-a != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:115: RuntimeWarning: invalid value encountered in not_equal if any(temp-a != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:117: RuntimeWarning: invalid value encountered in add tempa = a + beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:118: RuntimeWarning: invalid value encountered in add temp = tempa + betah /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:119: RuntimeWarning: invalid value encountered in subtract if irnd==0 and any(temp-tempa != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:119: RuntimeWarning: invalid value encountered in not_equal if irnd==0 and any(temp-tempa != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:124: RuntimeWarning: invalid value encountered in divide betain = one / beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:127: RuntimeWarning: invalid value encountered in multiply a = a * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:130: RuntimeWarning: invalid value encountered in subtract temp = one - a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:131: RuntimeWarning: invalid value encountered in subtract if any(temp-one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:131: RuntimeWarning: invalid value encountered in not_equal if any(temp-one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:133: RuntimeWarning: invalid value encountered in multiply a = a * beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:149: RuntimeWarning: invalid value encountered in add temp = one + a /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:150: RuntimeWarning: invalid value encountered in subtract if any(temp-one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:150: RuntimeWarning: invalid value encountered in not_equal if any(temp-one != zero): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:152: RuntimeWarning: invalid value encountered in multiply a = a * beta /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:160: RuntimeWarning: invalid value encountered in add temp = one + eps /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:168: RuntimeWarning: invalid value encountered in add t = one + eps /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:172: RuntimeWarning: invalid value encountered in multiply z = y*y /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:173: RuntimeWarning: invalid value encountered in multiply a = z*one # Check here for underflow /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:174: RuntimeWarning: invalid value encountered in multiply temp = z*t /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:175: RuntimeWarning: invalid value encountered in add if any(a+a == zero) or any(abs(z)>=y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:175: RuntimeWarning: invalid value encountered in equal if any(a+a == zero) or any(abs(z)>=y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:175: RuntimeWarning: invalid value encountered in absolute if any(a+a == zero) or any(abs(z)>=y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:175: RuntimeWarning: invalid value encountered in greater_equal if any(a+a == zero) or any(abs(z)>=y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:177: RuntimeWarning: invalid value encountered in multiply temp1 = temp * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:178: RuntimeWarning: invalid value encountered in multiply if any(temp1*beta == z): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:178: RuntimeWarning: invalid value encountered in equal if any(temp1*beta == z): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:174: RuntimeWarning: underflow encountered in multiply temp = z*t /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:184: RuntimeWarning: invalid value encountered in not_equal if ibeta != 10: /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:198: RuntimeWarning: invalid value encountered in multiply y = y * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:199: RuntimeWarning: invalid value encountered in multiply a = y * one /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:200: RuntimeWarning: invalid value encountered in multiply temp = y * t /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:201: RuntimeWarning: invalid value encountered in add if any(a+a != zero) and any(abs(y) < xmin): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:201: RuntimeWarning: invalid value encountered in not_equal if any(a+a != zero) and any(abs(y) < xmin): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:201: RuntimeWarning: invalid value encountered in absolute if any(a+a != zero) and any(abs(y) < xmin): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:201: RuntimeWarning: invalid value encountered in less if any(a+a != zero) and any(abs(y) < xmin): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:203: RuntimeWarning: invalid value encountered in multiply temp1 = temp * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:204: RuntimeWarning: invalid value encountered in multiply if any(temp1*beta == y) and any(temp != y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:204: RuntimeWarning: invalid value encountered in equal if any(temp1*beta == y) and any(temp != y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:203: RuntimeWarning: underflow encountered in multiply temp1 = temp * betain /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:204: RuntimeWarning: invalid value encountered in not_equal if any(temp1*beta == y) and any(temp != y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:215: RuntimeWarning: invalid value encountered in not_equal if mx <= k + k - 3 and ibeta != 10: /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:223: RuntimeWarning: invalid value encountered in equal if ibeta == 2 and not i: /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:227: RuntimeWarning: invalid value encountered in not_equal if any(a != y): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:229: RuntimeWarning: invalid value encountered in subtract xmax = one - epsneg /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:230: RuntimeWarning: invalid value encountered in multiply if any(xmax*one != xmax): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:230: RuntimeWarning: invalid value encountered in not_equal if any(xmax*one != xmax): /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:232: RuntimeWarning: invalid value encountered in multiply xmax = xmax / (xmin*beta*beta*beta) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:232: RuntimeWarning: invalid value encountered in divide xmax = xmax / (xmin*beta*beta*beta) /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:235: RuntimeWarning: invalid value encountered in equal if ibeta==2: /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:236: RuntimeWarning: invalid value encountered in add xmax = xmax + xmax /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:266: RuntimeWarning: invalid value encountered in add ten = two + two + two + two + two /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/machar.py:267: RuntimeWarning: invalid value encountered in power resolution = ten ** (-self.precision) On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Hi, > > > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > > fine and passes all tests. But 1.0rc3 and more recent give the > > following on import: > > > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in add > > Warning: invalid value encountered in not_equal > > Warning: invalid value encountered in absolute > > Warning: invalid value encountered in less > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in equal > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > Warning: invalid value encountered in multiply > > [lots more of this] > > > > The odd thing is that all tests pass. I have looked, but can't find > > where this Warning is coming from in the code. Any thoughts on where > > this is coming from? What can I do to help debug this? I am not sure > > what revision introduced this issue. > > > The reason that you are seeing this now is that the default error state > has been tightened up. There were some issues with tests failing as a > result of this, but I believe I fixed those already and you're seeing > this on import, not when running the tests correct? The first thing to > do is figure out where the invalids are occurring, and the natural way > to do that is to set the error state to raise, but you can't set the > error state till you import it, so that's not going to help here. > > I think the first thing that I would try is to throw in a > seterr(all='raise', under='ignore') right after the call to _setdef in > numeric.py. If you're lucky, this will point out where the invalids are > popping up. As a sanity check, you could instead make this > seterr(all='ignore'), which should make all the warnings go away, but > won't tell you anything about why there are warnings to begin with. > > Regards, > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Fri Oct 20 23:55:49 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 20:55:49 -0700 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> Message-ID: <45399A45.6050509@ieee.org> Brian Granger wrote: > Also, when I use seterr(all='ignore') the the tests fail: > > ====================================================================== > FAIL: Ticket #112 > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", > line 219, in check_longfloat_repr > assert(str(a)[1:9] == str(a[0])[:8]) > AssertionError > > ---------------------------------------------------------------------- > Ran 516 tests in 0.823s > > FAILED (failures=1) > > Thanks for helping out on this. > How recent is your version? I just a problem that was causing this same failure yesterday -- if you checkout is older than that, you may want to get the most recent stuff from SVN and see if that fixes this. -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Sat Oct 21 00:00:34 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 22:00:34 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45399A45.6050509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> <45399A45.6050509@ieee.org> Message-ID: <6ce0ac130610202100q1b13c2efm2e50fc3b32243447@mail.gmail.com> I have been doing these recent tests with 1.0rc3. I am building from trunk right now and we will see how that goes. Thanks for your help. Brian On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Also, when I use seterr(all='ignore') the the tests fail: > > > > ====================================================================== > > FAIL: Ticket #112 > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", > > line 219, in check_longfloat_repr > > assert(str(a)[1:9] == str(a[0])[:8]) > > AssertionError > > > > ---------------------------------------------------------------------- > > Ran 516 tests in 0.823s > > > > FAILED (failures=1) > > > > Thanks for helping out on this. > > > How recent is your version? I just a problem that was causing this same > failure yesterday -- if you checkout is older than that, you may want to > get the most recent stuff from SVN and see if that fixes this. > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Sat Oct 21 00:13:07 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 21:13:07 -0700 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610202049w6dd1e84dp29651624f1917c22@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202049w6dd1e84dp29651624f1917c22@mail.gmail.com> Message-ID: <45399E53.9080109@ieee.org> Brian Granger wrote: > When I set seterr(all='warn') I see the following: > > In [1]: import numpy > /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/ufunclike.py:46: > RuntimeWarning: invalid value encountered in log > _log2 = umath.log(2) > /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/scimath.py:19: > RuntimeWarning: invalid value encountered in log > _ln2 = nx.log(2.0) > [etc, etc] Wow! That looks pretty bad. What do you get if you try just "numpy.log(2)" or "numpy.log(2.0)"? Is it producing sane results for scalars at all? I suppose another possibility is that the error reporting is broken on AIX for some reason. Hmmm. I'm betting that is is. The macro UFUNC_CHECK_STATUS is very platform dependent. There is a version from AIX (ufuncobject.h line 301), but perhaps it's broken on your particular configuration and as a result is spitting out all kinds of bogus errors. This is only coming to light now because the default error checking level got cranked up. I gotta call it a night and I'll be out tomorrow, so I won't be much more help, but here's something that you might look into: have you compiled numarray sucessfully? If you haven't you might want to try it. It uses the same default error checking that numpy is now using. If you have, you might want to look for the equivalent of UFUNC_CHECK_STATUS (it might even have the same name) and splice it into numpy and see if it fixes your problems. Of course, if numpy.log(2) is spitting out something bogus, there's something much worse going on, but I suspect you would have noticed that by now. Good luck, -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Sat Oct 21 00:13:21 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 22:13:21 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45399A45.6050509@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> <45399A45.6050509@ieee.org> Message-ID: <6ce0ac130610202113u3d550aabld8157dac915c19db@mail.gmail.com> Tim, I just tried everything with r3375. I set seterr(all='warn') and the tests passed. But all the floating point warning are still there. With seterr(all='ignore') the warnings go away and all the tests pass. should I worry about the warnings? thanks Brian On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > Also, when I use seterr(all='ignore') the the tests fail: > > > > ====================================================================== > > FAIL: Ticket #112 > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", > > line 219, in check_longfloat_repr > > assert(str(a)[1:9] == str(a[0])[:8]) > > AssertionError > > > > ---------------------------------------------------------------------- > > Ran 516 tests in 0.823s > > > > FAILED (failures=1) > > > > Thanks for helping out on this. > > > How recent is your version? I just a problem that was causing this same > failure yesterday -- if you checkout is older than that, you may want to > get the most recent stuff from SVN and see if that fixes this. > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Sat Oct 21 00:21:07 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Fri, 20 Oct 2006 21:21:07 -0700 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610202113u3d550aabld8157dac915c19db@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202043s1fe16a16yd2876f0c0f07c3a4@mail.gmail.com> <45399A45.6050509@ieee.org> <6ce0ac130610202113u3d550aabld8157dac915c19db@mail.gmail.com> Message-ID: <4539A033.9050800@ieee.org> Brian Granger wrote: > Tim, > > I just tried everything with r3375. I set seterr(all='warn') and the > tests passed. But all the floating point warning are still there. > With seterr(all='ignore') the warnings go away and all the tests pass. > should I worry about the warnings? > Maybe. I just sent you some email on this. But my guess is that the code that checks for FP errors is broken on your particular system. Mainly I suspect this because I think you would have noticed by now if everything was as broken as the warnings seem to indicate. Assuming that's the case, and this will probably become clear if you test a bunch of computations that give correct (and non INF/NAN) results, but still spit out warnings, you have two choices: try to fix the warnings code or disable the warnings. The former would be preferable since then you could actually use the warnings code, but it may be a pain in the neck unless you can find some place to steal the relevant code from. -tim > thanks > > Brian > > > > On 10/20/06, Tim Hochberg wrote: > >> Brian Granger wrote: >> >>> Also, when I use seterr(all='ignore') the the tests fail: >>> >>> ====================================================================== >>> FAIL: Ticket #112 >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File "/usr/common/homes/g/granger/usr/local/lib/python/numpy/core/tests/test_regression.py", >>> line 219, in check_longfloat_repr >>> assert(str(a)[1:9] == str(a[0])[:8]) >>> AssertionError >>> >>> ---------------------------------------------------------------------- >>> Ran 516 tests in 0.823s >>> >>> FAILED (failures=1) >>> >>> Thanks for helping out on this. >>> >>> >> How recent is your version? I just a problem that was causing this same >> failure yesterday -- if you checkout is older than that, you may want to >> get the most recent stuff from SVN and see if that fixes this. >> >> -tim >> >> >> ------------------------------------------------------------------------- >> Using Tomcat but need to do more? Need to support web services, security? >> Get stuff done quickly with pre-integrated technology to make your job easier >> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Sat Oct 21 01:01:42 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 20 Oct 2006 23:01:42 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <45399E53.9080109@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <45398F97.2030509@ieee.org> <6ce0ac130610202049w6dd1e84dp29651624f1917c22@mail.gmail.com> <45399E53.9080109@ieee.org> Message-ID: <6ce0ac130610202201i406c8ea3r3e90cb519547a@mail.gmail.com> Thanks, I will investigate more on these things and get back to you early in the week. But for now numpy seems to be functioning pretty normally (log(2) gives the correct answer). thanks again. It would be great to figure this stuff out before 1.0, but we might not have time. Brian On 10/20/06, Tim Hochberg wrote: > Brian Granger wrote: > > When I set seterr(all='warn') I see the following: > > > > In [1]: import numpy > > /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/ufunclike.py:46: > > RuntimeWarning: invalid value encountered in log > > _log2 = umath.log(2) > > /usr/common/homes/g/granger/usr/local/lib/python/numpy/lib/scimath.py:19: > > RuntimeWarning: invalid value encountered in log > > _ln2 = nx.log(2.0) > > > [etc, etc] > > Wow! That looks pretty bad. What do you get if you try just > "numpy.log(2)" or "numpy.log(2.0)"? Is it producing sane results for > scalars at all? I suppose another possibility is that the error > reporting is broken on AIX for some reason. > > Hmmm. > > I'm betting that is is. The macro UFUNC_CHECK_STATUS is very platform > dependent. There is a version from AIX (ufuncobject.h line 301), but > perhaps it's broken on your particular configuration and as a result is > spitting out all kinds of bogus errors. This is only coming to light now > because the default error checking level got cranked up. > > I gotta call it a night and I'll be out tomorrow, so I won't be much > more help, but here's something that you might look into: have you > compiled numarray sucessfully? If you haven't you might want to try it. > It uses the same default error checking that numpy is now using. If you > have, you might want to look for the equivalent of UFUNC_CHECK_STATUS > (it might even have the same name) and splice it into numpy and see if > it fixes your problems. > > Of course, if numpy.log(2) is spitting out something bogus, there's > something much worse going on, but I suspect you would have noticed that > by now. > > Good luck, > > -tim > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Sat Oct 21 02:07:01 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 21 Oct 2006 00:07:01 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> Message-ID: <4539B905.3010705@ieee.org> Brian Granger wrote: > Hi, > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > fine and passes all tests. But 1.0rc3 and more recent give the > following on import: > Most likely the error-detection code is not working on your platform. The platform dependent stuff is not that difficult. I tried to implement something for AIX, but very likely got it wrong (and don't have a platform to test it on). It is the UFUNC_CHECK_STATUS that must be implemented. Perhaps, we can do a simple check and disable the error modes: seterr(all='ignore') will work and "turn-off" error-detection on your platform. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sebzur at pin.if.uz.zgora.pl Sat Oct 21 09:41:45 2006 From: sebzur at pin.if.uz.zgora.pl (=?UTF-8?B?U2ViYXN0aWFuIMW7dXJlaw==?=) Date: Sat, 21 Oct 2006 15:41:45 +0200 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: Robert Kern napisa?(a): > Your description is a bit vague. Possibly by my weak English... I'll try to make myself clearer now. Do you mean that you have some model function f > that maps X values to Y values? > > f(x) -> y > My model is quantum energy operator - spin hamiltonian (SH) with some additional assumption about so called 'line shape', 'line widths',etc. It describes various electron interactions, visible in electron paramagnetic resonance (EPR, ESR) experiment. The simplest SH can be written in a form: H = m B g S (1) where m is a constant (bohr magneton), B is magnetic field (my x-variable), g is so called 'zeeman matrix' and S is total spin angular momentum operator. Summing it all together: the simple model is parametrized by: - line shape, - line width, - zeeman matrix (3x3 diagonal matrix - the spatial dependence), - total spin S. After SH (1) diagonalization one can obtain so called 'resonance fields' and 'resonance intensities'. After a convolution with appropriate line shape function which is parametrized by the line width one can finally get the simulated EPR spectrum (simDat=[[X1,...,Xn],[Y1,...,Yn]]). This is a roughly, schematic description, appropriate to EPR spectra of monocrystals. In my situation the problem is more sophisticated - I have polycrystaline (powders) data, and to obtain a simulated EPR powder spectrum I need to sum up the EPR spectra of monocrystals that come from many possible spatial orientations, and the resultant spectrum is an envelope of all the monocrystals spectra. There's no simple model function that maps X -> Y. > If that is the case, is there some reason that you cannot run your simulation > using the same X points as your experimental data? > I can only demand a X range and number of X values within the range, there's no possibility to find the Y(X) for a specified X. These limitations on one hand come from the external program I'm using to simulate the EPR spectra, on the other are a result of spatial averaging of EPR data for powders, where a lot of interpolations are involved. > OTOH, is there some other independent variable (say Z) that *is* common between > your experimental and simulated data? > > f(z) -> (x, y) > This is probably the situation I'm in. These other variables are my model parameters, namely: line shape-width, zeeman matrix... and they're commen between the experiment and the simulation. To make it clear. I've already solved the problem by a simple linear interpolation of simulated points within the narrow neighborhood of experimental data point. The simulation points are uniformly distributed along the X-range, with a density I'm able to tune. It all works quite well but I'm founding it as a 'brute-force' method and I wonder, if there's any more sophisticated and maybe already incorporated into any Python module method? Anyway, it looks like it's impossible to compare two discrete 2D data sets without any interpolations included... :] A. M. Archibald has proposed spline fitting, which I'll try. I'll also look at the Numerical Recipes discussion he has proposed. Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From veomagxa at powerhill.com Sat Oct 21 09:54:45 2006 From: veomagxa at powerhill.com (India:) Date: Sat, 21 Oct 2006 21:54:45 +0800 Subject: browser we Message-ID: <000f01c6f518$777a93e0$955ea2dc@SUPER7ABF2EC2B> Cold addition less division or comfy futonsa lot or fuss or water in featurethe a rules ride lots. Todays events Advanced is Local in Nation World Business in Editorials a Opinion Columnists Sports in Comics Games Living Pacific Northwest Magazine or Travel Outdoors Real am estate Obituaries Special Alerts? Other Packages Related depends recommends suggests libc arm or hppa mk mips mipsel powerpcgnu Library Shared sparclibc in amd libc alphagnu ia a Download available of filessize. Boots Qayoure only one pain brain a Getting Startedno is excuse mom woes Microsofts Halo film is. Confident un action Iran is Shiites in vs Shiites in Iraq rift of parade west flock Jerusalem of brave is Israeli takes harder line North in leaders Darfur careful blame or either dogs fury? Offense a facing tough testso far flaws grow baseball cities Larry Stoneworld a Series brink Movies is. Somali women swimming Sunday am expanding canal births Japan adoption agencies of ask. Swimming Sunday expanding of canal births Japan a adoption agencies in ask why! Straggle Detroit Seriesmore is ap read emailed articles air Kuow Hear columnist Danny Westneat a review weeks Weekday is Fridays am is. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Groz.gif Type: image/gif Size: 10621 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From sebzur at pin.if.uz.zgora.pl Sat Oct 21 10:00:27 2006 From: sebzur at pin.if.uz.zgora.pl (=?ISO-8859-2?Q?Sebastian_=AFurek?=) Date: Sat, 21 Oct 2006 16:00:27 +0200 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: A. M. Archibald napisa?(a): > > In scipy there are some very convenient spline fitting tools which > will allow you to fit a nice smooth spline through the simulation data > points (or near, if they have some uncertainty); you can then easily > look at the RMS difference in the y values. You can also, less easily, > look at the distance from the curve allowing for some uncertainty in > the x values. > I'll try a spline fitting. I've already made some linear interpolations (see Robert Kern answer) which works well enough to use it. I'm working on a genetic algorithms application to the model parameters optimalization problem and this RMSe comparison serves me as 'fitness function'. This 'fitness function' is important element in whole procedure, so I'm trying to found the best solution to obtain it. > I suppose you could also fit a curve through the experimental points > and compare the two curves in some way. > Well, I can do it, indeed. But every single fitting procedure implicate some additional error, so when it comes to fit, I must use it very cautiously. The simulated data-points fitting should be the only acceptable fitting procedure, I guess. > If you want to avoid using an a priori model, Numerical Recipes > discuss some possible approaches ("Do two-dimensional distributions > differ?" at http://www.nrbook.com/a/bookcpdf.html is one) but it's not > clear how to turn the problem you describe into a solvable one - some > assumption about how the models vary between sampled x values appears > to be necessary, and that amounts to interpolation. > I'll look to this NR discussion. Thank You for these comments! Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 21 11:51:56 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 21 Oct 2006 09:51:56 -0600 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: On 10/21/06, Sebastian ?urek wrote: > > Robert Kern napisa?(a): > > > To make it clear. > > I've already solved the problem by a simple linear interpolation of > simulated points within the narrow neighborhood of experimental data > point. The simulation points are uniformly distributed along the > X-range, with a density I'm able to tune. It all works quite well but > I'm founding it as a 'brute-force' method and I wonder, if there's any > more sophisticated and maybe already incorporated into any Python module > method? > > Anyway, it looks like it's impossible to compare two discrete 2D data > sets without any interpolations included... :] I note that interpolation can be seen as a linear map from the data points to the interpolation points, so a lot of standard tools to be used. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From sebzur at pin.if.uz.zgora.pl Sat Oct 21 14:47:05 2006 From: sebzur at pin.if.uz.zgora.pl (=?UTF-8?B?U2ViYXN0aWFuIMW7dXJlaw==?=) Date: Sat, 21 Oct 2006 20:47:05 +0200 Subject: Model and experiment fitting. In-Reply-To: References: Message-ID: The problem seemed to be solved, by the A. M. Archibald clue. I've used splines to fit the simulation data. After that, I can easily find any Y(X) point, for all X in range (x_min,x_max) where x_min and x_max are the experiment independent variable. The experimental data stay untouched. Sorry for all the confusion I've made. Thanks a lot to all of You! Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nadavh at visionsense.com Sat Oct 21 15:05:21 2006 From: nadavh at visionsense.com (Nadav Horesh) Date: Sat, 21 Oct 2006 21:05:21 +0200 Subject: Model and experiment fitting. Message-ID: <07C6A61102C94148B8104D42DE95F7E8C8F0F7@exchange2k.envision.co.il> 1. If at least one of your data sets to be interpulated is on a grid, you can use numpy.ndimage.map function for fast interpolation for 2d (in fact for any dimensional) dataset. 2. Isn't there an analytic expression to average the expectration values of SH over all possible orientations between B and the crystal axis? My experience shows that some analytic work can save 99% of simulation time. Nadav -----Original Message----- From: numpy-discussion-bounces at lists.sourceforge.net on behalf of Sebastian Zurek Sent: Sat 21-Oct-06 15:41 To: numpy-discussion at lists.sourceforge.net Cc: Subject: Re: [Numpy-discussion] Model and experiment fitting. Robert Kern napisal(a): > Your description is a bit vague. Possibly by my weak English... I'll try to make myself clearer now. Do you mean that you have some model function f > that maps X values to Y values? > > f(x) -> y > My model is quantum energy operator - spin hamiltonian (SH) with some additional assumption about so called 'line shape', 'line widths',etc. It describes various electron interactions, visible in electron paramagnetic resonance (EPR, ESR) experiment. The simplest SH can be written in a form: H = m B g S (1) where m is a constant (bohr magneton), B is magnetic field (my x-variable), g is so called 'zeeman matrix' and S is total spin angular momentum operator. Summing it all together: the simple model is parametrized by: - line shape, - line width, - zeeman matrix (3x3 diagonal matrix - the spatial dependence), - total spin S. After SH (1) diagonalization one can obtain so called 'resonance fields' and 'resonance intensities'. After a convolution with appropriate line shape function which is parametrized by the line width one can finally get the simulated EPR spectrum (simDat=[[X1,...,Xn],[Y1,...,Yn]]). This is a roughly, schematic description, appropriate to EPR spectra of monocrystals. In my situation the problem is more sophisticated - I have polycrystaline (powders) data, and to obtain a simulated EPR powder spectrum I need to sum up the EPR spectra of monocrystals that come from many possible spatial orientations, and the resultant spectrum is an envelope of all the monocrystals spectra. There's no simple model function that maps X -> Y. > If that is the case, is there some reason that you cannot run your simulation > using the same X points as your experimental data? > I can only demand a X range and number of X values within the range, there's no possibility to find the Y(X) for a specified X. These limitations on one hand come from the external program I'm using to simulate the EPR spectra, on the other are a result of spatial averaging of EPR data for powders, where a lot of interpolations are involved. > OTOH, is there some other independent variable (say Z) that *is* common between > your experimental and simulated data? > > f(z) -> (x, y) > This is probably the situation I'm in. These other variables are my model parameters, namely: line shape-width, zeeman matrix... and they're commen between the experiment and the simulation. To make it clear. I've already solved the problem by a simple linear interpolation of simulated points within the narrow neighborhood of experimental data point. The simulation points are uniformly distributed along the X-range, with a density I'm able to tune. It all works quite well but I'm founding it as a 'brute-force' method and I wonder, if there's any more sophisticated and maybe already incorporated into any Python module method? Anyway, it looks like it's impossible to compare two discrete 2D data sets without any interpolations included... :] A. M. Archibald has proposed spline fitting, which I'll try. I'll also look at the Numerical Recipes discussion he has proposed. Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 5129 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From kwgoodman at gmail.com Sat Oct 21 17:05:42 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Sat, 21 Oct 2006 14:05:42 -0700 Subject: some work on arpack In-Reply-To: References: Message-ID: On 8/15/06, David Grant wrote: > My idea is (if I have time) to write an eigs-like function in python > that will only perform a subset of what Matlab's eigs does for. It > will, for example, compute a certain number of eigenvalues and > eigenvectors for a real, sparse, symmetric matrix (the case I'm > interested in)... I hope that this subset-of-matlab's-eigs function > will not be too hard to write. Then more functionality can be added on > to eigs.py later... Does this make sense? Did you, or anybody else on the list, have any luck making a numpy version of eigs? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Sat Oct 21 19:26:46 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Sat, 21 Oct 2006 20:26:46 -0300 Subject: Numpy-scalars vs Numpy 0-d arrays: copy or not copy? In-Reply-To: <453957F2.8070507@ee.byu.edu> References: <45389A02.30901@obs.u-bordeaux1.fr> <200610201255.57968.faltet@carabos.com> <4538CE5D.5000903@obs.u-bordeaux1.fr> <453957F2.8070507@ee.byu.edu> Message-ID: On 10/20/06, Travis Oliphant wrote: > > How about this. To get the i,j,k,l element > > a[i:i+1,j:j+1,k:k+1,l:l+1].squeeze() > > -Travis I think all this can be condensed in a method call or similar mehcanism, natively provided by ndarray type. Or should this be seen as a special use case? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Sat Oct 21 19:36:21 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 22 Oct 2006 08:36:21 +0900 Subject: some work on arpack In-Reply-To: References: Message-ID: Here's something I've been wondering: is it ok to port Matlab functions over to python? If so, then it's maybe an afternoon's work to get eigs working, given eigs.m and python Arpack wrappers. I'm 99% sure it would *not* be kosher to copy-paste their code directly, but it's not a copy paste job because of the differences between Python and .m. So it seems a grayer area to me. --bb On 10/22/06, Keith Goodman wrote: > On 8/15/06, David Grant wrote: > > > My idea is (if I have time) to write an eigs-like function in python > > that will only perform a subset of what Matlab's eigs does for. It > > will, for example, compute a certain number of eigenvalues and > > eigenvectors for a real, sparse, symmetric matrix (the case I'm > > interested in)... I hope that this subset-of-matlab's-eigs function > > will not be too hard to write. Then more functionality can be added on > > to eigs.py later... Does this make sense? > > Did you, or anybody else on the list, have any luck making a numpy > version of eigs? > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sat Oct 21 19:44:48 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sat, 21 Oct 2006 17:44:48 -0600 Subject: some work on arpack In-Reply-To: References: Message-ID: On 10/21/06, Bill Baxter wrote: > > Here's something I've been wondering: is it ok to port Matlab > functions over to python? If so, then it's maybe an afternoon's work > to get eigs working, given eigs.m and python Arpack wrappers. > > I'm 99% sure it would *not* be kosher to copy-paste their code > directly, but it's not a copy paste job because of the differences > between Python and .m. So it seems a grayer area to me. > > --bb I haven't the faintest idea. What did the EULA say? What about a copyright? This sounds like a job for a lawyer, but I would guess it would be a bad idea unless the code is from a third party with a free license. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From wbaxter at gmail.com Sat Oct 21 20:27:39 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Sun, 22 Oct 2006 09:27:39 +0900 Subject: some work on arpack In-Reply-To: References: Message-ID: On 10/22/06, Charles R Harris wrote: > On 10/21/06, Bill Baxter wrote: > > Here's something I've been wondering: is it ok to port Matlab > > functions over to python? If so, then it's maybe an afternoon's work > > to get eigs working, given eigs.m and python Arpack wrappers. > > [...] > > I haven't the faintest idea. What did the EULA say? What about a copyright? > This sounds like a job for a lawyer, but I would guess it would be a bad > idea unless the code is from a third party with a free license. Gee, there's an idea. :-) ---- from License.txt ---- "Except as expressly provided by this Agreement, including the attached Addendum, Licensee may not adapt, translate, or convert "M-files", "MDL-files" or "P-code" contained in the Programs in order to create software, a principal purpose of which is to perform the same or similar functions as Programs licensed by MathWorks or which is intended to replace any component of the Programs. The Licensee may not incorporate or use "M-files", "P-code", source code, or any other part of the Programs in or as part of another computer program without the consent of MathWorks. A Licensed User may modify pieces of MathWorks' code for the Licensed User's own use. A Licensed User may share such modified code with others provided each recipient is also a Licensed User for the original form of the code. " ------------------------- So. If you're a Matlab licensee it would be ok to use it in your own stuff, but putting it in Numpy is definitely out of the question. --bb ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From AGED at HOTMAIL.COM Sun Oct 22 01:41:58 2006 From: AGED at HOTMAIL.COM (=?iso-2022-jp?B?QUdFRA==?=) Date: Sat, 21 Oct 2006 22:41:58 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCJDMkcyRLJEEkTxsoQg==?= Message-ID: ?????????????????? http://11487.com/shi/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Sun Oct 22 05:43:01 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Sun, 22 Oct 2006 11:43:01 +0200 Subject: some work on arpack In-Reply-To: References: Message-ID: <20061022094301.GO10843@mentat.za.net> On Sun, Oct 22, 2006 at 09:27:39AM +0900, Bill Baxter wrote: > On 10/22/06, Charles R Harris wrote: > > On 10/21/06, Bill Baxter wrote: > > > Here's something I've been wondering: is it ok to port Matlab > > > functions over to python? If so, then it's maybe an afternoon's work > > > to get eigs working, given eigs.m and python Arpack wrappers. > > > [...] > > > > I haven't the faintest idea. What did the EULA say? What about a copyright? > > This sounds like a job for a lawyer, but I would guess it would be a bad > > idea unless the code is from a third party with a free license. > > Gee, there's an idea. :-) > > ---- from License.txt ---- [...] > ------------------------- > > So. If you're a Matlab licensee it would be ok to use it in your own > stuff, but putting it in Numpy is definitely out of the question. A lot of the code in the Octave Forge is licensed under the BSD license, so you can use that. Unfortunately, the main Octave code is released under the GPL. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sebzur at pin.if.uz.zgora.pl Sun Oct 22 06:39:24 2006 From: sebzur at pin.if.uz.zgora.pl (=?UTF-8?B?U2ViYXN0aWFuIMW7dXJlaw==?=) Date: Sun, 22 Oct 2006 12:39:24 +0200 Subject: Model and experiment fitting. In-Reply-To: <07C6A61102C94148B8104D42DE95F7E8C8F0F7@exchange2k.envision.co.il> References: <07C6A61102C94148B8104D42DE95F7E8C8F0F7@exchange2k.envision.co.il> Message-ID: Nadav Horesh napisa?(a): > 1. If at least one of your data sets to be interpulated is on a grid, > you can use numpy.ndimage.map function for fast interpolation for 2d (in fact for any dimensional) dataset. I've already used a splines to interpolate a missing simulated points. That procedure works great and is very fast. But I'll check the numpy.ndimage - I haven't used it, yet. > 2. Isn't there an analytic expression to average the expectration values of SH over all possible orientations > between B and the crystal axis? My experience shows that some analytic work can save 99% of simulation time. Well, the simulations are already very fast. The time consumption is approximately ~0.3s for a single powder spectrum (2.8GHz Pentium D). The calculations are held by an external, very fine EPR spectra simulation tool. The author must have incorporated into it a lot of rationalizations, but this is a binary tool (unfortunately) and I do not know, what exactly sits inside of it... All I know, is that the orientations are represented by a grid (with an increment step tunable by a user). From library documentation: "After having computed the spectrum for a number of orientations specified, the simulation function interpolates these spectra for additional orientations before summing up all spectra." The interpolation is accomplish with a splines. Thank you for your comment, best regards Sebastian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From hagberg at lanl.gov Sun Oct 22 10:08:51 2006 From: hagberg at lanl.gov (Aric Hagberg) Date: Sun, 22 Oct 2006 08:08:51 -0600 Subject: some work on arpack In-Reply-To: References: Message-ID: <20061022140850.GR10447@t7.lanl.gov> On Sat, Oct 21, 2006 at 02:05:42PM -0700, Keith Goodman wrote: > Did you, or anybody else on the list, have any luck making a numpy > version of eigs? I made a start at an ARPACK wrapper, see http://projects.scipy.org/scipy/scipy/ticket/231 and the short thread at scipy-dev http://thread.gmane.org/gmane.comp.python.scientific.devel/5166/focus=5175 In addition to the wrapper there is a Python interface (and some tests). I don't know if the interface is like "eigs" - I don't use Matlab. It will give you a few eigenvalues and eigenvectors for the standard eigenproblem (Ax=lx) for any type of A (symmetric/nonsymmetric, real/complex, single/double, sparse/nonsparse). The generalized and shifted modes are not implemented. I need to find some time (or some help) to get it finished. Regards, Aric ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Sun Oct 22 10:20:36 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sun, 22 Oct 2006 16:20:36 +0200 Subject: some work on arpack In-Reply-To: <20061022140850.GR10447@t7.lanl.gov> References: <20061022140850.GR10447@t7.lanl.gov> Message-ID: <000c01c6f5e5$3eadc7d0$0a83a8c0@ratbert> Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Aric Hagberg > Sent: Sunday, October 22, 2006 4:09 PM > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] some work on arpack > > On Sat, Oct 21, 2006 at 02:05:42PM -0700, Keith Goodman wrote: > > Did you, or anybody else on the list, have any luck making a numpy > > version of eigs? > > I made a start at an ARPACK wrapper, see > http://projects.scipy.org/scipy/scipy/ticket/231 > and the short thread at scipy-dev > http://thread.gmane.org/gmane.comp.python.scientific.devel/5166/focus=5175 > > In addition to the wrapper there is a Python interface (and some tests). > I don't know if the interface is like "eigs" - I don't use Matlab. If you're interested in making an interface that looks like MATLAB's, you can take a peek at: > > It will give you a few eigenvalues and eigenvectors for the standard > eigenproblem (Ax=lx) for any type of A (symmetric/nonsymmetric, > real/complex, > single/double, sparse/nonsparse). > > The generalized and shifted modes are not implemented. > I need to find some time (or some help) to get it finished. > > Regards, > Aric > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Sun Oct 22 10:21:40 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sun, 22 Oct 2006 16:21:40 +0200 Subject: some work on arpack In-Reply-To: <20061022140850.GR10447@t7.lanl.gov> References: <20061022140850.GR10447@t7.lanl.gov> Message-ID: <000d01c6f5e5$64551010$0a83a8c0@ratbert> Argh. > > In addition to the wrapper there is a Python interface (and some tests). > I don't know if the interface is like "eigs" - I don't use Matlab. http://www.mathworks.de/access/helpdesk/help/techdoc/ref/index.html?/access/ helpdesk/help/techdoc/ref/eigs.html should give you some idea of how MATLAB's eigs works. Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Sun Oct 22 11:34:58 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Sun, 22 Oct 2006 08:34:58 -0700 Subject: some work on arpack In-Reply-To: <20061022140850.GR10447@t7.lanl.gov> References: <20061022140850.GR10447@t7.lanl.gov> Message-ID: On 10/22/06, Aric Hagberg wrote: > On Sat, Oct 21, 2006 at 02:05:42PM -0700, Keith Goodman wrote: > > Did you, or anybody else on the list, have any luck making a numpy > > version of eigs? > > I made a start at an ARPACK wrapper, see > http://projects.scipy.org/scipy/scipy/ticket/231 > and the short thread at scipy-dev > http://thread.gmane.org/gmane.comp.python.scientific.devel/5166/focus=5175 That looks very promising! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Sun Oct 22 13:10:55 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Mon, 23 Oct 2006 02:10:55 +0900 Subject: [Numpy-discussion] some work on arpack In-Reply-To: <20061022140850.GR10447@t7.lanl.gov> References: <20061022140850.GR10447@t7.lanl.gov> Message-ID: Don't know if this is any use, but to me (not knowing Fortran nearly so well as C++) this looks pretty useful: http://www.caam.rice.edu/software/ARPACK/arpack++.html http://www.ime.unicamp.br/~chico/arpack++/ It provides a nice high-level interface on top of ARPACK. I could see it being useful on a number of levels: 1) actually use its wrappers directly instead of calling the f2py'ed fortran code - at least as a stop gap measure to cover the holes ("shifted modes" etc) - this could potentially even be faster since the reverse-communication interface requires some iteration, and this way the loops would run in compiled C++ instead of python. 2) use it's code just as a reference to help write more wrappers to the fortran code 3) if nothing else, it has pretty decent documentation. For instance it includes a nice table of the amount of storage you need to reserve for various different calling modes. (It seems ARPACK's main documentation is not freely available, so I'm not sure what's in it, but if you have that then ARPACK++'s docs may not be much help). As far as I can tell, it must be under the same licensing terms as ARPACK itself. It doesn't actually specify anything as far as I could see. But it seems to be linked prominently from the ARPACK website and appears as an offshoot project. --bb On 10/22/06, Aric Hagberg wrote: > On Sat, Oct 21, 2006 at 02:05:42PM -0700, Keith Goodman wrote: > > Did you, or anybody else on the list, have any luck making a numpy > > version of eigs? > > I made a start at an ARPACK wrapper, see > http://projects.scipy.org/scipy/scipy/ticket/231 > and the short thread at scipy-dev > http://thread.gmane.org/gmane.comp.python.scientific.devel/5166/focus=5175 > > In addition to the wrapper there is a Python interface (and some tests). > I don't know if the interface is like "eigs" - I don't use Matlab. > > It will give you a few eigenvalues and eigenvectors for the standard > eigenproblem (Ax=lx) for any type of A (symmetric/nonsymmetric, real/complex, > single/double, sparse/nonsparse). > > The generalized and shifted modes are not implemented. > I need to find some time (or some help) to get it finished. > > Regards, > Aric From simon at arrowtheory.com Sun Oct 22 17:38:21 2006 From: simon at arrowtheory.com (Simon Burton) Date: Sun, 22 Oct 2006 14:38:21 -0700 Subject: PyDX - announcement Message-ID: <20061022143821.3b085ca4.simon@arrowtheory.com> Apologies if this is too off-topic for these lists, but I hope some people here find this interesting! PyDX - first public announcement ================================ Overview -------- PyDX is a package for working with calculus (differential geometry), arbitrary precision arithmetic (using gmpy), and interval arithmetic. It provides: * multivariate automatic differentiation (to arbitrary order) * Tensor objects for computations in differential geometry * Interval scalars (based on libMPFI) for calculating rigorous bounds on arithmetic operations (validated numerics). * Arbitrary order validated ODE solver. PyDX uses lazy computation techniques to greatly enhance performance of the resulting functions. This code grew out of a research project at The Australian National University, Department of Physics, which involved computing bounds on geodesics in relativistic space-time's. Documentation ------------- http://gr.anu.edu.au/svn/people/sdburton/pydx/doc/user-guide.html http://gr.anu.edu.au/svn/people/sdburton/pydx/doc/api/index.html Subversion Repository --------------------- http://gr.anu.edu.au/svn/people/sdburton/pydx/ Simon Burton October 22 2006 The Australian National University From ellisonbg.net at gmail.com Sun Oct 22 17:37:46 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sun, 22 Oct 2006 15:37:46 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <4539B905.3010705@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <4539B905.3010705@ieee.org> Message-ID: <6ce0ac130610221437i66118e04p33d64f0616dd1665@mail.gmail.com> > Most likely the error-detection code is not working on your platform. > The platform dependent stuff is not that difficult. I tried to > implement something for AIX, but very likely got it wrong (and don't > have a platform to test it on). It is the UFUNC_CHECK_STATUS that must > be implemented. Perhaps, we can do a simple check and disable the > error modes: I looked at the UFUNC_CHECK_STATUS implementation for AIX and put in some print statements. The the fpstatus returned to fp_read_flag is always indicating an FP_INVALID so UFUNC_CHECK_STATUS return 8. But the code looks fine to me. It seems a little odd that it would always indicate this flag. Where else can I look to try to see what it goiing on? Do you think this is worth worrying about, or should we just have use seterr(all='ignore')? I am willing to try to hunt this down, but I don't know much about the internals of numpy. Thanks Brian > seterr(all='ignore') > > will work and "turn-off" error-detection on your platform. > > -Travis > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Sun Oct 22 18:19:55 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sun, 22 Oct 2006 16:19:55 -0600 Subject: Problem introduced after 1.0rc2 on AIX with xlc In-Reply-To: <4539B905.3010705@ieee.org> References: <6ce0ac130610201942p1643e7dcs22df9ef8b8a6454b@mail.gmail.com> <4539B905.3010705@ieee.org> Message-ID: <6ce0ac130610221519q42a4eb4odc119aab8e4a6bee@mail.gmail.com> So, I have figured out the problem and have a solution. I have submitted a ticket for this: http://projects.scipy.org/scipy/numpy/ticket/362 That describes the problem and solution. Thanks for everyones ideas on this. Brian On 10/21/06, Travis Oliphant wrote: > Brian Granger wrote: > > Hi, > > > > i am running numpy on aix compiling with xlc. Revision 1.0rc2 works > > fine and passes all tests. But 1.0rc3 and more recent give the > > following on import: > > > > Most likely the error-detection code is not working on your platform. > The platform dependent stuff is not that difficult. I tried to > implement something for AIX, but very likely got it wrong (and don't > have a platform to test it on). It is the UFUNC_CHECK_STATUS that must > be implemented. Perhaps, we can do a simple check and disable the > error modes: > > seterr(all='ignore') > > will work and "turn-off" error-detection on your platform. > > -Travis > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From humufr at yahoo.fr Sun Oct 22 18:45:42 2006 From: humufr at yahoo.fr (humufr at yahoo.fr) Date: Sun, 22 Oct 2006 18:45:42 -0400 Subject: bug? Message-ID: <200610221845.42894.humufr@yahoo.fr> Hello, the docstring for compress in numpy give this help(numpy.compress) compress(condition, m, axis=None, out=None) compress(condition, x, axis=None) = those elements of x corresponding to those elements of condition that are "true". condition must be the same size as the given dimension of x. So (but perhaps I can misundertand the help due to my english) I don't undersand the following error, for me a and c array does have the same dimension and size. So someone can explain me the result please? Thanks, N. In [86]: a = numpy.arange(9) In [87]: a = numpy.arange(9).reshape(3,3) In [88]: c = numpy.ones(9).reshape(3,3) In [89]: numpy.compress(c,a) --------------------------------------------------------------------------- exceptions.ValueError Traceback (most recent call last) /home/gruel/tmp/Astro/FATBOY/ /home/gruel/usr/lib/python2.4/site-packages/numpy/core/fromnumeric.py in compress(condition, m, axis, out) 353 except AttributeError: 354 return _wrapit(m, 'compress', condition, axis, out) --> 355 return compress(condition, axis, out) 356 357 def clip(m, m_min, m_max): ValueError: condition must be 1-d array ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Sun Oct 22 19:12:57 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 22 Oct 2006 18:12:57 -0500 Subject: bug? In-Reply-To: <200610221845.42894.humufr@yahoo.fr> References: <200610221845.42894.humufr@yahoo.fr> Message-ID: humufr at yahoo.fr wrote: > Hello, > > the docstring for compress in numpy give this > > help(numpy.compress) > > compress(condition, m, axis=None, out=None) > compress(condition, x, axis=None) = those elements of x corresponding > to those elements of condition that are "true". condition must be the > same size as the given dimension of x. > > > So (but perhaps I can misundertand the help due to my english) I don't > undersand the following error, for me a and c array does have the same > dimension and size. So someone can explain me the result please? The docstring is a bit underspecified. The condition array *must* be a 1D array with the same size *as the given axis* of the other array (using the convention that axis=None implies operating over the flattened array). There's simply no valid interpretation of this, for example: compress(array([[1, 0, 0], [1, 1, 0]]), arange(6).reshape(2,3)) since numpy arrays cannot be "ragged". -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Sun Oct 22 20:47:10 2006 From: fullung at gmail.com (Albert Strasheim) Date: Mon, 23 Oct 2006 02:47:10 +0200 Subject: Strange results when sorting array with fields Message-ID: <002001c6f63c$c63c0250$0a83a8c0@ratbert> Hello all I'm trying to sort an array with two fields, but I'm getting a result that doesn't seem to make sense. What I tried (first attempt): I have two 2-D arrays. I would like to sort one based on the sort of the other. I managed to do this with argsort. However, the fancy indexing required to get the sorted array using what argsort returned was very slow. I followed this example: http://www.scipy.org/Numpy_Example_List#head-9f8656795227e3c43e849c6c0435eee b32afd722 What I tried (second attempt): I created an array with two fields. I think/hope/expected that sorting the array would sort it on the first field in the dtype and then on the second. This is *much* faster than the fancy indexing approach. Code: import numpy as N print N.__version__ print dt = N.dtype([('aaa', N.float64), ('bbb', N.uint8)]) x = N.empty((2, 2), dt) xa = x[dt.names[0]] xa[:] = [[1.,2], [3.,4.]] xb = x[dt.names[1]] xb[:] = [[1, 2], [3, 4]] print 'before sort:' print x print x.sort(kind='quicksort') # other kinds not supported print 'after sort:' print x Output on my system: 1.0.dev3376 before sort: [[(1.0, 1) (2.0, 2)] [(3.0, 3) (4.0, 4)]] after sort: [[(2.0, 2) (1.0, 1)] [(4.0, 4) (3.0, 3)]] The already sorted array has been unsorted in some way. Any thoughts? Thanks! Regards, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jfishman at bowdoin.edu Sun Oct 22 22:05:40 2006 From: jfishman at bowdoin.edu (Jeremy R. Fishman) Date: Sun, 22 Oct 2006 22:05:40 -0400 Subject: Cross compilation Message-ID: <453C2374.5000208@bowdoin.edu> Hi, I was wondering if anyone can give me some advice on how to go about cross-compiling NumPy. I have been searching around and can't find any support in distutils for cross compilation. Is there some way I can still compile Numerical Python using a mipsel-linux compiler, on say a Cygwin host? The end result would be to use NumPy with an embedded Python interpreter running on a mipsel host. In specific, The OPEN-R system by Sony for use on Aibo robotic dogs. I need to figure out how to get setup to use the mipsel compiler, not how to set up the compiler. Thanks, Jeremy Fishman -------------- next part -------------- A non-text attachment was scrubbed... Name: jfishman.vcf Type: text/x-vcard Size: 158 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Sun Oct 22 22:44:07 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 22 Oct 2006 21:44:07 -0500 Subject: Cross compilation In-Reply-To: <453C2374.5000208@bowdoin.edu> References: <453C2374.5000208@bowdoin.edu> Message-ID: Jeremy R. Fishman wrote: > Hi, I was wondering if anyone can give me some advice on how to go about > cross-compiling NumPy. I have been searching around and can't find any > support in distutils for cross compilation. Is there some way I can > still compile Numerical Python using a mipsel-linux compiler, on say a > Cygwin host? I'm afraid that distutils really does not support cross-compilation. numpy adds some more complications in that it tries to configure itself by compiling and running small programs. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sun Oct 22 23:22:20 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 22 Oct 2006 21:22:20 -0600 Subject: Strange results when sorting array with fields In-Reply-To: <002001c6f63c$c63c0250$0a83a8c0@ratbert> References: <002001c6f63c$c63c0250$0a83a8c0@ratbert> Message-ID: On 10/22/06, Albert Strasheim wrote: > > Hello all > > I'm trying to sort an array with two fields, but I'm getting a result that > doesn't seem to make sense. > > What I tried (first attempt): I have two 2-D arrays. I would like to sort > one based on the sort of the other. I managed to do this with argsort. > However, the fancy indexing required to get the sorted array using what > argsort returned was very slow. I followed this example: It is certainly awkward. I am going to add a function to do this after 1.0comes out. Sounds like you need it now. http://www.scipy.org/Numpy_Example_List#head-9f8656795227e3c43e849c6c0435eee > b32afd722 > > What I tried (second attempt): I created an array with two fields. I > think/hope/expected that sorting the array would sort it on the first > field > in the dtype and then on the second. This is *much* faster than the fancy > indexing approach. I believe it sorts on the two fields together as one big binary blob. Output on my system: > > 1.0.dev3376 > > before sort: > [[(1.0, 1) (2.0, 2)] > [(3.0, 3) (4.0, 4)]] > > after sort: > [[(2.0, 2) (1.0, 1)] > [(4.0, 4) (3.0, 3)]] > > The already sorted array has been unsorted in some way. Any thoughts? I suspect something to do with the binary representation of the floats. Probably depends on big/little endian also. The floats 1.0, 2.0, and 4.0all have zero mantissas while 3.0 has a one. The sort behaves differently if uint8 is used for both fields. In [3]: dt = dtype([('aaa', uint8), ('bbb', uint8)]) In [4]: x = empty((2, 2), dt) In [5]: xa = x[dt.names[0]] In [6]: xa[:] = [[1,2], [3,4]] In [7]: xb = x[dt.names[1]] In [8]: xb[:] = [[1,2], [3,4]] In [9]: x.sort() In [11]: x Out[11]: array([[(1, 1), (2, 2)], [(3, 3), (4, 4)]], dtype=[('aaa', '|u1'), ('bbb', '|u1')]) Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Mon Oct 23 00:24:41 2006 From: pgmdevlist at gmail.com (P GM) Date: Mon, 23 Oct 2006 00:24:41 -0400 Subject: A reimplementation of MaskedArray + MaskedRecords Message-ID: <777651ce0610222124r40697b0eyaf4a7c107ea3e624@mail.gmail.com> Folks, I updated the alternative implementation of MaskedArray on the wiki, mainly to correct a couple of bugs. (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) In addition, I attached another file, maskedrecordarray, which introduce a new class, MaskedRecord, as a subclass of recarray and MaskedArray. An instance of this class accepts a recarray as data, and uses two masks: the 'recordmask' has as many entries as records in the array, each entry with the same fields as a record, but of boolean types, indicating whether a field is masked or not; an entry is flagged as masked in the 'mask' array if at least one field is masked. The 'mask' object is introduced mostly for compatibilty with MaskedArray, only 'recordmask' is really useful. A few examples in the file should give you an idea of what can be done. In particular, you can define a new maskedrecord array as simply as ; a = masked_record([('Alan',29,200.), ('Bill',31,260.0)], dtype=[('name','S30'),('age',int_),('weight',float_)], mask=[(1,0,0), (0,0,0)]) Note that maskedrecordarray is still quite experimental. As I'm not a regular user of records, I don't really know what should be implemented... The file can be accessed at http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedrecordarray.py Once again, I need your comments and suggestions ! Thanks. Pierre -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From lfriedri at imtek.de Mon Oct 23 02:25:11 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Mon, 23 Oct 2006 08:25:11 +0200 Subject: memory position of numpy arrays In-Reply-To: <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> Message-ID: <1161584711.6785.12.camel@localhost> Hello all, to interact with some hardware (data retrieval), I use the following scheme (Windows, Python 2.4): * in Python, I create a numpy array as a buffer * I pass this array to a self written .dll using ctypes * the C-code in the .dll passes the pointer to the buffer to the API of the hardware; then the API starts writing data to my buffer * from python I can use a helper function of the .dll to know which part of the buffer is safe to be read at the moment; so I can copy this part to a different numpy array and work with the data This works quite well, but today I got a blue screen, reporting some paging-problem. It occured after doing some other stuff like moving some windows on the screen and importing pylab, and I heard the harddisk working hard, so I assume that the part of memory my buffer is in, was paged to the harddisk. This is a problem, since the hardware driver will continuously try to write to this specific memory location I gave to it. My primary question is, how to avoid a numpy-array being paged. In the ideal case there would be a flag to set, that makes sure that this array is always at this position in physical memory. Of course I am also interested in other people's work on hardware access. Do you think the above discribed way is a good one? To me it was the best way because I could do as much as possible in Python and keep may C-coded .dll very small. Is anyone doing similar things? Thanks for every comment Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From strawman at astraw.com Mon Oct 23 03:45:19 2006 From: strawman at astraw.com (Andrew Straw) Date: Mon, 23 Oct 2006 00:45:19 -0700 Subject: memory position of numpy arrays In-Reply-To: <1161584711.6785.12.camel@localhost> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> Message-ID: <453C730F.4090000@astraw.com> It sounds like your hardware drivers may be buggy -- you should only get segfaults, not (the Windows equivalent of) kernel panics, when your userspace code accesses wrong memory. But if you have buggy hardware drivers, I suppose it's possible that locking the memory will help. This wouldn't be the domain of numpy, however. In Linux, this is acheived with a system call such as mlock() or mlockall(). You'll have to figure out the appropriate call in Windows. Thinking about it, if your hardware drivers require the memory to be locked, they should do it themselves. However, I'm not convinced this is the real issue. It seems at least equally likely that your hardware drivers were developed with particular pattern of timing when accessing the buffers, but now you may be attempting to hold a buffer longer (preventing the driver writing to it) than the developer ever tested. It shouldn't blue-screen, but it does... I think it quite likely that you have some buggy hardware drivers. What hardware is it? -Andrew Lars Friedrich wrote: > Hello all, > > to interact with some hardware (data retrieval), I use the following > scheme (Windows, Python 2.4): > > * in Python, I create a numpy array as a buffer > * I pass this array to a self written .dll using ctypes > * the C-code in the .dll passes the pointer to the buffer to the API of > the hardware; then the API starts writing data to my buffer > * from python I can use a helper function of the .dll to know which part > of the buffer is safe to be read at the moment; so I can copy this part > to a different numpy array and work with the data > > This works quite well, but today I got a blue screen, reporting some > paging-problem. It occured after doing some other stuff like moving some > windows on the screen and importing pylab, and I heard the harddisk > working hard, so I assume that the part of memory my buffer is in, was > paged to the harddisk. This is a problem, since the hardware driver will > continuously try to write to this specific memory location I gave to it. > > My primary question is, how to avoid a numpy-array being paged. In the > ideal case there would be a flag to set, that makes sure that this array > is always at this position in physical memory. > > Of course I am also interested in other people's work on hardware > access. Do you think the above discribed way is a good one? To me it was > the best way because I could do as much as possible in Python and keep > may C-coded .dll very small. Is anyone doing similar things? > > Thanks for every comment > > Lars > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mark at hymers.org.uk Mon Oct 23 06:50:27 2006 From: mark at hymers.org.uk (Mark Hymers) Date: Mon, 23 Oct 2006 11:50:27 +0100 Subject: Unit test error with numpy rc3 In-Reply-To: <45378BC6.70609@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> Message-ID: <20061023105027.GA6741@titus.mhy.org.uk> On Thu, 19, Oct, 2006 at 08:29:26AM -0600, Travis Oliphant spoke thus.. > Actually, you shouldn't be getting an INF at all. This is what the > test is designed to test for (so I guess it's working). The test was > actually written wrong and was never failing because previously keyword > arguments to ufuncs were ignored. > > Can you show us what 'a' is on your platform. Hi, I've just done a Mac OS X PPC build of the SVN trunk and am getting this failure too. nidesk046:~/scratch/upstream/scipy mark$ python Python 2.4.1 (#2, Mar 31 2005, 00:05:10) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy as N >>> N.__version__ '1.0.dev3378' >>> N.array([1000],dtype=N.float).dtype dtype('float64') >>> N.array([1000],dtype=N.longfloat).dtype dtype('float128') >>> N.test() ...snip... FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", line 220, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError >>> a = N.exp(N.array([1000],dtype=N.longfloat)) >>> str(a) '[inf]' Any ideas about this? Mark -- Mark Hymers "I once absent-mindedly ordered Three Mile Island dressing in a restaurant and, with great presence of mind, they brought Thousand Island Dressing and a bottle of chili sauce." Terry Pratchett, alt.fan.pratchett -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at mailcan.com Mon Oct 23 00:21:56 2006 From: pgmdevlist at mailcan.com (Pierre GM) Date: Mon, 23 Oct 2006 00:21:56 -0400 Subject: A reimplementation of MaskedArray + MaskedRecords In-Reply-To: <16761e100610171719we82588mf8138bdfadce3398@mail.gmail.com> References: <200610160232.20888.pgmdevlist@mailcan.com> <200610162242.58541.pgmdevlist@mailcan.com> <16761e100610171719we82588mf8138bdfadce3398@mail.gmail.com> Message-ID: <200610230021.56852.pgmdevlist@mailcan.com> Folks, I updated the alternative implementation of MaskedArray on the wiki, mainly to correct a couple of bugs. (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) In addition, I attached another file, maskedrecordarray, which introduce a new class, MaskedRecord, as a subclass of recarray and MaskedArray. An instance of this class accepts a recarray as data, and uses two masks: the 'recordmask' has as many entries as records in the array, each entry with the same fields as a record, but of boolean types, indicating whether a field is masked or not; an entry is flagged as masked in the 'mask' array if at least one field is masked. The 'mask' object is introduced mostly for compatibilty with MaskedArray, only 'recordmask' is really useful. A few examples in the file should give you an idea of what can be done. In particular, you can define a new maskedrecord array as simply as ; a = masked_record([('Alan',29,200.), ('Bill',31,260.0)], dtype=[('name','S30'),('age',int_),('weight',float_)], mask=[(1,0,0), (0,0,0)]) Note that maskedrecordarray is still quite experimental. As I'm not a regular user of records, I don't really know what should be implemented... The file can be accessed at http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedrecordarray.py Once again, I need your comments and suggestions ! Thanks. Pierre ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From sransom at nrao.edu Mon Oct 23 11:57:57 2006 From: sransom at nrao.edu (Scott Ransom) Date: Mon, 23 Oct 2006 11:57:57 -0400 Subject: ix_ problem changing input arrays? Message-ID: <200610231157.57181.sransom@nrao.edu> I believe that ix_() has recently begun modifying the shapes of its input arrays. For instance: Python 2.4.4c0 (#2, Jul 30 2006, 18:20:12) [GCC 4.1.2 20060715 (prerelease) (Debian 4.1.1-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy as N >>> a = N.array([1,2,3]) >>> b = N.array([7,6,5,4]) >>> ax, bx = N.ix_(a, b) >>> a array([[1], [2], [3]]) >>> b array([[7, 6, 5, 4]]) >>> N.__version__ '1.0.dev3379' Is this intended behaviour? Thanks, Scott -- Scott M. Ransom Address: NRAO Phone: (434) 296-0320 520 Edgemont Rd. email: sransom at nrao.edu Charlottesville, VA 22903 USA GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Mon Oct 23 12:32:54 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Mon, 23 Oct 2006 18:32:54 +0200 Subject: ix_ problem changing input arrays? In-Reply-To: <200610231157.57181.sransom@nrao.edu> References: <200610231157.57181.sransom@nrao.edu> Message-ID: <20061023163254.GE30511@mentat.za.net> On Mon, Oct 23, 2006 at 11:57:57AM -0400, Scott Ransom wrote: > I believe that ix_() has recently begun modifying the shapes of its > input arrays. For instance: [...] This should be fixed in SVN. Cheers St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cjw at sympatico.ca Mon Oct 23 14:22:49 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Mon, 23 Oct 2006 14:22:49 -0400 Subject: [Fwd: numpy error] Message-ID: <453D0879.4030407@sympatico.ca> -------- Original Message -------- Subject: numpy error Date: Mon, 23 Oct 2006 20:01:33 +0200 From: Juergen Kareta Newsgroups: gmane.comp.python.general Hello, this is my first try to get wxmpl-1.2.8 running. Therefor I installed: python 2.5 matplotlib-0.87.6.win32-py2.5.exe numpy-1.0rc3.win32-py2.5.exe on WinXP SP2 The result is a version mismatch (see below). Numpy version 1000002 seems to be numpy-1.0b5 which is not downloadable anymore. Any hints ? Thanks in advance. J?rgen traceback: from pylab import * RuntimeError: module compiled against version 1000002 of C-API but this version of numpy is 1000009 The import of the numpy version of the nxutils module, _nsnxutils, failed. This is is either because numpy was unavailable when matplotlib was compiled, because a dependency of _nsnxutils could not be satisfied, or because the build flag for this module was turned off in setup.py. If it appears that _nsnxutils was not built, make sure you have a working copy of numpy and then re-install matplotlib. Otherwise, the following traceback gives more details: Traceback (most recent call last): File "", line 1, in File "C:\Python25\Lib\site-packages\pylab.py", line 1, in from matplotlib.pylab import * File "C:\Python25\Lib\site-packages\matplotlib\pylab.py", line 199, in import mlab #so I can override hist, psd, etc... File "C:\Python25\Lib\site-packages\matplotlib\mlab.py", line 64, in import nxutils File "C:\Python25\Lib\site-packages\matplotlib\nxutils.py", line 17, in from matplotlib._ns_nxutils import * ImportError: numpy.core.multiarray failed to import -- http://mail.python.org/mailman/listinfo/python-list ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Mon Oct 23 16:50:44 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Mon, 23 Oct 2006 13:50:44 -0700 Subject: combinatorics Message-ID: <453D2B24.9020600@jpl.nasa.gov> Hi Is there any support for combinatorics in numpy or scipy? Actually, all I need is to evaluate is (n/m) i.e. n choose m. If numpy doesn't have it, anyone know of a good public domain prog? Mathew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 16:57:23 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 14:57:23 -0600 Subject: combinatorics In-Reply-To: <453D2B24.9020600@jpl.nasa.gov> References: <453D2B24.9020600@jpl.nasa.gov> Message-ID: <453D2CB3.5000504@ieee.org> Mathew Yeates wrote: > Hi > Is there any support for combinatorics in numpy or scipy? Actually, all > I need is to evaluate is (n/m) i.e. n choose m. > > In [3]: scipy.comb(109,54,exact=1) Out[3]: 49263609265046928387789436527216L 109 choose 54 -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Mon Oct 23 17:22:00 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Mon, 23 Oct 2006 14:22:00 -0700 Subject: combinatorics In-Reply-To: <453D2CB3.5000504@ieee.org> References: <453D2B24.9020600@jpl.nasa.gov> <453D2CB3.5000504@ieee.org> Message-ID: <453D3278.3010004@jpl.nasa.gov> Thank you! Travis Oliphant wrote: > Mathew Yeates wrote: > >> Hi >> Is there any support for combinatorics in numpy or scipy? Actually, all >> I need is to evaluate is (n/m) i.e. n choose m. >> >> >> > In [3]: scipy.comb(109,54,exact=1) > Out[3]: 49263609265046928387789436527216L > > 109 choose 54 > > -Travis > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 17:30:31 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 15:30:31 -0600 Subject: Strange results when sorting array with fields In-Reply-To: <002001c6f63c$c63c0250$0a83a8c0@ratbert> References: <002001c6f63c$c63c0250$0a83a8c0@ratbert> Message-ID: <453D3477.7080008@ieee.org> Albert Strasheim wrote: > Hello all > > I'm trying to sort an array with two fields, but I'm getting a result that > doesn't seem to make sense. > > What I tried (first attempt): I have two 2-D arrays. I would like to sort > one based on the sort of the other. I managed to do this with argsort. > However, the fancy indexing required to get the sorted array using what > argsort returned was very slow. I followed this example: > > http://www.scipy.org/Numpy_Example_List#head-9f8656795227e3c43e849c6c0435eee > b32afd722 > > What I tried (second attempt): I created an array with two fields. I > think/hope/expected that sorting the array would sort it on the first field > in the dtype and then on the second. This is *much* faster than the fancy > indexing approach. > That kind of sorting is what lexsort does (although with indexes) and is more complicated than what the sort routine does in NumPy. The sorting routines in NumPy use the output of the comparison operator for the type to compute the result. An array with fields is of type void. Right now, the VOID_compare routine is equivalent to the STRING_compare routine (i.e. raw bytes are compared). I doubt this will do what you want in most cases. It would be possible to adapt this compare routine when fields are present and so something like compare the first field first and the second field only if the first is equal. But, this would require a bit of work and is probably best left for 1.0.1 or later. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From loredo at astro.cornell.edu Mon Oct 23 17:32:11 2006 From: loredo at astro.cornell.edu (Tom Loredo) Date: Mon, 23 Oct 2006 17:32:11 -0400 Subject: Model and experiment fitting In-Reply-To: References: Message-ID: <1161639131.453d34db7de47@astrosun2.astro.cornell.edu> Hi Sebastian, I'm still unclear about the problem. From your last description, it sounds like the problem is *not* 2-D; you are trying to model a 1-d function of a 1-D argument (unless B is not the scalar field strength, but rather \vec B, the 3-D field vector). Also, your problem appears to be a regression (i.e., curve/surface fitting) problem, not a density estimation problem (comparing samples drawn from two distributions), so the Numerical Recipes suggestion (which addresses comparing densities) is probably not relevant. It sounds like you have some observed data that perhaps can be modeled as y_i = f(x_i; theta) + e_i where e_i represent measurement error, f(x;theta) is the "true model" that gives the intensity as a function of the field strength, which depends on some parameters theta, and y_i are the measured intensities. I presume you know something about the y_i measurement errors (like a std error for all or for each of them). You also have a computational model that gives you simulation data that can be modeled as Y_i = g(X_i; theta) with (presumably) no Y_i "measurement error" (though perhaps there is an equivalent if you use Monte Carlo in the calculation or have other quantifiable sources of error). As you phrased the problem, it appears you know theta exactly for both the observational and simulation data (an unusual situation!). You just want to ascertain whether g is a good approximation to f. Is this correct? There is a large literature on problems like this where theta is *unknown* and one wants to either calibrate the simulation or infer theta for the observations from a sparse set of runs of the simulation at various theta. But no papers come immediately to mind for the known theta case, though the "validation" stage of some of the available papers address it. For the unknown theta case, the relevant literature is fairly new and goes under various names (DACE: Design & Analysis of Computer Experiments; BACCO: Bayesian Analysis of Computer Code Output; MUCM: Managing Uncertainty in Complex Models). The main tools are interpolators that *quantify uncertainty* in the interpolation and machinery to propagate uncertainty through subsequent analyses. Gaussian processes (which include things like splines, kriging and random walks as special cases, but with "error propagation") are used to build an "emulator" for the simulation (an emulator is an interpolator that also gives you measures of uncertainty in the interpolation). There are free codes available for Matlab and R implementing this paradigm, but it's a tricky business (as I am slowly discovering, having just started to delve into it). It is unclear from your description why the X_i cannot be made equal to the x_i. If either or both are uncontrollable---esp. if you cannot set x_i but have to rely on "nature" providing you x_i values that would differ from one set of observations to the next---this adds a layer of complexity that is not trivial. It becomes a "measurement error problem" (aka "errors-in-the-variables problem"), with subtle aspects (you can easily go astray by simply ignoring measurement errors on the x's; they do *not* "average out" as you get more sample points). This can (and has) been incorporated into the emulator framework, though I don't know if any of the free code does this. Part of the MUCM/BACCO approach is estimation of a "bias" or "misfit" term between the simulation and calibration data (here it would be delta(x) = g(x)-f(x)). Perhaps your problem can be phrased in terms of whether the bias function is significantly nonzero anywhere, and if so, where. There are various places to go to learn about this stuff if it interests you; here are a few links. The first two are to the home pages of two working groups at SAMSI (an interdisciplinary statistics/appl. math institute), which is devoting this year to research on methods and applications in analysis of computer models. http://www.stat.duke.edu/~fei/samsi/index.html http://www4.stat.ncsu.edu/~gawhite/SAMSIweb/ http://mucm.group.shef.ac.uk/ http://cran.r-project.org/src/contrib/Descriptions/BACCO.html http://www2.imm.dtu.dk/~hbn/dace/ A particularly accessible paper is by the Los Alamos statistics group: http://portal.acm.org/citation.cfm?id=1039891.1039922 Again, I don't think this literature directly addresses your problem, but it may provide the "right" way to approach it, if you are willing to do some work to connect your problem to this framework. The short answer to your question is that I think you will be hard pressed to get a good answer to your question using off-the-shelf SciPy tools. Good luck, Tom Loredo ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 23 17:39:42 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 23 Oct 2006 15:39:42 -0600 Subject: Strange and hard to reproduce crash Message-ID: Hi all, two colleagues have been seeing occasional crashes from very long-running code which uses numpy. We've now gotten a backtrace from one such crash, unfortunately it uses a build from a few days ago: In [3]: numpy.__version__ Out[3]: '1.0b5.dev3097' In [4]: scipy.__version__ Out[4]: '0.5.0.2180' Because it takes so long to get the code to crash (several days of 100%CPU usage), I can't make a new one right now, but I'll be happy to restart the same run with a current SVN build if necessary, and post the results in a few days. In the meantime, here's a gdb backtrace we were able to get by setting MALLOC_CHECK_ to 2 and running the python process from within gdb: Program received signal SIGABRT, Aborted. [Switching to Thread 1073880896 (LWP 26280)] 0x40000402 in __kernel_vsyscall () (gdb) bt #0 0x40000402 in __kernel_vsyscall () #1 0x0042c7d5 in raise () from /lib/tls/libc.so.6 #2 0x0042e149 in abort () from /lib/tls/libc.so.6 #3 0x0046b665 in free_check () from /lib/tls/libc.so.6 #4 0x00466e65 in free () from /lib/tls/libc.so.6 #5 0x005a4ab7 in PyObject_Free () from /usr/lib/libpython2.3.so.1.0 #6 0x403f6336 in arraydescr_dealloc (self=0x40424020) at arrayobject.c:10455 #7 0x403fab3e in PyArray_FromArray (arr=0xe081cb0, newtype=0x40424020, flags=0) at arrayobject.c:7725 #8 0x403facc3 in PyArray_FromAny (op=0xe081cb0, newtype=0x0, min_depth=0, max_depth=0, flags=0, context=0x0) at arrayobject.c:8178 #9 0x4043bc45 in PyUFunc_GenericFunction (self=0x943a660, args=0xa9dbf2c, mps=0xbfc83730) at ufuncobject.c:906 #10 0x40440a04 in ufunc_generic_call (self=0x943a660, args=0xa9dbf2c) at ufuncobject.c:2742 #11 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 #12 0x0057d6d4 in PyObject_CallFunction () from /usr/lib/libpython2.3.so.1.0 #13 0x403eabb6 in PyArray_GenericBinaryFunction (m1=Variable "m1" is not available. ) at arrayobject.c:3296 #14 0x0057b7e1 in PyNumber_Check () from /usr/lib/libpython2.3.so.1.0 #15 0x0057c1e0 in PyNumber_Multiply () from /usr/lib/libpython2.3.so.1.0 #16 0x005d16a3 in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #17 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #18 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #19 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #20 0x00590e2e in PyFunction_SetClosure () from /usr/lib/libpython2.3.so.1.0 #21 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 #22 0x00584d98 in PyMethod_New () from /usr/lib/libpython2.3.so.1.0 #23 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 #24 0x005b584c in _PyObject_SlotCompare () from /usr/lib/libpython2.3.so.1.0 #25 0x005aec2c in PyType_IsSubtype () from /usr/lib/libpython2.3.so.1.0 #26 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 #27 0x005d2b7f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #28 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #29 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #30 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #31 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #32 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #33 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #34 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 #35 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 #36 0x005d5362 in PyEval_EvalCode () from /usr/lib/libpython2.3.so.1.0 #37 0x005ee817 in PyErr_Display () from /usr/lib/libpython2.3.so.1.0 #38 0x005ef942 in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.3.so.1.0 #39 0x005f0994 in PyRun_AnyFileExFlags () from /usr/lib/libpython2.3.so.1.0 #40 0x005f568e in Py_Main () from /usr/lib/libpython2.3.so.1.0 #41 0x080485b2 in main () # End of BT. This code is running on a Fedora Core 3 box, with python 2.3.4 and numpy/scipy compiled using gcc 3.4.4. I realize that it's extremely difficult to help with so little information, but unfortunately we have no small test that can reproduce the problem. Only our large research codes, when running for multiple days on a single run, cause this. Even very intensive uses of the same code but which last only a few hours never show this. This code is a long-runing iterative algorithm, so it's basically applying the same (complex) loop over and over until convergence, using numpy and scipy pretty extensively throughout. If super Travis (or anyone else) can have a Eureka moment from the above backtrace, that would be fantastic. If there's any other information you think I may be able to provide, I'll be happy to do my best. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 18:04:14 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 16:04:14 -0600 Subject: Strange results when sorting array with fields In-Reply-To: <002001c6f63c$c63c0250$0a83a8c0@ratbert> References: <002001c6f63c$c63c0250$0a83a8c0@ratbert> Message-ID: <453D3C5E.6080609@ieee.org> Albert Strasheim wrote: > Hello all > > I'm trying to sort an array with two fields, but I'm getting a result that > doesn't seem to make sense. > > What I tried (first attempt): I have two 2-D arrays. I would like to sort > one based on the sort of the other. I managed to do this with argsort. > However, the fancy indexing required to get the sorted array using what > argsort returned was very slow. I followed this example: > > http://www.scipy.org/Numpy_Example_List#head-9f8656795227e3c43e849c6c0435eee > b32afd722 > > What I tried (second attempt): I created an array with two fields. I > think/hope/expected that sorting the array would sort it on the first field > in the dtype and then on the second. This is *much* faster than the fancy > indexing approach. > O.K. I lied, I realized that my comments in the VOID_compare code were silly (about being unable to define > or <). It makes sense to just define them based on the first field, then the second field (if the first field is equal), and so forth. Obviously this is not the only way one could define sorting (any field could be used as "the first field", and so forth. But, is is a fairly obvious default to use the first field. It turns out, it was not so difficult to implement this and is now in SVN. So, the VOID_compare now does something a little more intelligent when fields are defined which means that record arrays can now be lexicographically sorted more easily than using lexsort and take (as long as the fields are ordered according to how you want the sort to proceed). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Mon Oct 23 18:07:02 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 24 Oct 2006 00:07:02 +0200 Subject: Strange and hard to reproduce crash In-Reply-To: References: Message-ID: <009b01c6f6ef$91aa6dc0$0a83a8c0@ratbert> Hey Fernando Maybe you can give the code a spin under Valgrind. It's going to be slow, but if the crash is being caused by memory corruption that happens all the time as the process is running, maybe Valgrind will show it. You need some Valgrind suppressions for Python. It seems the 2.3 source tree didn't contain these yet, so try the one from trunk: http://svn.python.org/view/python/trunk/Misc/valgrind-python.supp?rev=47113& view=auto I then run Valgrind as follows: valgrind \ --tool=memcheck \ --leak-check=yes \ --error-limit=no \ --suppressions=valgrind-python.supp \ --num-callers=20 \ --freelist-vol=536870912 \ -v \ python foo.py I recommend using the latest Valgrind (3.2.1) from here: http://www.valgrind.org/downloads/current.html#current A build from source should be as simple as ./configure && make. Cheers, Albert > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Fernando Perez > Sent: Monday, October 23, 2006 11:40 PM > To: Discussion of Numerical Python > Subject: [Numpy-discussion] Strange and hard to reproduce crash > > Hi all, > > two colleagues have been seeing occasional crashes from very > long-running code which uses numpy. We've now gotten a backtrace from > one such crash, unfortunately it uses a build from a few days ago: > > In [3]: numpy.__version__ > Out[3]: '1.0b5.dev3097' > > In [4]: scipy.__version__ > Out[4]: '0.5.0.2180' > > Because it takes so long to get the code to crash (several days of > 100%CPU usage), I can't make a new one right now, but I'll be happy to > restart the same run with a current SVN build if necessary, and post > the results in a few days. > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 18:10:31 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 16:10:31 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: References: Message-ID: <453D3DD7.50506@ieee.org> Fernando Perez wrote: > Hi all, > > two colleagues have been seeing occasional crashes from very > long-running code which uses numpy. We've now gotten a backtrace from > one such crash, unfortunately it uses a build from a few days ago: > This looks like a reference-count problem on the data-type objects (probably one of the builtin ones is trying to be released). The reference count problem is probably hard to track down. A quick fix is to not allow the built-ins to be "freed" (the attempt should never be made, but if it is, then we should just incref the reference count and continue rather than die). Ideally, the reference count problem should be found, but other-wise I'll just insert some print statements if the attempt is made, but not actually do it as a safety measure. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 18:32:00 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 16:32:00 -0600 Subject: Release of 1.0 coming In-Reply-To: <453D3DD7.50506@ieee.org> References: <453D3DD7.50506@ieee.org> Message-ID: <453D42E0.9040609@ieee.org> The long awaited day is coming....--- Wednesday is the target. Please submit problems before Tuesday (tomorrow). Nothing but bug-fixes are being changed right now. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 23 18:41:17 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 23 Oct 2006 16:41:17 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: <009b01c6f6ef$91aa6dc0$0a83a8c0@ratbert> References: <009b01c6f6ef$91aa6dc0$0a83a8c0@ratbert> Message-ID: On 10/23/06, Albert Strasheim wrote: > Hey Fernando > > Maybe you can give the code a spin under Valgrind. It's going to be slow, > but if the crash is being caused by memory corruption that happens all the > time as the process is running, maybe Valgrind will show it. > > You need some Valgrind suppressions for Python. It seems the 2.3 source tree > didn't contain these yet, so try the one from trunk: [...] Thanks, Albert. I can give it a try, though it will probably take ages to run. This already requires 3-4 days of non-stop execution to cause a crash, and valgrind can make execution times go up by a factor of 10. I'd like to have some info before a month :) Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 23 18:44:24 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 23 Oct 2006 16:44:24 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: <453D3DD7.50506@ieee.org> References: <453D3DD7.50506@ieee.org> Message-ID: On 10/23/06, Travis Oliphant wrote: > Fernando Perez wrote: > > Hi all, > > > > two colleagues have been seeing occasional crashes from very > > long-running code which uses numpy. We've now gotten a backtrace from > > one such crash, unfortunately it uses a build from a few days ago: > > > This looks like a reference-count problem on the data-type objects > (probably one of the builtin ones is trying to be released). The > reference count problem is probably hard to track down. > > A quick fix is to not allow the built-ins to be "freed" (the attempt > should never be made, but if it is, then we should just incref the > reference count and continue rather than die). > > Ideally, the reference count problem should be found, but other-wise > I'll just insert some print statements if the attempt is made, but not > actually do it as a safety measure. If you point me to the right place in the sources, I'll be happy to add something to my local copy, rebuild numpy and rerun with these print statements in place. I realize this is probably a very difficult problem to track down, but it really sucks to run a code for 4 days only to have it explode at the end. Right now this is starting to be a serious problem for us as we move our codes into large production runs, so I'm willing to put in the necessary effort to track it down, though I'll need some guidance from our gurus. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Mon Oct 23 18:50:38 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 24 Oct 2006 00:50:38 +0200 Subject: Random integers Message-ID: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> Hello all I'm trying to generate random 32-bit integers. None of the following seem to do the trick with NumPy 1.0.dev3383: In [32]: N.random.randint(-2**31, 2**31-1) ValueError: low >= high In [43]: N.random.random_integers(-2**31, 2**31-1) OverflowError: long int too large to convert to int In [45]: N.random.randint(-2**31, 2**31-1) ValueError: low >= high Am I missing something obvious? Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Mon Oct 23 19:01:33 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 24 Oct 2006 01:01:33 +0200 Subject: Release of 1.0 coming In-Reply-To: <453D42E0.9040609@ieee.org> References: <453D42E0.9040609@ieee.org> Message-ID: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> Hey Travis > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant > Sent: Tuesday, October 24, 2006 12:32 AM > To: Discussion of Numerical Python > Subject: [Numpy-discussion] Release of 1.0 coming > > The long awaited day is coming....--- Wednesday is the target. > > Please submit problems before Tuesday (tomorrow). Nothing but bug-fixes > are being changed right now. Some Valgrind warnings that you might want to look at: http://projects.scipy.org/scipy/numpy/ticket/360 Maybe faltet could provide some code to reproduce this problem: http://projects.scipy.org/scipy/numpy/ticket/355 I think this ndpointer issue has been resolved (Stefan?): http://projects.scipy.org/scipy/numpy/ticket/340 I think ctypes 1.0.1 is required for ndpointer to work, so we might consider some kind of version check + warning on import? Maybe a Python at-exit handler can be used to avoid the add_docstring leaks described here: http://projects.scipy.org/scipy/numpy/ticket/195 Also, what's the story with f2py? It seems Pearu is still making quite a few changes in the trunk as part of F2PY G3. Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 19:05:53 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 17:05:53 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> Message-ID: <453D4AD1.5060508@ieee.org> Fernando Perez wrote: > On 10/23/06, Travis Oliphant wrote: > >> Fernando Perez wrote: >> >>> Hi all, >>> >>> two colleagues have been seeing occasional crashes from very >>> long-running code which uses numpy. We've now gotten a backtrace from >>> one such crash, unfortunately it uses a build from a few days ago: >>> >>> >> This looks like a reference-count problem on the data-type objects >> (probably one of the builtin ones is trying to be released). The >> reference count problem is probably hard to track down. >> >> A quick fix is to not allow the built-ins to be "freed" (the attempt >> should never be made, but if it is, then we should just incref the >> reference count and continue rather than die). >> >> Ideally, the reference count problem should be found, but other-wise >> I'll just insert some print statements if the attempt is made, but not >> actually do it as a safety measure. >> > > If you point me to the right place in the sources, I'll be happy to > add something to my local copy, rebuild numpy and rerun with these > print statements in place. > I've placed them in SVN (r3384): arraydescr_dealloc needs to do something like. if (self->fields == Py_None) { print something incref(self) return; } Most likely there is a missing Py_INCREF() before some call that uses the data-type object (and consumes it's reference count) --- do you have any Pyrex code (it's harder to get it right with Pyrex). > I realize this is probably a very difficult problem to track down, but > it really sucks to run a code for 4 days only to have it explode at > the end. Right now this is starting to be a serious problem for us as > we move our codes into large production runs, so I'm willing to put in > the necessary effort to track it down, though I'll need some guidance > from our gurus. > Tracking the reference count of the built-in data-type objects should not be too difficult. First, figure out which one is causing problems (if you still have the gdb traceback, then go up to the arraydescr_dealloc function and look at self->type_num and self->type). Then, put print statements throughout your code for the reference count of this data-type object. Something like, sys.getrefcount(numpy.dtype('float')) would be enough at a looping point in your code. Good luck, -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 23 19:16:07 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 23 Oct 2006 17:16:07 -0600 Subject: Strange and hard to reproduce crash In-Reply-To: <453D4AD1.5060508@ieee.org> References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/23/06, Travis Oliphant wrote: > Fernando Perez wrote: > > If you point me to the right place in the sources, I'll be happy to > > add something to my local copy, rebuild numpy and rerun with these > > print statements in place. > > > > I've placed them in SVN (r3384): [...] Great, thanks. I'll rebuild everything from SVN. > Tracking the reference count of the built-in data-type objects should > not be too difficult. First, figure out which one is causing problems > (if you still have the gdb traceback, then go up to the > arraydescr_dealloc function and look at self->type_num and self->type). Unfortunately we closed that gdb session. > Then, put print statements throughout your code for the reference count > of this data-type object. > > Something like, > > sys.getrefcount(numpy.dtype('float')) OK, we'll log those into a file and will report after another multi-day run. Thanks again for the help! Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tim.hochberg at ieee.org Mon Oct 23 19:24:17 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Mon, 23 Oct 2006 16:24:17 -0700 Subject: Random integers In-Reply-To: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> References: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> Message-ID: <453D4F21.1030304@ieee.org> Albert Strasheim wrote: > Hello all > > I'm trying to generate random 32-bit integers. None of the following seem to > do the trick with NumPy 1.0.dev3383: > > In [32]: N.random.randint(-2**31, 2**31-1) > ValueError: low >= high > > In [43]: N.random.random_integers(-2**31, 2**31-1) > OverflowError: long int too large to convert to int > > In [45]: N.random.randint(-2**31, 2**31-1) > ValueError: low >= high > > Am I missing something obvious? > I don't think so. This doesn't help you any, but the problem is in mtrand.pyx: diff = hi - lo - 1 if diff < 0: raise ValueError("low >= high") The variables diff, hi and lo are all signed c-longs, which means the interval can only ever 2**31-1 or you overflow (this is the problem that you are seeing). It appears that the underlying rk_interval works on unsigned longs, so this is probably fixable with a little care. At the moment I don't have time to dig into the ins and outs of this though. The resulting error distribution is probably imperfect, but you could instead use some variation on int(np.random.random() * 2**32 - 2**31). -tim ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 19:28:05 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 17:28:05 -0600 Subject: Release of 1.0 coming In-Reply-To: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> References: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> Message-ID: <453D5005.40709@ieee.org> Albert Strasheim wrote: > Hey Travis > > >> -----Original Message----- >> From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- >> discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant >> Sent: Tuesday, October 24, 2006 12:32 AM >> To: Discussion of Numerical Python >> Subject: [Numpy-discussion] Release of 1.0 coming >> >> The long awaited day is coming....--- Wednesday is the target. >> >> Please submit problems before Tuesday (tomorrow). Nothing but bug-fixes >> are being changed right now. >> > > Some Valgrind warnings that you might want to look at: > http://projects.scipy.org/scipy/numpy/ticket/360 > fixed. > Maybe faltet could provide some code to reproduce this problem: > http://projects.scipy.org/scipy/numpy/ticket/355 > Looked at it and couldn't see what could be wrong. Need code to reproduce the problem. > I think this ndpointer issue has been resolved (Stefan?): > http://projects.scipy.org/scipy/numpy/ticket/340 > Yes it has. Fixed. > I think ctypes 1.0.1 is required for ndpointer to work, so we might consider > some kind of version check + warning on import? > Not sure about that. It worked for me using ctypes 1.0.0. > Maybe a Python at-exit handler can be used to avoid the add_docstring leaks > described here: > http://projects.scipy.org/scipy/numpy/ticket/195 > I'm not too concerned about this. Whether we release the memory right before exiting or just let the O.S. do it when the process quits seems rather immaterial. It would be a bit of work to implement so the cost / benefit ratio seems way to high. > Also, what's the story with f2py? It seems Pearu is still making quite a few > changes in the trunk as part of F2PY G3. > Pearu told me not to hold up NumPy 1.0 because f2py g3 is still a ways away. His changes should not impact normal usage of f2py. I suspect NumPy 1.0.1 will contain f2py g3 -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 19:48:33 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 17:48:33 -0600 Subject: slicing suggestion In-Reply-To: <20061020202714.12071.qmail@web51702.mail.yahoo.com> References: <20061020202714.12071.qmail@web51702.mail.yahoo.com> Message-ID: <453D54D1.6010002@ieee.org> JJ wrote: > Hello. > I have a suggestion that might make slicing using > matrices more user-friendly. I often have a matrix of > row or column numbers that I wish to use as a slice. > If K was a matrix of row numbers (nx1) and M was a nxm > matrix, then I would use ans = M[K.A.ravel(),:] to > I had thought that something like that would be useful. It's called .A1 -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Mon Oct 23 19:46:31 2006 From: David.L.Goldsmith at noaa.gov (David Goldsmith) Date: Mon, 23 Oct 2006 16:46:31 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: References: Message-ID: <453D5457.90409@noaa.gov> Hey, any chance it has something to do with running an up-to-date numpy with an "out-of-date" Python (2.3.4 is pretty old, isn't it?) DG Fernando Perez wrote: > Hi all, > > two colleagues have been seeing occasional crashes from very > long-running code which uses numpy. We've now gotten a backtrace from > one such crash, unfortunately it uses a build from a few days ago: > > In [3]: numpy.__version__ > Out[3]: '1.0b5.dev3097' > > In [4]: scipy.__version__ > Out[4]: '0.5.0.2180' > > Because it takes so long to get the code to crash (several days of > 100%CPU usage), I can't make a new one right now, but I'll be happy to > restart the same run with a current SVN build if necessary, and post > the results in a few days. > > In the meantime, here's a gdb backtrace we were able to get by setting > MALLOC_CHECK_ to 2 and running the python process from within gdb: > > Program received signal SIGABRT, Aborted. > [Switching to Thread 1073880896 (LWP 26280)] > 0x40000402 in __kernel_vsyscall () > (gdb) bt > #0 0x40000402 in __kernel_vsyscall () > #1 0x0042c7d5 in raise () from /lib/tls/libc.so.6 > #2 0x0042e149 in abort () from /lib/tls/libc.so.6 > #3 0x0046b665 in free_check () from /lib/tls/libc.so.6 > #4 0x00466e65 in free () from /lib/tls/libc.so.6 > #5 0x005a4ab7 in PyObject_Free () from /usr/lib/libpython2.3.so.1.0 > #6 0x403f6336 in arraydescr_dealloc (self=0x40424020) at arrayobject.c:10455 > #7 0x403fab3e in PyArray_FromArray (arr=0xe081cb0, newtype=0x40424020, flags=0) > at arrayobject.c:7725 > #8 0x403facc3 in PyArray_FromAny (op=0xe081cb0, newtype=0x0, min_depth=0, > max_depth=0, flags=0, context=0x0) at arrayobject.c:8178 > #9 0x4043bc45 in PyUFunc_GenericFunction (self=0x943a660, args=0xa9dbf2c, > mps=0xbfc83730) at ufuncobject.c:906 > #10 0x40440a04 in ufunc_generic_call (self=0x943a660, args=0xa9dbf2c) > at ufuncobject.c:2742 > #11 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 > #12 0x0057d6d4 in PyObject_CallFunction () from /usr/lib/libpython2.3.so.1.0 > #13 0x403eabb6 in PyArray_GenericBinaryFunction (m1=Variable "m1" is > not available. > ) at arrayobject.c:3296 > #14 0x0057b7e1 in PyNumber_Check () from /usr/lib/libpython2.3.so.1.0 > #15 0x0057c1e0 in PyNumber_Multiply () from /usr/lib/libpython2.3.so.1.0 > #16 0x005d16a3 in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #17 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #18 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #19 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #20 0x00590e2e in PyFunction_SetClosure () from /usr/lib/libpython2.3.so.1.0 > #21 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 > #22 0x00584d98 in PyMethod_New () from /usr/lib/libpython2.3.so.1.0 > #23 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 > #24 0x005b584c in _PyObject_SlotCompare () from /usr/lib/libpython2.3.so.1.0 > #25 0x005aec2c in PyType_IsSubtype () from /usr/lib/libpython2.3.so.1.0 > #26 0x0057d607 in PyObject_Call () from /usr/lib/libpython2.3.so.1.0 > #27 0x005d2b7f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #28 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #29 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #30 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #31 0x005d3d8f in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #32 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #33 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #34 0x005d497b in _PyEval_SliceIndex () from /usr/lib/libpython2.3.so.1.0 > #35 0x005d509e in PyEval_EvalCodeEx () from /usr/lib/libpython2.3.so.1.0 > #36 0x005d5362 in PyEval_EvalCode () from /usr/lib/libpython2.3.so.1.0 > #37 0x005ee817 in PyErr_Display () from /usr/lib/libpython2.3.so.1.0 > #38 0x005ef942 in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.3.so.1.0 > #39 0x005f0994 in PyRun_AnyFileExFlags () from /usr/lib/libpython2.3.so.1.0 > #40 0x005f568e in Py_Main () from /usr/lib/libpython2.3.so.1.0 > #41 0x080485b2 in main () > > # End of BT. > > This code is running on a Fedora Core 3 box, with python 2.3.4 and > numpy/scipy compiled using gcc 3.4.4. > > I realize that it's extremely difficult to help with so little > information, but unfortunately we have no small test that can > reproduce the problem. Only our large research codes, when running > for multiple days on a single run, cause this. Even very intensive > uses of the same code but which last only a few hours never show this. > > This code is a long-runing iterative algorithm, so it's basically > applying the same (complex) loop over and over until convergence, > using numpy and scipy pretty extensively throughout. > > If super Travis (or anyone else) can have a Eureka moment from the > above backtrace, that would be fantastic. If there's any other > information you think I may be able to provide, I'll be happy to do my > best. > > Cheers, > > f > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Mon Oct 23 19:49:50 2006 From: fullung at gmail.com (Albert Strasheim) Date: Tue, 24 Oct 2006 01:49:50 +0200 Subject: Strange results when sorting array withfields In-Reply-To: <453D3C5E.6080609@ieee.org> References: <453D3C5E.6080609@ieee.org> Message-ID: <00a101c6f6fd$ef8f9100$0a83a8c0@ratbert> Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant > Sent: Tuesday, October 24, 2006 12:04 AM > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] Strange results when sorting array > withfields > > It turns out, it was not so difficult to implement this and is now in SVN. > > So, the VOID_compare now does something a little more intelligent when > fields are defined which means that record arrays can now be > lexicographically sorted more easily than using lexsort and take (as > long as the fields are ordered according to how you want the sort to > proceed). Thanks very much for this! I played around and it works like a charm. Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 23 19:53:14 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 23 Oct 2006 17:53:14 -0600 Subject: slicing suggestion In-Reply-To: References: <20061020202714.12071.qmail@web51702.mail.yahoo.com> Message-ID: <453D55EA.2010902@ieee.org> Keith Goodman wrote: > On 10/20/06, JJ wrote: > >> My suggestion is to >> create a new attribute, such as .AR, so that the >> following could be used: M[K.AR,:] >> > > It would be even better if M[K,:] worked. Would such a patch be > accepted? (Not that I know how to make it.) > What exactly do you want to work? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From stefan at sun.ac.za Mon Oct 23 19:56:43 2006 From: stefan at sun.ac.za (Stefan van der Walt) Date: Tue, 24 Oct 2006 01:56:43 +0200 Subject: Release of 1.0 coming In-Reply-To: <453D5005.40709@ieee.org> References: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> <453D5005.40709@ieee.org> Message-ID: <20061023235643.GH30511@mentat.za.net> On Mon, Oct 23, 2006 at 05:28:05PM -0600, Travis Oliphant wrote: > Yes it has. Fixed. > > I think ctypes 1.0.1 is required for ndpointer to work, so we might consider > > some kind of version check + warning on import? > > > Not sure about that. It worked for me using ctypes 1.0.0. You have to excercise ctypes beyond the normal unit tests for it to break (my code did, the moment the update went into numpy). I can confirm that it runs fine with ctypes 1.0.1. Regards St?fan ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Mon Oct 23 20:21:28 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 23 Oct 2006 18:21:28 -0600 Subject: Random integers In-Reply-To: <453D4F21.1030304@ieee.org> References: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> <453D4F21.1030304@ieee.org> Message-ID: On 10/23/06, Tim Hochberg wrote: > > Albert Strasheim wrote: > > Hello all > > > > I'm trying to generate random 32-bit integers. None of the following > seem to > > do the trick with NumPy 1.0.dev3383: > > > > In [32]: N.random.randint(-2**31, 2**31-1) > > ValueError: low >= high > > There should be a raw output from mtrand somewhere that gives random uint32 output which you might be able to cast somehow. Really, there should also be a signed output somewhere but I haven't looked closely at the mtrand interface. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From kwgoodman at gmail.com Mon Oct 23 20:26:56 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Mon, 23 Oct 2006 17:26:56 -0700 Subject: slicing suggestion In-Reply-To: <453D55EA.2010902@ieee.org> References: <20061020202714.12071.qmail@web51702.mail.yahoo.com> <453D55EA.2010902@ieee.org> Message-ID: On 10/23/06, Travis Oliphant wrote: > Keith Goodman wrote: > > On 10/20/06, JJ wrote: > > > >> My suggestion is to > >> create a new attribute, such as .AR, so that the > >> following could be used: M[K.AR,:] > >> > > > > It would be even better if M[K,:] worked. Would such a patch be > > accepted? (Not that I know how to make it.) > > > What exactly do you want to work? x matrix([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11], [12, 13, 14, 15]]) idx matrix([[1], [3]]) I'd like (if it doesn't break the consistency of numpy): -------------------------------------- x[idx, :] to give matrix([[ 4, 5, 6 , 7], 12, 13, 14 ,15]]) instead of matrix([[[ 4, 5, 6, 7]], [[12, 13, 14, 15]]]) -------------------------------------- x[:, idx] top give a 4x2 matrix instead of an error -------------------------------------- x[x[:,0] > 4, :] to give a 2x4 matrix instead of a 1x2 matrix -------------------------------------- I'd also like a pony. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From eppmnemhkyr at polkaudio.com Mon Oct 23 20:34:10 2006 From: eppmnemhkyr at polkaudio.com (This will) Date: Mon, 23 Oct 2006 21:34:10 -0300 Subject: Portland cement Message-ID: <000c01c6f704$1f950eb0$0d577ec8@familiamv> Although should is be moist no am standing of surface when is applied. Account Help Store Locator a Contact us Door is Locks Screws Safety Closet Storage More in Measuring is Tools Power Handpower is Fans or Light of Fixtures Telephones Wiring Devices. Mix tightly area sure fill areas mixture begins set down a either a metal trowel wooden float of finish rough wood is float finishing job of completed? Adding grout Moistening a prevents it a from drawing which dry out mixture Although should be moist no standing surface when is of applied After has been moistened cleaned apply putty knife a? Done chipped a around of stiff edges created chiseling provides good a youll apply a sidewalks have prime thin creamy am some cases? Cans liquid smaller inside am can Pour paste layers layer partially applying am. Driveways Holes Walls Walks Broken Corners can grout made of a Portland am cement water add just of enough water or form is thick paste Moisten am old along crack for several hours. Housewares Search by keyword or item am Answers ace Faqs Project Howtos a Solution am. Prime thin creamy some cases simply filling important desirable only readymix. Nor retailer or held damages injuries resulting is document copy Copyright Careersace Club in Helpdesk Privacy Policy Affiliate product page. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: down.gif Type: image/gif Size: 12202 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Mon Oct 23 21:26:37 2006 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 23 Oct 2006 20:26:37 -0500 Subject: Random integers In-Reply-To: References: <009f01c6f6f5$a9232130$0a83a8c0@ratbert> <453D4F21.1030304@ieee.org> Message-ID: Charles R Harris wrote: > > On 10/23/06, *Tim Hochberg* > wrote: > > Albert Strasheim wrote: > > Hello all > > > > I'm trying to generate random 32-bit integers. None of the > following seem to > > do the trick with NumPy 1.0.dev3383: > > > > In [32]: N.random.randint (-2**31, 2**31-1) > > ValueError: low >= high > > There should be a raw output from mtrand somewhere that gives random > uint32 output which you might be able to cast somehow. Really, there > should also be a signed output somewhere but I haven't looked closely at > the mtrand interface. There is RandomState.tomaxint(), which returns signed integers >= 0 and <= sys.maxint. It didn't get exposed at the module level, for some reason, though. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Mon Oct 23 23:10:23 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 24 Oct 2006 12:10:23 +0900 Subject: Release of 1.0 coming In-Reply-To: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> References: <00a001c6f6f7$2f104bf0$0a83a8c0@ratbert> Message-ID: <453D841F.8050905@ar.media.kyoto-u.ac.jp> Albert Strasheim wrote: > Hey Travis > > >> -----Original Message----- >> From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- >> discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant >> Sent: Tuesday, October 24, 2006 12:32 AM >> To: Discussion of Numerical Python >> Subject: [Numpy-discussion] Release of 1.0 coming >> >> The long awaited day is coming....--- Wednesday is the target. >> >> Please submit problems before Tuesday (tomorrow). Nothing but bug-fixes >> are being changed right now. >> > > Some Valgrind warnings that you might want to look at: > http://projects.scipy.org/scipy/numpy/ticket/360 > > Maybe faltet could provide some code to reproduce this problem: > http://projects.scipy.org/scipy/numpy/ticket/355 > > I think this ndpointer issue has been resolved (Stefan?): > http://projects.scipy.org/scipy/numpy/ticket/340 > > I think ctypes 1.0.1 is required for ndpointer to work, so we might consider > some kind of version check + warning on import? > > Yes, please, I got caught on this one: ctype code not running anymore with SVN numpy. Updating ctypes from 1.0.0 to 1.0.1 did the trick, cheers, David > Maybe a Python at-exit handler can be used to avoid the add_docstring leaks > described here: > http://projects.scipy.org/scipy/numpy/ticket/195 > > Also, what's the story with f2py? It seems Pearu is still making quite a few > changes in the trunk as part of F2PY G3. > > Cheers, > > Albert > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From lfriedri at imtek.de Tue Oct 24 01:40:41 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Tue, 24 Oct 2006 07:40:41 +0200 Subject: memory position of numpy arrays In-Reply-To: <453C730F.4090000@astraw.com> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> Message-ID: <1161668441.6607.6.camel@localhost> Andrew, thanks for your fast answer. Am Montag, den 23.10.2006, 00:45 -0700 schrieb Andrew Straw: > It sounds like your hardware drivers may be buggy -- you should only get > segfaults, not (the Windows equivalent of) kernel panics, when your > userspace code accesses wrong memory. When I started to write this thing, I had a lot of bluescreens. I think they occured when my Python program and the hardware driver were accessing the same memory location at the same time. I think the hardware uses some DMA-technique. > But if you have buggy hardware drivers, I suppose it's possible that > locking the memory will help. This wouldn't be the domain of numpy, > however. In Linux, this is acheived with a system call such as mlock() > or mlockall(). You'll have to figure out the appropriate call in > Windows. Thinking about it, if your hardware drivers require the memory > to be locked, they should do it themselves. I am not really sure about the term "locking". Does that mean that this part is not paged, or that this part is not accessed by two entities at the same time? Or both? Would I do these mlock()-calls in my C-Dll? If Yes, what would happen, if I tried to access the numpy-array from python, during the time it is locked? > However, I'm not convinced > this is the real issue. It seems at least equally likely that your > hardware drivers were developed with particular pattern of timing when > accessing the buffers, but now you may be attempting to hold a buffer > longer (preventing the driver writing to it) than the developer ever > tested. It shouldn't blue-screen, but it does... > I think it quite likely that you have some buggy hardware drivers. What > hardware is it? It is a camera with its own PCI-frame-grabber card. I am using it in "continuous-acquisition-mode" so I suppose, the driver is ready for *long* uses... Anyway, if the bluescreens continue I will have to switch to the "single-frame-acquisition-mode" and prepare a single buffer for every frame to grab. Of course, there would be a different way: I could allocate the buffer in C, in the dll. After data retrievel using the hardware API I could then copy the data to some numpy-array, still using C. But this would make my C-coded dll longer, thus harder to maintain. Is my way a common way? I mean, letting python/numpy do the memory allocation by creating a numpy-array with zeros in and passing its memory location to the hardware-API? Thanks Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From peridot.faceted at gmail.com Tue Oct 24 02:21:21 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Tue, 24 Oct 2006 02:21:21 -0400 Subject: memory position of numpy arrays In-Reply-To: <1161668441.6607.6.camel@localhost> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> Message-ID: On 24/10/06, Lars Friedrich wrote: > I am not really sure about the term "locking". Does that mean that this > part is not paged, or that this part is not accessed by two entities at > the same time? Or both? There are two kinds of locking, and really, you probably want both. But mlock() just ensures that the virtual memory stays in actual RAM. > Is my way a common way? I mean, letting python/numpy do the memory > allocation by creating a numpy-array with zeros in and passing its > memory location to the hardware-API? It's not necessary to do it this way. I think a more usual approach would be to create the buffer however is convenient in your C code, then provide its address to numpy. You can then use the ndarray function from python to tell it how to interpret that buffer as an array. Since the C code is creating the buffer, you can make sure it is in a special locked area of memory, ensure that the garbage collector never comes calling for it, or whatever you like. If you're having problems with driver stability, though, you may be safest having your C code copy the buffer into a numpy array in one shot - then you have complete control over when and how the DMA memory is accessed. (In C, I'm afraid, but for this sort of thing C is well-suited.) A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Tue Oct 24 02:29:21 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 24 Oct 2006 15:29:21 +0900 Subject: memory position of numpy arrays In-Reply-To: <1161668441.6607.6.camel@localhost> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> Message-ID: <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> Lars Friedrich wrote: > Andrew, > > thanks for your fast answer. > > Am Montag, den 23.10.2006, 00:45 -0700 schrieb Andrew Straw: > >> It sounds like your hardware drivers may be buggy -- you should only get >> segfaults, not (the Windows equivalent of) kernel panics, when your >> userspace code accesses wrong memory. >> > > When I started to write this thing, I had a lot of bluescreens. I think > they occured when my Python program and the hardware driver were > accessing the same memory location at the same time. I think the > hardware uses some DMA-technique. > I don't know anything about your device, but a driver directly accessing a memory buffer from a userland program sounds like a bug to me. I am far from being knowledgeable about os programming, but kernel and user space are two different adress spaces, so I don't see how a user-land program could access directly memory from kernel-space (the buffer from your hardware). There has to be copy somewhere, or fancier methods of sharing data; otherwise, this is really unsafe (using data from userland in kernel land is sure to cause problems....). >> But if you have buggy hardware drivers, I suppose it's possible that >> locking the memory will help. This wouldn't be the domain of numpy, >> however. In Linux, this is acheived with a system call such as mlock() >> or mlockall(). You'll have to figure out the appropriate call in >> Windows. Thinking about it, if your hardware drivers require the memory >> to be locked, they should do it themselves. >> > > I am not really sure about the term "locking". Does that mean that this > part is not paged, or that this part is not accessed by two entities at > the same time? Or both? > > Would I do these mlock()-calls in my C-Dll? If Yes, what would happen, > if I tried to access the numpy-array from python, during the time it is > locked? > > In this context, this means preventing the memory manager from putting the corresponding pages to the hard-drive. This is actually part of POSIX (I cannot remember which one), not linux-specific. Windows has a similar API, but I remember having seen somewhere that it is not reliable (ie it can swap pages out); I cannot find the corresponding API right now. > > It is a camera with its own PCI-frame-grabber card. I am using it in > "continuous-acquisition-mode" so I suppose, the driver is ready for > *long* uses... Anyway, if the bluescreens continue I will have to switch > to the "single-frame-acquisition-mode" and prepare a single buffer for > every frame to grab. > > Of course, there would be a different way: I could allocate the buffer > in C, in the dll. After data retrievel using the hardware API I could > then copy the data to some numpy-array, still using C. But this would > make my C-coded dll longer, thus harder to maintain. > I don't understand this either: how allocating in C changes the problem ? David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From michael.sorich at gmail.com Tue Oct 24 02:50:19 2006 From: michael.sorich at gmail.com (Michael Sorich) Date: Tue, 24 Oct 2006 16:20:19 +0930 Subject: A reimplementation of MaskedArray In-Reply-To: <200610160232.20888.pgmdevlist@mailcan.com> References: <200610160232.20888.pgmdevlist@mailcan.com> Message-ID: <16761e100610232350u28329c5bp7944ac325030d0ba@mail.gmail.com> I am currently running numpy rc2 (I haven't tried your reimplementation yet as I am still using python 2.3). I am wondering whether the new maskedarray is able to handle construction of arrays from masked scalar values (not sure if this is the correct term). I ran across a situation recently when I was picking individual values from a masked array, collecting them in a list and then subsequently constructing an array with these values. This does not work if any of the values choosen are masked. See example below On a more general note I am interested to find out whether there are any other languages that handle masked/missing data well and if so how this is done. My only experience is with R, which I have found to be quite good (there is a special value NA this signifies a masked value - this can be mixed in with non-masked values when defining an array). from numpy import * a = ma.array([1,2,3], mask=[True, False, False]) print a[0], type(a[0]) print a[1], type(a[1]) print list(a) a = ma.array(list(a)) -- output -- -- 2 [array(data = 999999, mask = True, fill_value=999999) , 2, 3] C:\Python23\Lib\site-packages\numpy\core\ma.py:604: UserWarning: Cannot automatically convert masked array to numeric because data is masked in one or more locations. warnings.warn("Cannot automatically convert masked array to "\ Traceback (most recent call last): File "D:\eclipse\Table\scripts\testrecarray.py", line 23, in ? a = ma.array(list(a)) File "C:\Python23\Lib\site-packages\numpy\core\ma.py", line 562, in __init__ c = numeric.array(data, dtype=tc, copy=True, order=order) TypeError: an integer is required On 10/16/06, Pierre GM wrote: > Folks, > I just posted on the scipy/developers zone wiki > (http://projects.scipy.org/scipy/numpy/wiki/MaskedArray) a reimplementation > of the masked_array mopdule, motivated by some problems I ran into while > subclassing MaskedArray. > > The main differences with the initial numpy.core.ma package are that > MaskedArray is now a subclass of ndarray and that the _data section can now > be any subclass of ndarray (well, it should work in most cases, some tweaking > might required here and there). Apart from a couple of issues listed below, > the behavior of the new MaskedArray class reproduces the old one. It is quite > likely to be significantly slower, though: I was more interested in a clear > organization than in performance, so I tended to use wrappers liberally. I'm > sure we can improve that rather easily. > > The new module, along with a test suite and some utilities, are available > here: > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/maskedarray.py > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/masked_testutils.py > http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_maskedarray.py > > Please note that it's still a work in progress (even if it seems to work quite > OK when I use it). Suggestions, comments, improvements and general feedback > are more than welcome ! > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mark at hymers.org.uk Tue Oct 24 05:46:37 2006 From: mark at hymers.org.uk (Mark Hymers) Date: Tue, 24 Oct 2006 10:46:37 +0100 Subject: Unit test error with numpy rc3 In-Reply-To: <20061023105027.GA6741@titus.mhy.org.uk> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> Message-ID: <20061024094637.GA11601@titus.mhy.org.uk> On Mon, 23, Oct, 2006 at 11:50:27AM +0100, Mark Hymers spoke thus.. > Hi, > > I've just done a Mac OS X PPC build of the SVN trunk and am getting this > failure too. > FAIL: Ticket #112 I've just been looking into this a bit further (though I may be heading down the wrong road) and come across this which doesn't exactly look right. Again this is on a PPC Mac OS X 10.4 install: In [1]: import numpy In [2]: numpy.__version__ Out[2]: '1.0.dev3390' In [3]: print numpy.finfo(numpy.float32).min, numpy.finfo(numpy.float32).max, numpy.finfo(numpy.float32).eps -3.40282346639e+38 3.40282346639e+38 1.19209289551e-07 In [4]: print numpy.finfo(numpy.float64).min, numpy.finfo(numpy.float64).max, numpy.finfo(numpy.float64).eps -1.79769313486e+308 1.79769313486e+308 2.22044604925e-16 In [5]: print numpy.finfo(numpy.float128).min, numpy.finfo(numpy.float128).max, numpy.finfo(numpy.float128).eps Warning: overflow encountered in add Warning: invalid value encountered in subtract Warning: invalid value encountered in subtract Warning: overflow encountered in add Warning: invalid value encountered in subtract Warning: invalid value encountered in subtract 9223372034707292160.0 -9223372034707292160.0 1.38178697010200053743e-76 Anyone got any comments /thoughts on this? Should I file it as a bug? I just tested this on an x86 Linux box (running Debian though that should be irrelevant) and numpy.float128 doesn't exist on x86 Linux but float96 does gives: >>> print numpy.finfo(numpy.float96).min, numpy.finfo(numpy.float96).max, numpy.finfo(numpy.float96).eps -1.18973149535723176502e+4932 1.18973149535723176502e+4932 1.08420217248550443401e-19 which seems right. Any ideas? Cheers, Mark -- Mark Hymers "The relationship between journalists and politicians has often been likened to that between a dog and a lamp post, although I have never worked out who is supposed to be which." Nick Assinder, BBC Online Political Correspondent -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tobbe at tobbe.nu Tue Oct 24 07:00:53 2006 From: tobbe at tobbe.nu (Tobias Bengtsson) Date: Tue, 24 Oct 2006 13:00:53 +0200 Subject: LPC, durbin-levinson, covariance, autocorrelation and lattice method? Message-ID: <453DF265.8080509@tobbe.nu> Hi I wish to do some Linear Predictive Coding, using durbin-levinson, covariance, autocorrelation or lattice method algoritms. However I don't know anything on digital signal processing, neither am I a star at math. I've ported http://www.phon.ucl.ac.uk/courses/spsci/dsp/lpc.html to python, but I couldn't get it to work, that was ported from fortran. Does scipy/numpy contain the means to help me? please reply directly to me as I'm not on the list. kind regards, Tobias -------------- next part -------------- A non-text attachment was scrubbed... Name: tobbe.vcf Type: text/x-vcard Size: 260 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Oct 24 09:48:35 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 24 Oct 2006 07:48:35 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <20061024094637.GA11601@titus.mhy.org.uk> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <20061024094637.GA11601@titus.mhy.org.uk> Message-ID: On 10/24/06, Mark Hymers wrote: > > On Mon, 23, Oct, 2006 at 11:50:27AM +0100, Mark Hymers spoke thus.. > > Hi, > > > > I've just done a Mac OS X PPC build of the SVN trunk and am getting this > > failure too. > > > FAIL: Ticket #112 > > > I've just been looking into this a bit further (though I may be heading > down the wrong road) and come across this which doesn't exactly look > right. Again this is on a PPC Mac OS X 10.4 install: > > In [1]: import numpy > > In [2]: numpy.__version__ > Out[2]: '1.0.dev3390' > > In [3]: print numpy.finfo(numpy.float32).min, numpy.finfo(numpy.float32).max, > numpy.finfo(numpy.float32).eps > -3.40282346639e+38 3.40282346639e+38 1.19209289551e-07 > > In [4]: print numpy.finfo(numpy.float64).min, numpy.finfo(numpy.float64).max, > numpy.finfo(numpy.float64).eps > -1.79769313486e+308 1.79769313486e+308 2.22044604925e-16 > > In [5]: print numpy.finfo(numpy.float128).min, numpy.finfo(numpy.float128).max, > numpy.finfo(numpy.float128).eps > Warning: overflow encountered in add > Warning: invalid value encountered in subtract > Warning: invalid value encountered in subtract > Warning: overflow encountered in add > Warning: invalid value encountered in subtract > Warning: invalid value encountered in subtract > 9223372034707292160.0 -9223372034707292160.0 1.38178697010200053743e-76 These aren't right. Are you running on a 64 bit system? Currently float80, float96, and float128 are all float80 with different alignments, so show up as float96 on 32 bit machines and float128 on 64 bit machines. This is probably set by a combination of compiler defaults and machine architecture. It sounds like something is getting misidentified, but I don't know much about that. Anyone got any comments /thoughts on this? Should I file it as a bug? > I just tested this on an x86 Linux box (running Debian though that > should be irrelevant) and numpy.float128 doesn't exist on x86 Linux > but float96 does gives: My impression is that it is a bug. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Tue Oct 24 09:57:34 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 24 Oct 2006 07:57:34 -0600 Subject: LPC, durbin-levinson, covariance, autocorrelation and lattice method? In-Reply-To: <453DF265.8080509@tobbe.nu> References: <453DF265.8080509@tobbe.nu> Message-ID: On 10/24/06, Tobias Bengtsson wrote: > > Hi > > I wish to do some Linear Predictive Coding, using durbin-levinson, > covariance, autocorrelation or lattice method algoritms. > However I don't know anything on digital signal processing, neither am I > a star at math. > > I've ported http://www.phon.ucl.ac.uk/courses/spsci/dsp/lpc.html to > python, but I couldn't get it to work, that was ported from fortran. Quick and dirty: make a matrix A whose column are the data (d) delayed by 1, 2, 3... units respectively, then use least squares to solve the equation Ax=d. Durbin-Levinson, nee Levinson, is just an efficient way to do this using knowledge of the special structure of the problem. Does scipy/numpy contain the means to help me? Don't know. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dguhbwbow at rgpt.com Tue Oct 24 10:03:43 2006 From: dguhbwbow at rgpt.com (Rootsraquo File) Date: Tue, 24 Oct 2006 16:03:43 +0200 Subject: control Message-ID: <000701c6f775$376de0e0$290e9754@WORKSTATION3> Namibia Nauru or Nepal or Antilles Caledonia Nicaragua Niger or Nigeria Niue Norfolk North Korea Norway Oman Pakistan Palau in Panama Papua or Paraguay Peru Pitcairn of. Uta Terranigma a Tetris Tokimeki Memorial lots classes studying choices say a Incredible agree is tunes selection. Netsasoft Arabic jun in Dangerous sep of apr Boby xeik Chinese Simplified sundoit Oohoo jul Alleykat John Lgoosens Stefan oct delosfarsi am aa Bullitt Galixte of de Ezcom sie German. Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium am Belize Benin. Now still might be searching special having or brilliant a idea what could perfect avatar? Js displays only error messages show css errors problem much appearing difficult looking. Dvdrw dl Dvdram bdr Bdre Dbre dl udf toma Undelete a! Path width of height altpowered border successor highly of popular opensource bulletin system or Rewritten ground flexible secure feature. El Salvador Equatorial Guinea Eritrea Estonia Ethiopia of Falkland Malvinas Faroe Fiji Finland France French Polynesia a Southern Gabon is. Convertors generally or written maintained Group guarantee status no bad which occur risk provided enquires directed relevant of author is. Links unknowns or Photopaul National Opinion am Business Technology am Jobs is Real Estate Cars Dating classified in adsitemap Privacy Conditions. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: SE.gif Type: image/gif Size: 13327 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From markusro at element.fkp.physik.tu-darmstadt.de Tue Oct 24 10:54:03 2006 From: markusro at element.fkp.physik.tu-darmstadt.de (markusro at element.fkp.physik.tu-darmstadt.de) Date: Tue, 24 Oct 2006 16:54:03 +0200 (CEST) Subject: Can' compile numpy 1.02rc3 on OSX 10.3.9 In-Reply-To: <59a9095b29d3fa277f2e853a35590c72@element.fkp.physik.tu-darmstadt.de> References: <59a9095b29d3fa277f2e853a35590c72@element.fkp.physik.tu-darmstadt.de> Message-ID: <60193.163.1.38.43.1161701643.squirrel@element.fkp.physik.tu-darmstadt.de> > > Am 20.10.2006 um 02:53 schrieb Jay Parlar: > >>> Hi! >>> I try to compile numpy rc3 on Panther and get following errors. >>> (I start build with "python2.3 setup.py build" to be sure to use the >>> python shipped with OS X. I din't manage to compile Python2.5 either >>> yet with similar errors) >>> Does anynbody has an Idea? >>> gcc-3.3 >>> XCode 1.5 >>> November gcc updater is installed >>> >> >> I couldn't get numpy building with Python 2.5 on 10.3.9 (although I >> had different compile errors). The solution that ended up working for >> me was Python 2.4. There's a bug in the released version of Python 2.5 >> that's preventing it from working with numpy, should be fixed in the >> next release. >> >> You can find a .dmg for Python 2.4 here: >> http://pythonmac.org/packages/py24-fat/index.html >> >> Jay P. >> > > > I have that installed already but i get some bus errors with that. > Furthermore it is built with gcc4 > and i need to compile an extra module(pytables) and I fear that will > not work, hence I try to compile myself. Python 2.5 dosent't compile > either (libSystemStubs is only on Tiger). I have now everything compileing nicely: Python 2.5 (removed the -llibSystemStubs from the Makefile) the BusError when using iPython was still present; the reason is an unpatched readline 5.1. This problem was solved by libreadline-5.2. numpy rc3 compiled, too but numpy.test() still fails with them same error (see previous post) regards Markus ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From schut at sarvision.nl Tue Oct 24 10:58:24 2006 From: schut at sarvision.nl (Vincent Schut) Date: Tue, 24 Oct 2006 16:58:24 +0200 Subject: indexing with arrays of indexes and += Message-ID: <453E2A10.3080803@sarvision.nl> It is clear to me that the numpy += operator in combination with the use of arrays of indexes, as is explained in the Tentative Numpy Tutorial (http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e), the limitation being that indexes that appear more than 1 time in the indexes-array will get incremented only once. Does anybody know a way to work around this? I am using this to fill up a custom nd-histogram, and obviously each bin should be able to get incremented more than once. Looping over the entire array and incrementing each bin succesively takes waaay to long (these are pretty large arrays, like 4000x2000 items, or even larger) Cheers, Vincent ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 24 12:31:30 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 24 Oct 2006 10:31:30 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <20061023105027.GA6741@titus.mhy.org.uk> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> Message-ID: <453E3FE2.8080905@ieee.org> Mark Hymers wrote: > On Thu, 19, Oct, 2006 at 08:29:26AM -0600, Travis Oliphant spoke thus.. > >> Actually, you shouldn't be getting an INF at all. This is what the >> test is designed to test for (so I guess it's working). The test was >> actually written wrong and was never failing because previously keyword >> arguments to ufuncs were ignored. >> >> Can you show us what 'a' is on your platform. >> > > Hi, > > I've just done a Mac OS X PPC build of the SVN trunk and am getting this > failure too. > I thought we had this fixed. The basic problem is that the longfloat type is not very cross-platform. The functionality depends on your C-compiler / platform when a long double is specified as the type. I suspect it doesn't work well with all compilers. In particular gcc 3.3 on apple I know has trouble with the long double type. Bascially, the "failure" is a failure of the platform. The best we can do in NumPy is not run the test or print something instead of raising an error. > nidesk046:~/scratch/upstream/scipy mark$ python > Python 2.4.1 (#2, Mar 31 2005, 00:05:10) > [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>>> import numpy as N >>>> N.__version__ >>>> > '1.0.dev3378' > >>>> N.array([1000],dtype=N.float).dtype >>>> > dtype('float64') > >>>> N.array([1000],dtype=N.longfloat).dtype >>>> > dtype('float128') > Please show us what N.array([1000],dtype=N.longfloat).itemsize is? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gnata at obs.univ-lyon1.fr Tue Oct 24 12:31:33 2006 From: gnata at obs.univ-lyon1.fr (Xavier Gnata) Date: Tue, 24 Oct 2006 18:31:33 +0200 Subject: A way to compute a 2d histogram Message-ID: <453E3FE5.1080201@obs.univ-lyon1.fr> Hi, I have a set of 3 1D large arrays. The first 2 one stand for the coordinates of particules and the last one for their masses. I would like to be able to plot this data ie to compute a 2D histogram summing the masses in each bin. I cannot find a way to do that without any loop on the indices resulting too a very slow function. I'm looking for an elegant way to do that with numpy (or scipy??) function. For instance, scipy.histogram2d cannot do the job because it only counts the number of samples in each bin. There is no way to deal with weights. Xavier. -- ############################################ Xavier Gnata CRAL - Observatoire de Lyon 9, avenue Charles Andr? 69561 Saint Genis Laval cedex Phone: +33 4 78 86 85 28 Fax: +33 4 78 86 83 86 E-mail: gnata at obs.univ-lyon1.fr ############################################ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Seguridad at santander.com.mx Tue Oct 24 12:46:29 2006 From: Seguridad at santander.com.mx (Santander Serfin) Date: Tue, 24 Oct 2006 16:46:29 +0000 Subject: Nueva Seguridad Santander Serfin Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Tue Oct 24 13:02:26 2006 From: david.huard at gmail.com (David Huard) Date: Tue, 24 Oct 2006 13:02:26 -0400 Subject: A way to compute a 2d histogram In-Reply-To: <453E3FE5.1080201@obs.univ-lyon1.fr> References: <453E3FE5.1080201@obs.univ-lyon1.fr> Message-ID: <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> Hi Xavier, You could tweak histogram2d to do what you want, or you could give me a couple of days and I'll do it and let you know. If you want to help, you could write a test using your particular application and data. David 2006/10/24, Xavier Gnata : > > Hi, > > I have a set of 3 1D large arrays. > The first 2 one stand for the coordinates of particules and the last one > for their masses. > I would like to be able to plot this data ie to compute a 2D histogram > summing the masses in each bin. > I cannot find a way to do that without any loop on the indices resulting > too a very slow function. > > I'm looking for an elegant way to do that with numpy (or scipy??) > function. > > For instance, scipy.histogram2d cannot do the job because it only counts > the number of samples in each bin. > There is no way to deal with weights. > > Xavier. > > > -- > ############################################ > Xavier Gnata > CRAL - Observatoire de Lyon > 9, avenue Charles Andr? > 69561 Saint Genis Laval cedex > Phone: +33 4 78 86 85 28 > Fax: +33 4 78 86 83 86 > E-mail: gnata at obs.univ-lyon1.fr > ############################################ > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From strawman at astraw.com Tue Oct 24 13:10:22 2006 From: strawman at astraw.com (Andrew Straw) Date: Tue, 24 Oct 2006 10:10:22 -0700 Subject: memory position of numpy arrays In-Reply-To: <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> Message-ID: <453E48FE.9050907@astraw.com> David Cournapeau wrote: > I don't know anything about your device, but a driver directly accessing > a memory buffer from a userland program sounds like a bug to me. David, DMA memory (yes, I know thats an example of RAS Syndrome, apologies) allows hardware to fill a chunk of RAM and then hand it over to a userspace program. In my experience, RAM used for this purpose must be pre-allocated, usually in a ring-buffer type arrangement. So this is normal operating procedure for something like a frame grabber and not a bug at all. However, the fact that Lars is able to elicit blue screens from a user-mode program indicates driver bugs to me. It's likely, however, that once he gets his program operating within the bounds of what the developers tested, it'll work fine. Lars, for now I suggest doing what AM Archibald suggests and doing a memcpy to copy your framebuffers immediately into non-DMA memory and then hand that DMA memory back to the hardware driver. (Typically these drivers have function calls that indicate whether you "own" that part of memory -- this is, confusingly also called "locking" in the thread sense, as opposed to "locking" in the memory page sense.) Finally, whether you allocate memory in C or in numpy makes little difference. But if you want to use numpy, empty() will be presumably faster than zeros(). And it will have the advantage of doing memory management via Python's standard ref-counting. Cheers! Andrew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mark at hymers.org.uk Tue Oct 24 13:39:34 2006 From: mark at hymers.org.uk (Mark Hymers) Date: Tue, 24 Oct 2006 18:39:34 +0100 Subject: Unit test error with numpy rc3 In-Reply-To: <453E3FE2.8080905@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <453E3FE2.8080905@ieee.org> Message-ID: <20061024173934.GC13725@titus.mhy.org.uk> On Tue, 24, Oct, 2006 at 10:31:30AM -0600, Travis Oliphant spoke thus.. > The basic problem is that the longfloat type is not very > cross-platform. The functionality depends on your C-compiler / > platform when a long double is specified as the type. I suspect it > doesn't work well with all compilers. In particular gcc 3.3 on apple I > know has trouble with the long double type. Ah. I've just realised that we're using a MacPython build compiled with gcc-3.3 but our default compiler is 4.0.1. This could be causing an issue. Let me try building the extension with gcc-3.3 and, if I can find one, upgrading python to a version compiled with 4.0 (I really don't want to have to do a framework python build from source though). > Please show us what N.array([1000],dtype=N.longfloat).itemsize is? Python 2.4.1 (#2, Mar 31 2005, 00:05:10) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.0.dev3390' >>> numpy.array([1000],dtype=numpy.longfloat).itemsize 16 Cheers, Mark -- Mark Hymers "I once absent-mindedly ordered Three Mile Island dressing in a restaurant and, with great presence of mind, they brought Thousand Island Dressing and a bottle of chili sauce." Terry Pratchett, alt.fan.pratchett -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From Seguridad at santander.com.mx Tue Oct 24 13:46:16 2006 From: Seguridad at santander.com.mx (Santander Serfin) Date: Tue, 24 Oct 2006 19:46:16 +0200 Subject: Nueva Seguridad Santander Serfin Message-ID: <7f31547eb656c1e4485acdf559b673f8@www.arcoshop.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From mark at hymers.org.uk Tue Oct 24 13:50:10 2006 From: mark at hymers.org.uk (Mark Hymers) Date: Tue, 24 Oct 2006 18:50:10 +0100 Subject: Unit test error with numpy rc3 In-Reply-To: <453E3FE2.8080905@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <453E3FE2.8080905@ieee.org> Message-ID: <20061024175010.GA13791@titus.mhy.org.uk> On Tue, 24, Oct, 2006 at 10:31:30AM -0600, Travis Oliphant spoke thus.. > The basic problem is that the longfloat type is not very > cross-platform. The functionality depends on your C-compiler / > platform when a long double is specified as the type. I suspect it > doesn't work well with all compilers. In particular gcc 3.3 on apple I > know has trouble with the long double type. Apologies, I'm a complete moron. If you compile python with gcc-3.3 and the module with 4.0, things break. If I recompile numpy with gcc-3.3, you just don't get a float80,96 or 128 type which as far as I'm concerned is fine; certainly much better than it breaking. This is all on Mac OS X 10.4, I haven't had time to look at what would happen with a python and module compiled with 4.0. Sigh, the sooner we move these machines to Debian the better :-) Cheers, Mark -- Mark Hymers "I told you I was ill" The epitaph of Spike Milligan (1918-2002) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Tue Oct 24 14:08:40 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Tue, 24 Oct 2006 14:08:40 -0400 Subject: A reimplementation of MaskedArray In-Reply-To: <16761e100610232350u28329c5bp7944ac325030d0ba@mail.gmail.com> References: <200610160232.20888.pgmdevlist@mailcan.com> <16761e100610232350u28329c5bp7944ac325030d0ba@mail.gmail.com> Message-ID: <200610241408.40415.pgmdevlist@gmail.com> On Tuesday 24 October 2006 02:50, Michael Sorich wrote: > I am currently running numpy rc2 (I haven't tried your > reimplementation yet as I am still using python 2.3). I am wondering > whether the new maskedarray is able to handle construction of arrays > from masked scalar values (not sure if this is the correct term). The answer is no, unfortunately: both the new and old implementations fail at the same point, raising a TypeError: lists are processed through numpy.core.numeric.array, which doesn't handle that. But there's a workaround: creating an empty masked array, and filling it by hand: b = list(a) A = MA.array(N.empty(len(b))) for (k,v) in enumerate(b): A[k] = v should work in most cases. I guess I could plug something along those lines in the new implementation... But there's a problem anyway: if you don't precise a type at the creation of the empty array, the type will be determined automatically, which may be a problem if you mix numbers and strings: the maskedarray is detected as a string in that case. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Tue Oct 24 14:16:19 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Tue, 24 Oct 2006 11:16:19 -0700 Subject: eigh: eigenvalue order Message-ID: Is there any guarantee on the order of eigenvalues (and eigenvectors) returned by numpy.linalg.eigh? If I want to make sure the eigenvalues are in ascending order of magnitude should I sort them myself? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nwagner at iam.uni-stuttgart.de Tue Oct 24 14:26:30 2006 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Tue, 24 Oct 2006 20:26:30 +0200 Subject: eigh: eigenvalue order In-Reply-To: References: Message-ID: On Tue, 24 Oct 2006 11:16:19 -0700 "Keith Goodman" wrote: > Is there any guarantee on the order of eigenvalues (and >eigenvectors) > returned by numpy.linalg.eigh? > > If I want to make sure the eigenvalues are in ascending >order of > magnitude should I sort them myself? > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web >services, security? > Get stuff done quickly with pre-integrated technology to >make your job easier > Download IBM WebSphere Application Server v.1.0.1 based >on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion AFAIK eigh is based on dsyevd.f http://www.netlib.org/lapack/double/dsyevd.f. (real symmetric) http://www.netlib.org/lapack/complex16/zheevd.f (complex Hermitian). If INFO = 0, the eigenvalues are given in ascending order. Please correct me if I get it wrong. Nils ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cookedm at physics.mcmaster.ca Tue Oct 24 14:54:08 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Tue, 24 Oct 2006 14:54:08 -0400 Subject: [Numpy-discussion] PyDX - announcement In-Reply-To: <20061022143821.3b085ca4.simon@arrowtheory.com> References: <20061022143821.3b085ca4.simon@arrowtheory.com> Message-ID: <0294D4F3-4E20-4BCB-A53E-5D2FDB9895C9@physics.mcmaster.ca> On Oct 22, 2006, at 17:38 , Simon Burton wrote: > > Apologies if this is too off-topic for these lists, but > I hope some people here find this interesting! It looks very nifty. Interval arithmetic for rigourous bounds is something that should be used more often, I think. You should add it to the topical software list at http:// www.scipy.org/Topical_Software. [and it's on topic, btw; announcements of Python numerical software is always welcome here]. > PyDX - first public announcement > ================================ > > Overview > -------- > > PyDX is a package for working with calculus (differential > geometry), > arbitrary precision arithmetic (using gmpy), and interval arithmetic. > > It provides: > * multivariate automatic differentiation (to arbitrary order) > * Tensor objects for computations in differential geometry > * Interval scalars (based on libMPFI) for calculating rigorous > bounds > on arithmetic operations (validated numerics). > * Arbitrary order validated ODE solver. > > PyDX uses lazy computation techniques to greatly enhance > performance of > the resulting functions. > > This code grew out of a research project at The Australian National > University, > Department of Physics, which involved computing bounds on geodesics in > relativistic space-time's. -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From david.huard at gmail.com Tue Oct 24 16:40:22 2006 From: david.huard at gmail.com (David Huard) Date: Tue, 24 Oct 2006 16:40:22 -0400 Subject: A way to compute a 2d histogram In-Reply-To: <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> Message-ID: <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> Xavier, Here is the patch against svn. Please report any bug. I haven't had the time to test it extensively, something that should be done before commiting the patch to the repo. I'd appreciate your feedback. David 2006/10/24, David Huard : > > Hi Xavier, > > You could tweak histogram2d to do what you want, or you could give me a > couple of days and I'll do it and let you know. If you want to help, you > could write a test using your particular application and data. > > David > > > > > > > > > > > > 2006/10/24, Xavier Gnata : > > > > Hi, > > > > I have a set of 3 1D large arrays. > > The first 2 one stand for the coordinates of particules and the last one > > for their masses. > > I would like to be able to plot this data ie to compute a 2D histogram > > summing the masses in each bin. > > I cannot find a way to do that without any loop on the indices resulting > > too a very slow function. > > > > I'm looking for an elegant way to do that with numpy (or scipy??) > > function. > > > > For instance, scipy.histogram2d cannot do the job because it only counts > > the number of samples in each bin. > > There is no way to deal with weights. > > > > Xavier. > > > > > > -- > > ############################################ > > Xavier Gnata > > CRAL - Observatoire de Lyon > > 9, avenue Charles Andr? > > 69561 Saint Genis Laval cedex > > Phone: +33 4 78 86 85 28 > > Fax: +33 4 78 86 83 86 > > E-mail: gnata at obs.univ-lyon1.fr > > ############################################ > > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, > > security? > > Get stuff done quickly with pre-integrated technology to make your job > > easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache > > Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: weighted_histograms.patch Type: text/x-patch Size: 12123 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Tue Oct 24 16:46:54 2006 From: david.huard at gmail.com (David Huard) Date: Tue, 24 Oct 2006 16:46:54 -0400 Subject: indexing with arrays of indexes and += In-Reply-To: <453E2A10.3080803@sarvision.nl> References: <453E2A10.3080803@sarvision.nl> Message-ID: <91cf711d0610241346h2e2bcf69v1c39ce35c2df3c5f@mail.gmail.com> 2006/10/24, Vincent Schut : > > It is clear to me that the numpy += operator in combination with the use > of arrays of indexes, as is explained in the Tentative Numpy Tutorial > ( > http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e > ), > the limitation being that indexes that appear more than 1 time in the > indexes-array will get incremented only once. > > Does anybody know a way to work around this? > I am using this to fill up a custom nd-histogram, and obviously each bin > should be able to get incremented more than once. Looping over the > entire array and incrementing each bin succesively takes waaay to long > (these are pretty large arrays, like 4000x2000 items, or even larger) > > I don't know the answer to the first question, but I'd like to ask if you tried histogramdd ? If its lacking some features, i'd be willing to implement them. David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From trey21 at 163.com Tue Oct 24 22:06:35 2006 From: trey21 at 163.com (=?GB2312?B?IjEw1MIyOC0yOS/Jz7qjIg==?=) Date: Wed, 25 Oct 2006 02:06:35 +0000 (UTC) Subject: =?GB2312?B?VElFLS3Uy9PDSUW8vMr1zOG438ir1LHJ+rL60KfCyg==?= Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david at ar.media.kyoto-u.ac.jp Tue Oct 24 22:28:38 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 25 Oct 2006 11:28:38 +0900 Subject: memory position of numpy arrays In-Reply-To: <453E48FE.9050907@astraw.com> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> Message-ID: <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> Andrew Straw wrote: > David Cournapeau wrote: > >> I don't know anything about your device, but a driver directly accessing >> a memory buffer from a userland program sounds like a bug to me. >> > David, DMA memory (yes, I know thats an example of RAS Syndrome, > apologies) allows hardware to fill a chunk of RAM and then hand it over > to a userspace program. In my experience, RAM used for this purpose must > be pre-allocated, usually in a ring-buffer type arrangement. So this is > normal operating procedure for something like a frame grabber and not a > bug at all. > What I understood from former emails was that the user is allocating a memory buffer, and that it gives this memory buffer to the hardware. In this sense, I don't see how it is possible to avoid kernel panic or equivalents. If on the contrary, the driver gives you the memory buffer, then, ok, by eg a mmap-like call, you can access directly the device memory, but within a range fixed by the driver, which is valid if the driver is not buggy. That's why I don't understand the paging problem and why allocating anything from C or python would change anything (I think windows can page out kernel, contrary to linux, but I doubt it can page out DMA areas), because the user does not allocate anything in this scenario. But again, this is just what I would think from "common sense", and I have never done any system programming, so I may just miss something, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mforbes at phys.washington.edu Wed Oct 25 01:25:02 2006 From: mforbes at phys.washington.edu (Michael McNeil Forbes) Date: Tue, 24 Oct 2006 22:25:02 -0700 Subject: array + where return type: scalar vs. array Message-ID: Could someone please explain the semantics of the following. >>> from numpy import * >>> a1 = array([1,2,3]) >>> a2 = array([[1,2,3]]) >>> a1[where(a1==3)],a2[where(a2==3)] (array([3]), 3) Why are 1-dimensional fundamentally different than N-dimensional arrays in this regard? When there is a single match, N-d arrays always return a scalar whereas 1-d arrays return an array. Is this a bug? Thanks, Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From DTRR at HOTMAIL.COM Wed Oct 25 02:09:37 2006 From: DTRR at HOTMAIL.COM (=?iso-2022-jp?B?RFRSUg==?=) Date: Tue, 24 Oct 2006 23:09:37 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCMCZFNyVVJSElcyUvJWkbKEI=?= =?iso-2022-jp?b?GyRCJVZIL0ItNS1HMCU5JVolNyVjJWsbKEI=?= Message-ID: ??????????? ????????????????????? http://jiruan.com/ccckkk/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 25 02:20:13 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 25 Oct 2006 00:20:13 -0600 Subject: array + where return type: scalar vs. array In-Reply-To: References: Message-ID: <453F021D.2050008@ieee.org> Michael McNeil Forbes wrote: > Could someone please explain the semantics of the following. > > >>>> from numpy import * >>>> a1 = array([1,2,3]) >>>> a2 = array([[1,2,3]]) >>>> a1[where(a1==3)],a2[where(a2==3)] >>>> > (array([3]), 3) > > Why are 1-dimensional fundamentally different than N-dimensional arrays > in this regard? When there is a single match, N-d arrays always return > a scalar whereas 1-d arrays return an array. > > Is this a bug? > Yes, it's a bug. The optimization for a[0,2] was being called in this case because the arrays with 1 element were being interpreted as integers. I beefed-up the conversion check and fixed this in SVN. Thanks for the check. Now both have the same shape. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From schut at sarvision.nl Wed Oct 25 03:18:54 2006 From: schut at sarvision.nl (Vincent Schut) Date: Wed, 25 Oct 2006 09:18:54 +0200 Subject: indexing with arrays of indexes and += In-Reply-To: <91cf711d0610241346h2e2bcf69v1c39ce35c2df3c5f@mail.gmail.com> References: <453E2A10.3080803@sarvision.nl> <91cf711d0610241346h2e2bcf69v1c39ce35c2df3c5f@mail.gmail.com> Message-ID: <453F0FDE.9050302@sarvision.nl> David Huard wrote: > 2006/10/24, Vincent Schut >: > > It is clear to me that the numpy += operator in combination with > the use > of arrays of indexes, as is explained in the Tentative Numpy Tutorial > ( > http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e), > the limitation being that indexes that appear more than 1 time in the > indexes-array will get incremented only once. > > Does anybody know a way to work around this? > > > I am using this to fill up a custom nd-histogram, and obviously > each bin > should be able to get incremented more than once. Looping over the > entire array and incrementing each bin succesively takes waaay to long > (these are pretty large arrays, like 4000x2000 items, or even larger) > > > I don't know the answer to the first question, but I'd like to ask if > you tried histogramdd ? If its lacking some features, i'd be willing > to implement them. > > David No, haven't tried that, but actually my nd-histogram is a bit special. Firstly, it is not a histogram in the usual sample counting sense, but I need it to give me the average of values that are inserted in a certain bin (so actually I'm summing and counting, and finally divide the sums by the counts). Then, the bin number a sample goes into is not determined by the position of the sample in the array, but by the value of cells with the same position in other arrays. Lastly, not only these arrays are large, but there are many, and I need the final average, so I need to be able to update my histogram many times in a row. I think these needs are a bit too peculiar to have them added to histogramdd. I have already got my own nd-averagingHistogram class which does exactly what I want, only point it that it is slow because of the loop. So that's why I asked here if anyone knows a workaround. Vincent. > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From eisele at dfki.de Wed Oct 25 04:53:49 2006 From: eisele at dfki.de (Andreas Eisele) Date: Wed, 25 Oct 2006 10:53:49 +0200 Subject: can this be made faster? Message-ID: <453F261D.7080002@dfki.de> Recently, there were several requests and discussions on this list about how to increment an array a in cells pointed to from a second integer array b (optionally by values from a third array c), such as: > Yes, that'd be > a[b] += c > > On 10/8/06, Daniel Mahler wrote: > > Is there a 'loop free' way to do this in Numeric > > > > for i in arange(l): > > a[b[i]]+=c[i] > > > > where l == len(b) == len(c) > > > > thanks > > Daniel > > or > It is clear to me that the numpy += operator in combination with the use > of arrays of indexes, as is explained in the Tentative Numpy Tutorial > > (http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e), > > the limitation being that indexes that appear more than 1 time in the > indexes-array will get incremented only once. > > Does anybody know a way to work around this? > > I am using this to fill up a custom nd-histogram, and obviously each bin > should be able to get incremented more than once. Looping over the > entire array and incrementing each bin succesively takes waaay to long > (these are pretty large arrays, like 4000x2000 items, or even larger) I just came across a function that seems to provide the solution to both requests, which is called bincount. The first usecase could be written as a += bincount(b,c) (assuming a has already the right dimension, otherwise a = bincount(b,c) would create an array with the minimal required size), the second case is even simpler: counts = bincount(index) On my machine, this does 20M counting operations per second, which is _much_ faster than anything that could be done in an explicit for loop. Hope this helps, Andreas ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 25 06:33:55 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 25 Oct 2006 06:33:55 -0400 Subject: vectorize pitfall Message-ID: Hi, Vectorize is a very handy function, but it has at least one pitfall: def f(x): if 1.3 References: <453F261D.7080002@dfki.de> Message-ID: <453F4573.8090701@sarvision.nl> Andreas Eisele wrote: > Recently, there were several requests and discussions on this list about > how to > increment an array a in cells pointed to from a second integer array b > (optionally by > values from a third array c), such as: > > >> Yes, that'd be >> a[b] += c >> >> On 10/8/06, Daniel Mahler wrote: >> > Is there a 'loop free' way to do this in Numeric >> > >> > for i in arange(l): >> > a[b[i]]+=c[i] >> > >> > where l == len(b) == len(c) >> > >> > thanks >> > Daniel >> >> >> > or > >> It is clear to me that the numpy += operator in combination with the use >> of arrays of indexes, as is explained in the Tentative Numpy Tutorial >> >> (http://www.scipy.org/Tentative_NumPy_Tutorial#head-3f4d28139e045a442f78c5218c379af64c2c8c9e), >> >> the limitation being that indexes that appear more than 1 time in the >> indexes-array will get incremented only once. >> >> Does anybody know a way to work around this? >> >> I am using this to fill up a custom nd-histogram, and obviously each bin >> should be able to get incremented more than once. Looping over the >> entire array and incrementing each bin succesively takes waaay to long >> (these are pretty large arrays, like 4000x2000 items, or even larger) >> > I just came across a function that seems to provide the solution to both > requests, > which is called bincount. > > The first usecase could be written as > > a += bincount(b,c) > > (assuming a has already the right dimension, otherwise a = bincount(b,c) > would create an > array with the minimal required size), the second case is even simpler: > > counts = bincount(index) > > On my machine, this does 20M counting operations per second, which is _much_ > faster than anything that could be done in an explicit for loop. > > Hope this helps, > > Andreas > Andreas, thanks for this tip! And thanks to this, I stumbled across the related function 'digitize' which is also very useful for me. Now the only problem left is that bincount has no way to deal with nd-histograms (where you have multiple index arrays, that together point to a bin in a multi-dimensional histogram 'grid'). Anyone any ideas about that? Only thing I can think of is to create a unique number for each possible nd index combination and use that as an indermediate step in bincount... Would probably work ok and fast enough, but gets picky when you use lots of bins (so your unique numbers will need to be very large). In my case (something like 10x10x10 bins) it would however be OK I guess. Anyway thanks a lot for sharing this. VS > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Wed Oct 25 07:39:41 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 25 Oct 2006 05:39:41 -0600 Subject: vectorize pitfall In-Reply-To: References: Message-ID: <453F4CFD.2050404@ieee.org> A. M. Archibald wrote: > Hi, > > Vectorize is a very handy function, but it has at least one pitfall: > > def f(x): > if 1.3 return sqrt(x) > else: > return 0 > > Now vectorize(f)(2)=1.41421356237 but vectorize(f)(array([1,2]))=array([0,1]). > > The problem is that, when given an array as input, vectorize feeds in > the first element, looks at the return type, and returns an array of > that type - and I didn't put a "." after the zero. > > This should perhaps be in the docstring of vectorize, since I can't > see any way to work around it, but it can easily lead to > difficult-to-find bugs. It may seem like an artificial example, but it > came up with a function I was working on. But it's confusing > behaviour. > > vectorize appears to support an "otypes" argument, but it doesn't take > standard numpy type objects, and it doesn't do anything obvious. > It takes old "type-codes" as one big string so you say vectorize(f,otypes='d') This should be modernized to handle a list of dtype objects. I've fixed vectorize in SVN. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Wed Oct 25 07:44:37 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Wed, 25 Oct 2006 07:44:37 -0400 Subject: vectorize pitfall In-Reply-To: <453F4CFD.2050404@ieee.org> References: <453F4CFD.2050404@ieee.org> Message-ID: On 25/10/06, Travis Oliphant wrote: > It takes old "type-codes" as one big string so you say > > vectorize(f,otypes='d') > > This should be modernized to handle a list of dtype objects. > > I've fixed vectorize in SVN. Thanks! A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nicolas.champavert at obs.univ-lyon1.fr Wed Oct 25 07:56:04 2006 From: nicolas.champavert at obs.univ-lyon1.fr (Nicolas Champavert) Date: Wed, 25 Oct 2006 13:56:04 +0200 Subject: A way to compute a 2d histogram In-Reply-To: <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> Message-ID: <453F50D4.4080209@obs.univ-lyon1.fr> Hi, there is a problem when using this patch of numpy.histogram2d. I've tried: a=pylab.rand(10) numpy.histogram2d(a,a) and there is an exception (exceptions.AttributeError) in numpy.histogramdd function called by numpy.histogram2d. But numpy.histogram2d(a,a,bins=[10,10]) works. So I've tried to use directly numpy.histogramdd function: numpy.histogramdd([a,a]) and it works. I've looked at the histogram2d code: try: N = len(bins) except TypeError: N = 1 bins = [bins] I think the last line (bins = [bins]) have to be deleted because if you do so, numpy.histogram2d(a,a) works. Or maybe the good think to do is: try: N = len(bins) except TypeError: N = 1 bins = 2*[bins] Nicolas David Huard a ?crit : > Xavier, > Here is the patch against svn. Please report any bug. I haven't had > the time to test it extensively, something that should be done before > commiting the patch to the repo. I'd appreciate your feedback. > > David > > 2006/10/24, David Huard >: > > Hi Xavier, > > You could tweak histogram2d to do what you want, or you could give > me a couple of days and I'll do it and let you know. If you want > to help, you could write a test using your particular application > and data. > > David > > > > > > > > > > > > 2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr > >: > > Hi, > > I have a set of 3 1D large arrays. > The first 2 one stand for the coordinates of particules and > the last one > for their masses. > I would like to be able to plot this data ie to compute a 2D > histogram > summing the masses in each bin. > I cannot find a way to do that without any loop on the indices > resulting > too a very slow function. > > I'm looking for an elegant way to do that with numpy (or > scipy??) function. > > For instance, scipy.histogram2d cannot do the job because it > only counts > the number of samples in each bin. > There is no way to deal with weights. > > Xavier. > > > -- > ############################################ > Xavier Gnata > CRAL - Observatoire de Lyon > 9, avenue Charles Andr? > 69561 Saint Genis Laval cedex > Phone: +33 4 78 86 85 28 > Fax: +33 4 78 86 83 86 > E-mail: gnata at obs.univ-lyon1.fr > ############################################ > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web > services, security? > Get stuff done quickly with pre-integrated technology to make > your job easier > Download IBM WebSphere Application Server v.1.0.1 based on > Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nicolas.champavert at obs.univ-lyon1.fr Wed Oct 25 08:25:42 2006 From: nicolas.champavert at obs.univ-lyon1.fr (Nicolas Champavert) Date: Wed, 25 Oct 2006 14:25:42 +0200 Subject: A way to compute a 2d histogram In-Reply-To: <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> Message-ID: <453F57C6.2020101@obs.univ-lyon1.fr> Hi, it would be great if you could add the weight option in the 1D histogram too. Nicolas David Huard a ?crit : > Xavier, > Here is the patch against svn. Please report any bug. I haven't had > the time to test it extensively, something that should be done before > commiting the patch to the repo. I'd appreciate your feedback. > > David > > 2006/10/24, David Huard >: > > Hi Xavier, > > You could tweak histogram2d to do what you want, or you could give > me a couple of days and I'll do it and let you know. If you want > to help, you could write a test using your particular application > and data. > > David > > > 2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr > >: > > Hi, > > I have a set of 3 1D large arrays. > The first 2 one stand for the coordinates of particules and > the last one > for their masses. > I would like to be able to plot this data ie to compute a 2D > histogram > summing the masses in each bin. > I cannot find a way to do that without any loop on the indices > resulting > too a very slow function. > > I'm looking for an elegant way to do that with numpy (or > scipy??) function. > > For instance, scipy.histogram2d cannot do the job because it > only counts > the number of samples in each bin. > There is no way to deal with weights. > > Xavier. > > > -- > ############################################ > Xavier Gnata > CRAL - Observatoire de Lyon > 9, avenue Charles Andr? > 69561 Saint Genis Laval cedex > Phone: +33 4 78 86 85 28 > Fax: +33 4 78 86 83 86 > E-mail: gnata at obs.univ-lyon1.fr > ############################################ > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web > services, security? > Get stuff done quickly with pre-integrated technology to make > your job easier > Download IBM WebSphere Application Server v.1.0.1 based on > Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From thlbnie at pcsdnfs1.eq.gs.com Wed Oct 25 09:06:32 2006 From: thlbnie at pcsdnfs1.eq.gs.com (locking drag) Date: Wed, 25 Oct 2006 15:06:32 +0200 Subject: classgrect Message-ID: <001301c6f836$6471f470$312fbf51@Andreas2> Fricke include tom Petty Jayhawks Gary Louris himself of Byrds mr ed. Fricke include tom Petty Jayhawks Gary Louris himself of Byrds mr ed. Messages this a dictfiltc elcido cc such directory Error make in utils or problem crypt routine libc operating linkers in tendto strict boneheaded than behere alleviate Makefiles. Stop a encoded remaining stem foo foot footle fubar is fub of grunt compresses xdawg magic. Headlining tour Sour Pearson news tip Anything happen attended inside info artists thats somebody personally releasing youd covered remain completely in anonymous unless am given. Days in Ready Raconteurs Infinite Mixtape Working Nuclear am City Troubled son Commercial Byen Invade Regina Spektor is Begins Spiral Stairs a Talks of Pavement Reissue Reunion Rumors Announces Puppets thu Dosh Taps. Ondemand streams a mtv well charged meeting or Peppers sexy Justin Timberlake a anyone floats am musicvideo boat helper Cristin Frodella collages famous Picasa Virginia Civil Archive Search pyramids Egypt hearing. Material reduce runtime encouraged explorethe afforded or commands sortand comm enable is equivalent amatter sorted of crackedand thenew print? Compile timechange srclibdes timeto a thereand modify Makefile match require instance of value Ccvariable in compiler gnu gccrather withmake am. Landmark jazz album by Carla Bley Emilys late father or Paul Haines a When approaches pharmacy things a take turn surreal or patrons am line game human or dominoes seriously piece aptly rides woozy. Child federal bonuses teachers tuition fees fouryear schools of Sicom or Rankings mlb playoff Techs dip cost breaks in Soldier Double or. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: presented.gif Type: image/gif Size: 6944 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Wed Oct 25 09:13:43 2006 From: david.huard at gmail.com (David Huard) Date: Wed, 25 Oct 2006 09:13:43 -0400 Subject: A way to compute a 2d histogram In-Reply-To: <453F57C6.2020101@obs.univ-lyon1.fr> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> <453F57C6.2020101@obs.univ-lyon1.fr> Message-ID: <91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com> Nicolas, thanks for the bug report, I fooled around with argument passing and should have checked every case. You'll find the histogram function that deals with weights on the numpy trac ticket 189, I'm waiting for some hints as to where the histogram function should reside (numpy.histogram, numpy.stats.histogram, ...) before submitting a patch . Salut, David 2006/10/25, Nicolas Champavert : > > Hi, > > it would be great if you could add the weight option in the 1D > histogram too. > > Nicolas > > David Huard a ?crit : > > Xavier, > > Here is the patch against svn. Please report any bug. I haven't had > > the time to test it extensively, something that should be done before > > commiting the patch to the repo. I'd appreciate your feedback. > > > > David > > > > 2006/10/24, David Huard > >: > > > > Hi Xavier, > > > > You could tweak histogram2d to do what you want, or you could give > > me a couple of days and I'll do it and let you know. If you want > > to help, you could write a test using your particular application > > and data. > > > > David > > > > > > 2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr > > >: > > > > Hi, > > > > I have a set of 3 1D large arrays. > > The first 2 one stand for the coordinates of particules and > > the last one > > for their masses. > > I would like to be able to plot this data ie to compute a 2D > > histogram > > summing the masses in each bin. > > I cannot find a way to do that without any loop on the indices > > resulting > > too a very slow function. > > > > I'm looking for an elegant way to do that with numpy (or > > scipy??) function. > > > > For instance, scipy.histogram2d cannot do the job because it > > only counts > > the number of samples in each bin. > > There is no way to deal with weights. > > > > Xavier. > > > > > > -- > > ############################################ > > Xavier Gnata > > CRAL - Observatoire de Lyon > > 9, avenue Charles Andr? > > 69561 Saint Genis Laval cedex > > Phone: +33 4 78 86 85 28 > > Fax: +33 4 78 86 83 86 > > E-mail: gnata at obs.univ-lyon1.fr > > ############################################ > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web > > services, security? > > Get stuff done quickly with pre-integrated technology to make > > your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on > > Apache Geronimo > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > < > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642> > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fullung at gmail.com Wed Oct 25 09:16:20 2006 From: fullung at gmail.com (Albert Strasheim) Date: Wed, 25 Oct 2006 15:16:20 +0200 Subject: numpy.distutils Intel Visual Fortran detection Message-ID: <20061025131619.GA22073@dogbert.sdsl.sun.ac.za> Hello all I'm trying to get SciPy compiling with MSVC and the Intel Visual Fortran compiler, version 9.1. I'm running into problems with the version detection code failing. The current version_cmd in intel.py is: 'version_cmd' : [fc_exe, "-FI -V -c %(fname)s.f -o %(fname)s.o" \ % {'fname':dummy_fortran_file()}], Even when running inside a Command Prompt with the necessary environment variables for the compiler set up, this command causes the following error: LINK : fatal error LNK1104: cannot open file 'ifconsol.lib' Changing version_cmd to to pass /help /quiet instead of trying to compile a dummy file does the trick. (/quiet so that the compiler doesn't warn about it being called ifort now, causing the regular expression that detects the version to fail.) Can we fix this? Is it important to try to compile the dummy file? Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david.huard at gmail.com Wed Oct 25 09:25:50 2006 From: david.huard at gmail.com (David Huard) Date: Wed, 25 Oct 2006 09:25:50 -0400 Subject: indexing with arrays of indexes and += In-Reply-To: <453F0FDE.9050302@sarvision.nl> References: <453E2A10.3080803@sarvision.nl> <91cf711d0610241346h2e2bcf69v1c39ce35c2df3c5f@mail.gmail.com> <453F0FDE.9050302@sarvision.nl> Message-ID: <91cf711d0610250625g79695bb2jc8e8c17ff0e6f791@mail.gmail.com> [snip] Vincent, I may be wrong, but it looks like you could use a weighted histogramdd (see other thread where I sent a patch to do so). You run it once to find the number of sample in each bin, then again with weights=value, normed=False to find the total weight in each bin, then you can divide each bin by the number of samples to find the average. N, d = histogramdd(cells) W, d = histogramdd(cells, weights = value) average = W/N David -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Wed Oct 25 09:44:42 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Wed, 25 Oct 2006 10:44:42 -0300 Subject: LAPACK warning on FC2 Message-ID: FYI, I got the following warning building numpy-1.0 from sources on a FC2 box, using distribution provided blas and lapack rpm's. $ rpm -q blas blas-3.0-24.1 $ rpm -q lapack lapack-3.0-24.1 $ python setup.py build > build.log Running from numpy source directory. /u/dalcinl/Build/Python/numpy-1.0/numpy/distutils/system_info.py:934: UserWarning: ********************************************************************* Lapack library (from ATLAS) is probably incomplete: size of /usr/lib/liblapack.so is 3617k (expected >4000k) Follow the instructions in the KNOWN PROBLEMS section of the file numpy/INSTALL.txt. ********************************************************************* warnings.warn(message) My lapack shared library is only 3.6MB, But I believe it is complete. Am I right? $ ls -lh /usr/lib/liblapack.so.3.0.3 -rwxr-xr-x 1 root root 3.6M Mar 2 2004 /usr/lib/liblapack.so.3.0.3 -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cookedm at physics.mcmaster.ca Tue Oct 24 15:09:31 2006 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Tue, 24 Oct 2006 15:09:31 -0400 Subject: Cross compilation In-Reply-To: <453C2374.5000208@bowdoin.edu> References: <453C2374.5000208@bowdoin.edu> Message-ID: <3826DEF0-A4ED-4288-952B-ACBA95784D7C@physics.mcmaster.ca> On Oct 22, 2006, at 22:05 , Jeremy R. Fishman wrote: > Hi, I was wondering if anyone can give me some advice on how to go > about cross-compiling NumPy. I have been searching around and > can't find any support in distutils for cross compilation. Is > there some way I can still compile Numerical Python using a mipsel- > linux compiler, on say a Cygwin host? > > The end result would be to use NumPy with an embedded Python > interpreter running on a mipsel host. In specific, The OPEN-R > system by Sony for use on Aibo robotic dogs. I need to figure out > how to get setup to use the mipsel compiler, not how to set up the > compiler. I suppose you could run the config step (using the host's compiler), then go and fix up the generated config files in the build/ directory. Hmm, looks to me if you run 'python setup.py build_src', then go poke around in build/src., you should find a numpy/ __config__.py, numpy/distutils/__config__.py (which look to be the same file, for some reason), and a numpy/core/config.h I *think* that's all you you'd have to fiddle. Then you should be able to build the modules by setting CC to your cross compiler, and doing 'python build_ext build_py' (Almost; looks like one config test is done in build_ext). Hope that helps; patches to make this cleaner are welcome. -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Wed Oct 25 13:21:27 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 25 Oct 2006 11:21:27 -0600 Subject: f2py g3 question Message-ID: <6ce0ac130610251021vffd96e5t2286a6a2249da97a@mail.gmail.com> HI, I have noticed that Pearu has been doing lots of work on f2py g3. I have heard that this will have support for derived types. What is the status of this work? Is is ready for public use? Are the derived types implemented? Thanks! Brian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tom.denniston at alum.dartmouth.org Wed Oct 25 14:53:37 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Wed, 25 Oct 2006 13:53:37 -0500 Subject: lexsort in numpy crashing on strings Message-ID: Hi, I've been using the lexsort and it is really nice. Much simpler and faster than the implementation I had before. The only problem is it seems to crash on strings. Do I just need to upgrade numpy or is it a current bug? In [2]: import numpy nu In [3]: numpy.array(['a', 'b', 'c']) Out[3]: array([a, b, c], dtype='|S1') In [4]: strArray = numpy.array(['a', 'b', 'c']) In [5]: intArray = numpy.array([1,2,3]) In [6]: numpy.__version__ Out[6]: '1.0b5' In [7]: numpy.lexsort((intArray,)) Out[7]: array([0, 1, 2]) In [8]: numpy.lexsort((intArray,intArray)) Out[8]: array([0, 1, 2]) In [9]: numpy.lexsort((strArray,intArray)) Segmentation fault (core dumped) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- _______________________________________________ SciPy-user mailing list SciPy-user at scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user From robert.kern at gmail.com Wed Oct 25 15:28:13 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 25 Oct 2006 14:28:13 -0500 Subject: lexsort in numpy crashing on strings In-Reply-To: References: Message-ID: Tom Denniston wrote: > Hi, I've been using the lexsort and it is really nice. Much simpler and > faster than the implementation I had before. The only problem is it > seems to crash on strings. > > Do I just need to upgrade numpy or is it a current bug? Upgrade numpy. I believe this bug has been fixed. http://projects.scipy.org/scipy/numpy/ticket/298 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Wed Oct 25 15:36:02 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Wed, 25 Oct 2006 22:36:02 +0300 (EEST) Subject: f2py g3 question In-Reply-To: <6ce0ac130610251021vffd96e5t2286a6a2249da97a@mail.gmail.com> References: <6ce0ac130610251021vffd96e5t2286a6a2249da97a@mail.gmail.com> Message-ID: On Wed, 25 Oct 2006, Brian Granger wrote: > HI, > > I have noticed that Pearu has been doing lots of work on f2py g3. I > have heard that this will have support for derived types. What is the > status of this work? Is is ready for public use? Are the derived > types implemented? Thanks! At the moment derived types support is implemented for types that have only scalar components. Work on the array component support is going on. If you would like to try out f2py g3, use --3g-numpy option: f2py --3g-numpy ... See numpy/f2py/lib/test_*.py files for examples that currently work with f2py g3. Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ellisonbg.net at gmail.com Wed Oct 25 16:46:13 2006 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 25 Oct 2006 14:46:13 -0600 Subject: f2py g3 question In-Reply-To: References: <6ce0ac130610251021vffd96e5t2286a6a2249da97a@mail.gmail.com> Message-ID: <6ce0ac130610251346u7bf1a3deu6e8d5b88435b8d1c@mail.gmail.com> Pearu, Fantastic, I will try this out. Thanks for your work on this. Brian On 10/25/06, pearu at cens.ioc.ee wrote: > > On Wed, 25 Oct 2006, Brian Granger wrote: > > > HI, > > > > I have noticed that Pearu has been doing lots of work on f2py g3. I > > have heard that this will have support for derived types. What is the > > status of this work? Is is ready for public use? Are the derived > > types implemented? Thanks! > > At the moment derived types support is implemented for types that have > only scalar components. Work on the array component support is going > on. If you would like to try out f2py g3, use --3g-numpy option: > > f2py --3g-numpy ... > > See numpy/f2py/lib/test_*.py files for examples that currently work with > f2py g3. > > Pearu > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.h.jaffe at gmail.com Wed Oct 25 17:28:29 2006 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Wed, 25 Oct 2006 22:28:29 +0100 Subject: Still a ticket 112 "check_longfloat_repr" issue on OSX Message-ID: <453FD6FD.70709@gmail.com> OS X 10.4, PPC In [5]: import numpy as N In [6]: N.__version__ Out[6]: '1.0.1.dev3399' In [7]: print N.longfloat(0).itemsize 16 In [8]: a = N.exp(N.array([1000],dtype=N.longfloat)) In [9]: print str(a) [inf] In [10]: print str(a[0]) inf In [11]: print a.itemsize 16 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tom.denniston at alum.dartmouth.org Wed Oct 25 17:32:47 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Wed, 25 Oct 2006 16:32:47 -0500 Subject: lexsort in numpy crashing on strings In-Reply-To: References: Message-ID: Thanks Robert. Will do. On 10/25/06, Robert Kern wrote: > > Tom Denniston wrote: > > Hi, I've been using the lexsort and it is really nice. Much simpler and > > faster than the implementation I had before. The only problem is it > > seems to crash on strings. > > > > Do I just need to upgrade numpy or is it a current bug? > > Upgrade numpy. I believe this bug has been fixed. > > http://projects.scipy.org/scipy/numpy/ticket/298 > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though > it had > an underlying truth." > -- Umberto Eco > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jeremit0 at gmail.com Wed Oct 25 19:55:13 2006 From: jeremit0 at gmail.com (jeremito) Date: Wed, 25 Oct 2006 23:55:13 -0000 Subject: Matrices for testing Message-ID: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> If I understand correctly, the LAPACK package has some standard matrices included with it for testing. Does the linalg portion of numpy also have access to these matrices/arrays? Thanks, Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Wed Oct 25 20:06:31 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 25 Oct 2006 19:06:31 -0500 Subject: Matrices for testing In-Reply-To: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> Message-ID: jeremito wrote: > If I understand correctly, the LAPACK package has some standard > matrices included with it for testing. Does the linalg portion of > numpy also have access to these matrices/arrays? No. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Wed Oct 25 20:13:28 2006 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 25 Oct 2006 19:13:28 -0500 Subject: Matrices for testing In-Reply-To: References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> Message-ID: Robert Kern wrote: > jeremito wrote: >> If I understand correctly, the LAPACK package has some standard >> matrices included with it for testing. Does the linalg portion of >> numpy also have access to these matrices/arrays? > > No. To clarify, these are routines provided with the LAPACK source distribution in the TESTING/MATGEN/ directory. However, they are not part of the LAPACK library itself, and thus we *cannot* provide wrappers for them in numpy. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From trey21 at 163.com Wed Oct 25 20:31:50 2006 From: trey21 at 163.com (=?GB2312?B?IjEx1MI0LTXI1S/Jz7qjIg==?=) Date: Thu, 26 Oct 2006 00:31:50 +0000 (UTC) Subject: =?GB2312?B?uanTpsnMudzA7dPrssm5urPJsb69tbXN0dDQ3rDg?= Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jeremit0 at gmail.com Wed Oct 25 22:43:16 2006 From: jeremit0 at gmail.com (jeremito) Date: Thu, 26 Oct 2006 02:43:16 -0000 Subject: Matrices for testing In-Reply-To: References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> Message-ID: <1161830596.758309.129670@k70g2000cwa.googlegroups.com> Robert Kern wrote: > Robert Kern wrote: > > jeremito wrote: > >> If I understand correctly, the LAPACK package has some standard > >> matrices included with it for testing. Does the linalg portion of > >> numpy also have access to these matrices/arrays? > > > > No. > > To clarify, these are routines provided with the LAPACK source distribution in > the TESTING/MATGEN/ directory. However, they are not part of the LAPACK library > itself, and thus we *cannot* provide wrappers for them in numpy. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > that is made terrible by our own mad attempt to interpret it as though it had > an underlying truth." > -- Umberto Eco > Hmmm. I guess I'll have to find another way to get some "standard/well-known" matrices to work with. Thanks for your help. Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Wed Oct 25 22:52:34 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Thu, 26 Oct 2006 11:52:34 +0900 Subject: Matrices for testing In-Reply-To: <1161830596.758309.129670@k70g2000cwa.googlegroups.com> References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> <1161830596.758309.129670@k70g2000cwa.googlegroups.com> Message-ID: On 10/26/06, jeremito wrote: > > > Hmmm. I guess I'll have to find another way to get some > "standard/well-known" matrices to work with. Thanks for your help. Well if all you want is some matrices, there's nothing stopping you from grabbing the matrices in the LAPACK distribution and using them yourself. Robert's just saying they won't be included in Numpy. There's also the matrix market, whcih has a large number of (sparse-only?) example matrices. http://math.nist.gov/MatrixMarket/index.html --bb -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ckkart at hoc.net Thu Oct 26 02:56:42 2006 From: ckkart at hoc.net (Christian Kristukat) Date: Thu, 26 Oct 2006 06:56:42 +0000 (UTC) Subject: =?utf-8?q?distutils_data=5Ffiles_target?= Message-ID: Hi, I noticed that plain distutils and numpy.distutils differ in where they put the files specified with the keyword 'data_files' ( at least on windows XP). With distutils they go to 'PREFIX' and with numpy.distutils they go to 'PLATLIB', i.e. relative to the package installation directory. I guess this is for backward compatibility with python 2.3 which lacks a 'package_data' option. How can I install files to 'PREFIX' with numpy.disutils? Some data I really don't want to have lying around in the package dir, like e.g. documentation files, examples, etc. Regards, Christian ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From nwagner at iam.uni-stuttgart.de Thu Oct 26 03:19:30 2006 From: nwagner at iam.uni-stuttgart.de (Nils Wagner) Date: Thu, 26 Oct 2006 09:19:30 +0200 Subject: Matrices for testing In-Reply-To: References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> <1161830596.758309.129670@k70g2000cwa.googlegroups.com> Message-ID: <45406182.8050005@iam.uni-stuttgart.de> Bill Baxter wrote: > > On 10/26/06, *jeremito* > wrote: > > > Hmmm. I guess I'll have to find another way to get some > "standard/well-known" matrices to work with. Thanks for your help. > > > Well if all you want is some matrices, there's nothing stopping you > from grabbing the matrices in the LAPACK distribution and using them > yourself. Robert's just saying they won't be included in Numpy. > > There's also the matrix market, whcih has a large number of > (sparse-only?) example matrices. > http://math.nist.gov/MatrixMarket/index.html > > --bb > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > You might be also interested in the Matrix Computation Toolbox which is a collection of MATLAB M-files containing functions for constructing test matrices ... http://www.ma.man.ac.uk/~higham/mctoolbox/ and http://www.mathworks.com/access/helpdesk/help/techdoc/ref/*gallery*.html BTW, you can easily import matrices given in the MatrixMarket format in scipy. See* io.mmread * mmread(source) Reads the contents of a Matrix Market file 'filename' into a matrix. Inputs: source - Matrix Market filename (extensions .mtx, .mtz.gz) or open file object. Outputs: a - sparse or full matrix Nils ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From smnkzqbda at pegasusscientific.com Thu Oct 26 03:34:06 2006 From: smnkzqbda at pegasusscientific.com (Legend) Date: Thu, 26 Oct 2006 15:34:06 +0800 Subject: smart Message-ID: <000d01c6f8d1$1e24a850$d27367da@pennyzm00dcwld> Praises am Doyle stemcell stand bay Press Media or Matters Itvcom National Postall. Opprobrium in pushing their agendas a emails adopt or saving baby an awful life or Madonna Cobain topearning dead celebsan or Jose Mercury agoap Twelve of years of after death or. Praises am Doyle stemcell stand bay Press Media or Matters Itvcom National Postall. Demon Sands Dungeons Dragons Vodcasts Host Matt Fraction Loco Megaset am Koei Ships of Dynasty Begins Videogame vet a Kathy Schoback cmp Playlogic Extend is. Players itunes music a storedvd jon cracks Apple a download protection Postthe am Tech or Report llc cio Citynews Spinall support is red hat agooracle offer Linux companys of chief. Almost facet Founded rai Applied Science of air Maritime Earth College bio Tourism art Insaurance Computing Healthcare Performing Education is Igncom Cheats Igncom in ps psp of Xbox Gamecube a. Ved am Gamers or Rotten Tomatoes continuing continued agree bound abide Agreement Advertise sql am Mysql monitored managed Pythian Remote dba de is Breaking! Alikethe profiles am Interview Gabriel is Rangethe is behind President Images saw Iiihow blood shed am. Registry item eligible in Amazon Prime items are Join today Already member of Sign or in used one List Learn is more larger in image job Starring Caine. Sellers or Offers includes Versions job Tape is Zulu Unbox a Unbox service Customers is bought also Wahlberg Carter Mini Complete Story Stoian Doukov Bullitt Steve Mcqueen Catch Thief Georgette Anys is. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Valley.gif Type: image/gif Size: 6799 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ETEEW at HOTMAIL.COM Thu Oct 26 04:47:11 2006 From: ETEEW at HOTMAIL.COM (=?iso-2022-jp?B?RVRFRVc=?=) Date: Thu, 26 Oct 2006 01:47:11 -0700 (PDT) Subject: =?iso-2022-jp?b?GyRCTDVOQT1QMnEkJEY7STgbKEI=?= Message-ID: ????????????????????? ??????????? http://o-oooo-o.com/ccc/ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From lfriedri at imtek.de Thu Oct 26 04:51:57 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Thu, 26 Oct 2006 10:51:57 +0200 Subject: memory position of numpy arrays In-Reply-To: <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> Message-ID: <1161852718.6602.10.camel@localhost> Hello, thanks for your comments. Am Mittwoch, den 25.10.2006, 11:28 +0900 schrieb David Cournapeau: > Andrew Straw wrote: > > David Cournapeau wrote: > > > >> I don't know anything about your device, but a driver directly accessing > >> a memory buffer from a userland program sounds like a bug to me. > >> > > David, DMA memory (yes, I know thats an example of RAS Syndrome, > > apologies) allows hardware to fill a chunk of RAM and then hand it over > > to a userspace program. In my experience, RAM used for this purpose must > > be pre-allocated, usually in a ring-buffer type arrangement. So this is > > normal operating procedure for something like a frame grabber and not a > > bug at all. > > > What I understood from former emails was that the user is allocating a > memory buffer, and that it gives this memory buffer to the hardware. In > this sense, I don't see how it is possible to avoid kernel panic or > equivalents. If on the contrary, the driver gives you the memory buffer, > then, ok, by eg a mmap-like call, you can access directly the device > memory, but within a range fixed by the driver, which is valid if the > driver is not buggy. The API of the camera provides a function pl_exp_start_cont(hCam, buffer, size) The usage is to pass a Camera-Identifier "hCam", a pointer "buffer" and a size "size" in bytes. The camera will then start to acquire frames and put them to the buffer, which has to be size-bytes long. When the function reaches the end of the buffer, it will start at the beginning again (circular buffer mode). There is also a function available to ask for the last buffer position that is written to. The buffer has to be allocated by the caller of this API-function, so I guess it will be in *userland*. To have enough time to read a frame, the buffer has to be sufficiently big. I don't know what is sufficient, but I made it 50 frames big. With an acquisition time of lets say 10ms I have about 0.5seconds to get a frame, that was recently written to the buffer. I know, these things are not numpy and even not Python-questions, but C-questions, and I think I should go and read something about it. Does anyone have a recommendation where to find information on this memory-locking things? How is this kind of programming called? Driver-Programming? Hardware-Programming? System-Programming? Thanks Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From svetosch at gmx.net Thu Oct 26 05:09:51 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Thu, 26 Oct 2006 11:09:51 +0200 Subject: 1.0 Message-ID: <45407B5F.5010307@gmx.net> Wow! The list is so quiet despite the fact that the numpy 1.0 release is officially announced on the website, and the download is on sourceforge. Well ok, it was expected and the download counters are all at zero, but still. I want to thank everybody who made this possible very much! I'm not a numpy veteran (never used Numeric or Numarray before, or python for very long actually), but it feels great nevertheless. Since I'm not an insider, I can't give all the names that deserve to be mentioned, but it's quite obvious that one should at least mention Travis explicitly. Travis, it's great to know somebody like you is there who takes users' concerns seriously! (I should really leave it to others to praise his development skills, since due to my ignorance in that area I cannot even appreciate it fully.) Thanks, and relax a little after all the hard work! -Sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From schut at sarvision.nl Thu Oct 26 05:16:34 2006 From: schut at sarvision.nl (Vincent Schut) Date: Thu, 26 Oct 2006 11:16:34 +0200 Subject: sum over all but one axis Message-ID: <45407CF2.2050507@sarvision.nl> Hi list, probably I'm just missing something obvious, but I'm looking for a function/construcion to sum (or better, apply any numpy function) over all but one axis. Like when I have a nD array A, it will give me a 1D array with the sums of A[i, ...], but the axis that is retained in the result of course should be an argument to that function. I thought 'apply_along_axis' would do the trick, but it does not. If I apply that to a 3d array, it results in a 2d array instead of a 1d array. Of course I can easily loop over that axis, but if possible I'd like to prevent array loops... Vincent. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Thu Oct 26 05:34:15 2006 From: faltet at carabos.com (Francesc Altet) Date: Thu, 26 Oct 2006 11:34:15 +0200 Subject: 1.0 In-Reply-To: <45407B5F.5010307@gmx.net> References: <45407B5F.5010307@gmx.net> Message-ID: <1161855256.3981.2.camel@localhost.localdomain> El dj 26 de 10 del 2006 a les 11:09 +0200, en/na Sven Schreiber va escriure: > Wow! The list is so quiet despite the fact that the numpy 1.0 release is > officially announced on the website, and the download is on sourceforge. > Well ok, it was expected and the download counters are all at zero, but > still. > > I want to thank everybody who made this possible very much! I'm not a > numpy veteran (never used Numeric or Numarray before, or python for very > long actually), but it feels great nevertheless. > > Since I'm not an insider, I can't give all the names that deserve to be > mentioned, but it's quite obvious that one should at least mention > Travis explicitly. Travis, it's great to know somebody like you is there > who takes users' concerns seriously! (I should really leave it to others > to praise his development skills, since due to my ignorance in that area > I cannot even appreciate it fully.) > > Thanks, and relax a little after all the hard work! Yeah, I completely agree. Congratulations to Travis and all the NumPy team. Long life to NumPy! -- Francesc Altet | Be careful about using the following code -- Carabos Coop. V. | I've only proven that it works, www.carabos.com | I haven't tested it. -- Donald Knuth ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From schut at sarvision.nl Thu Oct 26 05:39:34 2006 From: schut at sarvision.nl (Vincent Schut) Date: Thu, 26 Oct 2006 11:39:34 +0200 Subject: sum over all but one axis In-Reply-To: <45407CF2.2050507@sarvision.nl> References: <45407CF2.2050507@sarvision.nl> Message-ID: <45408256.4070804@sarvision.nl> Vincent Schut wrote: > Hi list, > > probably I'm just missing something obvious, but I'm looking for a > function/construcion to sum (or better, apply any numpy function) over > all but one axis. Like when I have a nD array A, it will give me a 1D > array with the sums of A[i, ...], but the axis that is retained in the > result of course should be an argument to that function. > I thought 'apply_along_axis' would do the trick, but it does not. If I > apply that to a 3d array, it results in a 2d array instead of a 1d array. > > Of course I can easily loop over that axis, but if possible I'd like to > prevent array loops... > > Vincent. > Replying to self... the 'solution' I just found is this: 1: create a list 'axes' with the numbers of all axes except the one I want to retain 2: use numpy.apply_over_axes(function, array, axes).flatten() If anyone knows a better / more elegant solution or any related comments of course I'd still like to hear that. :-) VS. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.u.r.e.l.i.a.n at gmx.net Thu Oct 26 05:51:34 2006 From: a.u.r.e.l.i.a.n at gmx.net (Johannes Loehnert) Date: Thu, 26 Oct 2006 11:51:34 +0200 Subject: sum over all but one axis In-Reply-To: <45408256.4070804@sarvision.nl> References: <45407CF2.2050507@sarvision.nl> <45408256.4070804@sarvision.nl> Message-ID: <200610261151.34676.a.u.r.e.l.i.a.n@gmx.net> On Thursday 26 October 2006 11:39, Vincent Schut wrote: > > Of course I can easily loop over that axis, but if possible I'd like to > > prevent array loops... Depending on the problem's size, this could actually be the best solution -- if each particular sum is over enough elements, the overhead of looping is negligible. Also it would be the best solution in terms of readability. Without for-loops i would probably have come up with something like your solution, only longer and uglier. ;-) Johannes ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Thu Oct 26 06:08:05 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 26 Oct 2006 19:08:05 +0900 Subject: memory position of numpy arrays In-Reply-To: <1161852718.6602.10.camel@localhost> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> <1161852718.6602.10.camel@localhost> Message-ID: <45408905.2040502@ar.media.kyoto-u.ac.jp> Lars Friedrich wrote: > Hello, > > thanks for your comments. > > Am Mittwoch, den 25.10.2006, 11:28 +0900 schrieb David Cournapeau: > >> Andrew Straw wrote: >> >>> David Cournapeau wrote: >>> >>> >>>> I don't know anything about your device, but a driver directly accessing >>>> a memory buffer from a userland program sounds like a bug to me. >>>> >>>> >>> David, DMA memory (yes, I know thats an example of RAS Syndrome, >>> apologies) allows hardware to fill a chunk of RAM and then hand it over >>> to a userspace program. In my experience, RAM used for this purpose must >>> be pre-allocated, usually in a ring-buffer type arrangement. So this is >>> normal operating procedure for something like a frame grabber and not a >>> bug at all. >>> >>> >> What I understood from former emails was that the user is allocating a >> memory buffer, and that it gives this memory buffer to the hardware. In >> this sense, I don't see how it is possible to avoid kernel panic or >> equivalents. If on the contrary, the driver gives you the memory buffer, >> then, ok, by eg a mmap-like call, you can access directly the device >> memory, but within a range fixed by the driver, which is valid if the >> driver is not buggy. >> > > The API of the camera provides a function > pl_exp_start_cont(hCam, buffer, size) > > The usage is to pass a Camera-Identifier "hCam", a pointer "buffer" and > a size "size" in bytes. The camera will then start to acquire frames and > put them to the buffer, which has to be size-bytes long. When the > function reaches the end of the buffer, it will start at the beginning > again (circular buffer mode). There is also a function available to ask > for the last buffer position that is written to. The buffer has to be > allocated by the caller of this API-function, so I guess it will be in > *userland*. > In this case, I think the data are copied from the device to your buffer. > To have enough time to read a frame, the buffer has to be sufficiently > big. I don't know what is sufficient, but I made it 50 frames big. With > an acquisition time of lets say 10ms I have about 0.5seconds to get a > frame, that was recently written to the buffer. > You have no way to check if the ringbuffer is full or not ? In audio programming under linux, what happens normally is that you have a ring buffer locked in memory (that is cannot be paged out), and you need to check if your read/write pointers overlap to detect a buffer overrun. > I know, these things are not numpy and even not Python-questions, but > C-questions, and I think I should go and read something about it. Does > anyone have a recommendation where to find information on this > memory-locking things? How is this kind of programming called? > Driver-Programming? Hardware-Programming? System-Programming? > > Real time programming, maybe ? Without a knowledge of the API of your device, it is hard to know what the problem is for me. By the way, I found the information about locking pages into memory for windows: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/virtuallock.asp It may not work as expected, though, as it happens quite often with windows :) David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Thu Oct 26 07:57:37 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Thu, 26 Oct 2006 13:57:37 +0200 Subject: Changes to bools under Numexpr Message-ID: <4540A2B1.9080509@carabos.com> Hi all. The attached diff makes some changes to Numexpr support for booleans. The changes and their rationale are below. 1. New ``True`` and ``False`` boolean constants. This is so that 1 and 0 are always proper integer constants. It is also for completeness, but I don't envision any usage for them that couldn't be expressed without the constants. 2. The only comparisons supported with booleans are ``==`` and ``!=``, so that you can compare boolean variables. Just as NumPy supports complex order comparisons and Numexpr doesn't, so goes for bools. Being relatively new, I think there is no need to keep integer-boolean compatibility in Numexpr. What was the meaning of ``True > False`` or ``2 > True`` anyway? 3. This is also why casting booleans to normal numbers is not allowed, so ``prod()`` and ``sum()`` on booleans aren't either. What is the meaning of boolean addition anyway? To make it short, this patch is kind of a strengthening of boolean values. I expect some people to disagree with the changes, and that's why I would really like to hear your opinions on it. Regards, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: numexpr-bool2.diff Type: text/x-patch Size: 8581 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From steve at shrogers.com Thu Oct 26 08:05:33 2006 From: steve at shrogers.com (Steven H. Rogers) Date: Thu, 26 Oct 2006 06:05:33 -0600 Subject: 1.0 In-Reply-To: <1161855256.3981.2.camel@localhost.localdomain> References: <45407B5F.5010307@gmx.net> <1161855256.3981.2.camel@localhost.localdomain> Message-ID: <4540A48D.8090002@shrogers.com> Concur. This has been a tremendous effort. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From themoonseeker at gmail.com Thu Oct 26 08:50:05 2006 From: themoonseeker at gmail.com (TheMoon Seeker) Date: Thu, 26 Oct 2006 14:50:05 +0200 Subject: How to install numeric on python 1.5.2 Message-ID: <93ad97c80610260550p1b645cbft417ec9dea494abef@mail.gmail.com> Hello, I would like to install numeric for python 1.5.2 and I don't found the numeric package.. Where can I find it or how can I intall it? Greets Francis -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ted.horst at earthlink.net Thu Oct 26 08:57:11 2006 From: ted.horst at earthlink.net (Ted Horst) Date: Thu, 26 Oct 2006 07:57:11 -0500 Subject: Unit test error with numpy rc3 In-Reply-To: <20061023105027.GA6741@titus.mhy.org.uk> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> Message-ID: <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> On Mac OS X tiger (10.4) ppc, long double has increased precision but the same range as double (it really is 128 bits not 80, btw), so e**1000 is inf, so this is not really an error. I'm not sure what is the right thing to do in the test, check for overflow? Also, finfo has never worked properly for this type. Ted On Oct 23, 2006, at 05:50, Mark Hymers wrote: > On Thu, 19, Oct, 2006 at 08:29:26AM -0600, Travis Oliphant spoke > thus.. >> Actually, you shouldn't be getting an INF at all. This is what the >> test is designed to test for (so I guess it's working). The test was >> actually written wrong and was never failing because previously >> keyword >> arguments to ufuncs were ignored. >> >> Can you show us what 'a' is on your platform. > > Hi, > > I've just done a Mac OS X PPC build of the SVN trunk and am getting > this > failure too. > > nidesk046:~/scratch/upstream/scipy mark$ python > Python 2.4.1 (#2, Mar 31 2005, 00:05:10) > [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> import numpy as N >>>> N.__version__ > '1.0.dev3378' >>>> N.array([1000],dtype=N.float).dtype > dtype('float64') >>>> N.array([1000],dtype=N.longfloat).dtype > dtype('float128') >>>> N.test() > ...snip... > FAIL: Ticket #112 > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ > python2.4/site-packages/numpy/core/tests/test_regression.py", line > 220, in check_longfloat_repr > assert(str(a)[1:9] == str(a[0])[:8]) > AssertionError >>>> a = N.exp(N.array([1000],dtype=N.longfloat)) >>>> str(a) > '[inf]' > > > Any ideas about this? > > Mark > > -- > Mark Hymers > > "I once absent-mindedly ordered Three Mile Island dressing in a > restaurant > and, with great presence of mind, they brought Thousand Island > Dressing and > a bottle of chili sauce." > Terry Pratchett, alt.fan.pratchett > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642______________________________ > _________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From a.h.jaffe at gmail.com Thu Oct 26 10:46:50 2006 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Thu, 26 Oct 2006 15:46:50 +0100 Subject: Still a ticket 112 "check_longfloat_repr" issue on OSX In-Reply-To: <453FD6FD.70709@gmail.com> References: <453FD6FD.70709@gmail.com> Message-ID: <4540CA5A.5080607@gmail.com> Andrew Jaffe wrote: > OS X 10.4, PPC > > In [5]: import numpy as N > > In [6]: N.__version__ > Out[6]: '1.0.1.dev3399' > > In [7]: print N.longfloat(0).itemsize > 16 > > In [8]: a = N.exp(N.array([1000],dtype=N.longfloat)) > > In [9]: print str(a) > [inf] > > In [10]: print str(a[0]) > inf > > In [11]: print a.itemsize > 16 Are others still seeing this at this point, or just me? A ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From yves.frederix at gmail.com Thu Oct 26 10:50:06 2006 From: yves.frederix at gmail.com (Yves) Date: Thu, 26 Oct 2006 16:50:06 +0200 Subject: Behaviour of numpy.random.shuffle Message-ID: <20061026145006.GA15978@kotnet.org> Hi, It seems that random.shuffle does not 'shuffle' anymore when presented with an n-dimensional array. In [1]: import numpy as N In [2]: a = N.arange(5).reshape((5,1)) In [3]: a Out[3]: array([[0], [1], [2], [3], [4]]) In [4]: N.random.shuffle(a) In [5]: a Out[5]: array([[0], [1], [1], [3], [3]]) In [6]: N.__version__ Out[6]: '1.0.dev3390' After application some rows appear to be duplicated. Is this documented/intended behaviour? Cheers, YVES ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From chanley at stsci.edu Thu Oct 26 11:05:30 2006 From: chanley at stsci.edu (Christopher Hanley) Date: Thu, 26 Oct 2006 11:05:30 -0400 (EDT) Subject: Still a ticket 112 "check_longfloat_repr" issue on OSX Message-ID: <20061026110530.CLV71732@comet.stsci.edu> I just tried this on one of our G4's and see the same behavior. Chris ---- Original message ---- >Date: Thu, 26 Oct 2006 15:46:50 +0100 >From: Andrew Jaffe >Subject: Re: [Numpy-discussion] Still a ticket 112 "check_longfloat_repr" issue on OSX >To: Discussion of Numerical Python > >Andrew Jaffe wrote: >> OS X 10.4, PPC >> >> In [5]: import numpy as N >> >> In [6]: N.__version__ >> Out[6]: '1.0.1.dev3399' >> >> In [7]: print N.longfloat(0).itemsize >> 16 >> >> In [8]: a = N.exp(N.array([1000],dtype=N.longfloat)) >> >> In [9]: print str(a) >> [inf] >> >> In [10]: print str(a[0]) >> inf >> >> In [11]: print a.itemsize >> 16 > >Are others still seeing this at this point, or just me? > >A ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 26 11:20:34 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 26 Oct 2006 09:20:34 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> Message-ID: On 10/26/06, Ted Horst wrote: > > On Mac OS X tiger (10.4) ppc, long double has increased precision but > the same range as double (it really is 128 bits not 80, btw), Looks like you are right: http://www.freestandards.org/spec/ELF/ppc64/PPC-elf64abi-1.9.html#PREC Actually two doubles with some strange rules, the quad value is the sum. Hmmm, I can see why compiler support was problematic. The extended precision also has some oddities. This "Extended precision" differs from the IEEE 754 Standard in the following ways: - The software support is restricted to round-to-nearest mode. Programs that use extended precision must ensure that this rounding mode is in effect when extended-precision calculations are performed. - Does not fully support the IEEE special numbers NaN and INF. These values are encoded in the high-order double value only. The low-order value is not significant. - Does not support the IEEE status flags for overflow, underflow, and other conditions. These flag have no meaning in this format. so > e**1000 is inf, so this is not really an error. > > I'm not sure what is the right thing to do in the test, check for > overflow? Also, finfo has never worked properly for this type. See above. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Thu Oct 26 12:14:33 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 26 Oct 2006 10:14:33 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> Message-ID: <4540DEE9.6070304@ieee.org> Ted Horst wrote: > On Mac OS X tiger (10.4) ppc, long double has increased precision but > the same range as double (it really is 128 bits not 80, btw), so > e**1000 is inf, so this is not really an error. > > Thanks for the clarification. Long-double is not standard across platforms with different platforms choosing to do different things with the extra bytes. This helps explain one more platform. > I'm not sure what is the right thing to do in the test, check for > overflow? Also, finfo has never worked properly for this type. > In machar.py is the code that runs to detect all of the parameters. I think the code should be moved to C as detecting precision on a long-double takes too long. The overflow check is a good idea. The test should probably check for overflow and not try to run if it's detected. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 26 13:08:46 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 26 Oct 2006 11:08:46 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: <4540DEE9.6070304@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> <4540DEE9.6070304@ieee.org> Message-ID: On 10/26/06, Travis Oliphant wrote: > > Ted Horst wrote: > > On Mac OS X tiger (10.4) ppc, long double has increased precision but > > the same range as double (it really is 128 bits not 80, btw), so > > e**1000 is inf, so this is not really an error. > > > > > > Thanks for the clarification. Long-double is not standard across > platforms with different platforms choosing to do different things with > the extra bytes. This helps explain one more platform. > > > I'm not sure what is the right thing to do in the test, check for > > overflow? Also, finfo has never worked properly for this type. > > > In machar.py is the code that runs to detect all of the parameters. I > think the code should be moved to C as detecting precision on a > long-double takes too long. > > The overflow check is a good idea. The test should probably check for > overflow and not try to run if it's detected. How to check overflow? According to the documentation the flag is not set by the hardware. And the precision is variable! Somewhere in the neighborhood of 31 decimal digits, more or less, depending. So I think it is hard to figure out what to do here. Chuck. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Thu Oct 26 13:24:25 2006 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Thu, 26 Oct 2006 11:24:25 -0600 Subject: [ANN] NumPy 1.0 release Message-ID: <4540EF49.1040605@ieee.org> We are very pleased to announce the release of NumPy 1.0 available for download at http://www.numpy.org This release is the culmination of over 18 months of effort to allow unification of the Numeric and Numarray communities. NumPy provides the features of both packages as well as comparable speeds in the domains where both were considered fast --- often beating both packages on certain problems. If there is an area where we can speed up NumPy then we are interested in hearing about the solution. NumPy is essentially a re-write of Numeric to include the features of Numarray plus more. NumPy is a C-based extension module to Python that provides an N-dimensional array object (ndarray), a collection of fast math functions, basic linear algebra, array-producing random number generators, and basic Fourier transform capabilities. Also included with NumPy are: 1) A data-type object. The data-type of all NumPy arrays are defined by a data-type object that describes how the block of memory that makes up an element of the array is to be interpreted. Supported are all basic C-types, structures containing C-types, arrays of C-types, and structures containing structures of C-types. Data-types can also be in big or little-endian order. NumPy arrays can therefore be constructed from any regularly-sized chunk of data. A chunk of data can also be a pointer to a Python object and therefore Object arrays can be constructed (including record arrays with object members). 2) Array scalars: there is a Python scalar object (inheriting from the standard object where possible) defined for every basic data-type that an array can have. 2) A matrix object so that '*' is re-defined as matrix-multiplication and '**' as matrix-power. 3) A character array object that can replace Numarray's similarly-named object. It is basically an array of strings (or unicode) with methods matching the string and unicode methods. 4) A record array that builds on the advanced data-type support of the basic array object to allow field access using attribute look-up as well as to provide more ways to build-up a record-array. 5) A memory-map object that makes it easier to use memory-mapped areas as the memory for an array object. 6) A basic container class that uses the ndarray as a member. This often facilitates multiple-inheritance. 7) A large collection of basic functions on the array. 8) Compatibility layer for Numeric including code to help in the conversion to NumPy and full C-API support. 9) Compatibility layer for NumPy including code to help in the conversion to NumPy and full C-API support. NumPy can work with Numeric and Numarray installed and while the three array objects are different to Python, they can all share each other's data through the use of the array interface. As the developers for Numeric we can definitively say development of Numeric has ceased as has effective support. You may still find an answer to a question or two and Numeric will be available for download as long as Sourceforge is around so and code written to Numeric will still work, but there will not be "official" releases of Numeric for future versions of Python (including Python2.5). The development of NumPy has been supported by the people at STScI who created Numarray and support it. They have started to port their applications to NumPy and have indicated that support for Numarray will be phased out over the next year. You are strongly encouraged to move to NumPy. The whole point of NumPy is to unite the Numeric/Numarray development and user communities. We have done our part in releasing NumPy 1.0 and doing our best to make the transistion as easy as possible. Please support us by adopting NumPy. If you have trouble with that, please let us know why so that we can address the problems you identify. Even better, help us in fixing the problems. New users should download NumPy first unless they need an older package to work with third party code. Third-party package writers should migrate to use NumPy. Though it is not difficult, there are some things that have to be altered. Several people are available to help with that process, just ask (we will do it free for open source code and as work-for-hire for commercial code). This release would not have been possible without the work of many people. Thanks go to (if we have missed your contribution please let us know): * Travis Oliphant for the majority of the code adaptation (blame him for code problems :-) ) * Jim Hugunin, Paul Dubois, Konrad Hinsen, David Ascher, Jim Fulton and many others for Numeric on which the code is based. * Perry Greenfield, J Todd Miller, Rick White, Paul Barrett for Numarray which gave much inspiration and showed the way forward. * Paul Dubois for Masked Arrays * Pearu Peterson for f2py and numpy.distutils and help with code organization * Robert Kern for mtrand, bug fixes, help with distutils, code organization, and much more. * David Cooke for many code improvements including the auto-generated C-API and optimizations. * Alexander Belopolsky (Sasha) for Masked array bug-fixes and tests, rank-0 array improvements, scalar math help and other code additions * Francesc Altet for unicode and nested record tests and much help with rooting out nested record array bugs. * Tim Hochberg for getting the build working on MSVC, optimization improvements, and code review * Charles Harris for the sorting code originally written for Numarray and for improvements to polyfit, many bug fixes, and documentation strings. * Robert Cimrman for numpy.distutils help and the set-operations for arrays * David Huard for histogram code improvements including 2-d and d-d code * Eric Jones for sundry subroutines borrowed from scipy_base * Fernando Perez for code snippets, ideas, bugfixes, and testing. * Ed Schofield for matrix.py patches, bugfixes, testing, and docstrings. * John Hunter for code snippets (from matplotlib) * Chris Hanley for help with records.py, testing, and bug fixes. * Travis Vaught, Joe Cooper, Jeff Strunk for administration of numpy.org web site and SVN * Andrew Straw for bug-reports and help with www.scipy.org * Albert Strasheim for bug-reports, unit-testing and Valgrind runs * Stefan van der Walt for bug-reports, regression-testing, and bug-fixes. * Eric Firing for bugfixes. * Arnd Baecker for 64-bit testing * A.M. Archibald for code that decreases the number of times reshape makes a copy. More information is available at http://numpy.scipy.org and http://www.scipy.org. Bug-reports and feature requests should be submitted as tickets to the Trac pages at http://projects.scipy.org/scipy/numpy/ As an anti-SPAM measure, you must create an account in order to post tickets. Enjoy the new release, Sincerely, The NumPy Developers *Disclaimer*: The main author, Travis Oliphant, has written a 350+ page book entitled "Guide to NumPy" that documents the new system fairly thoroughly. The first two chapters of this book are available on-line for free, but the remainder must be purchased (until 2010 or a certain number of total sales has been reached). See http://www.trelgol.com for more details. There is plenty of free documentation available now for NumPy, however. Go to http://www.scipy.org for more details. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cicsxtxz at plco.shockware.com Thu Oct 26 13:41:40 2006 From: cicsxtxz at plco.shockware.com (pink border) Date: Fri, 27 Oct 2006 01:41:40 +0800 Subject: Gerald MatterSee Recovering Message-ID: <000b01c6f925$fe4d9000$08671f3c@cbc> Espaolmeet Study Guidesgift Cardshome Officedvd Setsmusic Setsprints Weeks Terrific Stephen King? Espaolmeet Study Guidesgift Cardshome Officedvd Setsmusic Setsprints Weeks Terrific Stephen King? Birthday either George Washington am or Abraham Lincoln Originally given patriotic card ribbon stars! Named Edward Burkhalter designer called decorative in inserts came sizes framing well fireplaces am walls floors am prominent Crescent Building. Government Fast is Free or Ratesstore Statuseasy Returnsall Help Desk Topics use Copyright Privacy a Policycopy llc. Consumer psychology ie must Deals magazine less savings least price Shoessave mens am. Us Millers a Antiques of Online Shopping of new Items Listed September Postcards of Great Medium for am Special or Days Review the! Known Presidents celebrated third is Monday or February or where stands today While never achieved there or number st Patricks am found or place years century with girl or holding. Samuel would last produce of todays is artware ceramics offered wide variety shapes glazes grouped together pieces or similar forms treatment lines collectors today Dickens Ware line introduced. Thousands brands question Service us update provide here comments site everyone note unable respond directly or submitted via am needed feedback appreciate form Wheres of Stufftrack recent or Returnssee heres Returns Helpforgot password am. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Spokane.gif Type: image/gif Size: 7647 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Thu Oct 26 13:57:18 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 26 Oct 2006 11:57:18 -0600 Subject: [ANN] NumPy 1.0 release In-Reply-To: <4540EF49.1040605@ieee.org> References: <4540EF49.1040605@ieee.org> Message-ID: On 10/26/06, Travis E. Oliphant wrote: > > > We are very pleased to announce the release of NumPy 1.0 available for > download at http://www.numpy.org Congratulations, Travis. Numpy was a needed unification and involved an enormous amount of work, most of it your own. Thanks for taking this on. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Thu Oct 26 13:26:14 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Thu, 26 Oct 2006 11:26:14 -0600 Subject: Unit test error with numpy rc3 In-Reply-To: References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> <4540DEE9.6070304@ieee.org> Message-ID: <4540EFB6.6080504@ieee.org> Charles R Harris wrote: > > > On 10/26/06, *Travis Oliphant* > wrote: > > Ted Horst wrote: > > On Mac OS X tiger (10.4) ppc, long double has increased > precision but > > the same range as double (it really is 128 bits not 80, btw), so > > e**1000 is inf, so this is not really an error. > > > > > > Thanks for the clarification. Long-double is not standard across > platforms with different platforms choosing to do different things > with > the extra bytes. This helps explain one more platform. > > > I'm not sure what is the right thing to do in the test, check for > > overflow? Also, finfo has never worked properly for this type. > > > In machar.py is the code that runs to detect all of the > parameters. I > think the code should be moved to C as detecting precision on a > long-double takes too long. > > The overflow check is a good idea. The test should probably check for > overflow and not try to run if it's detected. > > > How to check overflow? According to the documentation the flag is not > set by the hardware. And the precision is variable! Somewhere in the > neighborhood of 31 decimal digits, more or less, depending. So I think > it is hard to figure out what to do here. Let's drop the test. Long-double is available but is not consistent across platforms and NumPy has done nothing to try and make it so. Thus, let's just let the user beware. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cjw at sympatico.ca Thu Oct 26 14:35:06 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu, 26 Oct 2006 14:35:06 -0400 Subject: Changes to bools under Numexpr In-Reply-To: <4540A2B1.9080509@carabos.com> References: <4540A2B1.9080509@carabos.com> Message-ID: <4540FFDA.1020006@sympatico.ca> Ivan Vilata i Balaguer wrote: > Hi all. The attached diff makes some changes to Numexpr support for > booleans. The changes and their rationale are below. > > 1. New ``True`` and ``False`` boolean constants. This is so that 1 and > 0 are always proper integer constants. It is also for completeness, > but I don't envision any usage for them that couldn't be expressed > without the constants. > I'm puzzled. Python already has constants True and False of the bool type. bool is a subclass of the int type. Any instance of the bool type can be converted to the int type. >>> a=1==0 >>> type(a) >>> int(a) 0 >>> a False >>> Colin W. > 2. The only comparisons supported with booleans are ``==`` and ``!=``, > so that you can compare boolean variables. Just as NumPy supports > complex order comparisons and Numexpr doesn't, so goes for bools. > Being relatively new, I think there is no need to keep > integer-boolean compatibility in Numexpr. What was the meaning of > ``True > False`` or ``2 > True`` anyway? > > 3. This is also why casting booleans to normal numbers is not allowed, > so ``prod()`` and ``sum()`` on booleans aren't either. What is the > meaning of boolean addition anyway? > > To make it short, this patch is kind of a strengthening of boolean > values. I expect some people to disagree with the changes, and that's > why I would really like to hear your opinions on it. > > Regards, > > :: > > Ivan Vilata i Balaguer >qo< http://www.carabos.com/ > C??rabos Coop. V. V V Enjoy Data > "" > > ------------------------------------------------------------------------ > > Index: interp_body.c > =================================================================== > --- interp_body.c (revisi??n: 2299) > +++ interp_body.c (copia de trabajo) > @@ -130,21 +130,22 @@ > ci_dest = c1i); > > case OP_INVERT_BB: VEC_ARG1(b_dest = !b1); > + case OP_AND_BBB: VEC_ARG2(b_dest = (b1 && b2)); > + case OP_OR_BBB: VEC_ARG2(b_dest = (b1 || b2)); > > - case OP_AND_BBB: VEC_ARG2(b_dest = b1 && b2); > - case OP_OR_BBB: VEC_ARG2(b_dest = b1 || b2); > + case OP_EQ_BBB: VEC_ARG2(b_dest = (b1 == b2)); > + case OP_NE_BBB: VEC_ARG2(b_dest = (b1 != b2)); > > - case OP_GT_BII: VEC_ARG2(b_dest = (i1 > i2) ? 1 : 0); > - case OP_GE_BII: VEC_ARG2(b_dest = (i1 >= i2) ? 1 : 0); > - case OP_EQ_BII: VEC_ARG2(b_dest = (i1 == i2) ? 1 : 0); > - case OP_NE_BII: VEC_ARG2(b_dest = (i1 != i2) ? 1 : 0); > + case OP_GT_BII: VEC_ARG2(b_dest = (i1 > i2)); > + case OP_GE_BII: VEC_ARG2(b_dest = (i1 >= i2)); > + case OP_EQ_BII: VEC_ARG2(b_dest = (i1 == i2)); > + case OP_NE_BII: VEC_ARG2(b_dest = (i1 != i2)); > > - case OP_GT_BFF: VEC_ARG2(b_dest = (f1 > f2) ? 1 : 0); > - case OP_GE_BFF: VEC_ARG2(b_dest = (f1 >= f2) ? 1 : 0); > - case OP_EQ_BFF: VEC_ARG2(b_dest = (f1 == f2) ? 1 : 0); > - case OP_NE_BFF: VEC_ARG2(b_dest = (f1 != f2) ? 1 : 0); > + case OP_GT_BFF: VEC_ARG2(b_dest = (f1 > f2)); > + case OP_GE_BFF: VEC_ARG2(b_dest = (f1 >= f2)); > + case OP_EQ_BFF: VEC_ARG2(b_dest = (f1 == f2)); > + case OP_NE_BFF: VEC_ARG2(b_dest = (f1 != f2)); > > - case OP_CAST_IB: VEC_ARG1(i_dest = (long)b1); > case OP_ONES_LIKE_II: VEC_ARG1(i_dest = 1); > case OP_NEG_II: VEC_ARG1(i_dest = -i1); > > @@ -157,7 +158,6 @@ > > case OP_WHERE_IFII: VEC_ARG3(i_dest = f1 ? i2 : i3); > > - case OP_CAST_FB: VEC_ARG1(f_dest = (long)b1); > case OP_CAST_FI: VEC_ARG1(f_dest = (double)(i1)); > case OP_ONES_LIKE_FF: VEC_ARG1(f_dest = 1.0); > case OP_NEG_FF: VEC_ARG1(f_dest = -f1); > @@ -180,8 +180,6 @@ > case OP_FUNC_FF: VEC_ARG1(f_dest = functions_f[arg2](f1)); > case OP_FUNC_FFF: VEC_ARG2(f_dest = functions_ff[arg3](f1, f2)); > > - case OP_CAST_CB: VEC_ARG1(cr_dest = (double)b1; > - ci_dest = 0); > case OP_CAST_CI: VEC_ARG1(cr_dest = (double)(i1); > ci_dest = 0); > case OP_CAST_CF: VEC_ARG1(cr_dest = f1; > @@ -203,8 +201,8 @@ > ci_dest = (c1i*c2r - c1r*c2i) / fa; > cr_dest = fb); > > - case OP_EQ_BCC: VEC_ARG2(b_dest = (c1r == c2r && c1i == c2i) ? 1 : 0); > - case OP_NE_BCC: VEC_ARG2(b_dest = (c1r != c2r || c1i != c2i) ? 1 : 0); > + case OP_EQ_BCC: VEC_ARG2(b_dest = (c1r == c2r && c1i == c2i)); > + case OP_NE_BCC: VEC_ARG2(b_dest = (c1r != c2r || c1i != c2i)); > > case OP_WHERE_CFCC: VEC_ARG3(cr_dest = f1 ? c2r : c3r; > ci_dest = f1 ? c2i : c3i); > Index: compiler.py > =================================================================== > --- compiler.py (revisi??n: 2299) > +++ compiler.py (copia de trabajo) > @@ -217,6 +217,10 @@ > for name in c.co_names: > if name == "None": > names[name] = None > + elif name == "True": > + names[name] = True > + elif name == "False": > + names[name] = False > else: > t = types.get(name, float) > names[name] = expr.VariableNode(name, type_to_kind[t]) > Index: tests/test_numexpr.py > =================================================================== > --- tests/test_numexpr.py (revisi??n: 2299) > +++ tests/test_numexpr.py (copia de trabajo) > @@ -67,10 +67,6 @@ > x = x + 5j > assert_equal(evaluate("sum(x**2+2,axis=0)"), sum(x**2+2,axis=0)) > assert_equal(evaluate("prod(x**2+2,axis=0)"), prod(x**2+2,axis=0)) > - # Check boolean (should cast to integer) > - x = (arange(10) % 2).astype(bool) > - assert_equal(evaluate("prod(x,axis=0)"), prod(x,axis=0)) > - assert_equal(evaluate("sum(x,axis=0)"), sum(x,axis=0)) > > def check_axis(self): > y = arange(9.0).reshape(3,3) > Index: interpreter.c > =================================================================== > --- interpreter.c (revisi??n: 2299) > +++ interpreter.c (copia de trabajo) > @@ -25,6 +25,9 @@ > OP_AND_BBB, > OP_OR_BBB, > > + OP_EQ_BBB, > + OP_NE_BBB, > + > OP_GT_BII, > OP_GE_BII, > OP_EQ_BII, > @@ -35,7 +38,6 @@ > OP_EQ_BFF, > OP_NE_BFF, > > - OP_CAST_IB, > OP_COPY_II, > OP_ONES_LIKE_II, > OP_NEG_II, > @@ -47,7 +49,6 @@ > OP_MOD_III, > OP_WHERE_IFII, > > - OP_CAST_FB, > OP_CAST_FI, > OP_COPY_FF, > OP_ONES_LIKE_FF, > @@ -70,7 +71,6 @@ > OP_EQ_BCC, > OP_NE_BCC, > > - OP_CAST_CB, > OP_CAST_CI, > OP_CAST_CF, > OP_ONES_LIKE_CC, > @@ -115,6 +115,8 @@ > break; > case OP_AND_BBB: > case OP_OR_BBB: > + case OP_EQ_BBB: > + case OP_NE_BBB: > if (n == 0 || n == 1 || n == 2) return 'b'; > break; > case OP_GT_BII: > @@ -131,10 +133,6 @@ > if (n == 0) return 'b'; > if (n == 1 || n == 2) return 'f'; > break; > - case OP_CAST_IB: > - if (n == 0) return 'i'; > - if (n == 1) return 'b'; > - break; > case OP_COPY_II: > case OP_ONES_LIKE_II: > case OP_NEG_II: > @@ -152,10 +150,6 @@ > if (n == 0 || n == 2 || n == 3) return 'i'; > if (n == 1) return 'f'; > break; > - case OP_CAST_FB: > - if (n == 0) return 'f'; > - if (n == 1) return 'b'; > - break; > case OP_CAST_FI: > if (n == 0) return 'f'; > if (n == 1) return 'i'; > @@ -194,10 +188,6 @@ > if (n == 0) return 'b'; > if (n == 1 || n == 2) return 'c'; > break; > - case OP_CAST_CB: > - if (n == 0) return 'c'; > - if (n == 1) return 'b'; > - break; > case OP_CAST_CI: > if (n == 0) return 'c'; > if (n == 1) return 'i'; > @@ -1296,6 +1286,10 @@ > add_op("invert_bb", OP_INVERT_BB); > add_op("and_bbb", OP_AND_BBB); > add_op("or_bbb", OP_OR_BBB); > + > + add_op("eq_bbb", OP_EQ_BBB); > + add_op("ne_bbb", OP_NE_BBB); > + > add_op("gt_bii", OP_GT_BII); > add_op("ge_bii", OP_GE_BII); > add_op("eq_bii", OP_EQ_BII); > @@ -1306,7 +1300,6 @@ > add_op("eq_bff", OP_EQ_BFF); > add_op("ne_bff", OP_NE_BFF); > > - add_op("cast_ib", OP_CAST_IB); > add_op("ones_like_ii", OP_ONES_LIKE_II); > add_op("copy_ii", OP_COPY_II); > add_op("neg_ii", OP_NEG_II); > @@ -1318,7 +1311,6 @@ > add_op("mod_iii", OP_MOD_III); > add_op("where_ifii", OP_WHERE_IFII); > > - add_op("cast_fb", OP_CAST_FB); > add_op("cast_fi", OP_CAST_FI); > add_op("copy_ff", OP_COPY_FF); > add_op("ones_like_ff", OP_ONES_LIKE_FF); > @@ -1342,7 +1334,6 @@ > add_op("eq_bcc", OP_EQ_BCC); > add_op("ne_bcc", OP_NE_BCC); > > - add_op("cast_cb", OP_CAST_CB); > add_op("cast_ci", OP_CAST_CI); > add_op("cast_cf", OP_CAST_CF); > add_op("copy_cc", OP_COPY_CC); > Index: expressions.py > =================================================================== > --- expressions.py (revisi??n: 2299) > +++ expressions.py (copia de trabajo) > @@ -62,7 +62,13 @@ > return kind_rank[n] > > def bestConstantType(x): > - for converter in bool, int, float, complex: > + # Numeric conversion to boolean values is not tried because > + # ``bool(1) == True`` (same for 0 and False), so 0 and 1 would be > + # interpreted as booleans when ``False`` and ``True`` are already > + # supported. > + if isinstance(x, bool): > + return bool > + for converter in int, float, complex: > try: > y = converter(x) > except StandardError, err: > @@ -122,10 +128,7 @@ > return a > if isinstance(a, (bool, int, float, complex)): > a = ConstantNode(a) > - kind = a.astKind > - if kind == 'bool': > - kind = 'int' > - return FuncNode('sum', [a, axis], kind=kind) > + return FuncNode('sum', [a, axis], kind=a.astKind) > > def prod_func(a, axis=-1): > axis = encode_axis(axis) > @@ -133,10 +136,7 @@ > a = ConstantNode(a) > if isinstance(a, ConstantNode): > return a > - kind = a.astKind > - if kind == 'bool': > - kind = 'int' > - return FuncNode('prod', [a, axis], kind=kind) > + return FuncNode('prod', [a, axis], kind=a.astKind) > > @ophelper > def div_op(a, b): > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From erin.sheldon at gmail.com Thu Oct 26 14:47:36 2006 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Thu, 26 Oct 2006 14:47:36 -0400 Subject: [ANN] NumPy 1.0 release In-Reply-To: References: <4540EF49.1040605@ieee.org> Message-ID: <331116dc0610261147g2dfc7e35yfd6bf74abf66f67@mail.gmail.com> Hi all - I think I speak for many when I say that this is a huge step for those who have desired to switch to numerical python from other languages (IDL, MATLAB, etc) but have been waiting for that 1.0 release. Many thanks to everyone involved. Erin Sheldon On 10/26/06, Charles R Harris wrote: > > > On 10/26/06, Travis E. Oliphant wrote: > > > > We are very pleased to announce the release of NumPy 1.0 available for > > download at http://www.numpy.org > > Congratulations, Travis. Numpy was a needed unification and involved an > enormous amount of work, most of it your own. Thanks for taking this on. > > Chuck > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jontwang at gmail.com Thu Oct 26 18:26:47 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Thu, 26 Oct 2006 17:26:47 -0500 Subject: Defining custom types Message-ID: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> I'm trying to write a Numpy extension that will encapsulate mxDateTime as a native Numpy type. I've decided to use a type inherited from Numpy's scalar double. However, I'm running into all sorts of problems. I'm using numpy 1.0b5; I realize this is somewhat out of date. For all the examples below, assume that I've created a 1x1 array, mxArr, with my custom type. The interface used by Array_FromPyScalar does not conform with the documentation's claim that a negative return value indicates an error. The return code from setitem is not checked. Instead, the code depends on a Python error being set. I seem to be able to load values into the array, but I can't extract anything out of the array, even to print it. In gdb I've verified that loading DateTime.now() correctly puts a float representation of the date into my array. However, if I try to get the value out, I get an error: >>> mxArr[0] = DateTime.now() >>> mxArr[0] Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", line 391, in array_repr ', ', "array(") File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", line 204, in array2string separator, prefix) File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", line 160, in _array2string format = _floatFormat(data, precision, suppress_small) File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", line 281, in _floatFormat non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0))) TypeError: bad operand type for abs() I'm not sure why it's trying to call abs() on my object to print it. I have a separate PyNumberMethods attached to my object type, copied from the float scalar type, and nb_absolute is set to 0. When I break at the various functions I've registered, the last thing Numpy tries to do is cast my custom data type to an object type (which it does so successfully) via _broadcast_cast. Thanks, Jonathan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant at ee.byu.edu Thu Oct 26 19:19:11 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 26 Oct 2006 17:19:11 -0600 Subject: Defining custom types In-Reply-To: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> Message-ID: <4541426F.2070108@ee.byu.edu> Jonathan Wang wrote: > I'm trying to write a Numpy extension that will encapsulate mxDateTime > as a native Numpy type. I've decided to use a type inherited from > Numpy's scalar double. However, I'm running into all sorts of > problems. I'm using numpy 1.0b5; I realize this is somewhat out of date. > Cool. The ability to create your own data-types (and define ufuncs for them) is a feature that I'd like to see explored. But, it has not received a lot of attention and so you may find bugs along the way. We'll try to fix them quickly as they arise (and there will be bug fix releases for 1.0). But, what do you mean "inheriting" from NumPy's double for your scalar data-type. This has significant implications. To define a new data-type object (that doesn't build from the VOID data-type), you need to flesh out the PyArray_Descr * structure and this can only be done in C. Perhaps you are borrowing most entries in the structure builtin double type and then filling in a few differently like setitem and getitem? Is that accurate? > For all the examples below, assume that I've created a 1x1 array, > mxArr, with my custom type. > > The interface used by Array_FromPyScalar does not conform with the > documentation's claim that a negative return value indicates an error. You must be talking about a different function. Array_FromPyScalar is an internal function and not a C-API call. It also returns a PyObject * not an integer. So, which function are you actually referring to? > The return code from setitem is not checked. Instead, the code depends > on a Python error being set. This may be true, but how is it a problem? > > I seem to be able to load values into the array, but I can't extract > anything out of the array, even to print it. In gdb I've verified that > loading DateTime.now() correctly puts a float representation of the > date into my array. However, if I try to get the value out, I get an > error: > >>> mxArr[0] = DateTime.now() > >>> mxArr[0] > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", line > 391, in array_repr > ', ', "array(") > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > line 204, in array2string > separator, prefix) > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > line 160, in _array2string > format = _floatFormat(data, precision, suppress_small) > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > line 281, in _floatFormat > non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0))) > TypeError: bad operand type for abs() > > I'm not sure why it's trying to call abs() on my object to print it. Because that's the implication of inheriting from a double. It's just part of the code that tries to format your values into an array (notice the _floatFormat). I actually borrowed this code from numarray so I can't speak to exactly what it's doing without more study. > I have a separate PyNumberMethods attached to my object type, copied > from the float scalar type, and nb_absolute is set to 0. When I break > at the various functions I've registered, the last thing Numpy tries > to do is cast my custom data type to an object type (which it does so > successfully) via _broadcast_cast. Don't confuse the Python object associated when an element of the array is extracted and the data-type of the array. Also don't confuse the PyNumberMethods of the scalar object with the ufuncs. Defining PyNumberMethods won't usually give you the ability to calculate ufuncs. Perhaps you just want to construct an "object" array of mxDateTime's. What is the reason you want to define an mxDateTime data-type? -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jontwang at gmail.com Thu Oct 26 19:37:42 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Thu, 26 Oct 2006 18:37:42 -0500 Subject: Defining custom types In-Reply-To: <4541426F.2070108@ee.byu.edu> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> Message-ID: <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> On 10/26/06, Travis Oliphant wrote: > > But, what do you mean "inheriting" from NumPy's double for your scalar > data-type. This has significant implications. To define a new > data-type object (that doesn't build from the VOID data-type), you need > to flesh out the PyArray_Descr * structure and this can only be done in > C. Perhaps you are borrowing most entries in the structure builtin > double type and then filling in a few differently like setitem and > getitem? Is that accurate? Sorry, I should have been clearer. When I talk about inheritance, I mean of the type underlying the array. For example, the built-in scalar double array has an underlying type of PyDoubleArrType_Type. My underlying type is a separate PyTypeObject. The interesting changes here are to tp_repr, tp_str, and tp_as_number. The rest of the fields are inherited from PyDoubleArrType_Type using the tp_base field. The array itself has another statically defined type object of type PyArray_Descr, which I'm creating with a PyObject_New call and filling in with many of the entries from the descriptor returned by PyArray_DescrFromType(NPY_DOUBLE), while overriding getitem and setitem to handle PyObject* of type mxDateTime as you guessed. > The interface used by Array_FromPyScalar does not conform with the > > documentation's claim that a negative return value indicates an error. > > You must be talking about a different function. Array_FromPyScalar is > an internal function and not a C-API call. It also returns a PyObject * > not an integer. So, which function are you actually referring to? > > > The return code from setitem is not checked. Instead, the code depends > > on a Python error being set. This may be true, but how is it a problem? > It's just confusing as the documentation indicates that the setitem function should return 0 for success and a negative number for failure. But within Array_FromPyScalar, we have: ret->descr->f->setitem(op, ret->data, ret); if (PyErr_Occurred()) { Py_DECREF(ret); return NULL; } else { return (PyObject *)ret; } So, someone reading the documentation could return -1 on failure without setting the Python error flag, and the function would happily continue on its way and fail to perform the proper casts. > > > I seem to be able to load values into the array, but I can't extract > > anything out of the array, even to print it. In gdb I've verified that > > loading DateTime.now() correctly puts a float representation of the > > date into my array. However, if I try to get the value out, I get an > > error: > > >>> mxArr[0] = DateTime.now() > > >>> mxArr[0] > > Traceback (most recent call last): > > File "", line 1, in ? > > File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", line > > 391, in array_repr > > ', ', "array(") > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 204, in array2string > > separator, prefix) > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 160, in _array2string > > format = _floatFormat(data, precision, suppress_small) > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 281, in _floatFormat > > non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0))) > > TypeError: bad operand type for abs() > > > > I'm not sure why it's trying to call abs() on my object to print it. > > Because that's the implication of inheriting from a double. It's just > part of the code that tries to format your values into an array (notice > the _floatFormat). I actually borrowed this code from numarray so I > can't speak to exactly what it's doing without more study. Hmm, so does Numpy ignore the tp_repr and tp_str fields in the PyTypeObject of the underlying type? I admittedly haven't had a chance to look at this code closely yet. > I have a separate PyNumberMethods attached to my object type, copied > > from the float scalar type, and nb_absolute is set to 0. When I break > > at the various functions I've registered, the last thing Numpy tries > > to do is cast my custom data type to an object type (which it does so > > successfully) via _broadcast_cast. > > Don't confuse the Python object associated when an element of the array > is extracted and the data-type of the array. Also don't confuse the > PyNumberMethods of the scalar object with the ufuncs. Defining > PyNumberMethods won't usually give you the ability to calculate ufuncs. Okay, is my understanding here correct? I am defining two type descriptors: PyArray_Descr mxNumpyType - describes the Numpy array type. PyTypeObject mxNumpyDataType - describes the data type of the contents of the array (i.e. mxNumpyType->typeobj points to this), inherits from PyDoubleArrType_Type and overrides some fields as mentioned above. And the getitem and setitem functions are designed to only give/take PyObject* of type mxDateTime. I guess it's not clear to me whether the abs() referred to by the error is an abs() ufunc or the nb_absolute pointer in the PyNumberMethods. Let me try overriding ufuncs and get back to you... Perhaps you just want to construct an "object" array of mxDateTime's. > What is the reason you want to define an mxDateTime data-type? Currently I am using an object array of mxDateTime's, but it's rather frustrating that I can't treat them as normal floats internally since that's really all they are. Jonathan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Thu Oct 26 19:49:25 2006 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 26 Oct 2006 18:49:25 -0500 Subject: Defining custom types In-Reply-To: <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> Message-ID: Jonathan Wang wrote: > It's just confusing as the documentation indicates that the setitem > function should return 0 for success and a negative number for failure. > But within Array_FromPyScalar, we have: > > ret->descr->f->setitem(op, ret->data, ret); > > if (PyErr_Occurred()) { > Py_DECREF(ret); > return NULL; > } else { > return (PyObject *)ret; > } > > So, someone reading the documentation could return -1 on failure without > setting the Python error flag, and the function would happily continue > on its way and fail to perform the proper casts. That's a documentation vagueness, then. This is a convention established by the Python C API. If an error happens in a function that returns PyObject*, then it should return NULL to inform the caller that an error happened; other functions should return 0 for success and -1 for an error. However, the function must still set an exception object. The rest is just a convenient convention. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From djm at mindrot.org Thu Oct 26 19:50:07 2006 From: djm at mindrot.org (Damien Miller) Date: Fri, 27 Oct 2006 09:50:07 +1000 (EST) Subject: numpy-1.0 regress failure on OpenBSD Message-ID: Hi, I have just got around to updating OpenBSD's numpy port from 1.0b1 to 1.0 and am running into the following regress failure: > .........................................................................................................................................................................................................................................................Warning: overflow encountered in exp > F....................................................................................................................................................................................................................................................................... > ====================================================================== > FAIL: Ticket #112 > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/ports/math/py-numpy/w-py-numpy-1.0/fake-i386/usr/local/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", line 220, in check_longfloat_repr > assert(str(a)[1:9] == str(a[0])[:8]) > AssertionError > > ---------------------------------------------------------------------- > Ran 513 tests in 3.471s > > FAILED (failures=1) The variable 'a' seems to contain '[Inf]', so the failure appears related to the overflow warning. Any clues on how I can debug this further? thanks, Damien Miller ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 26 19:56:39 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 26 Oct 2006 17:56:39 -0600 Subject: numpy-1.0 regress failure on OpenBSD In-Reply-To: References: Message-ID: <45414B37.8050402@ee.byu.edu> Damien Miller wrote: >Hi, > >I have just got around to updating OpenBSD's numpy port from 1.0b1 to >1.0 and am running into the following regress failure: > > > >>.........................................................................................................................................................................................................................................................Warning: overflow encountered in exp >>F....................................................................................................................................................................................................................................................................... >>====================================================================== >>FAIL: Ticket #112 >>---------------------------------------------------------------------- >>Traceback (most recent call last): >> File "/usr/ports/math/py-numpy/w-py-numpy-1.0/fake-i386/usr/local/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", line 220, in check_longfloat_repr >> assert(str(a)[1:9] == str(a[0])[:8]) >>AssertionError >> >>---------------------------------------------------------------------- >>Ran 513 tests in 3.471s >> >>FAILED (failures=1) >> >> > >The variable 'a' seems to contain '[Inf]', so the failure appears related to >the overflow warning. > >Any clues on how I can debug this further? > > Unless you want to help with tracking how long double is interpreted on several platforms, then just ignore the test. (It actually wasn't being run in 1.0b1). -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Thu Oct 26 20:15:08 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu, 26 Oct 2006 18:15:08 -0600 Subject: Defining custom types In-Reply-To: <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> Message-ID: <45414F8C.8090002@ee.byu.edu> > It's just confusing as the documentation indicates that the setitem > function should return 0 for success and a negative number for > failure. But within Array_FromPyScalar, we have: > > ret->descr->f->setitem(op, ret->data, ret); > > if (PyErr_Occurred()) { > Py_DECREF(ret); > return NULL; > } else { > return (PyObject *)ret; > } > I see the problem. We are assuming an error is set on failure, so both -1 should be returned and an error condition set for your own setitem function. This is typical Python behavior. I'll fix the documentation. > > > > > I seem to be able to load values into the array, but I can't extract > > anything out of the array, even to print it. In gdb I've > verified that > > loading DateTime.now() correctly puts a float representation of the > > date into my array. However, if I try to get the value out, I get an > > error: > > >>> mxArr[0] = DateTime.now() > > >>> mxArr[0] > > Traceback (most recent call last): > > File "", line 1, in ? > > File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", > line > > 391, in array_repr > > ', ', "array(") > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 204, in array2string > > separator, prefix) > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 160, in _array2string > > format = _floatFormat(data, precision, suppress_small) > > File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py", > > line 281, in _floatFormat > > non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0))) > > TypeError: bad operand type for abs() > > > > I'm not sure why it's trying to call abs() on my object to print it. > > Because that's the implication of inheriting from a double. It's > just > part of the code that tries to format your values into an array > (notice > the _floatFormat). I actually borrowed this code from numarray so I > can't speak to exactly what it's doing without more study. > > > Hmm, so does Numpy ignore the tp_repr and tp_str fields in the > PyTypeObject of the underlying type? admittedly haven't had a chance > to look at this code closely yet. How arrays print is actually user-settable. The default printing function does indeed ignore tp_repr and tp_str of the underlying scalar objects in order to be able to set precisions. Now, we could probably fix the default printing function to actually use the tp_repr and/or tp_str fields of the corresponding scalar objects. This is worth filing a ticket about. In the mean time you can create a new array print function that checks for your data-type as the type of the array and then does something different otherwise it calls the old function. Then, register this function as the print function for arrays. > > > > I have a separate PyNumberMethods attached to my object type, copied > > from the float scalar type, and nb_absolute is set to 0. When I > break > > at the various functions I've registered, the last thing Numpy > tries > > to do is cast my custom data type to an object type (which it > does so > > successfully) via _broadcast_cast. > > Don't confuse the Python object associated when an element of the > array > is extracted and the data-type of the array. Also don't confuse the > PyNumberMethods of the scalar object with the ufuncs. Defining > PyNumberMethods won't usually give you the ability to calculate > ufuncs. > > > Okay, is my understanding here correct? I am defining two type > descriptors: > PyArray_Descr mxNumpyType - describes the Numpy array type. > PyTypeObject mxNumpyDataType - describes the data type of the contents > of the array (i.e. mxNumpyType->typeobj points to this), inherits from > PyDoubleArrType_Type and overrides some fields as mentioned above. > The nomenclature is that mxNumPyType is the data-type of the array and your PyTypeObject is the "type" of the elements of the array. So, you have the names a bit backward. So, to correspond with the way I use the words "type" and "data-type", I would name them: PyArray_Descr mxNumpyDataType PyTypeObject mxNumpyType > And the getitem and setitem functions are designed to only give/take > PyObject* of type mxDateTime. > These are in the 'f' member of the PyArray_Descr structure, so presumably you have also filled in your PyArray_Descr structure with items from PyArray_DOUBLE? > I guess it's not clear to me whether the abs() referred to by the > error is an abs() ufunc or the nb_absolute pointer in the > PyNumberMethods. Let me try overriding ufuncs and get back to you... > > Perhaps you just want to construct an "object" array of mxDateTime's. > What is the reason you want to define an mxDateTime data-type? > > > Currently I am using an object array of mxDateTime's, but it's rather > frustrating that I can't treat them as normal floats internally since > that's really all they are. Ah, I see. So, you would like to be able to say view the array of mxDateTimes as an array of "floats" (using the .view method). You are correct that this doesn't make sense when you are talking about objects, but might if mxDateTime objects are really just floats. I just wanted to make sure you were aware of the object array route. The new data-type route is less well traveled but I'm anxious to smooth the wrinkles out. Your experiences will help. Bascially we are moving from Numeric being a "builtin data-types" only to a NumPy that has "arbitrary" data-types with a few special-cased "builtins" We need more experience to clarify the issues. Your identification of problems in the default printing, for example, is one thing that will help. Keep us posted. I'd love to here how things went and what can be done to improve. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ted.horst at earthlink.net Thu Oct 26 20:55:36 2006 From: ted.horst at earthlink.net (Ted Horst) Date: Thu, 26 Oct 2006 19:55:36 -0500 Subject: Unit test error with numpy rc3 In-Reply-To: <4540EFB6.6080504@ieee.org> References: <97670e910610190535u2aa1da25x2cdd756e8dd8194d@mail.gmail.com> <45377E6E.7040006@ieee.org> <45378BC6.70609@ieee.org> <20061023105027.GA6741@titus.mhy.org.uk> <8CF9293B-11B4-418B-8C22-050165586578@earthlink.net> <4540DEE9.6070304@ieee.org> <4540EFB6.6080504@ieee.org> Message-ID: <5FCC8177-0926-43CC-BD2B-05EE57FDC6B6@earthlink.net> On Oct 26, 2006, at 12:26, Travis Oliphant wrote: > Charles R Harris wrote: >> >> >> On 10/26/06, *Travis Oliphant* > > wrote: >> >> Ted Horst wrote: >>> On Mac OS X tiger (10.4) ppc, long double has increased >> precision but >>> the same range as double (it really is 128 bits not 80, btw), so >>> e**1000 is inf, so this is not really an error. >>> >>> >> >> Thanks for the clarification. Long-double is not standard >> across >> platforms with different platforms choosing to do different >> things >> with >> the extra bytes. This helps explain one more platform. >> >>> I'm not sure what is the right thing to do in the test, check for >>> overflow? Also, finfo has never worked properly for this type. >>> >> In machar.py is the code that runs to detect all of the >> parameters. I >> think the code should be moved to C as detecting precision on a >> long-double takes too long. >> >> The overflow check is a good idea. The test should probably >> check for >> overflow and not try to run if it's detected. >> >> >> How to check overflow? According to the documentation the flag is not >> set by the hardware. And the precision is variable! Somewhere in the >> neighborhood of 31 decimal digits, more or less, depending. So I >> think >> it is hard to figure out what to do here. > > Let's drop the test. Long-double is available but is not consistent > across platforms and NumPy has done nothing to try and make it so. > Thus, let's just let the user beware. > > -Travis Yeah, that seem like the thing to do. Just for completeness: >>> N.seterr(all = 'raise') >>> fa = N.array([1e308], dtype=N.float) >>> lfa = N.array([1e308], dtype=N.longfloat) >>> fa + fa Traceback (most recent call last): File "", line 1, in ? FloatingPointError: overflow encountered in add >>> lfa + lfa Traceback (most recent call last): File "", line 1, in ? FloatingPointError: overflow encountered in add >>> N.exp(fa) Traceback (most recent call last): File "", line 1, in ? FloatingPointError: overflow encountered in exp >>> N.exp(lfa) array([inf], dtype=float128) Ted ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Thu Oct 26 21:03:31 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Thu, 26 Oct 2006 18:03:31 -0700 Subject: distutils question Message-ID: <45415AE3.8080908@jpl.nasa.gov> Hi I am trying to compile a 64 bit version of numpy with gcc. When building, numpy tries to figure out the lapack/atlas version. Up to this point, everything has been compiled with gcc -m64 and all is groovy. But, when an attempt is made to get the atlas version, the link fails because the command "gcc _configtest.o -L/u/fuego0b/myeates/lib -llapack -lcblas -latlas -o _configtest" is being run (Note the lack of -m64) This generates an error ld: fatal: file _configtest.o: wrong ELF class: ELFCLASS64 ld: fatal: File processing errors. No output written to _configtest collect2: ld returned 1 exit status ld: fatal: file _configtest.o: wrong ELF class: ELFCLASS64 ld: fatal: File processing errors. No output written to _configtest collect2: ld returned 1 exit status LinkError: LinkErro...us 1',),) Anybody know how I can force gcc -m64 when linking? I already have the environment variables CFLAGS=-m64 LDFLAGS=-64 Mathew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From myeates at jpl.nasa.gov Thu Oct 26 21:42:52 2006 From: myeates at jpl.nasa.gov (Mathew Yeates) Date: Thu, 26 Oct 2006 18:42:52 -0700 Subject: build errors Solaris x86 64 bit python2.5 Message-ID: <4541641C.6050004@jpl.nasa.gov> yes, I got around the problem from my previous posting "distutils question". I added ld_args[:0] = ['-m64'] to line 209 of python2.5/distutils/unixcompiler.py. Lovely, yes I know. I now get an error "numpy/core/src/multiarraymodule.c:7230: error: `NPY_ALLOW_THREADS' undeclared". This is after several billion warning messages of the form numpy/core/src/multiarraymodule.c:5010: warning: dereferencing type-punned pointer will break strict-aliasing rules Any help, suggestions gratefully accepted. Mathew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Thu Oct 26 23:34:01 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 26 Oct 2006 21:34:01 -0600 Subject: build errors Solaris x86 64 bit python2.5 In-Reply-To: <4541641C.6050004@jpl.nasa.gov> References: <4541641C.6050004@jpl.nasa.gov> Message-ID: On 10/26/06, Mathew Yeates wrote: > > yes, I got around the problem from my previous posting "distutils > question". I added ld_args[:0] = ['-m64'] to line 209 of > python2.5/distutils/unixcompiler.py. Lovely, yes I know. > > I now get an error "numpy/core/src/multiarraymodule.c:7230: error: > `NPY_ALLOW_THREADS' undeclared". This is after several billion warning > messages of the form > numpy/core/src/multiarraymodule.c:5010: warning: dereferencing > type-punned pointer will break strict-aliasing rules GCC? Needs the -no-strict-aliasing flag. Everybody hates the default except the compiler writers because you can't cast pointers between different sized types, something the linux kernel and numpy do a lot. Things can fail badly if you don't set the flag and ignore the warnings. I don't know about the other warning, maybe some syntax error causing the declaration to be missed. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From george.sakkis at gmail.com Thu Oct 26 23:58:28 2006 From: george.sakkis at gmail.com (George Sakkis) Date: Fri, 27 Oct 2006 03:58:28 +0000 (UTC) Subject: numpy - scipy version hell References: <200610191314.54482.jkgruet@sandia.gov> Message-ID: Robert Kern gmail.com> writes: > It looks like you linked against a FORTRAN LAPACK, but didn't manage to link the > FORTRAN runtime library libg2c. Can you give us the output of your build? > I just installed Numpy, ATLAS and LAPACK on Centos a few hours ago and I got the exact same error. You're right, libg2c is never linked. Here's a sample line from the linking: gcc -pthread -shared build/temp.linux-i686-2.4/numpy/linalg/lapack_litemodule.o -L/usr/local/lib/atlas/ -llapack -lptf77blas -lptcblas -latlas -o build/lib.linux-i686-2.4/numpy/linalg/lapack_lite.so I've been looking into numpy's distutils for the last hour or so but didn't track down the problem yet; something seems to be broken with Redhat's setup... George ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jk985 at tom.com Fri Oct 27 00:24:19 2006 From: jk985 at tom.com (=?GB2312?B?IjEx1MI0LTXI1S+xsb6pIg==?=) Date: Fri, 27 Oct 2006 04:24:19 +0000 (UTC) Subject: =?GB2312?B?RVhDRUy6zVBQVNTaxvPStbncwO3Kx7XEuN+8ttOm08M=?= Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Fri Oct 27 01:04:57 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 27 Oct 2006 00:04:57 -0500 Subject: numpy - scipy version hell In-Reply-To: References: <200610191314.54482.jkgruet@sandia.gov> Message-ID: George Sakkis wrote: > Robert Kern gmail.com> writes: > >> It looks like you linked against a FORTRAN LAPACK, but didn't manage to link > the >> FORTRAN runtime library libg2c. Can you give us the output of your build? >> > > I just installed Numpy, ATLAS and LAPACK on Centos a few hours ago and I got the > exact same error. You're right, libg2c is never linked. Here's a sample line > from the linking: > > gcc -pthread -shared build/temp.linux-i686-2.4/numpy/linalg/lapack_litemodule.o > -L/usr/local/lib/atlas/ -llapack -lptf77blas -lptcblas -latlas -o > build/lib.linux-i686-2.4/numpy/linalg/lapack_lite.so > > I've been looking into numpy's distutils for the last hour or so but didn't > track down the problem yet; something seems to be broken with Redhat's setup... Did you do anything to configure the libraries for ATLAS? Like editing site.cfg? If so, you will need to add -lg2c after -latlas. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From lfriedri at imtek.de Fri Oct 27 01:55:06 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Fri, 27 Oct 2006 07:55:06 +0200 Subject: memory position of numpy arrays In-Reply-To: <45408905.2040502@ar.media.kyoto-u.ac.jp> References: <49f6c4ae0610111330h2458739cxfea7548627b54552@mail.gmail.com> <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> <1161852718.6602.10.camel@localhost> <45408905.2040502@ar.media.kyoto-u.ac.jp> Message-ID: <1161928506.6608.4.camel@localhost> Am Donnerstag, den 26.10.2006, 19:08 +0900 schrieb David Cournapeau: > By the way, I found the information about locking pages into memory for > windows: > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/virtuallock.asp > Thanks for the link, I now use a virtuallock in my Dll. I don't get the "paging-error"-bluescreens anymore, but now I get other ones ("DRIVER_IRQL_NOT_LESS_OR_EQUAL"), but I think this is another issue, and I am working on it... If anyone is using python / numpy / ctypes for hardware control (say, Cameras with grabber-cards or fire-wire / DCAM; National Instruments acquisition cards using NIDAQmx, ...) I am interested in discussion! Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gael.varoquaux at normalesup.org Fri Oct 27 02:46:13 2006 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 27 Oct 2006 08:46:13 +0200 Subject: memory position of numpy arrays In-Reply-To: <1161928506.6608.4.camel@localhost> References: <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> <1161852718.6602.10.camel@localhost> <45408905.2040502@ar.media.kyoto-u.ac.jp> <1161928506.6608.4.camel@localhost> Message-ID: <20061027064613.GF17064@clipper.ens.fr> On Fri, Oct 27, 2006 at 07:55:06AM +0200, Lars Friedrich wrote: > If anyone is using python / numpy / ctypes for hardware control (say, > Cameras with grabber-cards or fire-wire / DCAM; National Instruments > acquisition cards using NIDAQmx, ...) I am interested in discussion! Worked great for me ! My approach was to write a small wrapper C (actually C++, with "extern C" linking) library that exposed only what I needed of the camera interface, in a "python-friendly" way, and to wrap it with ctypes. I controlled a "Pixis" princeton instruments camera this way. As I said, it worked surprisingly well. I can send the code as an example if you wish. Ga?l ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Fri Oct 27 03:21:18 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Fri, 27 Oct 2006 09:21:18 +0200 Subject: Changes to bools under Numexpr In-Reply-To: <4540FFDA.1020006@sympatico.ca> References: <4540A2B1.9080509@carabos.com> <4540FFDA.1020006@sympatico.ca> Message-ID: <4541B36E.7080403@carabos.com> En/na Colin J. Williams ha escrit: > Ivan Vilata i Balaguer wrote: >> Hi all. The attached diff makes some changes to Numexpr support for >> booleans. The changes and their rationale are below. >> >> 1. New ``True`` and ``False`` boolean constants. This is so that 1 and >> 0 are always proper integer constants. It is also for completeness, >> but I don't envision any usage for them that couldn't be expressed >> without the constants. >> > I'm puzzled. > Python already has constants True and False of the bool type. bool is a > subclass of the int type. > Any instance of the bool type can be converted to the int type. > >>> a=1==0 > >>> type(a) > > >>> int(a) > 0 > >>> a > False > >>> > > Colin W. > >> 2. The only comparisons supported with booleans are ``==`` and ``!=``, >> so that you can compare boolean variables. Just as NumPy supports >> complex order comparisons and Numexpr doesn't, so goes for bools. >> Being relatively new, I think there is no need to keep >> integer-boolean compatibility in Numexpr. What was the meaning of >> ``True > False`` or ``2 > True`` anyway? Well, the ``True`` and ``False`` constants where not previously supported in Numexpr because they had to be defined somewhere. Now they are. Regarding the Python int and bool types and their relationships, it is a very elegant solution introduced in Python 2.3 since previous versions didn't have a proper boolean type, so the 0 and 1 ints where used for that. What I'm proposing here is, since Numexpr has a recent story and most probably there isn't much code affected by the change, why not define the boolean type as a purely logical one and leave its numeric compatibility issues out? By the way, it simplifies Numexpr's virtual machine (less casting opcodes). I admit again this looks a little baffling, of course, but I don't think it would mean many noticeable changes to the user. Regards, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Fri Oct 27 03:36:59 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 27 Oct 2006 01:36:59 -0600 Subject: build errors Solaris x86 64 bit python2.5 In-Reply-To: References: <4541641C.6050004@jpl.nasa.gov> Message-ID: On 10/26/06, Charles R Harris wrote: > > > > On 10/26/06, Mathew Yeates wrote: > > > > yes, I got around the problem from my previous posting "distutils > > question". I added ld_args[:0] = ['-m64'] to line 209 of > > python2.5/distutils/unixcompiler.py. Lovely, yes I know. > > > > I now get an error "numpy/core/src/multiarraymodule.c:7230: error: > > `NPY_ALLOW_THREADS' undeclared". This is after several billion warning > > messages of the form > > numpy/core/src/multiarraymodule.c:5010: warning: dereferencing > > type-punned pointer will break strict-aliasing rules > > > GCC? Needs the -no-strict-aliasing flag. Everybody hates the default > except the compiler writers because you can't cast pointers between > different sized types, something the linux kernel and numpy do a lot. Things > can fail badly if you don't set the flag and ignore the warnings. > Make that -fno-strict-aliasing. The whole command line on x86 looks like: gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fPIC Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ivilata at carabos.com Fri Oct 27 03:43:32 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Fri, 27 Oct 2006 09:43:32 +0200 Subject: [ANN] NumPy 1.0 release In-Reply-To: <4540EF49.1040605@ieee.org> References: <4540EF49.1040605@ieee.org> Message-ID: <4541B8A4.3000201@carabos.com> En/na Travis E. Oliphant ha escrit:: > We are very pleased to announce the release of NumPy 1.0 available for > download at http://www.numpy.org > > This release is the culmination of over 18 months of effort to allow > unification of the Numeric and Numarray communities. [...] Wow, let me say this must be for so many people in this group like having a child at last! Congratulations and lots of thanks to all developers (and very especially to Travis) and contributors for the impressive designing, coding, testing and, last but not least, community driving of the last months. Let's hope this goes on like this for a looong time! Cheers, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david at ar.media.kyoto-u.ac.jp Fri Oct 27 05:53:28 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 27 Oct 2006 18:53:28 +0900 Subject: pyaudio, a module to make noise from numpy arrays Message-ID: <4541D718.6040308@ar.media.kyoto-u.ac.jp> Hi, I announce the first release of pyaudio, a module to make noise from numpy arrays (read, write and play audio files with numpy arrays). * WHAT FOR ?: The Goal is to give to a numpy/scipy environmenet some basic audio IO facilities (ala sound, wavread, wavwrite of matlab). With pyaudio, you should be able to read and write most common audio files from and to numpy arrays. The underlying IO operations are done using libsndfile from Erik Castro Lopo (http://www.mega-nerd.com/libsndfile/), so he is the one who did the hard work. As libsndfile has support for a vast number of audio files (including wav, aiff, but also htk, ircam, and flac, an open source lossless codec), pyaudio enables the import from and export to a fairly large number of file formats. There is also a really crude player, which uses tempfile to play audio, and which only works for linux-alsa anyway. I intend to add better support at least for linux, and for other platforms if this does not involve too much hassle. So basically, if you are lucky enough to use a recent linux system, pyaudio already gives you the equivalent of wavread, wavwrite and sound. * DOWNLOAD: http://www.ar.media.kyoto-u.ac.jp/members/david/pyaudio.tar.gz * INSTALLATION INSTRUCTIONS: Just untar the package and drop it into scipy/Lib/sandbox, and add the two following lines to scipy/Lib/sandbox/setup.py: # Package to make some noise using numpy config.add_subpackage('pyaudio') (if libsndfile.so is not in /usr/lib, a fortiori if you are a windows user, you should also change set the right location for libsndfile in pyaudio/pysndfile.py, at the line _snd.cdll.LoadLibrary('/usr/lib/libsndfile.so') ) * EXAMPLE USAGE == Reading example == # Reading from '/home/david/blop.flac' from scipy.sandbox.pyaudio import sndfile a = sndfile('/home/david/blop.flac') print a tmp = a.read_frames_float(1024) --> Prints: File : /home/david/blop.flac Sample rate : 44100 Channels : 2 Frames : 9979776 Format : 0x00170002 Sections : 1 Seekable : True Duration : 00:03:46.298 And put into tmp the 1024 first frames (a frame is the equivalent of a sample, but taking into account the number of channels: so 1024 frames gives you 2048 samples here). == Writing example == # Writing to a wavfile: from scipy.sandbox.pyaudio import sndfile import numpy as N noise = N.random.randn((44100)) a = sndfile('/home/david/blop.flac', sfm['SFM_WRITE'], sf_format['SF_FORMAT_WAV'] | sf_format['SF_FORMAT_PCM16'], 1, 44100) a.write_frames(noise, 44100) a.close() -> should gives you a lossless compressed white noise ! This is really a first release, not really tested, not much documentation, I can just say it works for me. I haven't found a good way to emulate enumerations, which libsndfile uses a lot, so I am using dictionaries generated from the library C header to get a relation enum label <=> value. If someone has a better idea, I am open to suggestions ! Cheers, David From lfriedri at imtek.de Fri Oct 27 07:04:00 2006 From: lfriedri at imtek.de (Lars Friedrich) Date: Fri, 27 Oct 2006 13:04:00 +0200 Subject: memory position of numpy arrays In-Reply-To: <20061027064613.GF17064@clipper.ens.fr> References: <49f6c4ae0610131222t78a6fbecmd0725a087510be25@mail.gmail.com> <1161584711.6785.12.camel@localhost> <453C730F.4090000@astraw.com> <1161668441.6607.6.camel@localhost> <453DB2C1.3000201@ar.media.kyoto-u.ac.jp> <453E48FE.9050907@astraw.com> <453ECBD6.4020407@ar.media.kyoto-u.ac.jp> <1161852718.6602.10.camel@localhost> <45408905.2040502@ar.media.kyoto-u.ac.jp> <1161928506.6608.4.camel@localhost> <20061027064613.GF17064@clipper.ens.fr> Message-ID: <1161947040.6608.9.camel@localhost> Hello Gael (sorry, I just don't get the dots...), Am Freitag, den 27.10.2006, 08:46 +0200 schrieb Gael Varoquaux: > Worked great for me ! My approach was to write a small wrapper C > (actually C++, with "extern C" linking) library that exposed only what I > needed of the camera interface, in a "python-friendly" way, and to wrap > it with ctypes. I controlled a "Pixis" princeton instruments camera this > way. As I said, it worked surprisingly well. I can send the code as an > example if you wish. Yes, that would be really nice! I think, I am doing the same thing. In the attachment, you find my dll-code, which I am still working on --so it is not ready to use!--, if you have any comments I will be happy. Lars -- Dipl.-Ing. Lars Friedrich Optical Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-K?hler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri at imtek.de -------------- next part -------------- A non-text attachment was scrubbed... Name: pvcamDLL.cpp Type: text/x-c++src Size: 4061 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cjw at sympatico.ca Fri Oct 27 09:33:35 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri, 27 Oct 2006 09:33:35 -0400 Subject: Changes to bools under Numexpr In-Reply-To: <4541B36E.7080403@carabos.com> References: <4540A2B1.9080509@carabos.com> <4540FFDA.1020006@sympatico.ca> <4541B36E.7080403@carabos.com> Message-ID: <45420AAF.1080605@sympatico.ca> Ivan Vilata i Balaguer wrote: > En/na Colin J. Williams ha escrit: > > >> Ivan Vilata i Balaguer wrote: >> >>> Hi all. The attached diff makes some changes to Numexpr support for >>> booleans. The changes and their rationale are below. >>> >>> 1. New ``True`` and ``False`` boolean constants. This is so that 1 and >>> 0 are always proper integer constants. It is also for completeness, >>> but I don't envision any usage for them that couldn't be expressed >>> without the constants. >>> >>> >> I'm puzzled. >> Python already has constants True and False of the bool type. bool is a >> subclass of the int type. >> Any instance of the bool type can be converted to the int type. >> >>> a=1==0 >> >>> type(a) >> >> >>> int(a) >> 0 >> >>> a >> False >> >>> >> >> Colin W. >> >> >>> 2. The only comparisons supported with booleans are ``==`` and ``!=``, >>> so that you can compare boolean variables. Just as NumPy supports >>> complex order comparisons and Numexpr doesn't, so goes for bools. >>> Being relatively new, I think there is no need to keep >>> integer-boolean compatibility in Numexpr. What was the meaning of >>> ``True > False`` or ``2 > True`` anyway? >>> > > Well, the ``True`` and ``False`` constants where not previously > supported in Numexpr because they had to be defined somewhere. Now they > are. > > Regarding the Python int and bool types and their relationships, it is a > very elegant solution introduced in Python 2.3 since previous versions > didn't have a proper boolean type, so the 0 and 1 ints where used for > that. What I'm proposing here is, since Numexpr has a recent story and > most probably there isn't much code affected by the change, why not > define the boolean type as a purely logical one and leave its numeric > compatibility issues out? By the way, it simplifies Numexpr's virtual > machine (less casting opcodes). > > I admit again this looks a little baffling, of course, but I don't think > it would mean many noticeable changes to the user. > > Regards, > > Ivan, I'm afraid I'm still baffled. Are you saying that your proposal is necessary to preserve compatibility with Python versions before 2.3? Otherwise, it appears to introduce clutter with no clear benefit. I don't find Numexpr in NumPy, are you referring to a scipy module? Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jeremit0 at gmail.com Fri Oct 27 09:43:54 2006 From: jeremit0 at gmail.com (jeremito) Date: Fri, 27 Oct 2006 13:43:54 -0000 Subject: Matrices for testing In-Reply-To: <45406182.8050005@iam.uni-stuttgart.de> References: <1161820513.989550.265110@i42g2000cwa.googlegroups.com> <1161830596.758309.129670@k70g2000cwa.googlegroups.com> <45406182.8050005@iam.uni-stuttgart.de> Message-ID: <1161956634.143048.4230@e3g2000cwe.googlegroups.com> > Well if all you want is some matrices, there's nothing stopping you > from grabbing the matrices in the LAPACK distribution and using them > yourself. Robert's just saying they won't be included in Numpy. > There's also the matrix market, whcih has a large number of > (sparse-only?) example matrices. > http://math.nist.gov/MatrixMarket/index.html > --bb _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > You might be also interested in the Matrix Computation Toolbox which is > a collection of MATLAB M-files containing functions for constructing > test matrices ... > http://www.ma.man.ac.uk/~higham/mctoolbox/ > > and > http://www.mathworks.com/access/helpdesk/help/techdoc/ref/*gallery*.html > > BTW, you can easily import matrices given in the MatrixMarket format in > scipy. See* io.mmread * > > mmread(source) > Reads the contents of a Matrix Market file 'filename' into a matrix. > > Inputs: > > source - Matrix Market filename (extensions .mtx, .mtz.gz) > or open file object. > > Outputs: > > a - sparse or full matrix > > Nils Thanks Bill and Nils. After my response, I had discovered the Matrix Market and realized it would be easy for me create some of the matrices myself. However having a way to read in the files already is really helpful. Thanks for pointing that out. Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From xtdcplp at pianometz.com Fri Oct 27 09:58:35 2006 From: xtdcplp at pianometz.com (Helsing was) Date: Fri, 27 Oct 2006 15:58:35 +0200 Subject: Routines moves Message-ID: <000d01c6f9cf$fea88000$80991d53@domjwd5vafgfgp> Walmart States Drug or Plan is Kfrom cbs News is worlds biggest retailer is said Thursday extending its generic another states added were am. Walmart States Drug or Plan is Kfrom cbs News is worlds biggest retailer is said Thursday extending its generic another states added were am. Help or Pivoting Small is pivotsleft or foot pivot pivot point Move heel upward Take pivots or backwards Repeat right foot degree Left half in way back Routines of. Explore jr Catalog am Gift Guides of Match Parts Phone Calling in Rent an This Weeks Specials see all the. Backwards ball that backward four There want work a Remember long defense short number could Increase range row miss in continue distances maximum range. Unveil colorful Tetris Russia Kfound via kottkeorg or Video find bbc or creator Alexey Pajitnov covers rivalries between Atari buying country doesnt believe property am much less property Ultimately great very important! Via kottkeorg Video find bbc creator Alexey Pajitnov covers rivalries or between Atari buying country doesnt of believe property much less property Ultimately great a very important gamemore. Default Gothic Dreams Green Fields am Cowboy Memories Orange Flowers Pink Rose Purple red! Generation schools will in teach boys girls rule under Title ix gives school districts a latitude create allgirl of allboy job Board is. Useful Search Kthe Offical takes or look using verbmore Seven reports generates timely bulletins presented characters current affairs say system produce tailored persons or. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: step.gif Type: image/gif Size: 7101 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jontwang at gmail.com Fri Oct 27 10:35:10 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Fri, 27 Oct 2006 09:35:10 -0500 Subject: Defining custom types In-Reply-To: <45414F8C.8090002@ee.byu.edu> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> Message-ID: <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> On 10/26/06, Travis Oliphant wrote: > > > Okay, is my understanding here correct? I am defining two type > > descriptors: > > PyArray_Descr mxNumpyType - describes the Numpy array type. > > PyTypeObject mxNumpyDataType - describes the data type of the contents > > of the array (i.e. mxNumpyType->typeobj points to this), inherits from > > PyDoubleArrType_Type and overrides some fields as mentioned above. > > > The nomenclature is that mxNumPyType is the data-type of the array and > your PyTypeObject is the "type" of the elements of the array. So, you > have the names a bit backward. > > So, to correspond with the way I use the words "type" and "data-type", I > would name them: > > PyArray_Descr mxNumpyDataType > PyTypeObject mxNumpyType Okay, I will use this convention going forwards. > And the getitem and setitem functions are designed to only give/take > > PyObject* of type mxDateTime. > > > These are in the 'f' member of the PyArray_Descr structure, so > presumably you have also filled in your PyArray_Descr structure with > items from PyArray_DOUBLE? That's correct. I have all members of the 'f' member identical to that from PyArray_DOUBLE, except: mxNumpyType->f->dotfunc = NULL; mxNumpyType->f->getitem = date_getitem; mxNumpyType->f->setitem = date_setitem; mxNumpyType->f->cast[PyArray_DOUBLE] = (PyArray_VectorUnaryFunc*) dateToDouble; mxNumpyType->f->cast[PyArray_OBJECT] = (PyArray_VectorUnaryFunc*) dateToObject; All other cast functions are NULL. If I redefine the string function, I encounter another, perhaps more serious problem leading to a segfault. I've defined my string function to be extremely simple: >>> def printer(arr): ... return str(arr[0]) Now, if I try to print an element of the array: >>> mxArr[0] I get to this stack trace: #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at scalartypes.inc.src :68 #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, base=0x814e7a8) at arrayobject.c:1419 #2 0x007d259f in array_subscript_nice (self=0x814e7a8, op=0x804eb8c) at arrayobject.c:1985 #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at Objects/abstract.c:94 (Note: for some reason gdb claims that arrayobject.c:1985 is array_subscript_nice, but looking at my source this line is actually in array_item_nice. *boggle*) But scalar_value returns NULL for all non-native types. So, destptr in PyArray_Scalar is set to NULL, and the call the copyswap segfaults. Perhaps scalar_value should be checking the scalarkind field of PyArray_Descr, or using the elsize and alignment fields to figure out the pointer to return if scalarkind isn't set? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cjw at sympatico.ca Fri Oct 27 11:55:29 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri, 27 Oct 2006 11:55:29 -0400 Subject: Python Enhancement Proposal (PEP) - Current status? Message-ID: <45422BF1.2090201@sympatico.ca> A July posting sets out the intent: http://scipy.org/BaseArray Version 3 of the draft: http://numpy.scipy.org/array_interface.shtml There is a description, from a C Structure perspective: http://svn.scipy.org/svn/PEP/PEP_basearray.txt What is the current status of the plan to develop a PEP? Is there a more recent version of the PEP, from a Python user's perspective? Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 12:02:18 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 10:02:18 -0600 Subject: Python Enhancement Proposal (PEP) - Current status? In-Reply-To: <45422BF1.2090201@sympatico.ca> References: <45422BF1.2090201@sympatico.ca> Message-ID: <45422D8A.8060805@ieee.org> Colin J. Williams wrote: > A July posting sets out the intent: > http://scipy.org/BaseArray > Version 3 of the draft: > http://numpy.scipy.org/array_interface.shtml > There is a description, from a C Structure perspective: > http://svn.scipy.org/svn/PEP/PEP_basearray.txt > > What is the current status of the plan to develop a PEP? > > Is there a more recent version of the PEP, from a Python user's perspective? > > I'm working on two PEP's after discussing this with Guido at SciPy 2006. 1) A data-type object PEP 2) An extension of the buffer protocol to include the array interface. These are being done instead of the basearray PEP for now. I'm hoping to have a draft of these ready for discussion in a few weeks. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 12:05:10 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 10:05:10 -0600 Subject: Defining custom types In-Reply-To: <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> Message-ID: <45422E36.3010502@ieee.org> Jonathan Wang wrote: > On 10/26/06, *Travis Oliphant* > wrote: > > > Okay, is my understanding here correct? I am defining two type > > descriptors: > > PyArray_Descr mxNumpyType - describes the Numpy array type. > > PyTypeObject mxNumpyDataType - describes the data type of the > contents > > of the array (i.e. mxNumpyType->typeobj points to this), > inherits from > > PyDoubleArrType_Type and overrides some fields as mentioned above. > > > The nomenclature is that mxNumPyType is the data-type of the array and > your PyTypeObject is the "type" of the elements of the array. > So, you > have the names a bit backward. > > So, to correspond with the way I use the words "type" and > "data-type", I > would name them: > > PyArray_Descr mxNumpyDataType > PyTypeObject mxNumpyType > > > Okay, I will use this convention going forwards. > > > And the getitem and setitem functions are designed to only > give/take > > PyObject* of type mxDateTime. > > > These are in the 'f' member of the PyArray_Descr structure, so > presumably you have also filled in your PyArray_Descr structure with > items from PyArray_DOUBLE? > > > That's correct. I have all members of the 'f' member identical to that > from PyArray_DOUBLE, except: > > mxNumpyType->f->dotfunc = NULL; > mxNumpyType->f->getitem = date_getitem; > mxNumpyType->f->setitem = date_setitem; > mxNumpyType->f->cast[PyArray_DOUBLE] = (PyArray_VectorUnaryFunc*) > dateToDouble; > mxNumpyType->f->cast[PyArray_OBJECT] = (PyArray_VectorUnaryFunc*) > dateToObject; > > All other cast functions are NULL. > > If I redefine the string function, I encounter another, perhaps more > serious problem leading to a segfault. I've defined my string function > to be extremely simple: > >>> def printer(arr): > ... return str(arr[0]) > > Now, if I try to print an element of the array: > >>> mxArr[0] > > I get to this stack trace: > #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at > scalartypes.inc.src:68 > #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, > base=0x814e7a8) at arrayobject.c:1419 > #2 0x007d259f in array_subscript_nice (self=0x814e7a8, op=0x804eb8c) > at arrayobject.c:1985 > #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at > Objects/abstract.c:94 > > (Note: for some reason gdb claims that arrayobject.c:1985 is > array_subscript_nice, but looking at my source this line is actually > in array_item_nice. *boggle*) > > But scalar_value returns NULL for all non-native types. So, destptr in > PyArray_Scalar is set to NULL, and the call the copyswap segfaults. > > Perhaps scalar_value should be checking the scalarkind field of > PyArray_Descr, or using the elsize and alignment fields to figure out > the pointer to return if scalarkind isn't set? Hmmm... It looks like the modifications to scalar_value did not take into account user-defined types. I've added a correction so that user-defined types will use setitem to set the scalar value into the array. Presumably your setitem function can handle setting the array with scalars of your new type? I've checked the changes into SVN. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cjw at sympatico.ca Fri Oct 27 12:09:57 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri, 27 Oct 2006 12:09:57 -0400 Subject: Python Enhancement Proposal (PEP) - Current status? In-Reply-To: <45422D8A.8060805@ieee.org> References: <45422BF1.2090201@sympatico.ca> <45422D8A.8060805@ieee.org> Message-ID: <45422F55.7040901@sympatico.ca> Travis Oliphant wrote: > Colin J. Williams wrote: > >> A July posting sets out the intent: >> http://scipy.org/BaseArray >> Version 3 of the draft: >> http://numpy.scipy.org/array_interface.shtml >> There is a description, from a C Structure perspective: >> http://svn.scipy.org/svn/PEP/PEP_basearray.txt >> >> What is the current status of the plan to develop a PEP? >> >> Is there a more recent version of the PEP, from a Python user's perspective? >> >> >> > I'm working on two PEP's after discussing this with Guido at SciPy 2006. > > 1) A data-type object PEP > 2) An extension of the buffer protocol to include the array interface. > > These are being done instead of the basearray PEP for now. I'm hoping > to have a draft of these ready for discussion in a few weeks. > > > -Travis > > Travis, Many thanks, I'll watch out. In an earlier draft, I believe it was proposed that the type specifier be a single character. I hope that thought will be given to the use of something similar to dtype as the mnemonic codes are much more user-friendly than the letter codes. Thanks also for all the slogging you have done over the last many months. Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From joris at ster.kuleuven.be Fri Oct 27 11:58:54 2006 From: joris at ster.kuleuven.be (Joris De Ridder) Date: Fri, 27 Oct 2006 17:58:54 +0200 Subject: astype() behaviour Message-ID: <200610271758.54379.joris@ster.kuleuven.be> Hi, Is the following behaviour of astype() intentional in NumPy 1.0? >>> x = array([1,2,3]) >>> x.astype(None) array([ 1., 2., 3.]) That is, the int32 is converted to float64. J. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From joris at ster.kuleuven.be Fri Oct 27 12:13:21 2006 From: joris at ster.kuleuven.be (Joris De Ridder) Date: Fri, 27 Oct 2006 18:13:21 +0200 Subject: dtype() examples that used to work Message-ID: <200610271813.21500.joris@ster.kuleuven.be> Hi, The following did once work in NumPy: >>> dtype([int16]) >>> dtype([[int16]]) >>> dtype([uint,int32]) >>> dtype(['f8','S10']) but now they all generate a "TypeError: data type not understood". Why? I could not find the answer in help(dtype), the RecordArrays tutorial on scipy.org, the NumPy release notes, nor in the NumPy book. J. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jeremit0 at gmail.com Fri Oct 27 10:44:46 2006 From: jeremit0 at gmail.com (jeremito) Date: Fri, 27 Oct 2006 14:44:46 -0000 Subject: Sorting eigenvalues and vectors together Message-ID: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> I am using a = numpy.linalg.eig(A) to get the eigenvalues and eigenvectors. I am interested only in the largest eigenvalue so I would like to sort the first element of a. But if I do that, I won't know what is the associated eigenvector. Is there a function that will sort the values and vectors together or do I need to write it myself. Thanks, Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Fri Oct 27 13:01:38 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 27 Oct 2006 19:01:38 +0200 Subject: astype() behaviour In-Reply-To: <200610271758.54379.joris@ster.kuleuven.be> References: <200610271758.54379.joris@ster.kuleuven.be> Message-ID: <200610271901.39809.faltet@carabos.com> A Divendres 27 Octubre 2006 17:58, Joris De Ridder va escriure: > Hi, > > Is the following behaviour of astype() intentional in NumPy 1.0? > > >>> x = array([1,2,3]) > >>> x.astype(None) > > array([ 1., 2., 3.]) > > That is, the int32 is converted to float64. Yes, I think the behaviour is intended. This is because 'float64' is the default type in NumPy from some months ago (before the default was 'int_') HTH, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 13:02:00 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 11:02:00 -0600 Subject: Python Enhancement Proposal (PEP) - Current status? In-Reply-To: <45423A03.7010401@noaa.gov> References: <45422BF1.2090201@sympatico.ca> <45422D8A.8060805@ieee.org> <45423A03.7010401@noaa.gov> Message-ID: <45423B88.60401@ieee.org> Christopher Barker wrote: > Travis Oliphant wrote: > >> I'm working on two PEP's after discussing this with Guido at SciPy 2006. >> >> 1) A data-type object PEP >> 2) An extension of the buffer protocol to include the array interface. >> >> These are being done instead of the basearray PEP for now. >> > > Those are a great starts, and I think it is a good idea to break these > out. Thanks for working on this. > > However, I still think it is a VERY good idea to have an n-d array in > the standard lib -- I hope that idea won't get forgotten. > I think any nd-array in the standard library will just be an "Object" nd-array, though. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From faltet at carabos.com Fri Oct 27 13:06:13 2006 From: faltet at carabos.com (Francesc Altet) Date: Fri, 27 Oct 2006 19:06:13 +0200 Subject: dtype() examples that used to work In-Reply-To: <200610271813.21500.joris@ster.kuleuven.be> References: <200610271813.21500.joris@ster.kuleuven.be> Message-ID: <200610271906.14099.faltet@carabos.com> A Divendres 27 Octubre 2006 18:13, Joris De Ridder va escriure: > Hi, > > The following did once work in NumPy: > >>> dtype([int16]) > >>> dtype([[int16]]) > >>> dtype([uint,int32]) > >>> dtype(['f8','S10']) > > but now they all generate a "TypeError: data type not understood". Why? Should be intended as well. If you try to set a dtype from a list, it has to follow the format of a description as specified in: http://numpy.scipy.org/array_interface.shtml for example: In [67]: dtype([('f1', int16)]) Out[67]: dtype([('f1', '0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Fri Oct 27 12:55:31 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Fri, 27 Oct 2006 09:55:31 -0700 Subject: Python Enhancement Proposal (PEP) - Current status? In-Reply-To: <45422D8A.8060805@ieee.org> References: <45422BF1.2090201@sympatico.ca> <45422D8A.8060805@ieee.org> Message-ID: <45423A03.7010401@noaa.gov> Travis Oliphant wrote: > I'm working on two PEP's after discussing this with Guido at SciPy 2006. > > 1) A data-type object PEP > 2) An extension of the buffer protocol to include the array interface. > > These are being done instead of the basearray PEP for now. Those are a great starts, and I think it is a good idea to break these out. Thanks for working on this. However, I still think it is a VERY good idea to have an n-d array in the standard lib -- I hope that idea won't get forgotten. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Fri Oct 27 13:19:40 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Fri, 27 Oct 2006 13:19:40 -0400 Subject: Sorting eigenvalues and vectors together In-Reply-To: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> Message-ID: On 27/10/06, jeremito wrote: > I am using a = numpy.linalg.eig(A) to get the eigenvalues and > eigenvectors. I am interested only in the largest eigenvalue so I > would like to sort the first element of a. But if I do that, I won't > know what is the associated eigenvector. Is there a function that will > sort the values and vectors together or do I need to write it myself. I think that they are sorted, but argsort() will do what you want (or argmax(), even). A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Fri Oct 27 13:33:52 2006 From: David.L.Goldsmith at noaa.gov (David L. Goldsmith) Date: Fri, 27 Oct 2006 10:33:52 -0700 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <4541D718.6040308@ar.media.kyoto-u.ac.jp> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> Message-ID: <45424300.60805@noaa.gov> I'm sure some others _might_ regard this as frivolous, so let me just say: "Way Cool"! Thanks! DG David Cournapeau wrote: > Hi, > I announce the first release of pyaudio, a module to make noise from > numpy arrays (read, write and play audio files with numpy arrays). > > * WHAT FOR ?: > > The Goal is to give to a numpy/scipy environmenet some basic audio IO > facilities (ala sound, wavread, wavwrite of matlab). > > With pyaudio, you should be able to read and write most common audio > files from and to numpy arrays. The underlying IO operations are done > using libsndfile from Erik Castro Lopo > (http://www.mega-nerd.com/libsndfile/), so he is the one who did the > hard work. As libsndfile has support for a vast number of audio files > (including wav, aiff, but also htk, ircam, and flac, an open source > lossless codec), pyaudio enables the import from and export to a fairly > large number of file formats. > There is also a really crude player, which uses tempfile to play > audio, and which only works for linux-alsa anyway. I intend to add > better support at least for linux, and for other platforms if this does > not involve too much hassle. > > So basically, if you are lucky enough to use a recent linux system, > pyaudio already gives you the equivalent of wavread, wavwrite and sound. > > * DOWNLOAD: > > http://www.ar.media.kyoto-u.ac.jp/members/david/pyaudio.tar.gz > > * INSTALLATION INSTRUCTIONS: > > Just untar the package and drop it into scipy/Lib/sandbox, and add > the two following lines to scipy/Lib/sandbox/setup.py: > > # Package to make some noise using numpy > config.add_subpackage('pyaudio') > > (if libsndfile.so is not in /usr/lib, a fortiori if you are a windows > user, you should also change set the right location for libsndfile in > pyaudio/pysndfile.py, at the line > _snd.cdll.LoadLibrary('/usr/lib/libsndfile.so') ) > > * EXAMPLE USAGE > > == Reading example == > > # Reading from '/home/david/blop.flac' > from scipy.sandbox.pyaudio import sndfile > > a = sndfile('/home/david/blop.flac') > print a > > tmp = a.read_frames_float(1024) > > --> Prints: > > File : /home/david/blop.flac > Sample rate : 44100 > Channels : 2 > Frames : 9979776 > Format : 0x00170002 > Sections : 1 > Seekable : True > Duration : 00:03:46.298 > > And put into tmp the 1024 first frames (a frame is the equivalent of > a sample, but taking into account the number of channels: so 1024 frames > gives you 2048 samples here). > > == Writing example == > # Writing to a wavfile: > from scipy.sandbox.pyaudio import sndfile > import numpy as N > > noise = N.random.randn((44100)) > a = sndfile('/home/david/blop.flac', sfm['SFM_WRITE'], > sf_format['SF_FORMAT_WAV'] | sf_format['SF_FORMAT_PCM16'], > 1, 44100) > > a.write_frames(noise, 44100) > a.close() > > -> should gives you a lossless compressed white noise ! > > This is really a first release, not really tested, not much > documentation, I can just say it works for me. I haven't found a good > way to emulate enumerations, which libsndfile uses a lot, so I am using > dictionaries generated from the library C header to get a relation enum > label <=> value. If someone has a better idea, I am open to suggestions ! > > Cheers, > > David > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From kwgoodman at gmail.com Fri Oct 27 13:23:46 2006 From: kwgoodman at gmail.com (Keith Goodman) Date: Fri, 27 Oct 2006 10:23:46 -0700 Subject: Sorting eigenvalues and vectors together In-Reply-To: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> Message-ID: On 10/27/06, jeremito wrote: > I am using a = numpy.linalg.eig(A) to get the eigenvalues and > eigenvectors. I am interested only in the largest eigenvalue so I > would like to sort the first element of a. But if I do that, I won't > know what is the associated eigenvector. Is there a function that will > sort the values and vectors together or do I need to write it myself. You can get the ranking of the eigenvalues with idx = a[0].argsort().argsort() or idx = (-a[0]).argsort().argsort(). And then use that to sort both the eigenvalues and eigenvectors. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From djm at mindrot.org Fri Oct 27 13:37:10 2006 From: djm at mindrot.org (Damien Miller) Date: Sat, 28 Oct 2006 03:37:10 +1000 (EST) Subject: numpy-1.0 regress failure on OpenBSD In-Reply-To: <45414B37.8050402@ee.byu.edu> References: <45414B37.8050402@ee.byu.edu> Message-ID: On Thu, 26 Oct 2006, Travis Oliphant wrote: > Unless you want to help with tracking how long double is interpreted on > several platforms, then just ignore the test. (It actually wasn't being > run in 1.0b1). I'm happy to help - do you have a testcase that I can run on the various platforms that OpenBSD supports? -d ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 13:01:05 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 11:01:05 -0600 Subject: dtype() examples that used to work In-Reply-To: <200610271813.21500.joris@ster.kuleuven.be> References: <200610271813.21500.joris@ster.kuleuven.be> Message-ID: <45423B51.3080006@ieee.org> Joris De Ridder wrote: > Hi, > > The following did once work in NumPy: > > >>>> dtype([int16]) >>>> dtype([[int16]]) >>>> dtype([uint,int32]) >>>> dtype(['f8','S10']) >>>> > > but now they all generate a "TypeError: data type not understood". Why? > I could not find the answer in help(dtype), the RecordArrays tutorial on > scipy.org, the NumPy release notes, nor in the NumPy book. > It's in the release notes. Look in the section from 1.0b2 to 1.0b3. It's highlighted with a Note: *NOTE: * When creating data-type objects from lists, the list must be in array descriptor format. The "list of data-type objects" approach is no longer supported due to its ambiguity with the very common array descriptor approach. You can still get this behavior using a comma-separated string or changing your list to an array_descriptor: [('', x) for x in oldlist] -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Fri Oct 27 14:17:16 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 27 Oct 2006 12:17:16 -0600 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <45424300.60805@noaa.gov> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> <45424300.60805@noaa.gov> Message-ID: On 10/27/06, David L. Goldsmith wrote: > I'm sure some others _might_ regard this as frivolous, so let me just > say: "Way Cool"! Thanks! +1, and not frivolous at all. It's /really/ neat to be able to pull in data from standard image formats (say jpeg) into arrays to quickly do numerics on images. Adding similar capabilities to audio signals is, IMHO, a great contribution. So I heartily join the congratulation. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From robert.kern at gmail.com Fri Oct 27 13:22:29 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 27 Oct 2006 12:22:29 -0500 Subject: Sorting eigenvalues and vectors together In-Reply-To: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> Message-ID: jeremito wrote: > I am using a = numpy.linalg.eig(A) to get the eigenvalues and > eigenvectors. I am interested only in the largest eigenvalue so I > would like to sort the first element of a. But if I do that, I won't > know what is the associated eigenvector. Is there a function that will > sort the values and vectors together or do I need to write it myself. Use argsort() on the eigenvalues and fancy indexing on the eigenvectors. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Fri Oct 27 14:33:03 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 12:33:03 -0600 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <4541D718.6040308@ar.media.kyoto-u.ac.jp> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> Message-ID: <454250DF.3060402@ieee.org> David Cournapeau wrote: > Hi, > I announce the first release of pyaudio, a module to make noise from > numpy arrays (read, write and play audio files with numpy arrays). > Very nice. Thank you. I'd like to see exactly this kind of thing for video files too. We can get a lot of mileage out of ctypes. By the way. I noticed your setup.py file is missing a .todict() to convert the configuration object into a dictionary that setup(** ) can handle. if __name__ == "__main__": from numpy.distutils.core import setup setup(**configuration(top_path='').todict()) You might also check to see if libsndfile can be imported during the setup phase and then warn the user before installing that they need it. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pearu at cens.ioc.ee Fri Oct 27 14:57:14 2006 From: pearu at cens.ioc.ee (pearu at cens.ioc.ee) Date: Fri, 27 Oct 2006 21:57:14 +0300 (EEST) Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <454250DF.3060402@ieee.org> References: <454250DF.3060402@ieee.org> Message-ID: On Fri, 27 Oct 2006, Travis Oliphant wrote: > David Cournapeau wrote: > > Hi, > > I announce the first release of pyaudio, a module to make noise from > > numpy arrays (read, write and play audio files with numpy arrays). > > > > Very nice. Thank you. I'd like to see exactly this kind of thing for > video files too. We can get a lot of mileage out of ctypes. > > By the way. I noticed your setup.py file is missing a .todict() to > convert the configuration object into a dictionary that setup(** ) can > handle. > > if __name__ == "__main__": > from numpy.distutils.core import setup > setup(**configuration(top_path='').todict()) Actually, one should use here setup(configuration=configuration) Pearu ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Fri Oct 27 15:11:11 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Fri, 27 Oct 2006 12:11:11 -0700 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <454250DF.3060402@ieee.org> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> <454250DF.3060402@ieee.org> Message-ID: <454259CF.9090608@noaa.gov> > David Cournapeau wrote: >> Hi, >> I announce the first release of pyaudio, a module to make noise from >> numpy arrays (read, write and play audio files with numpy arrays). Does this have anything to do with this pyaudio? http://people.csail.mit.edu/hubert/pyaudio/ -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jontwang at gmail.com Fri Oct 27 15:37:01 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Fri, 27 Oct 2006 14:37:01 -0500 Subject: Defining custom types In-Reply-To: <45422E36.3010502@ieee.org> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> <45422E36.3010502@ieee.org> Message-ID: <4f7636bf0610271237p57a7b83bkb3b6cf00a738a833@mail.gmail.com> On 10/27/06, Travis Oliphant wrote: > > > If I redefine the string function, I encounter another, perhaps more > > serious problem leading to a segfault. I've defined my string function > > to be extremely simple: > > >>> def printer(arr): > > ... return str(arr[0]) > > > > Now, if I try to print an element of the array: > > >>> mxArr[0] > > > > I get to this stack trace: > > #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at > > scalartypes.inc.src:68 > > #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, > > base=0x814e7a8) at arrayobject.c:1419 > > #2 0x007d259f in array_subscript_nice (self=0x814e7a8, op=0x804eb8c) > > at arrayobject.c:1985 > > #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at > > Objects/abstract.c:94 > > > > (Note: for some reason gdb claims that arrayobject.c:1985 is > > array_subscript_nice, but looking at my source this line is actually > > in array_item_nice. *boggle*) > > > > But scalar_value returns NULL for all non-native types. So, destptr in > > PyArray_Scalar is set to NULL, and the call the copyswap segfaults. > > > > Perhaps scalar_value should be checking the scalarkind field of > > PyArray_Descr, or using the elsize and alignment fields to figure out > > the pointer to return if scalarkind isn't set? > > Hmmm... It looks like the modifications to scalar_value did not take > into account user-defined types. I've added a correction so that > user-defined types will use setitem to set the scalar value into the > array. Presumably your setitem function can handle setting the array > with scalars of your new type? > > I've checked the changes into SVN. Do there also need to be changes in scalartypes.inc.src to use getitem if a user-defined type does not inherit from a Numpy scalar? i.e. at scalartypes.inc.src:114 we should return some pointer calculated from the PyArray_Descr's elsize and alignment field to get the destination for the "custom scalar" type to be copied. As it stands, if the user-defined type does not inherit from a Numpy scalar, lots of things continue to break. Furthermore it seems like the scalar conversions prefer the builtin types, but it seems to me that the user-defined type should be preferred. i.e. if I try to get an element from my mxDateTime array, I get a float back: >>> mxArr[0] = DateTime.now() >>> mxArr[0][0] 732610.60691268521 But what I really want is the mxDateTime, which, oddly enough, is what happens if I use tolist(): >>> mxArr.tolist()[0] [] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Fri Oct 27 16:02:56 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Fri, 27 Oct 2006 14:02:56 -0600 Subject: Defining custom types In-Reply-To: <4f7636bf0610271237p57a7b83bkb3b6cf00a738a833@mail.gmail.com> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> <45422E36.3010502@ieee.org> <4f7636bf0610271237p57a7b83bkb3b6cf00a738a833@mail.gmail.com> Message-ID: <454265F0.7000805@ieee.org> Jonathan Wang wrote: > On 10/27/06, *Travis Oliphant* > wrote: > > > If I redefine the string function, I encounter another, perhaps more > > serious problem leading to a segfault. I've defined my string > function > > to be extremely simple: > > >>> def printer(arr): > > ... return str(arr[0]) > > > > Now, if I try to print an element of the array: > > >>> mxArr[0] > > > > I get to this stack trace: > > #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at > > scalartypes.inc.src:68 > > #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, > > base=0x814e7a8) at arrayobject.c:1419 > > #2 0x007d259f in array_subscript_nice (self=0x814e7a8, > op=0x804eb8c) > > at arrayobject.c:1985 > > #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at > > Objects/abstract.c:94 > > > > (Note: for some reason gdb claims that arrayobject.c:1985 is > > array_subscript_nice, but looking at my source this line is > actually > > in array_item_nice. *boggle*) > > > > But scalar_value returns NULL for all non-native types. So, > destptr in > > PyArray_Scalar is set to NULL, and the call the copyswap segfaults. > > > > Perhaps scalar_value should be checking the scalarkind field of > > PyArray_Descr, or using the elsize and alignment fields to > figure out > > the pointer to return if scalarkind isn't set? > > Hmmm... It looks like the modifications to scalar_value did not take > into account user-defined types. I've added a correction so that > user-defined types will use setitem to set the scalar value into the > array. Presumably your setitem function can handle setting the array > with scalars of your new type? > > I've checked the changes into SVN. > > > Do there also need to be changes in scalartypes.inc.src to use getitem > if a user-defined type does not inherit from a Numpy scalar? This needs to be clarified. I don't think it's possible to do it without inheriting from a numpy scalar at this point (the void numpy scalar can be inherited from and is pretty generic). I know I was not considering that case when I wrote the code. > i.e. at scalartypes.inc.src:114 we should return some pointer > calculated from the PyArray_Descr's elsize and alignment field to get > the destination for the "custom scalar" type to be copied. I think this is a good idea. I doubt it's enough to fix all places that don't inherit from numpy scalars, but it's a start. It seems like we need to figure out where the beginning of the data is for the type which is assumed to be defined on alignment boundaries after a PyObject_HEAD (right)? This could actually be used for everything and all the switch and if statements eliminated. I think the alignment field is the only thing needed, though. I don't see how I would use the elsize field? > > As it stands, if the user-defined type does not inherit from a Numpy > scalar, lots of things continue to break. Not surprising, I did not make sure and support this. > Furthermore it seems like the scalar conversions prefer the builtin > types, but it seems to me that the user-defined type should be preferred. I'm not sure what this means. > > > i.e. if I try to get an element from my mxDateTime array, I get a > float back: > >>> mxArr[0] = DateTime.now() > >>> mxArr[0][0] > 732610.60691268521 Why can you index mxArr[0]? What is mxArr[0]? If it's a scalar, then why can you index it? What is type(mxArr[0])? > > But what I really want is the mxDateTime, which, oddly enough, is what > happens if I use tolist(): > >>> mxArr.tolist()[0] > [] That's not surprising because tolist just calls getitem on each element in the array to construct the list. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jeremit0 at gmail.com Fri Oct 27 16:23:38 2006 From: jeremit0 at gmail.com (jeremito) Date: Fri, 27 Oct 2006 20:23:38 -0000 Subject: Sorting eigenvalues and vectors together In-Reply-To: References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> Message-ID: <1161980618.503699.300800@e3g2000cwe.googlegroups.com> argsort() will do the trick. Thanks once again. Jeremy ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From svetosch at gmx.net Fri Oct 27 17:04:13 2006 From: svetosch at gmx.net (Sven Schreiber) Date: Fri, 27 Oct 2006 23:04:13 +0200 Subject: Sorting eigenvalues and vectors together In-Reply-To: <1161980618.503699.300800@e3g2000cwe.googlegroups.com> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> <1161980618.503699.300800@e3g2000cwe.googlegroups.com> Message-ID: <4542744D.6040801@gmx.net> jeremito schrieb: > argsort() will do the trick. Thanks once again. > Jeremy > I was a bit confused by your question, maybe you can clarify what you did in the end. IIRC, if the eigenvalues returned by numpy are real numbers (due to the type of the underlying matrix and algorithm), then they are automatically returned ascending (again, IIRC). That would mean you don't have to do anything. But if they're complex, it's not obvious how to sort them. Probably by modulus. So what exactly was your problem? ciao, Sven ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jontwang at gmail.com Fri Oct 27 17:08:08 2006 From: jontwang at gmail.com (Jonathan Wang) Date: Fri, 27 Oct 2006 16:08:08 -0500 Subject: Defining custom types In-Reply-To: <454265F0.7000805@ieee.org> References: <4f7636bf0610261526t5b7b4191x370308df775799a9@mail.gmail.com> <4541426F.2070108@ee.byu.edu> <4f7636bf0610261637y7f2e0db1xc4e6e2fb080ad200@mail.gmail.com> <45414F8C.8090002@ee.byu.edu> <4f7636bf0610270735t4596d796xae9f18cbdc439103@mail.gmail.com> <45422E36.3010502@ieee.org> <4f7636bf0610271237p57a7b83bkb3b6cf00a738a833@mail.gmail.com> <454265F0.7000805@ieee.org> Message-ID: <4f7636bf0610271408p7adcedc7h547cd33434456a2c@mail.gmail.com> On 10/27/06, Travis Oliphant wrote: > > Jonathan Wang wrote: > > On 10/27/06, *Travis Oliphant* > > wrote: > > > > > If I redefine the string function, I encounter another, perhaps > more > > > serious problem leading to a segfault. I've defined my string > > function > > > to be extremely simple: > > > >>> def printer(arr): > > > ... return str(arr[0]) > > > > > > Now, if I try to print an element of the array: > > > >>> mxArr[0] > > > > > > I get to this stack trace: > > > #0 scalar_value (scalar=0x814be10, descr=0x5079e0) at > > > scalartypes.inc.src:68 > > > #1 0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0, > > > base=0x814e7a8) at arrayobject.c:1419 > > > #2 0x007d259f in array_subscript_nice (self=0x814e7a8, > > op=0x804eb8c) > > > at arrayobject.c:1985 > > > #3 0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at > > > Objects/abstract.c:94 > > > > > > (Note: for some reason gdb claims that arrayobject.c:1985 is > > > array_subscript_nice, but looking at my source this line is > > actually > > > in array_item_nice. *boggle*) > > > > > > But scalar_value returns NULL for all non-native types. So, > > destptr in > > > PyArray_Scalar is set to NULL, and the call the copyswap > segfaults. > > > > > > Perhaps scalar_value should be checking the scalarkind field of > > > PyArray_Descr, or using the elsize and alignment fields to > > figure out > > > the pointer to return if scalarkind isn't set? > > > > Hmmm... It looks like the modifications to scalar_value did not take > > into account user-defined types. I've added a correction so that > > user-defined types will use setitem to set the scalar value into the > > array. Presumably your setitem function can handle setting the > array > > with scalars of your new type? > > > > I've checked the changes into SVN. > > > > > > Do there also need to be changes in scalartypes.inc.src to use getitem > > if a user-defined type does not inherit from a Numpy scalar? > This needs to be clarified. I don't think it's possible to do it > without inheriting from a numpy scalar at this point (the void numpy > scalar can be inherited from and is pretty generic). I know I was not > considering that case when I wrote the code. > > i.e. at scalartypes.inc.src:114 we should return some pointer > > calculated from the PyArray_Descr's elsize and alignment field to get > > the destination for the "custom scalar" type to be copied. > I think this is a good idea. I doubt it's enough to fix all places that > don't inherit from numpy scalars, but it's a start. > > It seems like we need to figure out where the beginning of the data is > for the type which is assumed to be defined on alignment boundaries > after a PyObject_HEAD (right)? This could actually be used for > everything and all the switch and if statements eliminated. > > I think the alignment field is the only thing needed, though. I don't > see how I would use the elsize field? Hmm, yeah, I guess alignment would be sufficient. Worst case, you could delegate to setitem, right? It would be useful to support arbitrary types. Suppose, for example, that I wanted to make an array of structs. In keeping with the date/time example, I might want to store a long and a double, the long for days in the Gregorian calendar and the double for seconds from midnight on that day. > Furthermore it seems like the scalar conversions prefer the builtin > > types, but it seems to me that the user-defined type should be > preferred. > I'm not sure what this means. > > > > > > i.e. if I try to get an element from my mxDateTime array, I get a > > float back: > > >>> mxArr[0] = DateTime.now() > > >>> mxArr[0][0] > > 732610.60691268521 > Why can you index mxArr[0]? What is mxArr[0]? If it's a scalar, then > why can you index it? What is type(mxArr[0])? Ah, I am mistaken here - I am correctly getting my mxNumpyDateTime type back: mxArr is a 1x1 matrix: >>> mxArr = numpy.empty((1,1), dtype = libMxNumpy.type) >>> mxArr[0] = DateTime.now() >>> type(mxArr) >>> type(mxArr[0]) >>> type(mxArr[0][0]) >>> mxArr.shape (1, 1) > But what I really want is the mxDateTime, which, oddly enough, is what > > happens if I use tolist(): > > >>> mxArr.tolist()[0] > > [] > > That's not surprising because tolist just calls getitem on each element > in the array to construct the list. I guess this is a degenerate case, since I have getitem returning a mxDateTime while the actual type of the elements in the array is mxNumpyDateTime (i.e. mxNumpyType). Would the correct behavior, then, be for getitem to return a mxNumpyDateTime and register the object cast function to return a mxDateTime? If I try to do math on the array, it seems like the operation is performed via object pointers (mxDateTime - mxDateTime returns a DateTimeDelta object, and mxNumpyDateTime is a float): >>> mxArr = numpy.empty((1,1), dtype = libMxNumpy.type) >>> mxArr[0][0] = DateTime.now() >>> mxArr2 = numpy.empty((1,1), dtype = libMxNumpy.type) >>> mxArr2[0][0] = DateTime.DateTimeFrom('2006-01-01') >>> type(mxArr[0][0]) >>> type(mxArr2[0][0]) >>> sub = mxArr - mxArr2 >>> type(sub[0][0]) I'm guessing I need to register ufunc loops for all the basic math on my types? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From robert.kern at gmail.com Fri Oct 27 18:11:15 2006 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 27 Oct 2006 17:11:15 -0500 Subject: Sorting eigenvalues and vectors together In-Reply-To: <4542744D.6040801@gmx.net> References: <1161960286.071066.310700@b28g2000cwb.googlegroups.com> <1161980618.503699.300800@e3g2000cwe.googlegroups.com> <4542744D.6040801@gmx.net> Message-ID: Sven Schreiber wrote: > IIRC, if the eigenvalues returned by numpy are real numbers (due to the > type of the underlying matrix and algorithm), then they are > automatically returned ascending (again, IIRC). This should be considered an implementation detail that may change in the future, so don't rely on it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Fri Oct 27 18:37:19 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sat, 28 Oct 2006 00:37:19 +0200 Subject: adding an attribute to an nd-array In-Reply-To: <45379D7E.30609@ee.byu.edu> References: <45379D7E.30609@ee.byu.edu> Message-ID: <000301c6fa18$76fed890$0a83a8c0@ratbert> Hello all > -----Original Message----- > From: numpy-discussion-bounces at lists.sourceforge.net [mailto:numpy- > discussion-bounces at lists.sourceforge.net] On Behalf Of Travis Oliphant > Sent: Thursday, October 19, 2006 5:45 PM > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] adding an attribute to an nd-array > > Stefan van der Walt wrote: > > >On Wed, Oct 18, 2006 at 09:17:49PM -0400, Pierre GM wrote: > >>On Wednesday 18 October 2006 20:29, Stefan van der Walt wrote: > >>>A quick question on extending numpy arrays: is it possible to easily > >>>add an attribute to an ndarray? > >>It might be easier to create a subclass: pleasehave a look here: > >>http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/test_s > ubclasses.py > >>That's a tiny example of subclassing ndarrays, with some extra > attributes. > >>(BTW, I'm not sure that's the most obvious place where to look: if it > turns > >>out to be useful, I'll put it on the scipy wiki) > > > >Thanks very much, Pierre. > > > >If I understand correctly, the following should work: > > > >import numpy as N > > > >class InfoArray(N.ndarray): > > def __new__(info_arr_cls,arr,info={}): > > info_arr_cls.info = info > > return N.array(arr).view(info_arr_cls) > > > >When does __array_finalize__ get called, and is it always necessary to > >specify it? > > Actually something as simple as > > class InfoArray(N.ndarray): > pass > > will allow you to add attributes to InfoArray. I would like to create an "info" array like this to attach some metadata, mostly sampling frequency, to some of my arrays. The first problem I'm running into is that this new InfoArray type needs something like the array function so that I can create it from an existing "normal" array. It would also be useful if metadata could survive through type conversions like this: y = array(x, 'f4') One frequently wants to do this where you start with a 8-bit or 16-bit samples of a speech signal and you then want to convert to a floating point type for further processing. I browsed through the NumPy book and looked at the __new__ and __array_finalize__ stuff in Pierre GM's new MaskedArray, but I'm still thoroughly confused about how to create a simple array+metadata subclass that Just Works(TM). Does anybody have some code that they would be willing to share? Thanks. Regards, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fullung at gmail.com Fri Oct 27 20:59:36 2006 From: fullung at gmail.com (Albert Strasheim) Date: Sat, 28 Oct 2006 02:59:36 +0200 Subject: NumPy on Intel C/MKL Message-ID: <000401c6fa2c$57041c30$0a83a8c0@ratbert> Hello all I managed to get NumPy to compile with the free for non-commercial use Linux versions of the Intel C compiler, version 9.1 and Intel MKL 8.1. Instructions at the bottom of this page: http://www.scipy.org/Installing_SciPy/Linux I ran the NumPy test suite, and it turned up a few test failures in test_ufunclike -- it seems the Intel C/MKL combo comes up with different answers for the sign of NaN. Compiling with -Wall turned up a few warnings, but nothing too serious. I submitted a ticket for the ones that seem easily fixable and might warrant a quick look: http://projects.scipy.org/scipy/numpy/ticket/366 I recently started building some of my other C code on Windows with the Intel compiler, and the speed increases in my application are quite dramatic. The Intel code is about 4 times faster than the code produced by Visual Studio .NET 2003. I suspect the speed increase is due to some loop vectorization. When compiling NumPy on Linux, some interesting messages pop up: numpy/core/src/arraytypes.inc.src(658) : (col. 9) remark: LOOP WAS VECTORIZED. numpy.distutils doesn't seem to support the Intel compiler on Windows, but if you have other C code lying around, you can easily use SCons to compile it with the Intel compiler (Intel offers 30-day evaluation versions of most of their products). Have fun! Cheers, Albert ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mforbes at phys.washington.edu Fri Oct 27 22:18:52 2006 From: mforbes at phys.washington.edu (Michael McNeil Forbes) Date: Fri, 27 Oct 2006 19:18:52 -0700 Subject: recarray.__setattr__ bug? Message-ID: Is the following the desired behaviour for setting recarray attributes?? This seems to clash with the semantics for arrays. ?>>> from numpy import * ?>>> a = array([1,2,3]) ?>>> b = a.view([('x',int),('y',int),('z',int)]) ?>>> r = b.view(recarray) ?>>> b['x'] = 0 ?>>> r.y = 0 ?>>> a ?array([0, 2, 3]) ?>>> r.x ?array([0]) ?>>> r.y ?0 Setting r.y creates a local variable in r.__dict__ which is then accessed rather than the array element referred to by fielddict. I am not sure how to fix this: trying to set an attribute that does not exist in an ndarray raises an error, so I figured that changing the second line of recarray.__setattr__ in numpy/core/records.py to ?? ?def __setattr__(self, attr, val): ? ? ? ? try: ? ? ? ? ? ? return sb.ndarray.__setattr__(self, attr, val) would work, but this sets the variable in r.__dict__ (and I am not exactly sure where this is defined, so I can't comment further).? Is there a simple solution like this or does one need to explicitly check '__dict__'. One could reverse the order and call 'setfield' if the value is in the 'fielddict' first and then revert to the base setter, but this would potentially hide other members like 'shape'.? (Making this change does not break any tests however.) A test case for this would be: ?? ?def check_recarray_setattr1(self): """Make sure __setattr__ sets array fields.""" a = array([1,2]) r = rec.fromarrays(a,names=['x','y']) b = a.view(r.dtype).view(recarray) b.x = 0 assert 0 == a[0] ? ? def check_recarray_setattr2(self): """Make sure shape attribute is not hidden.""" a = array([[1,2],[3,4]]) r = rec.fromarrays(a,names=['x','shape']) b = a.view(r.dtype).view(recarray) assert 3 == b[1,0][0] b.shape = (1,2) assert 3 == b[0,1][0] Michael. -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Sat Oct 28 05:14:14 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sat, 28 Oct 2006 05:14:14 -0400 Subject: adding an attribute to an nd-array In-Reply-To: <000301c6fa18$76fed890$0a83a8c0@ratbert> References: <000301c6fa18$76fed890$0a83a8c0@ratbert> Message-ID: <200610280514.14604.pgmdevlist@gmail.com> On Friday 27 October 2006 18:37, Albert Strasheim wrote: > Hello all ... > I would like to create an "info" array like this to attach some metadata, > mostly sampling frequency, to some of my arrays. > ... > I browsed through the NumPy book and looked at the __new__ and > __array_finalize__ stuff in Pierre GM's new MaskedArray, but I'm still > thoroughly confused about how to create a simple array+metadata subclass > that Just Works(TM). > > Does anybody have some code that they would be willing to share? I'm not sure about the JustWorks(R)(TM), but I just created a page on the Wiki describing a simple example of InfoArray class http://www.scipy.org/Subclasses That should do what you want, but let me know if you have some more specific requests. I guess I should go more into the details of subclassing on that page, but I'm quite far from being a specialist... ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Sat Oct 28 06:19:06 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sat, 28 Oct 2006 06:19:06 -0400 Subject: recarray.__setattr__ bug? In-Reply-To: References: Message-ID: <200610280619.06841.pgmdevlist@gmail.com> On Friday 27 October 2006 22:18, Michael McNeil Forbes wrote: > Is the following the desired behaviour for setting recarray attributes?? I ran into the same problem recently... What about modifying __setattr__ to the following ? def __setattr__(self, attr, val): fielddict = sb.ndarray.__getattribute__(self,'dtype').fields if attr in fielddict.keys(): return sb.ndarray.__setitem__(self, attr, val) try: return object.__setattr__(self, attr, val) except AttributeError: # Must be a fieldname raise AttributeError, "record array has no attribute %s" % attr The order is changed, as in Michael's suggestion, but that doesn't set any variable in the __dict__, nor does it mask anything. Instead of modifying the an attribute, we modify the field directly... ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jk985 at tom.com Sat Oct 28 13:56:54 2006 From: jk985 at tom.com (=?GB2312?B?IjEx1MI0LTXI1S+xsb6pIg==?=) Date: Sat, 28 Oct 2006 17:56:54 +0000 (UTC) Subject: =?GB2312?B?RVhDRUy6zVBQVNTaxvPStbncwO3Kx7XEuN+8ttOm08M=?= Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Sat Oct 28 14:28:35 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 28 Oct 2006 12:28:35 -0600 Subject: recarray.__setattr__ bug? In-Reply-To: References: Message-ID: <4543A153.4030505@ieee.org> Michael McNeil Forbes wrote: > Is the following the desired behaviour for setting recarray attributes? > This seems to clash with the semantics for arrays. > > >>> from numpy import * > >>> a = array([1,2,3]) > >>> b = a.view([('x',int),('y',int),('z',int)]) > >>> r = b.view(recarray) > >>> b['x'] = 0 > >>> r.y = 0 > >>> a > array([0, 2, 3]) > >>> r.x > array([0]) > >>> r.y > 0 > > Setting r.y creates a local variable in r.__dict__ which is then > accessed rather than the array element referred to by fielddict. > Hmm.... I know that the code was changed at some point a few months ago specifically to this behavior because of some concerns Perry, Chris (people at STScI) had. Originally, field names came first, but we changed it so they could set known attributes of a record array even if they were also field names. This may be an unintentional side-effect. So, let's not just change things again and create problems for them. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From vdeelocbad at planetprint.com Sat Oct 28 15:03:37 2006 From: vdeelocbad at planetprint.com (life.We an) Date: Sat, 28 Oct 2006 21:03:37 +0200 Subject: Seal Campaign aquatic life.We Message-ID: <000b01c6fac3$c61d6620$851336c2@mama> Hints avoid common would like help of distribute of fliers club meetings events feel download in current flier. Bottom Balticon Wyndham downtown Baltimore in main hallway across entrance in first Here some advice which provides or useful hints avoid common would. Hints avoid common would like help of distribute of fliers club meetings events feel download in current flier. One or more writers in how to am improve own writing Past panels included writing about the part Trek express a point view of in story or trials being in writer planets of stars am galaxies that. Door or telling where pick up or wi ll starting October check page now am may obtain forms of clicking now closed refer is. More writers how to improve own writing Past panels included writing about in. Atthedoor bring take in place location posted should of arrive prior close start possible or There door telling am where pick up wi in ll starting October check page now may obtain. Useful hints avoid common would like help distribute fliers club meetings events feel download current flier pdf format make am copies Thanks much. Location posted or should arrive prior close start is possible There door telling where pick up wi ll starting October check page now is may obtain forms clicking. May obtain forms clicking now closed refer Hotel or Hunt Valley inn well overflow or hotelsthe has begun am accepting or person Farpoint February same Leave is visit table in bottom am. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: with.gif Type: image/gif Size: 8199 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From pgmdevlist at gmail.com Sat Oct 28 15:30:22 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sat, 28 Oct 2006 15:30:22 -0400 Subject: recarray.__setattr__ bug? In-Reply-To: <4543A153.4030505@ieee.org> References: <4543A153.4030505@ieee.org> Message-ID: <200610281530.22929.pgmdevlist@gmail.com> On Saturday 28 October 2006 14:28, Travis Oliphant wrote: > > Hmm.... I know that the code was changed at some point a few months ago > specifically to this behavior because of some concerns Perry, Chris > (people at STScI) had. Originally, field names came first, but we > changed it so they could set known attributes of a record array even if > they were also field names. So that I could have a field named 'shape', and modifying r.shape would change the shape of the array, not the content of the field ? That makes sense, but isn't it a bit dangerous ? Shouldn't we have a list of reserved keywords (dtype, shape...) that would raise an exception if used as field names ? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Sat Oct 28 15:36:23 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sat, 28 Oct 2006 15:36:23 -0400 Subject: A recarray of arrays Message-ID: <200610281536.23189.pgmdevlist@gmail.com> Folks, What is the easiest way to define a recarray of arrays ? For example, I'd need something like that: Given three arrays >>> x = N.arange(5) >>> y = x+1 >>> z = N.sqrt(x) and a list of names: >>> n = ['x','y','z'] Define a 3-record array with two fields: the first one being a ndarray (in x, y or), the second a string (in n). I'm a bit at loss with the definition of the corresponding dtype... ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From service at visa.com Sat Oct 28 17:10:27 2006 From: service at visa.com (service at visa.com) Date: Sat, 28 Oct 2006 17:10:27 -0400 Subject: VISA Update Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Sat Oct 28 20:08:53 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sat, 28 Oct 2006 18:08:53 -0600 Subject: A recarray of arrays In-Reply-To: <200610281536.23189.pgmdevlist@gmail.com> References: <200610281536.23189.pgmdevlist@gmail.com> Message-ID: <4543F115.4030004@ieee.org> Pierre GM wrote: > Folks, > What is the easiest way to define a recarray of arrays ? > For example, I'd need something like that: > Given three arrays > >>>> x = N.arange(5) >>>> y = x+1 >>>> z = N.sqrt(x) >>>> > and a list of names: > >>>> n = ['x','y','z'] >>>> > Define a 3-record array with two fields: the first one being a ndarray (in x, > y or), the second a string (in n). > > I'm a bit at loss with the definition of the corresponding dtype... > I'm a little unclear one what you want. Here is a data-type whose first field is a name and whose second field is a 5-element array: dt = N.dtype([('name', 'S10'), ('data', float, (5,))]) Then: a = array([('x',x),('y',y),('z',z)],dt) gives array([('x', [0.0, 1.0, 2.0, 3.0, 4.0]), ('y', [1.0, 2.0, 3.0, 4.0, 5.0]), ('z', [0.0, 1.0, 1.4142135623730951, 1.7320508075688772, 2.0])], dtype=[('name', '|S10'), ('data', ' <4543A153.4030505@ieee.org> Message-ID: In article <4543A153.4030505 at ieee.org>, Travis Oliphant wrote: > Hmm.... I know that the code was changed at some point a few months ago > specifically to this behavior because of some concerns Perry, Chris > (people at STScI) had. Originally, field names came first, but we > changed it so they could set known attributes of a record array even if > they were also field names. > > This may be an unintentional side-effect. So, let's not just change > things again and create problems for them. Were any test cases generated for this? Changing the code did not break anything: if this is important, a test should probably be added. Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From mforbes at phys.washington.edu Sat Oct 28 22:45:27 2006 From: mforbes at phys.washington.edu (Michael McNeil Forbes) Date: Sat, 28 Oct 2006 19:45:27 -0700 Subject: recarray.__setattr__ bug? References: <4543A153.4030505@ieee.org> <200610281530.22929.pgmdevlist@gmail.com> Message-ID: In article <200610281530.22929.pgmdevlist at gmail.com>, Pierre GM wrote: > So that I could have a field named 'shape', and modifying > r.shape would change the shape of the array, not the content of the field ? > > That makes sense, but isn't it a bit dangerous ? Shouldn't we have a list of > reserved keywords (dtype, shape...) that would raise an exception if used as > field names ? Since the field names can be used with regular arrays without any consequences, I think it would be bad to raise an exception with recarrays where there was no problem with regular arrays, and regular arrays should allow field names such as 'shape'. A note should be added to the recarray documentation, however, pointing out this pitfall. Perhaps a warning could be raised when a recarray is constructed with poor fieldname choices, but I would not raise an exception. In any case, I think is is dangerous to allow the fieldnames to hide standard array members. This would break the array interface for such recarrays causing many subtle problems for any algorithm that assumed the recarray to behave as a standard array: This code could be deeply buried. Instead, the user's code would behave 'strangely', but it is also the user's code that assigned bad fieldnames, and a warning would alert of this. Another concern that I found with the recarray interface is that the current implementation exhibits quite poor performance, but I am not sure how to fix this yet. Is there any documentation about the general design decisions behind recarrays? I think they could be very useful for simplifying code, but could use several improvements. Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Sun Oct 29 01:00:38 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Sun, 29 Oct 2006 14:00:38 +0900 Subject: pyaudio, a module to make noise from numpy arrays In-Reply-To: <454259CF.9090608@noaa.gov> References: <4541D718.6040308@ar.media.kyoto-u.ac.jp> <454250DF.3060402@ieee.org> <454259CF.9090608@noaa.gov> Message-ID: <45443576.9030908@ar.media.kyoto-u.ac.jp> Christopher Barker wrote: > > Does this have anything to do with this pyaudio? > > http://people.csail.mit.edu/hubert/pyaudio/ > > -Chris > Not at all. I should have looked for pyaudio as a name on google :) Before coding my small package, I looked at other python bindings for audio, but either they were not cross platform, or not uptodate, or depended on too much external code. I didn't find this one, though. I think using something like portaudio just to be able to play or record data is a bit overkill: those libraries are supposed to be used for audio applications, with all the constraints: real time capabilities, mixing capabilities, being able to list soundcards, etc... My scope is really much simpler: I just want to be able to import audio files as numpy arrays, process them, and to listen to the result; it really just intend to be an equivalent of wavread, wavwrite and sound/soundsc of matlab. libsndfile is the defacto standard on linux (almost all audio applications on linux with IO needs use it), is really high quality, and is available on linux, windows and mac OSX. cheers, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Sun Oct 29 01:09:27 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sun, 29 Oct 2006 01:09:27 -0400 Subject: recarray.__setattr__ bug? In-Reply-To: References: <200610281530.22929.pgmdevlist@gmail.com> Message-ID: <200610290109.27473.pgmdevlist@gmail.com> On Saturday 28 October 2006 22:45, Michael McNeil Forbes wrote: > Since the field names can be used with regular arrays without any > consequences, I think it would be bad to raise an exception with > recarrays where there was no problem with regular arrays, and regular > arrays should allow field names such as 'shape'. I agree to a certain extent: as switching from regular arrays to recarrays is straightforward, any field can potentially can be accessed as an attribute and this can brings some serious surprises. If an exception is too restrictive, a warning should at least be raised (just as the new warnings about dividing by zeros) when an array is created with field names in a exclude list. As a starter, the exclude list could be most (all ?) of the basic attributes/methods of a regular ndarray, along with 'fields', 'mask'... > Another concern that I found with the recarray interface is that the > current implementation exhibits quite poor performance, but I am not > sure how to fix this yet. Well, there's a lot of __getattribute__ access: that tends to slow things down. But wouldn't checking whether the fields are in the 'exclude' list degrade the performance even more ? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Sun Oct 29 02:40:32 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sun, 29 Oct 2006 01:40:32 -0600 Subject: recarray.__setattr__ bug? In-Reply-To: References: <4543A153.4030505@ieee.org> Message-ID: <45445AF0.9010503@ieee.org> Michael McNeil Forbes wrote: > In article <4543A153.4030505 at ieee.org>, > Travis Oliphant wrote: > >> Hmm.... I know that the code was changed at some point a few months ago >> specifically to this behavior because of some concerns Perry, Chris >> (people at STScI) had. Originally, field names came first, but we >> changed it so they could set known attributes of a record array even if >> they were also field names. >> >> This may be an unintentional side-effect. So, let's not just change >> things again and create problems for them. >> > > Were any test cases generated for this? Changing the code did not break > anything: if this is important, a test should probably be added. > You are right. In fact there are lots of tests that need to be added. Stefan as been doing a great job of regression testing. I've committed some fixes to this particular issue that allows setattr to be used to set field names. Now 1) The standard setattr is tried first 2) If it fails, then the field-dictionary is checked to see if there is a field with that name. a) If not, then the error created by 1 is raised b) if the name is a field, then an attempt is made to set the field with that name 3) If the standard setattr succeeds then either i) a built-in attribute was correctly set or ii) a new attribute was created and attached to the dictionary. a) If the attribute is not in the fields, then return b) Otherwise, see if this name was in the .__dict__ before the call (if it's not then we just added it so delete it). i) If the deletion succeeds then set set the field ii If the deletion fails, then return the result of standard setattr setting. I've added a test for some of these behaviors. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ivilata at carabos.com Sun Oct 29 04:15:49 2006 From: ivilata at carabos.com (Ivan Vilata i Balaguer) Date: Sun, 29 Oct 2006 10:15:49 +0100 Subject: Changes to bools under Numexpr In-Reply-To: <45420AAF.1080605@sympatico.ca> References: <4540A2B1.9080509@carabos.com> <4540FFDA.1020006@sympatico.ca> <4541B36E.7080403@carabos.com> <45420AAF.1080605@sympatico.ca> Message-ID: <45447145.1010100@carabos.com> En/na Colin J. Williams ha escrit:: > I'm afraid I'm still baffled. Are you saying that your proposal is > necessary > to preserve compatibility with Python versions before 2.3? Otherwise, it > appears > to introduce clutter with no clear benefit. > > I don't find Numexpr in NumPy, are you referring to a scipy module? From http://www.scipy.org/SciPyPackages/NumExpr: The scipy.sandbox.numexpr package supplies routines for the fast evaluation of array expressions elementwise by using a vector-based virtual machine. It's comparable to scipy.weave.blitz (in Weave), but doesn't require a separate compile step of C or C++ code. If your SciPy package doesn't include it, you can get a checkout from the repository:: $ svn co http://svn.scipy.org/svn/scipy/trunk/Lib/sandbox/numexpr Numexpr is not usable under Python <= 2.4 since it uses decorators (and maybe for other reasons), so my proposal clearly hasn't the intention on Python 2.3 compatibility. Instead, it is a simplification of Numexpr's type system by stricter boolean types. I suggest that you try Numexpr. It is a great piece of software. Cheers, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ C?rabos Coop. V. V V Enjoy Data "" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 309 bytes Desc: OpenPGP digital signature URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From gerard.vermeulen at grenoble.cnrs.fr Sun Oct 29 04:49:26 2006 From: gerard.vermeulen at grenoble.cnrs.fr (Gerard Vermeulen) Date: Sun, 29 Oct 2006 10:49:26 +0100 Subject: ANN: PyQwt3D-0.1.2 Message-ID: <20061029104926.6388c04e.gerard.vermeulen@grenoble.cnrs.fr> What is PyQwt3D? - it is a set of Python bindings for the QwtPlot3D C++ class library which extends the Qt framework with widgets for 3D data visualization. PyQwt3D inherits the snappy feel from QwtPlot3D. The examples at http://pyqwt.sourceforge.net/pyqwt3d-examples.html show how easy it is to make a 3D plot and how to save a 3D plot to an image or an (E)PS/PDF file. - it requires and extends PyQt, a set of Python bindings for Qt. - it supports the use of PyQt, Qt, Qwt, the Numerical Python extensions (any combination of NumPy, numarray, and numarray) and optionally SciPy in a GUI Python application or in an interactive Python session. - it runs on POSIX, Mac OS X and Windows platforms (practically any platform supported by Qt and Python). New features: - support for Python-2.5 (requires PyQt and SIP snapshots or the future PyQt-3.17, PyQt-4.1 and SIP-4.5) - support for Qt-4.2.x and -4.1.x - support for NumPy-1.0, see http://numpy.scipy.org - PyQwt3D.OpenGL gives access to a small subset of the OpenGL API - EnrichmentDemo.py example demonstrates PyQwt3D.OpenGL The home page of PyQwt3D is http://pyqwt.sourceforge.net. PyQwt3D-0.1.2 requires: 1. Python-2.5.x,-2.4.x, or -2.3. 2. PyQt-4.0.x, or -3.16.x (or the future PyQt-4.1 or PyQt-3.17) 3. SIP-4.4.x (or the future SIP-4.5) 4. Qt-4.2.x, Qt-4.1.x, Qt-3.3.x, or -3.2.x. 5. QwtPlot3D-0.2.6. Have fun -- Gerard Vermeulen ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From hivktdnzyc at openheart.com Sun Oct 29 06:37:27 2006 From: hivktdnzyc at openheart.com (to manage) Date: Sun, 29 Oct 2006 20:37:27 +0900 Subject: series Message-ID: <001101c6fb4e$9ca66810$ceb30d3b@yourkbvfzpdt4n> Corporate is welfare Adaware secures business dangerous identity is is important at school home or office now can. Enough to manage worry about without added stress or protecting private company Well am relieve war against spy is ware requires is heavy defense protect corporate welfare am Adaware is secures. Corporate is welfare Adaware secures business dangerous identity is is important at school home or office now can. Freeskins free Support Education Lists the Press us Header our a? About without am added stress protecting in private company Well relieve war against is spy ware of requires heavy defense in protect corporate welfare a Adaware secures business dangerous is identity. Foundation Incread Quick Center Forums or What News aim new is Blog Software in Reuters oct Spoofing am bug Found am ie or Cnet News Footer links Useprivacy. Enough to manage worry about without added stress protecting private company Well relieve war against spy ware requires heavy defense a protect. In finding right here see theproduct Chart or Project eco Clean up malicious threats a growing series standalone toolsread is More ser a Awards takes? Our Mission yourself is family are top is priority provides you in with safety security privacy giving real peace. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Mission..gif Type: image/gif Size: 8552 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cjw at sympatico.ca Sun Oct 29 06:55:20 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Sun, 29 Oct 2006 06:55:20 -0500 Subject: Changes to bools under Numexpr In-Reply-To: <45447145.1010100@carabos.com> References: <4540A2B1.9080509@carabos.com> <4540FFDA.1020006@sympatico.ca> <4541B36E.7080403@carabos.com> <45420AAF.1080605@sympatico.ca> <45447145.1010100@carabos.com> Message-ID: <454496A8.1090002@sympatico.ca> Ivan Vilata i Balaguer wrote: > En/na Colin J. Williams ha escrit:: > > >> I'm afraid I'm still baffled. Are you saying that your proposal is >> necessary >> to preserve compatibility with Python versions before 2.3? Otherwise, it >> appears >> to introduce clutter with no clear benefit. >> >> I don't find Numexpr in NumPy, are you referring to a scipy module? >> > > From http://www.scipy.org/SciPyPackages/NumExpr: > > The scipy.sandbox.numexpr package supplies routines for the fast > evaluation of array expressions elementwise by using a vector-based > virtual machine. It's comparable to scipy.weave.blitz (in Weave), > but doesn't require a separate compile step of C or C++ code. > > If your SciPy package doesn't include it, you can get a checkout from > the repository:: > > $ svn co http://svn.scipy.org/svn/scipy/trunk/Lib/sandbox/numexpr > > Numexpr is not usable under Python <= 2.4 since it uses decorators (and > maybe for other reasons), so my proposal clearly hasn't the intention on > Python 2.3 compatibility. Instead, it is a simplification of Numexpr's > type system by stricter boolean types. > > I suggest that you try Numexpr. It is a great piece of software. Cheers, > > :: > > Ivan Vilata i Balaguer >qo< http://www.carabos.com/ > C??rabos Coop. V. V V Enjoy Data > > Ivan, Many thanks for this. It's a long time since I looked at SciPy, though I will look at Numexpr. Currently, I'm focusing on Numpy. Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From pgmdevlist at gmail.com Sun Oct 29 10:52:02 2006 From: pgmdevlist at gmail.com (Pierre GM) Date: Sun, 29 Oct 2006 10:52:02 -0500 Subject: recarray.__setattr__ bug? In-Reply-To: <45445AF0.9010503@ieee.org> References: <45445AF0.9010503@ieee.org> Message-ID: <200610291052.03425.pgmdevlist@gmail.com> On Sunday 29 October 2006 02:40, Travis Oliphant wrote: > I've committed some fixes to this particular issue that allows setattr > to be used to set field names. I just started playing with the new recarray: that works great ! Travis, Thanks a lot ! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From tom.denniston at alum.dartmouth.org Sun Oct 29 17:53:37 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Sun, 29 Oct 2006 16:53:37 -0600 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. Message-ID: I recently upgraded to numpy 1.0 from 1.0b5. I noticed that numpy.argmaxbehavior is very strange on object arrays. See below: (Pdb) numpy.__version__ '1.0' (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object)) 0 (Pdb) numpy.argmax(numpy.array([2, 3], dtype=int)) 1 (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object), axis=0) 0 I would expect the argmax to behave the same on the dtype=int and dtype=object examples but it doesn't. Am I missing some subtelty or is this just a bug? 1.0 is the most recent version, right? --Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cjw at sympatico.ca Sun Oct 29 18:22:13 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Sun, 29 Oct 2006 18:22:13 -0500 Subject: matrixdef.py Message-ID: <454537A5.1030202@sympatico.ca> Line 71 has: data.view(subtype) This appears to involve a call to __array_finalize__. Is this an unconditional call? If not, what are the conditions? Why not use __init__ to handle these things, since it is always called after the __new__? Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Sun Oct 29 19:05:37 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 29 Oct 2006 17:05:37 -0700 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: On 10/29/06, Tom Denniston wrote: > > I recently upgraded to numpy 1.0 from 1.0b5. I noticed that numpy.argmaxbehavior is very strange on object arrays. See below: > > (Pdb) numpy.__version__ > '1.0' > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object)) > 0 > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=int)) > 1 > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object), axis=0) > 0 > > > I would expect the argmax to behave the same on the dtype=int and > dtype=object examples but it doesn't. Am I missing some subtelty or is this > just a bug? 1.0 is the most recent version, right? > Suppose In [22]: array([1,[2,3]], dtype=object) Out[22]: array([1, [2, 3]], dtype=object) How would you compare the elements? In [27]: 2 < [0,0] Out[27]: True In [28]: [0,0] > 2 Out[28]: True Compares memory locations? In [28]: [2] < [0,0] Out[28]: False Lexical ordering? I don't know how python interprets these things. That said, I suspect your example should behave better, but it might give strange results sometimes anyway. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tom.denniston at alum.dartmouth.org Sun Oct 29 20:10:05 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Sun, 29 Oct 2006 19:10:05 -0600 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: Oh. My mistake. I thought I had an array of 2 objects which were ints. I actually had an array of one list of 2 ints. It works properly if I construct the array properly. In [14]: a = numpy.empty((2), dtype=object) In [15]: a[0:]=[2,3] In [16]: a Out[16]: array([2, 3], dtype=object) In [17]: a.shape Out[17]: (2,) In [18]: a[0] Out[18]: 2 In [19]: a[1] Out[19]: 3 In [20]: numpy.argmax(a) Out[20]: 1 On 10/29/06, Charles R Harris wrote: > > > > On 10/29/06, Tom Denniston wrote: > > > > I recently upgraded to numpy 1.0 from 1.0b5. I noticed that > > numpy.argmax behavior is very strange on object arrays. See below: > > > > (Pdb) numpy.__version__ > > '1.0' > > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object)) > > 0 > > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=int)) > > 1 > > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object), axis=0) > > 0 > > > > > > I would expect the argmax to behave the same on the dtype=int and > > dtype=object examples but it doesn't. Am I missing some subtelty or is this > > just a bug? 1.0 is the most recent version, right? > > > > Suppose > > In [22]: array([1,[2,3]], dtype=object) > Out[22]: array([1, [2, 3]], dtype=object) > > How would you compare the elements? > > In [27]: 2 < [0,0] > Out[27]: True > > In [28]: [0,0] > 2 > Out[28]: True > > Compares memory locations? > > In [28]: [2] < [0,0] > Out[28]: False > > Lexical ordering? > > > I don't know how python interprets these things. That said, I suspect your > example should behave better, but it might give strange results sometimes > anyway. > > Chuck > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From charlesr.harris at gmail.com Sun Oct 29 21:17:34 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 29 Oct 2006 19:17:34 -0700 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: On 10/29/06, Tom Denniston wrote: > > Oh. My mistake. I thought I had an array of 2 objects which were ints. I > actually had an array of one list of 2 ints. It works properly if I > construct the array properly. > I think there is actually a bug here: In [61]: sort(array([3,2], dtype=object)) Out[61]: array([2, 3], dtype=object) In [62]: argmax(array([2,3], dtype=object)) Out[62]: 0 See, the sort works fine. I suspect argmax is using the wrong comparison function. I was just pointing out that sometimes it is hard to know what is going on with objects. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david at ar.media.kyoto-u.ac.jp Mon Oct 30 03:26:36 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 30 Oct 2006 17:26:36 +0900 Subject: A few questions about numpy.distutils Message-ID: <4545B73C.8040005@ar.media.kyoto-u.ac.jp> Hi there, I would like to improve a bit the packaging of pyaudio, and for that, I need to do the following when configuring my package: - to be able to retrieve a header file (sndfile.h). I need the full pathname, because I need to read its content - to be able to retrieve a library file. I need the full pathname, including the library prefix and suffix/extension, because I need to load it with ctypes. I've shamelessly copied and pasted code from numpy.distutils.system_info, and I manage to retrieve informations for my library, and to retrieve the full pathname for sndfile.h, but I don't know how to get the full shared library file. For now, my sndfile_info looks like that: class sndfile_info(system_info): #variables to override section = 'sndfile' notfounderror = SndfileNotFoundError libname = 'sndfile' header = 'sndfile.h' def __init__(self): system_info.__init__(self) def calc_info(self): """ Compute the informations of the library """ # Look for the shared library sndfile_libs = self.get_libs('sndfile_libs', self.libname) lib_dirs = self.get_lib_dirs() for i in lib_dirs: tmp = self.check_libs(i, sndfile_libs) if tmp is not None: info = tmp break else: return # Look for the header file include_dirs = self.get_include_dirs() inc_dir = None for d in include_dirs: p = self.combine_paths(d,self.header) if p: inc_dir = os.path.dirname(p[0]) headername = os.path.abspath(p[0]) break if inc_dir is not None: dict_append(info, include_dirs=[inc_dir], headername = headername) self.set_info(**info) return Any help to get the full library name would be appreciated (I cannot use find_library from ctypes.util, because I have no guarantee that it will return the same one than distutils), Thanks, David ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From yves.frederix at gmail.com Mon Oct 30 05:47:57 2006 From: yves.frederix at gmail.com (Yves) Date: Mon, 30 Oct 2006 11:47:57 +0100 Subject: How to check for nan in an array? Message-ID: <20061030104757.GA6967@kotnet.org> Hi, Is there a way of checking for nan's in an array, as comparing to N.nan doesn't seem to work. In [1]: import numpy as N In [2]: a = N.asarray(0.)/0 Warning: invalid value encountered in divide In [3]: a Out[3]: nan In [4]: a==N.nan Out[4]: False In [5]: b = a.copy() In [6]: a==b Out[6]: False In [7]: b Out[7]: nan In [8]: N.__version__ Out[8]: '1.0.dev3390' Many thanks, YVES ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From yves.frederix at gmail.com Mon Oct 30 05:53:45 2006 From: yves.frederix at gmail.com (Yves) Date: Mon, 30 Oct 2006 11:53:45 +0100 Subject: How to check for nan in an array? In-Reply-To: <20061030104757.GA6967@kotnet.org> References: <20061030104757.GA6967@kotnet.org> Message-ID: <20061030105345.GA7117@kotnet.org> Forget it. N.isnan() does the job. YVES On Mon, Oct 30, 2006 at 11:47:57AM +0100, Yves wrote: > Hi, > > Is there a way of checking for nan's in an array, as comparing to N.nan > doesn't seem to work. > > In [1]: import numpy as N > > In [2]: a = N.asarray(0.)/0 > Warning: invalid value encountered in divide > > In [3]: a > Out[3]: nan > > In [4]: a==N.nan > Out[4]: False > > In [5]: b = a.copy() > > In [6]: a==b > Out[6]: False > > In [7]: b > Out[7]: nan > > In [8]: N.__version__ > Out[8]: '1.0.dev3390' > > > Many thanks, > YVES > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From eddkrne at pc119.net Mon Oct 30 06:16:25 2006 From: eddkrne at pc119.net (GMT Hours) Date: Mon, 30 Oct 2006 12:16:25 +0100 Subject: porting Message-ID: <001001c6fc14$d8598a10$bc0a53d9@SS> Timestamp stack trace etc so too in However am happen before am fixed in are correctly? Its displayed n v notably par am Consoles im getting or two know help much figuring problem thought let know Quote Error am exception returned failure a nsresult is location frame data getvisible latter appears am switching after something havent yet is figured! Pri msg in Quote asqueella is Russia Moscow Nice thinks in that logged though worldquot is pm wrote am flaws uploaded update which allows these together fri updatexml or pointing Mass Installer am note Sorry that is fd dt am wow sweet thanks do or think accesskey dropdown its displayed n v notably a par Consoles im getting two know is help much figuring problem or thought let am know Quote. Faq dont like a default layout in simply right click a select Customize have am your way please keyboard shortcut displaying wont happen Either use of keyconfig or in Developer adding shortcut id of get more detailed timestamp stack trace etc so too However am happen am before fixed are of correctly am look tell me where agree colored messages Just mrtechs modify them in. Trace etc so too However happen is before am fixed are correctly look tell in me or where agree colored a messages! Selected eg selective copying deleted reopen Complete easier porting single patches dialog is visible least a out a Included Winstripe themes appearance of available four additional languages German flickering in gone filtering loading Categories applied agree in. Single patches dialog visible least out Included Winstripe in themes appearance available four additional in languages German flickering gone filtering loading Categories applied a agree a refer such a updown preview handler occasional funny up down Patch didnt official Pinstripe rotates a mode features Collapse is recent or one Quick access a Copy is clipboard of Lang related a Evaluating expression twice produces am results remains am changing. Do think accesskey dropdown a its is displayed n v notably par am. Slovak Czech locales thanks Infoman or Leszek is teo or yczkowski Rastislav Huboan Michal Gebauer Claiming or Flock button open sidebar respected currently Trunk Portuguese French Teboga Menet from is blocked domains is slip when opening possible a several consoles is rare of break Xpcom were Content Ported Lithuanian locale Edgaras Januauskas ignore until closed a. Needed am about replacing am original see a Testing in Version a Revisions or Jumps correct number am an external of editor requires form is issue introduced! Logo Copyright Rights a Reserved Privacy Policy log Register my Posts Watched Topics Login Rules or Forums Knowledge Base submit news. Default layout is simply is right click select Customize have your way please keyboard shortcut displaying wont happen Either use keyconfig or or Developer adding a shortcut id get more of. Changed accesskeys is Context of gt Evaluate problems hopefully hardly any or never Initial is Last edited sat times or total? Huboan Michal Gebauer Claiming a Flock button is open sidebar respected currently Trunk Portuguese in French a Teboga Menet from blocked domains slip. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: though..gif Type: image/gif Size: 7941 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From winkliesrp at oyakeuro.com Mon Oct 30 07:19:23 2006 From: winkliesrp at oyakeuro.com (reserved. are) Date: Mon, 30 Oct 2006 13:19:23 +0100 Subject: our Message-ID: <000801c6fc1d$a2195c60$17bdb651@b00df3904d6014> Jewel Quest Sandscript Classic Sudoku Express Bejeweled or Tumblebugs see Classicsgo a retro am Atari Hasbro Signin Scrabble Monopoly Yahtzee Centipede is Favorite. Hotties at Bellagios upscale is nightclub Book or entry a Hiro Late is night action is this exotic club the hip Maritime Hotel Opium Smoke line in South a Beachs ultimate garden a Tanned a beauties flock Halloween Take turn around track Mario a Andrettis racing of school Make tracks or across Mojave our Hummer Glide over gators wet wild Everglades airboat ride sideways in. Travelsite Welcome Gamescom games plus community gaming forums friends gamescom llc a Aolgames gdc America or ecc Vmware virtual in machine of pc Network Vmtn Forumsuser Groupsvmtn Store in Nonstore am Orders in Cart Pricing Stor a ffb Faqs. Music News Finance Estate Research Learn Shopping is Business Television Travelsite Welcome of Gamescom games plus community gaming forums friends gamescom llc Aolgames gdc America ecc Vmware a virtual machine pc Network a Vmtn Forumsuser Groupsvmtn Store Nonstore Orders Cart Pricing of Stor ffb in Faqs am Find Reseller in Company or Account Product Licensing Sign Register Logout of Help advanced Homeintro Interfaces Homeserver of. Wild Everglades airboat ride sideways in in Wine Country Have minute speedboat Manhattan Scope! Mexico Antigua Aruba Atlanta Bahamas Bahama Island Barbados a Bermuda Boston Cabo Lucaslos Cabos Cancun Cayman in Islands Grand Chicago Cozumel Daytona Beach Disney of World Orlando Dominican rep Puerto a Plata Punta Cana Santo of Domingo Fort Lauderdale of big Kauai Maui Oahu is Honolulu Houston Ixtapa Jamaica in all! Diner is Dash is Restaurant Rescue Defender of Gold Miner Fish Tycoon Panda in Craze Board two Kind Solitaire Mahjong Escape of Ancient China is Mahjomino Aloha Tripeaks am Casino Poker pop Spanish Poker Wsop or Texas Holdem Limit Blackjack Three Download Luxor Scuba of Jewel. Yahtzee Centipede in Favorite Gamesin Expedition mission recover is lost artifacts Titanic is trial check out popular Vegasfish Video. Company Account of Product Licensing Sign in Register or Logout Help advanced Homeintro Interfaces Homeserver am Test Homedata Center Desktop Paperstech nical Success Forumsyour Customer Alliance Consulting vac Signinfind Usabout Homenews Investor Email Page of Print Appliances or Steps Compare Browser Resources Support in Legal or Requests in Usage. Mediocre next or levelnew own private some matching fun Hidden Titanic Mystery Files Prime Suspects am Rumble Cube am Deluxe Tropix Plantasia zen Scrabble Blast Travels Texttwist of Flip Words Arcade Diner! Key in West a via eyw Lake am Tahoe is London Mazatlan Myrtle Orleans City Paris Phoenix Playa del Carmen Rico Vallarta Reno. Many things look when am player use it can be real hassle scanning through loads in small ones that fit needs am come want Buyer Guide has made. Angeles san Francisco Nightlife a Adventure Travel Special Events Nightclubs Stuff Magazine Home is get There Stay am Party. Maya in Rome Italy of Antonio Diego am Scottsdale Arizona Seattle st a Croix Usvi st of Lucia Castries Vieux. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: scanning.gif Type: image/gif Size: 7791 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From joris at ster.kuleuven.be Mon Oct 30 07:53:32 2006 From: joris at ster.kuleuven.be (Joris De Ridder) Date: Mon, 30 Oct 2006 13:53:32 +0100 Subject: astype() behaviour In-Reply-To: <200610271901.39809.faltet@carabos.com> References: <200610271758.54379.joris@ster.kuleuven.be> <200610271901.39809.faltet@carabos.com> Message-ID: <200610301353.32926.joris@ster.kuleuven.be> On Friday 27 October 2006 19:01, Francesc Altet wrote: [FA]: A Divendres 27 Octubre 2006 17:58, Joris De Ridder va escriure: [FA]: > Hi, [FA]: > [FA]: > Is the following behaviour of astype() intentional in NumPy 1.0? [FA]: > [FA]: > >>> x = array([1,2,3]) [FA]: > >>> x.astype(None) [FA]: > [FA]: > array([ 1., 2., 3.]) [FA]: > [FA]: > That is, the int32 is converted to float64. [FA]: [FA]: Yes, I think the behaviour is intended. This is because 'float64' is the [FA]: default type in NumPy from some months ago (before the default was 'int_') OK, updated the astype() example in the www.scipy.org/Numpy_Example_List . J. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From joris at ster.kuleuven.be Mon Oct 30 08:58:39 2006 From: joris at ster.kuleuven.be (Joris De Ridder) Date: Mon, 30 Oct 2006 14:58:39 +0100 Subject: dtype() examples that used to work In-Reply-To: <200610271906.14099.faltet@carabos.com> References: <200610271813.21500.joris@ster.kuleuven.be> <200610271906.14099.faltet@carabos.com> Message-ID: <200610301458.39353.joris@ster.kuleuven.be> On Friday 27 October 2006 19:06, Francesc Altet wrote: [FA]: for example: [FA]: [FA]: In [67]: dtype([('f1', int16)]) [FA]: Out[67]: dtype([('f1', ' Hi, I have a script that crashes, but only if it runs over 9~10 hours, with the following backtrace from gdb. The script uses PyMC, and repeatedly calls (> 1000000) likelihood functions written in fortran and wrapped with f2py. Numpy: 1.0.dev3327 Python: 2.4.3 Does this backtrace give enough info to track the problem or do the gurus need more ? Thanks, David *** glibc detected *** free(): invalid pointer: 0x00002aaaac1257e0 *** Program received signal SIGABRT, Aborted. [Switching to Thread 46912504440528 (LWP 25269)] 0x00002aaaab09011d in raise () from /lib/libc.so.6 (gdb) backtrace #0 0x00002aaaab09011d in raise () from /lib/libc.so.6 #1 0x00002aaaab09184e in abort () from /lib/libc.so.6 #2 0x00002aaaab0c4e41 in __fsetlocking () from /lib/libc.so.6 #3 0x00002aaaab0ca90e in malloc_usable_size () from /lib/libc.so.6 #4 0x00002aaaab0cac56 in free () from /lib/libc.so.6 #5 0x00002aaaabff7770 in PyArray_FromArray (arr=0x1569500, newtype=0x2aaaac1257e0, flags=0) at arrayobject.c:7804 #6 0x00002aaaabfece56 in PyArray_FromAny (op=0x1569500, newtype=0x0, min_depth=0, max_depth=0, flags=0, context=0x0) at arrayobject.c:8257 #7 0x00002aaaabff40b1 in PyArray_MultiIterNew (n=2) at arrayobject.c:10253 #8 0x00002aaaabff44bc in _broadcast_cast (out=0x62b5, in=0x6, castfunc=0x2aaaabfbf5a0 , iswap=-1, oswap=6) at arrayobject.c:7445 #9 0x00002aaaabffe301 in PyArray_CastToType (mp=0x156dca0, at=, fortran_=0) at arrayobject.c:7344 #10 0x00002aaaabffe785 in PyArray_FromScalar (scalar=0x1573b30, outcode=0x2aaaac1257e0) at scalartypes.inc.src:219 #11 0x00002aaaabfecff5 in PyArray_FromAny (op=0x1573b30, newtype=0x2aaaac1257e0, min_depth=0, max_depth=, flags=0, context=0x0) at arrayobject.c:8260 #12 0x00002aaab6038b7b in array_from_pyobj (type_num=11, dims=0x7fffff8f6200, rank=1, intent=, obj=0x1573b30) at build/src.linux-x86_64-2.4/fortranobject.c:653 #13 0x00002aaab6034aa9 in f2py_rout_flib_beta ( capi_self=, capi_args=, capi_keywds=, f2py_func=0x2aaab603e830 ) at build/src.linux-x86_64-2.4/PyMC/flibmodule.c:2601 #14 0x0000000000414490 in PyObject_Call () #15 0x0000000000475de5 in PyEval_EvalFrame () #16 0x00000000004bdf69 in PyDescr_NewGetSet () #17 0x00000000004143eb in PyIter_Next () #18 0x000000000046ba53 in _PyUnicodeUCS4_IsNumeric () #19 0x0000000000477ab1 in PyEval_EvalFrame () #20 0x00000000004783ff in PyEval_EvalCodeEx () #21 0x000000000047699b in PyEval_EvalFrame () #22 0x0000000000476ab6 in PyEval_EvalFrame () #23 0x0000000000476ab6 in PyEval_EvalFrame () #24 0x00000000004783ff in PyEval_EvalCodeEx () #25 0x000000000047699b in PyEval_EvalFrame () #26 0x00000000004783ff in PyEval_EvalCodeEx () #27 0x000000000047699b in PyEval_EvalFrame () #28 0x00000000004783ff in PyEval_EvalCodeEx () #29 0x000000000047699b in PyEval_EvalFrame () #30 0x00000000004783ff in PyEval_EvalCodeEx () #31 0x0000000000478512 in PyEval_EvalCode () #32 0x000000000049c222 in PyRun_FileExFlags () #33 0x000000000049c4ae in PyRun_SimpleFileExFlags () #34 0x0000000000410a80 in Py_Main () #35 0x00002aaaab07d49b in __libc_start_main () from /lib/libc.so.6 #36 0x000000000040ffba in _start () -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From tom.denniston at alum.dartmouth.org Mon Oct 30 11:32:20 2006 From: tom.denniston at alum.dartmouth.org (Tom Denniston) Date: Mon, 30 Oct 2006 10:32:20 -0600 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: Yes I definately agree about the latter! On 10/29/06, Charles R Harris wrote: > > > > On 10/29/06, Tom Denniston wrote: > > > > Oh. My mistake. I thought I had an array of 2 objects which were ints. > > I actually had an array of one list of 2 ints. It works properly if I > > construct the array properly. > > > > I think there is actually a bug here: > > In [61]: sort(array([3,2], dtype=object)) > Out[61]: array([2, 3], dtype=object) > > In [62]: argmax(array([2,3], dtype=object)) > Out[62]: 0 > > See, the sort works fine. I suspect argmax is using the wrong comparison > function. I was just pointing out that sometimes it is hard to know what is > going on with objects. > > > Chuck > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From faltet at carabos.com Mon Oct 30 11:49:03 2006 From: faltet at carabos.com (Francesc Altet) Date: Mon, 30 Oct 2006 17:49:03 +0100 Subject: dtype() examples that used to work In-Reply-To: <200610301458.39353.joris@ster.kuleuven.be> References: <200610271813.21500.joris@ster.kuleuven.be> <200610271906.14099.faltet@carabos.com> <200610301458.39353.joris@ster.kuleuven.be> Message-ID: <1162226943.3965.7.camel@localhost.localdomain> El dl 30 de 10 del 2006 a les 14:58 +0100, en/na Joris De Ridder va escriure: > IMO, record arrays seem powerful, but also intimidating at a first glance. Agreed, specially if you start to nest datatypes. > I think many didactical examples will help getting them into common use. > I made some effort to get updated examples in the Numpy Example List: > > www.scipy.org/Numpy_Example_List#dtype > www.scipy.org/Numpy_Example_List#array > > Could people who already have some experience with it, have a look at > them and give me their opinion? Looks good. I've taken the freedom to add some examples of nested types and recarrays. -- Francesc Altet | Be careful about using the following code -- Carabos Coop. V. | I've only proven that it works, www.carabos.com | I haven't tested it. -- Donald Knuth ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From jmakem01 at qub.ac.uk Mon Oct 30 12:14:31 2006 From: jmakem01 at qub.ac.uk (Jonathan Makem) Date: Mon, 30 Oct 2006 17:14:31 -0000 Subject: Abaqus Python Message-ID: Hi, I work with Abaqus 6.6.1 finite element software. To access the results from a simulation, the Abaqus scripting interface is used in Python by importing OdbAccess modules. These modules can only be accessed using the version of Python that is installed with Abaqus. However, I cannot install numpy for this version of Python. And I can't use Python 2.4 or 2.5 because then I can't use the odbAcces modules. Is it possible to use numpy with the Abaqus version of Python? If so, how? Regards, Jonny. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From oliphant.travis at ieee.org Mon Oct 30 14:03:48 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 30 Oct 2006 12:03:48 -0700 Subject: Conversion about getting array interface into Python Message-ID: <45464C94.4080900@ieee.org> If anybody has a desire to see the array interface into Python, please help by voicing an opinion on python-dev in the discussion about adding data-type objects to Python. There are a few prominent people who don't get why applications would need to share data-type information about memory areas. I need help giving reasons why. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 14:18:52 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 12:18:52 -0700 Subject: glibc invalid pointer error In-Reply-To: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> Message-ID: On 10/30/06, David Huard wrote: > Hi, > I have a script that crashes, but only if it runs over 9~10 hours, with the > following backtrace from gdb. The script uses PyMC, and repeatedly calls (> > 1000000) likelihood functions written in fortran and wrapped with f2py. > Numpy: 1.0.dev3327 > Python: 2.4.3 This sounds awfully reminiscent of the bug I recently mentioned: http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 We left a fresh run over the weekend, but my office mate is currently out of the office and his terminal is locked, so I don't know what the result is. I'll report shortly: we followed Travis' instructions and ran with a fresh SVN build which includes the extra warnings he added to the dealloc routines. You may want to try the same advice, perhaps with information from both of us the gurus may zero in on the problem, if indeed it is the same. Note that I'm not positive it's the same problem, and our backtraces aren't quite the same. But the rest of the scenario is similar: low-level memory crash from glibc, very long run is needed to fire the bug, potentially millions of calls to both numpy and to f2py-wrapped in-house libraries. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From xqmlkttvz at pason.com Mon Oct 30 14:28:25 2006 From: xqmlkttvz at pason.com (programs) Date: Mon, 30 Oct 2006 14:28:25 -0500 Subject: Subaru Discount Message-ID: <000c01c6fc59$91bdc5e0$a12cffcf@homem9afjgt740> Mark is overrated Joel is Stretch Welcome why Many taken in pine. Simply is exactly what minimum skill practice in every is far a should camp am from water sources pitch tent how build in fire if yo first place Rather foremost attitude ethic respecting caring doing part. Spread Have question Want work a for Membership Benefits Overview am Principles Traveling Trainers Team Bios Program or Details Events Requests East West Subaru Discount State is Advocate a Partner Front Country Tools Teaching Awareness Workshops of Trainer Courses Master Educator Course Schedule Locate in Resources Roster Submission Corporate or Nonprofit Education Outfitter Guide of Service Retailer Parks Agencies Staff Contact us History Media Room. Online Today Learn about or Peak and other innovative is Trace programs get the training you is need find out where can it over partners helping to spread Have am question Want work for Membership a Benefits is Overview am Principles Traveling Trainers Team is Bios or Program Details Events Requests or East West Subaru Discount State Advocate Partner Front Country Tools of Teaching or Awareness Workshops is. Enjoy am are critical risk am Also risk continued access wildlands as. Board of Directors Archived lnt Site or Leaving your or mark am is of overrated Joel is Stretch Welcome why of Many taken pine cone or of rock veered off trail dodge mud puddles in gotten in too close wildlife tossed or an apple of core into woods While these in actions may seem or harmless at time until we reduce our impact quality outdoor! Resources Roster is Submission a Corporate Nonprofit Education of Outfitter Guide or Service a Retailer am Parks Agencies Staff Contact us History Media Room Board of Directors in Archived lnt Site of Leaving in your mark is overrated in Joel Stretch Welcome why Many taken of pine a cone or am rock veered. Rock veered of off trail dodge of mud puddles gotten too close! Time is until we reduce our impact quality in outdoor enjoy are critical risk is Also risk continued access wildlands as land management sometimes is take is action protect they manage Unless catches up of with behavior in all outdoors unchanged is possible by presence Solution a Center am Ethics believes. Durable Surfaces a Dispose Waste Properly Minimize am Campfire Respect be Visitors Click of here am detailed am Seven Give back? Can it over is partners helping to spread Have question Want a work for a Membership is Benefits Overview Principles Traveling am Trainers Team is. Submission Corporate Nonprofit Education Outfitter Guide a Service Retailer Parks in Agencies in. Far should camp from water sources pitch a tent how build fire if yo in first place Rather a. That impacts or widespread causes complex simple change am through a research one or person of time set rules nor simply or exactly am what. Courses Master Educator Course Schedule Locate Resources Roster Submission Corporate Nonprofit. Need find out where can of it over partners helping or to or spread Have am question Want work for! Quality am outdoor enjoy are critical risk of Also risk am continued access wildlands am as. From water sources pitch tent how build fire if yo first place Rather foremost attitude is ethic. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: protect.gif Type: image/gif Size: 7946 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From david.huard at gmail.com Mon Oct 30 14:57:43 2006 From: david.huard at gmail.com (David Huard) Date: Mon, 30 Oct 2006 14:57:43 -0500 Subject: glibc invalid pointer error In-Reply-To: References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> Message-ID: <91cf711d0610301157o19f2ba70o214ec1d08adfc406@mail.gmail.com> Ok, I'll update numpy and give it another try tonight. Regards, David 2006/10/30, Fernando Perez : > > On 10/30/06, David Huard wrote: > > Hi, > > I have a script that crashes, but only if it runs over 9~10 hours, with > the > > following backtrace from gdb. The script uses PyMC, and repeatedly calls > (> > > 1000000) likelihood functions written in fortran and wrapped with f2py. > > Numpy: 1.0.dev3327 > > Python: 2.4.3 > > This sounds awfully reminiscent of the bug I recently mentioned: > > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > > We left a fresh run over the weekend, but my office mate is currently > out of the office and his terminal is locked, so I don't know what the > result is. I'll report shortly: we followed Travis' instructions and > ran with a fresh SVN build which includes the extra warnings he added > to the dealloc routines. You may want to try the same advice, perhaps > with information from both of us the gurus may zero in on the problem, > if indeed it is the same. > > Note that I'm not positive it's the same problem, and our backtraces > aren't quite the same. But the rest of the scenario is similar: > low-level memory crash from glibc, very long run is needed to fire the > bug, potentially millions of calls to both numpy and to f2py-wrapped > in-house libraries. > > Cheers, > > f > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez.net at gmail.com Mon Oct 30 15:08:00 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 13:08:00 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: <453D4AD1.5060508@ieee.org> References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/23/06, Travis Oliphant wrote: > I've placed them in SVN (r3384): > > arraydescr_dealloc needs to do something like. > > if (self->fields == Py_None) { > print something > incref(self) > return; > } Here is some more info. We left a long-running job over the weekend with the prints you suggested. Oddly, something happened at the OS level which killed our SSH connection to that machine, but the above numpy dealloc() warning never printed (we logged this). What did happen is that the refcount you suggested we print: sys.getrefcount(numpy.dtype('float')) eventually seems to have wrapped around and gone negative. I'm attaching the log file with those print statements, the key point is that this happens eventually: PSVD Iteration 19 Ref count 1989827662 bar 444 PSVD Iteration 0 Ref count 2021353399 PSVD Iteration 1 Ref count 2143386207 PSVD Iteration 2 Ref count -2001245193 PSVD Iteration 3 Ref count -1915816437 PSVD Iteration 4 Ref count -1902698473 That refcount is for dtype('float') as indicated above. Is it not a problem that this particular refcount goes negative? Eventually it may continue increasing and hit a zero, point at which I imagine that the bad dealloc will occur. Are refcounts stored in signed 32-bit ints? Why? I'd have naively expected them to be stored in unsigned longs to avoid wraparound problems, but maybe I'm completely missing the real problem here. We've started another run to see if we can get the actual crash to happen, will report. Cheers, f -------------- next part -------------- A non-text attachment was scrubbed... Name: numpy_report Type: application/octet-stream Size: 3307 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From dalcinl at gmail.com Mon Oct 30 16:08:23 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 30 Oct 2006 18:08:23 -0300 Subject: [Numpy-discussion] Conversion about getting array interface into Python In-Reply-To: <45464C94.4080900@ieee.org> References: <45464C94.4080900@ieee.org> Message-ID: On 10/30/06, Travis Oliphant wrote: > > If anybody has a desire to see the array interface into Python, please > help by voicing an opinion on python-dev in the discussion about adding > data-type objects to Python. There are a few prominent people who > don't get why applications would need to share data-type information > about memory areas. I need help giving reasons why. > > -Travis Python-Dev is sometimes a hard place to get into. I remember posting some proposals or even bugs (considered bugs for me and others) and getting rather crude responses. Travis said "a few prominent people...". I consider Travis a *very* prominent people in Python world. His **terrific** contribution to NumPy reveals a really smart, always-looking-ahead way of doing things. However, I've seen before strong and disparate opposition to his proposals in Python-Dev. Perhaps the reason for this is simple: few Python core developers are involved in scientific computing and do not have a clear idea of what it is needed for this. I really believe that NumPy/Scipy community should try to raise his voice on Python-Dev. Many NumPy/Scipy users/developers really want to run high-performance Python code. Python is being used in supercomputers, some applications taking advantage of Python (SPaSM) have even won the Gordon Bell Performance Prize. An 25 Tflop/s application involving Python programing language is really a good example of what can be achieved with Python and compiled code interaction. In short, I fully support Travis in his initiative to standardize access to low level binary data, and encourage others like me who really want this to post to Python-Dev. From my part, I will try to post my reasons in connection with my (small) experience developing MPI for Python. Regars, -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From dalcinl at gmail.com Mon Oct 30 16:15:28 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 30 Oct 2006 18:15:28 -0300 Subject: Abaqus Python In-Reply-To: References: Message-ID: On 10/30/06, Jonathan Makem wrote: > > Hi, > > I work with Abaqus 6.6.1 finite element software. To access the results > from a simulation, the Abaqus scripting interface is used in Python by > importing OdbAccess modules. These modules can only be accessed using the > version of Python that is installed with Abaqus. However, I cannot install > numpy for this version of Python. And I can't use Python 2.4 or 2.5 because > then I can't use the odbAcces modules. Is it possible to use numpy with the > Abaqus version of Python? If so, how? > Jonny. Could you specify which version of Python is used with ABAQUS? can you launch the ABAQUS-provided Python interpreter and do: >>> import sys >>> print sys.version Perhaps the 'odbAcces' module is a module provided by ABAQUS? Which platform are you using? Linux? Windows? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Mon Oct 30 16:54:10 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Mon, 30 Oct 2006 14:54:10 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Fernando Perez wrote: > > On 10/23/06, Travis Oliphant wrote: > > > I've placed them in SVN (r3384): > > > > arraydescr_dealloc needs to do something like. > > > > if (self->fields == Py_None) { > > print something > > incref(self) > > return; > > } > > Here is some more info. We left a long-running job over the weekend > with the prints you suggested. Oddly, something happened at the OS > level which killed our SSH connection to that machine, but the above > numpy dealloc() warning never printed (we logged this). > > What did happen is that the refcount you suggested we print: > > sys.getrefcount(numpy.dtype('float')) > > eventually seems to have wrapped around and gone negative. I'm > attaching the log file with those print statements, the key point is > that this happens eventually: > > PSVD Iteration 19 > Ref count 1989827662 > bar 444 > PSVD Iteration 0 > Ref count 2021353399 > PSVD Iteration 1 > Ref count 2143386207 > PSVD Iteration 2 > Ref count -2001245193 > PSVD Iteration 3 > Ref count -1915816437 > PSVD Iteration 4 > Ref count -1902698473 > > That refcount is for dtype('float') as indicated above. Is it not a > problem that this particular refcount goes negative? Eventually it > may continue increasing and hit a zero, point at which I imagine that > the bad dealloc will occur. > > Are refcounts stored in signed 32-bit ints? Why? I'd have naively > expected them to be stored in unsigned longs to avoid wraparound > problems, but maybe I'm completely missing the real problem here. I suspect the real problem is that the refcount keeps going up. Even if it was unsigned it would eventually wrap to zero and with a bit of luck get garbage collected. So probably something isn't decrementing the refcount. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez.net at gmail.com Mon Oct 30 17:01:51 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 15:01:51 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Charles R Harris wrote: > I suspect the real problem is that the refcount keeps going up. Even if it > was unsigned it would eventually wrap to zero and with a bit of luck get > garbage collected. So probably something isn't decrementing the refcount. Oops, my bad: I meant *unsigned long long*, so that the refcount is a 64-bit object. By the time it wraps around, you'll have run out of memory long ago. Having 32 bit ref counters can potentially mean you run out of the counter before you run out of RAM on a system with sufficient memory. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Mon Oct 30 17:13:37 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 30 Oct 2006 15:13:37 -0700 Subject: glibc invalid pointer error In-Reply-To: References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> Message-ID: <45467911.2060207@ee.byu.edu> Fernando Perez wrote: >On 10/30/06, David Huard wrote: > > >>Hi, >>I have a script that crashes, but only if it runs over 9~10 hours, with the >>following backtrace from gdb. The script uses PyMC, and repeatedly calls (> >>1000000) likelihood functions written in fortran and wrapped with f2py. >>Numpy: 1.0.dev3327 >>Python: 2.4.3 >> >> > >This sounds awfully reminiscent of the bug I recently mentioned: > >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > > It actually looks very much like it. I think the problem may be in f2py or in one of the C-API calls where-in there is a reference-count problem with the built-in data-type objects. NumPy won't try to free those anymore which will solve the immediate problem, but there is still a reference-count problem somewhere. The reference to the data-type objects is consumed by constructors that take PyArray_Descr * arguments. So, you often need to INCREF before passing to those constructors. It looks like this INCREF is forgotten in some extension module (perhaps in f2py or PyMC). It's possible it's in NumPy itself, though I've re-checked the code lots of times looking for that specific problem. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 17:31:51 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 15:31:51 -0700 Subject: glibc invalid pointer error In-Reply-To: <45467911.2060207@ee.byu.edu> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> <45467911.2060207@ee.byu.edu> Message-ID: On 10/30/06, Travis Oliphant wrote: > Fernando Perez wrote: > >This sounds awfully reminiscent of the bug I recently mentioned: > > > >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > > > > > > It actually looks very much like it. I think the problem may be in f2py > or in one of the C-API calls where-in there is a reference-count problem > with the built-in data-type objects. > > NumPy won't try to free those anymore which will solve the immediate > problem, but there is still a reference-count problem somewhere. > > The reference to the data-type objects is consumed by constructors that > take PyArray_Descr * arguments. So, you often need to INCREF before > passing to those constructors. It looks like this INCREF is forgotten > in some extension module (perhaps in f2py or PyMC). It's possible it's > in NumPy itself, though I've re-checked the code lots of times looking > for that specific problem. As a data point, our code has almost no manual memory management in C, but lots and lots of f2py-generated wrappers, as well as a lot of weave.inline-generated code. We do have hand-written C extensions, but most of them operate on externally allocated arrays. The one little snippet where we manually manage memory is a copy of numpy's innerproduct() which I simplified and tuned for our purposes; it just does: ret = (PyArrayObject *)PyArray_SimpleNew(nd,dimensions, ap1->descr->type_num); if (ret == NULL) goto fail; [ do computational loop to fill in ret array, no memory management here ] return (PyObject *)ret; fail: Py_XDECREF(ret); return NULL; That's the full extent of our manual memory management, and I don't see any problem with it, but maybe there is: I copied this from numpy months ago and haven't really looked again. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Mon Oct 30 17:36:32 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 30 Oct 2006 15:36:32 -0700 Subject: glibc invalid pointer error In-Reply-To: References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> Message-ID: <45467E70.60700@ee.byu.edu> Fernando Perez wrote: >On 10/30/06, David Huard wrote: > > >>Hi, >>I have a script that crashes, but only if it runs over 9~10 hours, with the >>following backtrace from gdb. The script uses PyMC, and repeatedly calls (> >>1000000) likelihood functions written in fortran and wrapped with f2py. >>Numpy: 1.0.dev3327 >>Python: 2.4.3 >> >> > >This sounds awfully reminiscent of the bug I recently mentioned: > >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > >We left a fresh run over the weekend, but my office mate is currently >out of the office and his terminal is locked, so I don't know what the >result is. I'll report shortly: we followed Travis' instructions and >ran with a fresh SVN build which includes the extra warnings he added >to the dealloc routines. You may want to try the same advice, perhaps >with information from both of us the gurus may zero in on the problem, >if indeed it is the same. > I talked about the reference counting issue. One problem is not incrementing the reference count when it needs to be. The other problem could occur if the reference-count was not decremented when it needed to be and the reference count wrapped from MAX_LONG to 0. This could also create the problem and would be expected for "long-running" processes. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 17:41:22 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 15:41:22 -0700 Subject: glibc invalid pointer error In-Reply-To: <45467E70.60700@ee.byu.edu> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> <45467E70.60700@ee.byu.edu> Message-ID: On 10/30/06, Travis Oliphant wrote: > Fernando Perez wrote: > > >On 10/30/06, David Huard wrote: > > > > > >>Hi, > >>I have a script that crashes, but only if it runs over 9~10 hours, with the > >>following backtrace from gdb. The script uses PyMC, and repeatedly calls (> > >>1000000) likelihood functions written in fortran and wrapped with f2py. > >>Numpy: 1.0.dev3327 > >>Python: 2.4.3 > >> > >> > > > >This sounds awfully reminiscent of the bug I recently mentioned: > > > >http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 > > > >We left a fresh run over the weekend, but my office mate is currently > >out of the office and his terminal is locked, so I don't know what the > >result is. I'll report shortly: we followed Travis' instructions and > >ran with a fresh SVN build which includes the extra warnings he added > >to the dealloc routines. You may want to try the same advice, perhaps > >with information from both of us the gurus may zero in on the problem, > >if indeed it is the same. > > > I talked about the reference counting issue. One problem is not > incrementing the reference count when it needs to be. The other problem > could occur if the reference-count was not decremented when it needed to > be and the reference count wrapped from MAX_LONG to 0. This could also > create the problem and would be expected for "long-running" processes. I just posted the log from that run in the other thread. I'm not sure if that helps you any though. I'm running the code again to see if we see your new warning fire, and will report back. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From peridot.faceted at gmail.com Mon Oct 30 17:49:38 2006 From: peridot.faceted at gmail.com (A. M. Archibald) Date: Mon, 30 Oct 2006 17:49:38 -0500 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 30/10/06, Fernando Perez wrote: > On 10/30/06, Charles R Harris wrote: > > > I suspect the real problem is that the refcount keeps going up. Even if it > > was unsigned it would eventually wrap to zero and with a bit of luck get > > garbage collected. So probably something isn't decrementing the refcount. > > Oops, my bad: I meant *unsigned long long*, so that the refcount is a > 64-bit object. By the time it wraps around, you'll have run out of > memory long ago. Having 32 bit ref counters can potentially mean you > run out of the counter before you run out of RAM on a system with > sufficient memory. Yes, this is a feature(?) of python as it currently stands (I checked 2.5) - reference counts are 32-bit signed integers, so if you have an object that has enough references, python will be exceedingly unhappy: http://mail.python.org/pipermail/python-dev/2002-September/028679.html It is of course possible that you actually have that many references to some object, but it seems to me you'd notice twenty-four gigabytes of pointers floating around... A. M. Archibald ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From matthew.brett at gmail.com Mon Oct 30 17:52:26 2006 From: matthew.brett at gmail.com (Matthew Brett) Date: Mon, 30 Oct 2006 22:52:26 +0000 Subject: dtype.hasobject value Message-ID: <1e2af89e0610301452r7d79aad7pa47b64ae906ebe31@mail.gmail.com> Hi, I notice that the value for: zeros((1,), dtype=object).dtype.hasobject is now 63, whereas previously it had been 1. Is this intended? Thanks, Matthew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Mon Oct 30 18:02:34 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon, 30 Oct 2006 16:02:34 -0700 Subject: glibc invalid pointer error In-Reply-To: <91cf711d0610301157o19f2ba70o214ec1d08adfc406@mail.gmail.com> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> <91cf711d0610301157o19f2ba70o214ec1d08adfc406@mail.gmail.com> Message-ID: <4546848A.60008@ee.byu.edu> David Huard wrote: > Ok, > I'll update numpy and give it another try tonight. > I just fixed some reference-count problems in f2py today. These were of the variety that there was a missing decref that would cause the reference count of certain often-used data-types to increase without bound and eventually wrap (to 0) in long-running processes using f2py. I suspect this is the fundamental problem in both cases. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Mon Oct 30 18:05:49 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 30 Oct 2006 20:05:49 -0300 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Fernando Perez wrote: > On 10/30/06, Charles R Harris wrote: > > > I suspect the real problem is that the refcount keeps going up. Even if it > > was unsigned it would eventually wrap to zero and with a bit of luck get > > garbage collected. So probably something isn't decrementing the refcount. > > Oops, my bad: I meant *unsigned long long*, so that the refcount is a > 64-bit object. By the time it wraps around, you'll have run out of > memory long ago. Having 32 bit ref counters can potentially mean you > run out of the counter before you run out of RAM on a system with > sufficient memory. > > Cheers, FYI, this is what is defined in Include/object.h /* PyObject_HEAD defines the initial segment of every PyObject. */ #define PyObject_HEAD \ _PyObject_HEAD_EXTRA \ Py_ssize_t ob_refcnt; \ struct _typeobject *ob_type; #define Py_INCREF(op) ( \ _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \ (op)->ob_refcnt++) #define Py_DECREF(op) \ if (_Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA \ --(op)->ob_refcnt != 0) \ _Py_CHECK_REFCNT(op) \ else \ _Py_Dealloc((PyObject *)(op)) And '_Py_CHECK_REFCNT' macro will finally call Py_FatalError 'ob_refcnt' is a Py_ssize_t integer, so I think you will not be able to overflow it, unless in case of C code with refcounting bugs. Am I right? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From dalcinl at gmail.com Mon Oct 30 18:10:09 2006 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 30 Oct 2006 20:10:09 -0300 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Lisandro Dalcin wrote: > FYI, this is what is defined in Include/object.h > I forgoy to say in Python-2.5 -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Chris.Barker at noaa.gov Mon Oct 30 18:48:12 2006 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Mon, 30 Oct 2006 15:48:12 -0800 Subject: Conversion about getting array interface into Python In-Reply-To: References: <45464C94.4080900@ieee.org> Message-ID: <45468F3C.8020407@noaa.gov> Lisandro Dalcin wrote: > Perhaps the reason for this is simple: > few Python core developers are involved in scientific computing and do > not have a clear idea of what it is needed for this. Perhaps true, but..... I imagine one of the issues that the python-dev crowd has is that they don't want to add features solely for the scientific computing crowd. Indeed, they don't need to. The features are there in numpy, everyone who uses numpy has access to them. The reason I want to have these features in the standard library is that there are a lot of good uses for them outside the scientific computing community, and, in particular, for interaction between scientific computing and more general purpose packages (GUI toolkits, Image processing, web frameworks, who knows?) Specifically, this is useful stuff for exchanging data with PIL, wxPython (two specific uses I have), and Matplotlib could really use it with all of its back-ends (TK, GTK, QT, etc....). In addition, I don't think it's on the table at the moment, but an n-d array has a lot of use well outside of number crunching. I've introduced a handful of users to numpy just to get access to a nice n-d object array type for completely no-numeric uses. Off to go post on python-dev..... -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 19:23:42 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 17:23:42 -0700 Subject: glibc invalid pointer error In-Reply-To: <4546848A.60008@ee.byu.edu> References: <91cf711d0610300626l63cc5d9ew3a0ff26ace1428f3@mail.gmail.com> <91cf711d0610301157o19f2ba70o214ec1d08adfc406@mail.gmail.com> <4546848A.60008@ee.byu.edu> Message-ID: On 10/30/06, Travis Oliphant wrote: > David Huard wrote: > > > Ok, > > I'll update numpy and give it another try tonight. > > > > I just fixed some reference-count problems in f2py today. These were of > the variety that there was a missing decref that would cause the > reference count of certain often-used data-types to increase without > bound and eventually wrap (to 0) in long-running processes using f2py. > > I suspect this is the fundamental problem in both cases. Many thanks, Travis. We're rebuilding numpy and all of our f2py-generated wrappers, and will start a new run. I'll report on the results as well. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Mon Oct 30 19:28:39 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 30 Oct 2006 17:28:39 -0700 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: On 10/30/06, Lisandro Dalcin wrote: > FYI, this is what is defined in Include/object.h > > /* PyObject_HEAD defines the initial segment of every PyObject. */ > #define PyObject_HEAD \ > _PyObject_HEAD_EXTRA \ > Py_ssize_t ob_refcnt; \ > struct _typeobject *ob_type; > > #define Py_INCREF(op) ( \ > _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \ > (op)->ob_refcnt++) > > #define Py_DECREF(op) \ > if (_Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA \ > --(op)->ob_refcnt != 0) \ > _Py_CHECK_REFCNT(op) \ > else \ > _Py_Dealloc((PyObject *)(op)) > > And '_Py_CHECK_REFCNT' macro will finally call Py_FatalError > > 'ob_refcnt' is a Py_ssize_t integer, so I think you will not be able > to overflow it, unless in case of C code with refcounting bugs. Am I > right? I think you are, and fortunately this indicates that they /did/ change to a longer data type for refcounting in newer pythons. The box where we have this problem is running 2.3 though, and obviously a runaway refcount in f2py can still die even if it's a longer data type. However, Travis mentioned he just fixed precisely a bug like that in f2py, so I'm optimistic, and I'm currently making a new test. Thanks for the info, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From cjw at sympatico.ca Mon Oct 30 20:01:27 2006 From: cjw at sympatico.ca (Colin J. Williams) Date: Mon, 30 Oct 2006 20:01:27 -0500 Subject: Conversion about getting array interface into Python In-Reply-To: <45464C94.4080900@ieee.org> References: <45464C94.4080900@ieee.org> Message-ID: <4546A067.6000405@sympatico.ca> Travis Oliphant wrote: > If anybody has a desire to see the array interface into Python, please > help by voicing an opinion on python-dev in the discussion about adding > data-type objects to Python. There are a few prominent people who > don't get why applications would need to share data-type information > about memory areas. I need help giving reasons why. > > -Travis > It seems to make sense to me. Perhaps a pointer to some document on just what is being proposed would help. Colin W. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From strawman at astraw.com Mon Oct 30 20:40:41 2006 From: strawman at astraw.com (Andrew Straw) Date: Mon, 30 Oct 2006 17:40:41 -0800 Subject: Strange and hard to reproduce crash In-Reply-To: References: <453D3DD7.50506@ieee.org> <453D4AD1.5060508@ieee.org> Message-ID: <4546A999.4000003@astraw.com> Fernando Perez wrote: > Here is some more info. We left a long-running job over the weekend > with the prints you suggested. Oddly, something happened at the OS > level which killed our SSH connection to that machine, but the above > numpy dealloc() warning never printed (we logged this). As an aside, I always use GNU screen when starting long-running jobs just in case something like this happens. screen lets you reconnect to a session from any login to a machine. Just a point of information... -Andrew ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Mon Oct 30 23:47:25 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 30 Oct 2006 21:47:25 -0700 Subject: dtype.hasobject value In-Reply-To: <1e2af89e0610301452r7d79aad7pa47b64ae906ebe31@mail.gmail.com> References: <1e2af89e0610301452r7d79aad7pa47b64ae906ebe31@mail.gmail.com> Message-ID: <4546D55D.30402@ieee.org> Matthew Brett wrote: > Hi, > > I notice that the value for: > > zeros((1,), dtype=object).dtype.hasobject > > is now 63, whereas previously it had been 1. Is this intended? > > Yes. We are using hasobject as an 8-bit flag now and separating out some of the concepts that make object arrays object arrays. This will be in 1.0.1 which will come out soon. It should be binary compatible with 1.0. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 31 00:19:45 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 30 Oct 2006 22:19:45 -0700 Subject: Strange numpy.argmax behavior on object arrays in numpy 1.0. In-Reply-To: References: Message-ID: <4546DCF1.3080105@ieee.org> Tom Denniston wrote: > I recently upgraded to numpy 1.0 from 1.0b5. I noticed that > numpy.argmax behavior is very strange on object arrays. See below: > > (Pdb) numpy.__version__ > '1.0' > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object)) > 0 > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=int)) > 1 > (Pdb) numpy.argmax(numpy.array([2, 3], dtype=object), axis=0) > 0 > > > I would expect the argmax to behave the same on the dtype=int and > dtype=object examples but it doesn't. Am I missing some subtelty or > is this just a bug? 1.0 is the most recent version, right? This is a bug. I've fixed in in SVN. Thanks for the test. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 31 00:22:32 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Mon, 30 Oct 2006 22:22:32 -0700 Subject: matrixdef.py In-Reply-To: <454537A5.1030202@sympatico.ca> References: <454537A5.1030202@sympatico.ca> Message-ID: <4546DD98.5020307@ieee.org> Colin J. Williams wrote: > Line 71 has: data.view(subtype) > > This appears to involve a call to __array_finalize__. Is this an > unconditional call? > Yes. Every-time an array is constructed the __array_finalize__ method is called unless the __array_finalize__ attribute is None > If not, what are the conditions? > > Why not use __init__ to handle these things, since it is always called > after the __new__? > Because __init__ must have the same calling signature as __new__ and we have no way of knowing what it is. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From gnata at obs.univ-lyon1.fr Tue Oct 31 06:04:41 2006 From: gnata at obs.univ-lyon1.fr (Xavier Gnata) Date: Tue, 31 Oct 2006 12:04:41 +0100 Subject: A way to compute a 2d histogram In-Reply-To: <91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com> References: <453E3FE5.1080201@obs.univ-lyon1.fr> <91cf711d0610241002u765170b5hcb9bacc850a205e0@mail.gmail.com> <91cf711d0610241340o3fbcc93x235849b8d63ffc16@mail.gmail.com> <453F57C6.2020101@obs.univ-lyon1.fr> <91cf711d0610250613q58056568p98dbc342728212fc@mail.gmail.com> Message-ID: <45472DC9.2090802@obs.univ-lyon1.fr> Hi, Your histograms functions look fine for me :) As it is a quite usual operation on an array, I would suggest to put it in numpy as numpy.histogram. IMHO, there is no point to create an numpy.stats only for histograms (or do you have plans to move other stats related function to numpy.stats?) Xavier. > Nicolas, thanks for the bug report, I fooled around with argument > passing and should have checked every case. > > You'll find the histogram function that deals with weights on the > numpy trac ticket 189, > I'm waiting for some hints as to where the histogram function should > reside (numpy.histogram, numpy.stats.histogram, ...) before submitting > a patch . > > Salut, > David > > > 2006/10/25, Nicolas Champavert >: > > Hi, > > it would be great if you could add the weight option in the 1D > histogram too. > > Nicolas > > David Huard a ?crit : > > Xavier, > > Here is the patch against svn. Please report any bug. I haven't had > > the time to test it extensively, something that should be done > before > > commiting the patch to the repo. I'd appreciate your feedback. > > > > David > > > > 2006/10/24, David Huard < david.huard at gmail.com > > > >>: > > > > Hi Xavier, > > > > You could tweak histogram2d to do what you want, or you > could give > > me a couple of days and I'll do it and let you know. If you want > > to help, you could write a test using your particular > application > > and data. > > > > David > > > > > > 2006/10/24, Xavier Gnata < gnata at obs.univ-lyon1.fr > > > >>: > > > > Hi, > > > > I have a set of 3 1D large arrays. > > The first 2 one stand for the coordinates of particules and > > the last one > > for their masses. > > I would like to be able to plot this data ie to compute > a 2D > > histogram > > summing the masses in each bin. > > I cannot find a way to do that without any loop on the > indices > > resulting > > too a very slow function. > > > > I'm looking for an elegant way to do that with numpy (or > > scipy??) function. > > > > For instance, scipy.histogram2d cannot do the job because it > > only counts > > the number of samples in each bin. > > There is no way to deal with weights. > > > > Xavier. > > > > > > -- > > ############################################ > > Xavier Gnata > > CRAL - Observatoire de Lyon > > 9, avenue Charles Andr? > > 69561 Saint Genis Laval cedex > > Phone: +33 4 78 86 85 28 > > Fax: +33 4 78 86 83 86 > > E-mail: gnata at obs.univ-lyon1.fr > > > > ############################################ > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web > > services, security? > > Get stuff done quickly with pre-integrated technology to > make > > your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on > > Apache Geronimo > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > > > > > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > > > > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ------------------------------------------------------------------------ > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- ############################################ Xavier Gnata CRAL - Observatoire de Lyon 9, avenue Charles Andr? 69561 Saint Genis Laval cedex Phone: +33 4 78 86 85 28 Fax: +33 4 78 86 83 86 E-mail: gnata at obs.univ-lyon1.fr ############################################ ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant.travis at ieee.org Tue Oct 31 07:57:37 2006 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 31 Oct 2006 05:57:37 -0700 Subject: Need more comments from scientific community on python-dev Message-ID: <45474841.9030802@ieee.org> I'm recruiting more comments on python-dev regarding my two proposals for improving Python's native ability to share ndarray-like information. There is a dearth of scientific-computing and number-crunching-aware people on python-dev. The result is that I sound like a lone voice arguing for something that nobody cares about. When, I don't think that is true. Please, please. If you want Python to grow support for the array interface (or something like it), then please speak up on python-dev. Even something as simple as I really see the need for a way to exchange data-format information between two objects sharing the buffer protocol can be helpful. You can post through the gmane newsgroup interface: gmane.comp.python.devel Find any of the posts on the PEP's I've introduced. Thanks for your help. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From david at ar.media.kyoto-u.ac.jp Tue Oct 31 08:37:31 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Tue, 31 Oct 2006 22:37:31 +0900 Subject: pyaudio 0.3, with docs ! Message-ID: <4547519B.4040201@ar.media.kyoto-u.ac.jp> Hi, I improved pyaudio last WE using indications given by various people on the list or privately, and as I finally got the motivation to set-up something which looks like a webpage, there is a doc with examples which show how to use it. The API to open files for writing is much saner, and the setup.py should be smart enough to grab all informations necessary to the wrapper, including the location of the shared libsndfile: download: http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pyaudio/#installation doc + examples: http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/pyaudio/ I would appreciate to hear reports on platforms which are not linux (windows, mac os X) to see if my the setup.py works there, Cheers, David From aisaac at american.edu Tue Oct 31 11:14:00 2006 From: aisaac at american.edu (Alan Isaac) Date: Tue, 31 Oct 2006 11:14:00 -0500 Subject: [Numpy-discussion] Need more comments from scientific community on python-dev In-Reply-To: <45474841.9030802@ieee.org> References: <45474841.9030802@ieee.org> Message-ID: On Tue, 31 Oct 2006, Travis Oliphant wrote: > Please, please. If you want Python to grow support for > the array interface (or something like it), then please > speak up on python-dev. The easiest access to this discussion for me was http://news.gmane.org/group/gmane.comp.python.devel/ I cannot add to this discussion, but I REALLY hope others will help Travis out here. (A few have.) He is fielding a lot of questions, some of which look to me to be from individuals who are ready to have fairly strong opinions without really understanding the "why" of his proposals. The good news is, there seems to be (on my naive reading) some sympathy for what Travis is trying to do. I think more motivating examples would prove helpful in swinging things. Cheers, Alan Isaac From oliphant at ee.byu.edu Tue Oct 31 12:04:48 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 10:04:48 -0700 Subject: [Numpy-discussion] Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> Message-ID: <45478230.2050709@ee.byu.edu> Fernando Perez wrote: >On 10/31/06, Rich Shepard wrote: > > >>On Tue, 31 Oct 2006, Alan Isaac wrote: >> >> >> >>>The easiest access to this discussion for me was >>>http://news.gmane.org/group/gmane.comp.python.devel/ I cannot add to this >>>discussion, but I REALLY hope others will help Travis out here. (A few >>>have.) He is fielding a lot of questions, some of which look to me to be >>>from individuals who are ready to have fairly strong opinions without >>>really understanding the "why" of his proposals. >>> >>> >> All this is sufficiently far from my areas of expertise that I cannot >>contribute anything useful. Otherwise, I'd be happy to lend support. >> >> > >I actually worry about the same: I really would like to help, but >after reading the whole discussion, I realized that the low-level >details being asked and discussed are something I don't really know >enough to say anything. And I don't want to sound simply saying 'Hey, >Travis is great, listen to him!' to python-dev, since that (asides >from looking silly) can be somewhat counter-productive. > > >How does that sound, Travis? Is that something you think might help >you, esp. since so many of us are feeling woefully underqualified to >lend a useful hand in the actual discussion on python-dev? > > That would be great. I think a couple of things would also be useful. 1) Some way to indicate to python-dev that I'm actually speaking for more than just myself. So, while I agree that just supporting my PEP (which probably in reality needs work) without understanding it is counter-productive, a voice that says. "We really do need this kind of functionality" is at least one more voice. 2) Examples of sharing memory between two objects. PIL is the classic example and has some merit, but because the internal memory layout of the PIL is 'pointer-to-pointers' instead of 'big-chunk-of-memory' it's not a 1-1 match to NumPy and the array interface only can comunicate information about the "mode." But, I can see other examples. PyMedia, PyGame, PyVideo? CVXOPT, PyVoxel. All of these seem to define their own objects which are basically just interpretations of chunks of memory. At one time, we might have said "these should all be sub-classes of the ndarray". Now, we are thinking more along the lines of "these should all expose an array interface". The array interface is still more bulky then it needs to be (it has to go through the attribute-lookup process which can be slow). It would be much better if the extended buffer protocol were available as a function-pointer on the type object of the type. If you have an application where you've ever wanted NumPy in the core. See if the extended buffer protocol serves your purposes and if you agree, voice your approval for the PEP. In my mind, the data-format PEP does not need to go through if there really is a better way to pass data-format information through the buffer protocol. But, the extended buffer protocol we *do* need. -Travis >Regards, > >f >_______________________________________________ >SciPy-user mailing list >SciPy-user at scipy.org >http://projects.scipy.org/mailman/listinfo/scipy-user > > From jdhunter at ace.bsd.uchicago.edu Tue Oct 31 12:13:24 2006 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Tue, 31 Oct 2006 11:13:24 -0600 Subject: [Numpy-discussion] Need more comments from scientific community on python-dev In-Reply-To: <45478230.2050709@ee.byu.edu> (Travis Oliphant's message of "Tue, 31 Oct 2006 10:04:48 -0700") References: <45474841.9030802@ieee.org> <45478230.2050709@ee.byu.edu> Message-ID: <87r6wo5ryj.fsf@peds-pc311.bsd.uchicago.edu> >>>>> "Travis" == Travis Oliphant writes: Travis> All of these seem to define their own objects which are Travis> basically just interpretations of chunks of memory. At Travis> one time, we might have said "these should all be Travis> sub-classes of the ndarray". Now, we are thinking more What about blitting pixel buffers from mpl or chaco agg into various GUI windows, GTK, Tk, WX, etc.... This seems like a ready made case for the array interface. I could pipe in with an example like this if it would help. JDH From oliphant at ee.byu.edu Tue Oct 31 12:25:20 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 10:25:20 -0700 Subject: dtype.hasobject value In-Reply-To: <4546D55D.30402@ieee.org> References: <1e2af89e0610301452r7d79aad7pa47b64ae906ebe31@mail.gmail.com> <4546D55D.30402@ieee.org> Message-ID: <45478700.10207@ee.byu.edu> Travis Oliphant wrote: >Matthew Brett wrote: > > >>Hi, >> >>I notice that the value for: >> >>zeros((1,), dtype=object).dtype.hasobject >> >>is now 63, whereas previously it had been 1. Is this intended? >> >> >> >> > >Yes. We are using hasobject as an 8-bit flag now and separating out >some of the concepts that make object arrays object arrays. > >This will be in 1.0.1 which will come out soon. It should be binary >compatible with 1.0. > > Let me follow up with this. I've changed back dtype.hasobject so that it returns a Python boolean object. Thus comparisons to 1 should work fine. Under the covers it is (dtype->hasobject & NPY_ITEM_HASOBJECT) == NPY_ITEM_HASOBJECT) There is a new attribute (flags) that contains all the flags in the hasobject variable. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From David.L.Goldsmith at noaa.gov Tue Oct 31 12:46:22 2006 From: David.L.Goldsmith at noaa.gov (David L Goldsmith) Date: Tue, 31 Oct 2006 09:46:22 -0800 Subject: Need more comments from scientific community on python-dev In-Reply-To: <45474841.9030802@ieee.org> References: <45474841.9030802@ieee.org> Message-ID: <45478BEE.5040008@noaa.gov> Perhaps you could please elaborate on what you regard as "the issue(s)" and "what's at stake"? I'm still comparatively new (been using python and numpy circa 8-9 months), but so far I haven't encountered any problems I'd attribute to "Python's ... [in]ability to share ndarray-like information". (The most vexing problems I've had have all been with matplotlib, its dual-mode API, and its rather clumsy and haphazard documentation.) I'd like to be supportive, but not blindly-so. DG Travis Oliphant wrote: > I'm recruiting more comments on python-dev regarding my two proposals > for improving Python's native ability to share ndarray-like information. > > There is a dearth of scientific-computing and number-crunching-aware > people on python-dev. The result is that I sound like a lone voice > arguing for something that nobody cares about. When, I don't think that > is true. Please, please. If you want Python to grow support for the > array interface (or something like it), then please speak up on > python-dev. > > Even something as simple as I really see the need for a way to exchange > data-format information between two objects sharing the buffer protocol > can be helpful. > > You can post through the gmane newsgroup interface: > > gmane.comp.python.devel > > Find any of the posts on the PEP's I've introduced. Thanks for your help. > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -- HMRD/ORR/NOS/NOAA ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From charlesr.harris at gmail.com Tue Oct 31 12:55:51 2006 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 31 Oct 2006 10:55:51 -0700 Subject: [SciPy-user] Need more comments from scientific community on python-dev In-Reply-To: <87r6wo5ryj.fsf@peds-pc311.bsd.uchicago.edu> References: <45474841.9030802@ieee.org> <45478230.2050709@ee.byu.edu> <87r6wo5ryj.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: On 10/31/06, John Hunter wrote: > > >>>>> "Travis" == Travis Oliphant writes: > Travis> All of these seem to define their own objects which are > Travis> basically just interpretations of chunks of memory. At > Travis> one time, we might have said "these should all be > Travis> sub-classes of the ndarray". Now, we are thinking more > > What about blitting pixel buffers from mpl or chaco agg into various > GUI windows, GTK, Tk, WX, etc.... This seems like a ready made case > for the array interface. I could pipe in with an example like this if > it would help. I would also like to see the socket object take a more general type than python strings. I suspect there are a lot of such things in python -- interfaces to services and such -- that would benefit from using a more general interface to memory blocks. But I am not a python developer and don't really know how to make that case. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From fperez.net at gmail.com Tue Oct 31 13:18:03 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 31 Oct 2006 11:18:03 -0700 Subject: [Numpy-discussion] Need more comments from scientific community on python-dev In-Reply-To: <45478230.2050709@ee.byu.edu> References: <45474841.9030802@ieee.org> <45478230.2050709@ee.byu.edu> Message-ID: On 10/31/06, Travis Oliphant wrote: > Fernando Perez wrote: > >I actually worry about the same: I really would like to help, but > >after reading the whole discussion, I realized that the low-level > >details being asked and discussed are something I don't really know > >enough to say anything. And I don't want to sound simply saying 'Hey, > >Travis is great, listen to him!' to python-dev, since that (asides > >from looking silly) can be somewhat counter-productive. > > > > > >How does that sound, Travis? Is that something you think might help > >you, esp. since so many of us are feeling woefully underqualified to > >lend a useful hand in the actual discussion on python-dev? > > > > > > That would be great. I think a couple of things would also be useful. OK, my experience so far has been that there's a certain 'activation barrier' with Wikis, but once pages are there, people for some reason feel more comfortable filling in. So even though it's mostly a place holder, I went ahead and made this: http://www.scipy.org/ArrayInterfacePEP Hopefully as the discussion evolves, this page can be filled in with all the necessary info in one place, and it will become in a few days a solid, organized repository of all the key points in this discussion. This will be a LOT easier to refer to in the python-dev battle than randomly scattered bits of emails in the discussion thread. I'll try to spend more time on it over the next few days to fill in, but I'm pretty busy with other things as well, so hopefully others can pitch in as well. Cheers, f ps - one more thing. This guy: http://blog.vrplumber.com/ has been rewriting the OpenGL bindings using ctypes, and I've seen posts from him about numpy (in his blog). He might be able to contribute something... From aisaac at american.edu Tue Oct 31 13:44:02 2006 From: aisaac at american.edu (Alan Isaac) Date: Tue, 31 Oct 2006 13:44:02 -0500 Subject: installing numpy on Tiger Message-ID: If other users will not profit from this question, please consider it OT and ignore it. But perhaps it will prove useful to some potential numpy users. I am a long time Windows user who has been happily using the Python and numpy installers. At work I need to use a Mac for a while. I installed Python 2.5 from python.org painlessly, but it looks like I'll have to compile numpy from source. Do I understand that right? I have used interpreted languages for pretty much all my computing needs, so I am hoping someone patient will point me to or lead me through the steps I need to take to successfully install numpy on this platform. Thank you, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From fperez.net at gmail.com Tue Oct 31 13:54:28 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 31 Oct 2006 11:54:28 -0700 Subject: installing numpy on Tiger In-Reply-To: References: Message-ID: On 10/31/06, Alan Isaac wrote: > If other users will not profit from this question, > please consider it OT and ignore it. But perhaps > it will prove useful to some potential numpy users. > > I am a long time Windows user who has been happily > using the Python and numpy installers. At work I > need to use a Mac for a while. I installed Python 2.5 > from python.org painlessly, but it looks like > I'll have to compile numpy from source. > Do I understand that right? > > I have used interpreted languages for pretty much > all my computing needs, so I am hoping someone patient > will point me to or lead me through the steps I need > to take to successfully install numpy on this platform. This is probably a good starting point: http://www.scipy.org/Installing_SciPy/Mac_OS_X There have been numerous threads on this issue recently, so you may also want to do a bit of searching of the mailing list archives. Cheers, f ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From lists.steve at arachnedesign.net Tue Oct 31 13:58:40 2006 From: lists.steve at arachnedesign.net (Steve Lianoglou) Date: Tue, 31 Oct 2006 13:58:40 -0500 Subject: installing numpy on Tiger In-Reply-To: References: Message-ID: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> Hi Alan, > I installed Python 2.5 > from python.org painlessly, but it looks like > I'll have to compile numpy from source. > Do I understand that right? I believe you'll have to if you want to use Python 2.5. > ... so I am hoping someone patient > will point me to or lead me through the steps I need > to take to successfully install numpy on this platform. I think the instructions on the scipy wiki are pretty thorough: http://scipy.org/Installing_SciPy/Mac_OS_X One thing to note is that I think I've seen on this list that people have been having problems compiling and using num/scipy on Python 2.5 (on any platform (?)) ... I haven't tried it myself, but that's the general feeling I've gotten from some of the emails on that subject. I could be wrong though. Another thing to consider would be to install the "Superpack" which is linked to in the mac section from the scipy downloads page: http://www.scipy.org/Download I also remember seeing on the ML that there were some problems w/ that superpack it not including some config file for ipython or something, but I'm not sure. For what it's worth, I'm using MacPorts (http://macports.org) to manage my python (and a other) installations, and have been compiling scipy/numpy/matplotlib/ipython from svn checkouts every now and again ... this has been working well for me. The version of python you get from macports is 2.4.3 Hope that helps. -steve ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From erin.sheldon at gmail.com Tue Oct 31 14:38:35 2006 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Tue, 31 Oct 2006 14:38:35 -0500 Subject: installing numpy on Tiger In-Reply-To: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> References: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> Message-ID: <331116dc0610311138m2eb27c23g4404a14a7084ad45@mail.gmail.com> Hi Alan - I have not had luck with the binary distros. There is always something that doesn't work, so I will be interested in the results of your efforts. The biggest problem with compiling things yourself is going to be dealing with LAPACK and fortran issues, and the backends in matplotlib. FWIW, the most stable solution I have found that doesn't involve dealing with dependencies yourself is darwinports (now macports). It currently has 1.0b5 numpy and scipy 0.5.1, which depend on python 2.4. Matplotlib worked with the wxPython backend but not others. While not the latest versions it will get you started, and I would guess 1.0 will be available soon (anyone?). I had absolutely zero problems using this once I realized that I had to use wxPython for the plotting backend. The snag is that gfortran is needed, which requires compiling gcc 4.0 which took 6 hours on my powerbook, so you will need some patience. Erin On 10/31/06, Steve Lianoglou wrote: > Hi Alan, > > > > I installed Python 2.5 > > from python.org painlessly, but it looks like > > I'll have to compile numpy from source. > > Do I understand that right? > > I believe you'll have to if you want to use Python 2.5. > > > ... so I am hoping someone patient > > will point me to or lead me through the steps I need > > to take to successfully install numpy on this platform. > > I think the instructions on the scipy wiki are pretty thorough: > http://scipy.org/Installing_SciPy/Mac_OS_X > > One thing to note is that I think I've seen on this list that people > have been having problems compiling and using num/scipy on Python 2.5 > (on any platform (?)) ... I haven't tried it myself, but that's the > general feeling I've gotten from some of the emails on that subject. > I could be wrong though. > > Another thing to consider would be to install the "Superpack" which > is linked to in the mac section from the scipy downloads page: > http://www.scipy.org/Download > > I also remember seeing on the ML that there were some problems w/ > that superpack it not including some config file for ipython or > something, but I'm not sure. > > For what it's worth, I'm using MacPorts (http://macports.org) to > manage my python (and a other) installations, and have been compiling > scipy/numpy/matplotlib/ipython from svn checkouts every now and > again ... this has been working well for me. The version of python > you get from macports is 2.4.3 > > Hope that helps. > -steve > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From ndarray at mac.com Tue Oct 31 15:10:01 2006 From: ndarray at mac.com (Sasha) Date: Tue, 31 Oct 2006 16:10:01 -0400 Subject: Need more comments from scientific community on python-dev In-Reply-To: <45474841.9030802@ieee.org> References: <45474841.9030802@ieee.org> Message-ID: On 10/31/06, Travis Oliphant wrote: > > I'm recruiting more comments on python-dev regarding my two proposals > for improving Python's native ability to share ndarray-like information. > I would love to help, but I feel that I will be on the other side of the disagreement. (That's why I reply here rather than on python-dev first.) I've suggested that numpy could use ctypes way to describe binary data a long time ago: http://sourceforge.net/mailarchive/message.php?msg_id=15395017 I agree that it would not work out of the box, but I don't see any major obstacles to adding the missing features. I think we should attempt to implement a ctypes approach and compare the result to the dtype object approach. If we discover any unimplementable features along the way, this will be a strong argument to abandon ctypes as a standard. The last time I looked at the issue, the only feature that ctypes were lacking for me was the itemsize information (endianness is probably lacking as well, but I did not need it.) I am not sure, however that this information belongs to the type description object rather than carried separately. Note that it is trivial to add an itemsize attribute to ctypes because each primitive ctype has a fixed itemsize and Structure can easily compute its itemsize. Doubling the number of ctypes to allow for different endianness is a more troubling proposition. In any case, I think we should seriously discuss the relative merits of the two approaches on this list and present our view to python-dev when we reach some consensus. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From aisaac at american.edu Tue Oct 31 15:23:27 2006 From: aisaac at american.edu (Alan G Isaac) Date: Tue, 31 Oct 2006 15:23:27 -0500 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> Message-ID: On Tue, 31 Oct 2006, Sasha apparently wrote: > I think we should attempt to implement a ctypes approach You are probably aware that Travis has addressed this at some length on comp.python.devel (for example http://article.gmane.org/gmane.comp.python.devel/84661 http://article.gmane.org/gmane.comp.python.devel/84670 ) but just in case ... Cheers, Alan Isaac ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 31 15:29:41 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 13:29:41 -0700 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> Message-ID: <4547B235.6050904@ee.byu.edu> Sasha wrote: >On 10/31/06, Travis Oliphant wrote: > > >>I'm recruiting more comments on python-dev regarding my two proposals >>for improving Python's native ability to share ndarray-like information. >> >> >> > >I would love to help, but I feel that I will be on the other side of >the disagreement. (That's why I reply here rather than on python-dev >first.) > > Please read my posts about the Python type-object verses normal Python Object situation. That really is the crux of the matter. Ctypes uses a Python type object for every data-format. NumPy uses an instance of a data-type object for every data-format. What advantage do we gain by making every instance of a data-type object *also* a Python type object? We get a lot of head-ache. Have you seen what ctypes had to do? It had to define a new Dictionary object so it could attach it to the tp_dict parameter because you can't just inherit from the PyTypeObject and add the fields you want to the structure. This is my argument. I for one am not going to put any effort in that direction. People are free to do it, if they want, of course. But, it's no small change. I would, however, put effort into "undertstanding ctypes objects" as data-type objects. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 31 15:30:39 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 13:30:39 -0700 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> Message-ID: <4547B26F.80103@ee.byu.edu> Sasha wrote: >On 10/31/06, Travis Oliphant wrote: > > >>I'm recruiting more comments on python-dev regarding my two proposals >>for improving Python's native ability to share ndarray-like information. >> >> >> > >I would love to help, but I feel that I will be on the other side of >the disagreement. (That's why I reply here rather than on python-dev >first.) > > There's actually two issues here as well. 1) The extended buffer protocol 2) How to exchange data-format information through it. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From Beating_Back_the_Shorters_nathaniel at gmail.com Tue Oct 31 16:30:43 2006 From: Beating_Back_the_Shorters_nathaniel at gmail.com (Crystal Xiong) Date: Tue, 31 Oct 2006 15:30:43 -0600 Subject: Haley Padgett Message-ID: <160519020459.XAA13898Beating_Back_the_Shorters_nathaniel@gmail.com> A non-text attachment was scrubbed... Name: not available Type: multipart/alternative Size: 20 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From ndarray at mac.com Tue Oct 31 16:36:58 2006 From: ndarray at mac.com (Sasha) Date: Tue, 31 Oct 2006 17:36:58 -0400 Subject: Need more comments from scientific community on python-dev In-Reply-To: <4547B235.6050904@ee.byu.edu> References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu> Message-ID: On 10/31/06, Travis Oliphant wrote: > Please read my posts about the Python type-object verses normal Python > Object situation. That really is the crux of the matter. > I read the whole python-dev thread before replying. I may be a little biased because I never liked somewhat cryptic letter codes in Numeric and the change of codes from Numeric to numpy did not contribute to my sympathy particularly when the convert script changed all unrelated instances of 'b' in my code to something else. I am also not a big fan of record arrays. I believe that numeric data should be stored in "inverted tables," where columns of homogeneous data are stored contiguously. With this disclaimer, I will address a few issues below. > Ctypes uses a Python type object for every data-format. > NumPy uses an instance of a data-type object for every data-format. > Yes, but AFAIK this is a recent innovation. Numarray used type objects and Numeric simply used letter codes. > What advantage do we gain by making every instance of a data-type object > *also* a Python type object? I think the main advantage is that you can have instances: >>> c_int(42) c_int(42) Of course, numpy has scalars for that, but ctypes also has fixed-length arrays, that are somewhat different from ndarrays: >>> a10 = c_int*10 >>> a10() <__main__.c_int_Array_10 object at 0x2a95816958> > We get a lot of head-ache. Have you seen > what ctypes had to do? It had to define a new Dictionary object so it > could attach it to the tp_dict parameter because you can't just inherit > from the PyTypeObject and add the fields you want to the structure. > This is my argument. > But, isn't this someone else's head-ache? Someone has already gone through all these contortions, why not reuse the effort? Others on python-dev described some specific needs of ctypes that your datatype object does not address. Your point seems to be that numpy does not share these needs and could use a much simpler approach. > I for one am not going to put any effort in that direction. People are > free to do it, if they want, of course. But, it's no small change. Can you present some use cases that illustrate the advantage of your datatype approach best? I will try to implement them with ctypes. > I would, however, put effort into "undertstanding ctypes objects" as > data-type objects. Yes, this is the key. I think we should better understand ctypes limitations before proposing an alternative. At the end of the day, it is better to have buffer protocol that describes the data using ctypes types than to have no standard type information at all. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 31 17:34:01 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 15:34:01 -0700 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu> Message-ID: <4547CF59.4020905@ee.byu.edu> Sasha wrote: >On 10/31/06, Travis Oliphant wrote: > > >Yes, this is the key. I think we should better understand ctypes >limitations before proposing an alternative. > I already understand it's practical limitations --- type objects as data-type instances is too bulky and too restrictive. You have to "be a Python type object" in order to be a data-type object. I wish others would actually understand the difference. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From oliphant at ee.byu.edu Tue Oct 31 17:48:19 2006 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue, 31 Oct 2006 15:48:19 -0700 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu> Message-ID: <4547D2B3.5000808@ee.byu.edu> Sasha wrote: >On 10/31/06, Travis Oliphant wrote: > > > >>Please read my posts about the Python type-object verses normal Python >>Object situation. That really is the crux of the matter. >> >> >> >I read the whole python-dev thread before replying. I may be a little >biased because I never liked somewhat cryptic letter codes in Numeric >and the change of codes from Numeric to numpy did not contribute to my >sympathy particularly when the convert script changed all unrelated >instances of 'b' in my code to something else. > The letter codes are not data-type objects. The letter codes are only there for historical reasons (and they are in the struct and array modules too so blame Python...) >I am also not a big >fan of record arrays. I believe that numeric data should be stored in >"inverted tables," where columns of homogeneous data are stored >contiguously. > Sure. I understand this. This was my argument too when Numarray was first proposed. But, how do you memory-map a record where the data is actually stored differently? Numeric's answer is "you don't" but that is un-acceptable. This is why record-arrays were created and the whole pandora's box of data-types was opened. > > >>Ctypes uses a Python type object for every data-format. >>NumPy uses an instance of a data-type object for every data-format. >> >> >> >Yes, but AFAIK this is a recent innovation. Numarray used type >objects and Numeric simply used letter codes. > > > Recent if 1 year is recent, and only if you worry precisely about when they were Python objects. Numeric always used a PyArray_Descr * structure (which was a PyObject_HEAD away from being a Python object) to describe data. The letter codes were just simple ways to "represent" those underlying structures (which could have been exposed to Python as "data-types" from the very beginning. >>What advantage do we gain by making every instance of a data-type object >>*also* a Python type object? >> >> > >I think the main advantage is that you can have instances: > > >>>>c_int(42) >>>> >>>> >c_int(42) > >Of course, numpy has scalars for that, but ctypes also has >fixed-length arrays, that are somewhat different from ndarrays: > > Sure, but you don't "need" an instance for every memory-layout description. If you want one, then great, ctypes gives it to you. But "requiring" data-type to be encoded in a type object is over-kill. >> We get a lot of head-ache. Have you seen >>what ctypes had to do? It had to define a new Dictionary object so it >>could attach it to the tp_dict parameter because you can't just inherit >>from the PyTypeObject and add the fields you want to the structure. >>This is my argument. >> >> >> > >But, isn't this someone else's head-ache? Someone has already gone >through all these contortions, why not reuse the effort? > No, it's not that simple. We have a headache whenever we want to do something like I just did and separate out the concepts of what makes a Python Object a Python object. Now, we don't just modify a simple C-structure (PyArray_Descr *), we have to modify a "meta-type" or a altered dictionary and get that change put in to ctypes. Inheriting from Python type objects is harder. People who have apparently never tried seem to think it's not but it is. >Others on >python-dev described some specific needs of ctypes that your datatype >object does not address. Your point seems to be that numpy does not >share these needs and could use a much simpler approach. > > No, that's not the point. The data-type object could easily be extended to accomodate those needs. The point is using data-types as instances of a regular Python object or data-types as instances of a "type" object. >>I for one am not going to put any effort in that direction. People are >>free to do it, if they want, of course. But, it's no small change. >> >> > >Can you present some use cases that illustrate the advantage of your >datatype approach best? > > >I will try to implement them with ctypes. > > This is missing the point. I have no doubt ctypes "could" be used on the Python side to describe a data-type. But, we need the PyArray_Descr * structure in NumPy. What are you going to replace the PyArray_Descr * structure with? That is the point. >> I would, however, put effort into "undertstanding ctypes objects" as >>data-type objects. >> >> > >Yes, this is the key. I think we should better understand ctypes >limitations before proposing an alternative. > Please understand what I meant. I meant putting effort into getting PyArray_DescrConverter to allow ctypes inputs and convert to the appropriate PyArray_Descr * structure. I already understand ctypes objects. I want the dtype() command to also understand them. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From josh.p.marshall at gmail.com Tue Oct 31 17:51:16 2006 From: josh.p.marshall at gmail.com (Josh Marshall) Date: Wed, 1 Nov 2006 09:51:16 +1100 Subject: Need more comments from scientific community on python-dev In-Reply-To: References: Message-ID: On 11/31/06, Fernando Perez wrote: > Fernando Perez wrote: > ps - one more thing. This guy: > > http://blog.vrplumber.com/ > > has been rewriting the OpenGL bindings using ctypes, and I've seen > posts from him about numpy (in his blog). He might be able to > contribute something... I've been working on the OSX port and extensions for OpenGL-ctypes. (now released as PyOpenGL-3.00a, please test!). What Mike has done is define setuptools plugins to interface to different kinds of array data. The default type is ctypes arrays, or numpy arrays if numpy is installed. The data types handled are ctypes sized arrays, ctypes pointers, strings (read-only), and Python lists. (and old Numeric/ numarray via a non-default build.) Summaries of development and usage, resp, are found: http://pyopengl.sourceforge.net/ctypes/development.html http://pyopengl.sourceforge.net/ctypes/using.html Notes on array handling from the above: > Perhaps the most complex mechanisms in OpenGL-ctypes are those > which implement the array-based operations which allow for using > low-level blocks of formatted data to communicate with the OpenGL > implementation. OpenGL-ctypes preferred basic array implementation > is the (new) numpy reimplementation of the original Numeric Python. > > The array handling functionality provided within OpenGL-ctypes is > localised to the OpenGL.arrays sub-package. Within the package, > there are two major classes, one (the FormatHandler) which > implements an interface to a way of storing data in Python, and > another (the ArrayDatatype) which models an OpenGL array format. > The ArrayDatatype classes use FormatHandlers to manipulate array- > compatible objects for use in the system. and on FormatHandlers, for each of the datatypes I mentioned earlier: > Each format handler is responsible for implementing an API that > ArrayDatatypes can use to work with the Python data-format. Data- > formats can support a subset of the API, they only need to support > those aspects of the data-format which make sense. Now, I haven't spent much time looking at these parts of OpenGL- ctypes, as they have just worked for me. I would think that it would be trivial to write a FormatHandler which uses the ndarray interface and data type description to use any object implenting it as an input for OpenGL. This would include things such as PIL images. Mike, can you give us your opinion on how a standardised data type descriptor would be helpful for PyOpenGL? The PEP and some information about it can be found here: http://www.scipy.org/ArrayInterfacePEP Cheers, Josh ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From wbaxter at gmail.com Tue Oct 31 19:26:22 2006 From: wbaxter at gmail.com (Bill Baxter) Date: Wed, 1 Nov 2006 09:26:22 +0900 Subject: Need more comments from scientific community on python-dev In-Reply-To: <45474841.9030802@ieee.org> References: <45474841.9030802@ieee.org> Message-ID: One thing I see with PIL and Matplotlib and PyOpenGL is that they're having to add specific support for Numpy and numeric and numarray and PIL and ctypes etc. I'm also in the camp that doesn't really understand this stuff well enough to really argue in favor of it, but would the proposed extensions make it possible for all these different packages to just have to support "buffer protocol" and magically be able to work with numpy or numeric without having to introduce explicit dependencies and support code for all those other packages? If I've got the facts straight then I'd be happy to go and say that on comp.lang.python. --bb On 10/31/06, Travis Oliphant wrote: > > > I'm recruiting more comments on python-dev regarding my two proposals > for improving Python's native ability to share ndarray-like information. > > There is a dearth of scientific-computing and number-crunching-aware > people on python-dev. The result is that I sound like a lone voice > arguing for something that nobody cares about. When, I don't think that > is true. Please, please. If you want Python to grow support for the > array interface (or something like it), then please speak up on > python-dev. > > Even something as simple as I really see the need for a way to exchange > data-format information between two objects sharing the buffer protocol > can be helpful. > > You can post through the gmane newsgroup interface: > > gmane.comp.python.devel > > Find any of the posts on the PEP's I've introduced. Thanks for your help. > > -Travis > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From alexander.belopolsky at gmail.com Tue Oct 31 18:51:57 2006 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 31 Oct 2006 18:51:57 -0500 Subject: Need more comments from scientific community on python-dev In-Reply-To: <4547D2B3.5000808@ee.byu.edu> References: <45474841.9030802@ieee.org> <4547B235.6050904@ee.byu.edu> <4547D2B3.5000808@ee.byu.edu> Message-ID: On 10/31/06, Travis Oliphant wrote: ... > Please understand what I meant. I meant putting effort into getting > PyArray_DescrConverter to allow ctypes inputs and convert to the > appropriate PyArray_Descr * structure. I already understand ctypes > objects. I want the dtype() command to also understand them. > I think I am starting to understand. Forgive me for being slow. Is it correct that you don't mind writing c_int * 10 instead of dtype((' References: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> <331116dc0610311138m2eb27c23g4404a14a7084ad45@mail.gmail.com> Message-ID: <6ce0ac130610311646m79374541l57a5e1753af56e81@mail.gmail.com> Nice binaries for gfortran can be found here: hpc.sourceforge.net/ On 10/31/06, Erin Sheldon wrote: > Hi Alan - > > I have not had luck with the binary distros. There is always > something that doesn't work, so I will be interested in > the results of your efforts. > > The biggest problem with compiling things yourself is going > to be dealing with LAPACK and fortran issues, and the > backends in matplotlib. > > FWIW, the most stable solution I have found that doesn't involve > dealing with dependencies yourself is darwinports (now > macports). It currently has 1.0b5 numpy and scipy 0.5.1, which > depend on python 2.4. Matplotlib worked with the wxPython > backend but not others. While not the latest versions it will get > you started, and I would guess 1.0 will be available soon (anyone?). > I had absolutely zero problems using this once I realized that I > had to use wxPython for the plotting backend. The snag > is that gfortran is needed, which requires compiling gcc 4.0 which > took 6 hours on my powerbook, so you will need some patience. > > Erin > > On 10/31/06, Steve Lianoglou wrote: > > Hi Alan, > > > > > > > I installed Python 2.5 > > > from python.org painlessly, but it looks like > > > I'll have to compile numpy from source. > > > Do I understand that right? > > > > I believe you'll have to if you want to use Python 2.5. > > > > > ... so I am hoping someone patient > > > will point me to or lead me through the steps I need > > > to take to successfully install numpy on this platform. > > > > I think the instructions on the scipy wiki are pretty thorough: > > http://scipy.org/Installing_SciPy/Mac_OS_X > > > > One thing to note is that I think I've seen on this list that people > > have been having problems compiling and using num/scipy on Python 2.5 > > (on any platform (?)) ... I haven't tried it myself, but that's the > > general feeling I've gotten from some of the emails on that subject. > > I could be wrong though. > > > > Another thing to consider would be to install the "Superpack" which > > is linked to in the mac section from the scipy downloads page: > > http://www.scipy.org/Download > > > > I also remember seeing on the ML that there were some problems w/ > > that superpack it not including some config file for ipython or > > something, but I'm not sure. > > > > For what it's worth, I'm using MacPorts (http://macports.org) to > > manage my python (and a other) installations, and have been compiling > > scipy/numpy/matplotlib/ipython from svn checkouts every now and > > again ... this has been working well for me. The version of python > > you get from macports is 2.4.3 > > > > Hope that helps. > > -steve > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, security? > > Get stuff done quickly with pre-integrated technology to make your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From iwncfbs at pcibuf.com Tue Oct 31 21:19:13 2006 From: iwncfbs at pcibuf.com (always remind) Date: Wed, 1 Nov 2006 10:19:13 +0800 Subject: RatesStore StatusEasy ReturnsAll Message-ID: <000a01c6fd5c$1fcee870$b6b0a8dc@cyserver> Might got is some! All abilities asking me is questions about many different things. Bar During California Ladies State! Rulings tax Tournament vol? Gambling Live Coverage of Schedule Results Database of. Additional blind after winning preceding pot am. Display of among in them Noble Powells in Syndetic Solutions or Ebsco Thomson of. Washington in Tavis Smiley of inspire am mentors achieving success. Action seven five days Come am View Article? >From players all a. Solutions Ebsco Thomson Gale Statement photocopy material? Put you too in Thats need of ask anyones permission go? Card Player Magazine am What if Wanna Leave. Pay not getting. Analyst Issues for Gift of Blogs a. Around he ready a reader Irene s. Analyst is Issues for Gift Blogs is it is of. Ccc Rosewood Danvers ma provides licenses variety. Might got is some! Rookies Holdem Humor a Legal Lowball Money. Inspire mentors achieving success Clean. Soonbampn Annex Oreilly Computing Gamespc Video Statefind. Support Press Contact Terms Copyright is copy rights reserved. Pooh Little Einstein Princess favorites! Makes fair a table remain protect them. Rulings tax Tournament vol? Librarians in serious buyers holds. Just do a anytime? Humor Legal Lowball Money Management Omaha Math Psychology! All abilities asking me is questions about many different things. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Us.gif Type: image/gif Size: 8792 bytes Desc: not available URL: -------------- next part -------------- ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 -------------- next part -------------- _______________________________________________ Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From george.sakkis at gmail.com Tue Oct 31 18:38:43 2006 From: george.sakkis at gmail.com (George Sakkis) Date: Tue, 31 Oct 2006 23:38:43 -0000 Subject: Reading records from file and sorting Message-ID: <1162337923.747767.323100@b28g2000cwb.googlegroups.com> Is there a more elegant and/or faster way to read some records from a file and then sort them by different fields ? What I have now is too specific and error-prone in general: import numpy as N records = N.fromfile(a_file, dtype=N.dtype('i2,i4')) records_by_f0 = records.take(records.getfield('i2').argsort()) records_by_f1 = records.take(records.getfield('i4',2).argsort()) If there's a better way, I'd like to see it; bonus points for in-place sorting. Thanks, George ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 From erin.sheldon at gmail.com Tue Oct 31 23:02:51 2006 From: erin.sheldon at gmail.com (Erin Sheldon) Date: Tue, 31 Oct 2006 23:02:51 -0500 Subject: installing numpy on Tiger In-Reply-To: <6ce0ac130610311646m79374541l57a5e1753af56e81@mail.gmail.com> References: <0CC02B41-653E-4294-A357-F28349FCECDF@arachnedesign.net> <331116dc0610311138m2eb27c23g4404a14a7084ad45@mail.gmail.com> <6ce0ac130610311646m79374541l57a5e1753af56e81@mail.gmail.com> Message-ID: <331116dc0610312002x38212d63o1d495745186607eb@mail.gmail.com> On 10/31/06, Brian Granger wrote: > Nice binaries for gfortran can be found here: > > hpc.sourceforge.net/ Do you know how to make macports recognize this compiler if it is outside the macports tree? I had a version of gfortran (4.2.0) already on my system, but it insisted on compiling its own version (4.1.1) Erin ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642