From Peterlist at eyou.com Mon Nov 1 00:00:37 2004 From: Peterlist at eyou.com (Tim) Date: Mon Nov 1 00:00:37 2004 Subject: [Numpy-discussion] Promote Your Marketing Message-ID: An HTML attachment was scrubbed... URL: From jmiller at stsci.edu Mon Nov 1 07:37:34 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Nov 1 07:37:34 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: References: Message-ID: <1099323100.32767.116.camel@halloween.stsci.edu> On Sun, 2004-10-31 at 12:31, Perry Greenfield wrote: [SNIP] > > the wrong choice. Not sure what is really best, although personally > > feel breaking compatibility is fine if the result is better. Is there > > not already a sub-package numeric within numarray that provides Numeric > > compatibility? Such a package could at least provide wrappers with > > compatible behavior for people who need that. > > > At the moment the numeric module provides more Numeric compatibility > (but not complete). In matplotlib we use a module called numerix to > provide a uniform interface to both Numeric and numerix (along with In anyone was scratching their head, I think Perry meant to say "both Numeric and numarray" here. In particular, matplotlib's array package proxy, numerix, uses numarray.numeric and some of the add-ons to supply Numeric-like functionality. Because numarray.numeric is Numeric-like, it is actually a subset of numarray, with differences in put(), take(), and nonzero() among other things. On the plus side, the simpler numeric put(), take(), and nonzero() are ports (can't get more compatible... any difference is a bug) and pure C (so they're faster for small arrays... but still slower than Numeric). Regards, Todd From Chris.Barker at noaa.gov Mon Nov 1 15:35:24 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Nov 1 15:35:24 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: References: Message-ID: <4186C686.5030407@noaa.gov> Perry Greenfield wrote: > This aspect of flat can be considered a wart. There are three different > desired behaviors depending on who you talk to. For efficiency reasons, > some only want flat (and even ravel) to work if the array is already > contiguous; that is, they don't want copies unless they ask for them. This isn't just efficiency: having a function (or method) that sometimes returns a copy, and sometimes a reference is asking for bugs. What happens if I make a change to the result in a function? Sometimes it will change the parent array, sometimes not. > otherwise for it to return a view. Yet others always want a copy. > So, are three different versions needed? Or options to a function? > The drawback of .flat (as an attribute) is there is only one choice > for behavior. A agree. I vote for a method. By the way, is it really impossible to have a discontiguous 1-d array? I'm not wizard at C or C++ but I've worked with the Numeric api enough to see what the problem is. However, it seems that there should be way to have a "get the n-th element" function or method to the Numarray object that should then work on polymorphic types, one of which would be a rank-1 non-contiguous array. Perhaps there is way too much existing code that relies on the array->strides[n] approach to introduce this now, but I think this kind of thing would be the key to making it easier to write optimized Numarray functions. > I'd > very much prefer not proliferate any more flavors of behavior > and just settle on one. +5 on this. I'd really like SciPy and numarray both to have the goal of merging the two. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From strawman at astraw.com Mon Nov 1 15:38:17 2004 From: strawman at astraw.com (Andrew Straw) Date: Mon Nov 1 15:38:17 2004 Subject: [Numpy-discussion] Re: floating point exception weirdness (Andrew Straw) In-Reply-To: <1099025806.2742.23.camel@f1> References: <1099025806.2742.23.camel@f1> Message-ID: <4186C7A1.6080107@astraw.com> Steve Chaplin wrote: >>Just a small addendum, (which I hope will spur on bug-fixing once Todd >>et al. are back from the conference -- let me know if I should file a >>sourceforge bug report): >> >> > >I've not read all this thread so I don't know the full background. But I >had a floating point / SSE problem using numarray. > >It turned out to be a glibc not numarray problem and was solved by >upgrading glibc. >http://sources.redhat.com/bugzilla/show_bug.cgi?id=10 >There was also a SourceForge bug report but I can't locate it. > > > The test code you filed with the RedHat bugzilla report at the URL you gave (above) does indeed produce the bug you mention in that bug report. So, I now think this is not a numarray bug, but a debian/libc bug. Thanks for your help, Steve -- I never would have found this myself. Now thinking about how to fix my glibc... Andrew From strawman at astraw.com Mon Nov 1 22:12:15 2004 From: strawman at astraw.com (Andrew Straw) Date: Mon Nov 1 22:12:15 2004 Subject: [Numpy-discussion] Re: floating point exception weirdness (Andrew Straw) In-Reply-To: <4186C7A1.6080107@astraw.com> References: <1099025806.2742.23.camel@f1> <4186C7A1.6080107@astraw.com> Message-ID: <8535C234-2C95-11D9-9367-003065D64CEA@astraw.com> I filed a bug report on this with the Debian bug tracking system. (See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=279294 ) It looks like it's fixed in the upstream glibc sources -- the Debian maintainers just have to apply the patch, so hopefully it won't take too long. Cheers! Andrew On Nov 1, 2004, at 3:32 PM, Andrew Straw wrote: > Steve Chaplin wrote: > >>> Just a small addendum, (which I hope will spur on bug-fixing once >>> Todd et al. are back from the conference -- let me know if I should >>> file a sourceforge bug report): >>> >> >> I've not read all this thread so I don't know the full background. >> But I >> had a floating point / SSE problem using numarray. >> >> It turned out to be a glibc not numarray problem and was solved by >> upgrading glibc. >> http://sources.redhat.com/bugzilla/show_bug.cgi?id=10 >> There was also a SourceForge bug report but I can't locate it. >> >> > The test code you filed with the RedHat bugzilla report at the URL you > gave (above) does indeed produce the bug you mention in that bug > report. So, I now think this is not a numarray bug, but a debian/libc > bug. > > Thanks for your help, Steve -- I never would have found this myself. > > Now thinking about how to fix my glibc... > Andrew > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader's Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From strawman at astraw.com Tue Nov 2 09:03:33 2004 From: strawman at astraw.com (Andrew Straw) Date: Tue Nov 2 09:03:33 2004 Subject: [Numpy-discussion] floating point exception weirdness In-Reply-To: <1092221365.3752.32.camel@localhost.localdomain> References: <4119BBFC.6020304@astraw.com> <1092221365.3752.32.camel@localhost.localdomain> Message-ID: <69922EEF-2CF0-11D9-9367-003065D64CEA@astraw.com> I hunted down the error and modified Doc/INSTALL.txt (patch below, please modify numarray sources) to indicate that the problem is not confined to Fedora Core1 but to any i386 linux with libc < 2.3.3 and SSE-using code. With my newly patched and compiled libc, my system runs great, and no more mysterious Floating Point Exceptions! Cheers! Andrew Index: Doc/INSTALL.txt =================================================================== RCS file: /cvsroot/numpy/numarray/Doc/INSTALL.txt,v retrieving revision 1.9 diff -r1.9 INSTALL.txt 209,212c209,212 < 1. Fedora Core1 -- if compiling against GNU libc on i386 and enabling < SSE processor functions (with something like "-march=athlon-xp") then < libc versions above 2.3.3 will be needed. < --- > 1. i386 linux -- if compiling against GNU libc on i386 and enabling > SSE processor functions (with something like "-march=athlon-xp" or > using other libraries that utilize SSE such as atlas or Intel IPP) > then libc version 2.3.3 or above will be needed. From jmiller at stsci.edu Tue Nov 2 14:34:57 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Nov 2 14:34:57 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: <4186C686.5030407@noaa.gov> References: <4186C686.5030407@noaa.gov> Message-ID: <1099434444.2706.137.camel@halloween.stsci.edu> On Mon, 2004-11-01 at 18:28, Chris Barker wrote: > Perry Greenfield wrote: > > This aspect of flat can be considered a wart. There are three different > > desired behaviors depending on who you talk to. For efficiency reasons, > > some only want flat (and even ravel) to work if the array is already > > contiguous; that is, they don't want copies unless they ask for them. > > This isn't just efficiency: having a function (or method) that sometimes > returns a copy, and sometimes a reference is asking for bugs. What > happens if I make a change to the result in a function? Sometimes it > will change the parent array, sometimes not. I looked at this some more and discovered we're doing what Numeric does with the .flat attribute: raise an exception for non-contiguous arrays. So backward compatibility is one motive for keeping .flat the way it is now. > > otherwise for it to return a view. Yet others always want a copy. > > So, are three different versions needed? Or options to a function? > > The drawback of .flat (as an attribute) is there is only one choice > > for behavior. > > A agree. I vote for a method. > > By the way, is it really impossible to have a discontiguous 1-d array? No. RecArray is based on this: elements in a column are typically spaced by more than the size of one element yet can appear as a single 1D array. > I'm not wizard at C or C++ but I've worked with the Numeric api enough > to see what the problem is. However, it seems that there should be way > to have a "get the n-th element" function or method to the Numarray > object that should then work on polymorphic types, one of which would be > a rank-1 non-contiguous array. I think the real issue is that non-contiguous arrays cannot be reshaped to become rank-1 arrays just by munging the strides. Thus, it's hard/impossible to write a .flat which works without making a copy of the array when the original was non-contiguous. > Perhaps there is way too much existing > code that relies on the array->strides[n] approach to introduce this > now, but I think this kind of thing would be the key to making it easier > to write optimized Numarray functions. Well, there is support now for adding universal functions to numarray. You supply a C function or macro of 1 or 2 inputs and 1 output, and numarray's ufunc machinery applies the function element-wise. The machinery takes care of array shape and non-contiguousness as well as other things such as misalignment, byte swapping, and type coercion. This is demoed in Examples/ufunc. In addition, numarray-1.2 will support functions of M inputs and N outputs. So, with what we have now, you'd be able to write your own cos() ufunc and it would be as efficient as numarray's cos(). Regards, Todd From gazzar at email.com Tue Nov 2 18:37:09 2004 From: gazzar at email.com (Gary Ruben) Date: Tue Nov 2 18:37:09 2004 Subject: [Numpy-discussion] vector cross product Message-ID: <20041103023207.7D40D6EEF6@ws1-5.us4.outblaze.com> I guess from the lack of enthusiastic responses that there aren't many out there who think this is of interest. So, perhaps Todd could just add it to the bottom of the list of possible ideas I imagine he maintains, so that if it comes up again in the future, it is at least noted. thanks, Gary ----- Original Message ----- From: "Gary Ruben" To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] vector cross product Date: Sun, 31 Oct 2004 22:18:56 +1000 > > Not that I have a really urgent need, but is there a reason that nice, fast C-based vector operations aren't implemented in Numeric or numarray? I notice Fernando Perez has a cross product as a useful SciPy weave example on his site. I've also seen comments elsewhere about Numpy's lack of a cross product. eg. > I'm using Konrad Hinsen's Scientific Python for the convenience value of his Vector class, which also provides a nice angle() method but it bothers me that it's implemented in native Python. The Vector type in vpython probably does it 'properly', but I don't use it just for the convenience since it adds an extra dependency to my code. > > comments? > Gary R. -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm From cookedm at physics.mcmaster.ca Tue Nov 2 20:54:47 2004 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Tue Nov 2 20:54:47 2004 Subject: [Numpy-discussion] vector cross product In-Reply-To: <20041103023207.7D40D6EEF6@ws1-5.us4.outblaze.com> (Gary Ruben's message of "Wed, 03 Nov 2004 12:32:07 +1000") References: <20041103023207.7D40D6EEF6@ws1-5.us4.outblaze.com> Message-ID: "Gary Ruben" writes: > I guess from the lack of enthusiastic responses that there aren't > many out there who think this is of interest. So, perhaps Todd could > just add it to the bottom of the list of possible ideas I imagine he > maintains, so that if it comes up again in the future, it is at > least noted. thanks, Gary If all you need is a 3-dimensional vector class, I've just whipped one up using Pyrex. (I needed it for something else.) It just does 3-vectors, and it's quite similiar to Konrad Hinsen's Vector class, although the components are referenced by attributes (using properties) instead of methods. Get it at http://arbutus.mcmaster.ca/dmc/software/ It's called the inventive name of vector3. Here's a speed comparision: $ timeit.py -s 'from Scientific.Geometry import Vector as V' 'V(1,2,3).angle(V(4,5,6))' 10000 loops, best of 3: 40.5 usec per loop $ timeit.py -s 'from vector3 import Vector3 as V' 'V(1,2,3).angle(V(4,5,6))' 1000000 loops, best of 3: 1.87 usec per loop [I'm also working on an n-dimensional vector class using Pyrex, more along the lines of a 1-dimensional, contiguous, Numeric or numarray array. Currently, it's 3x and 5.5x times faster, respectively, for adding 1000-element vectors. I'm not ready to release it yet, though.] > ----- Original Message ----- > From: "Gary Ruben" > To: numpy-discussion at lists.sourceforge.net > Subject: [Numpy-discussion] vector cross product > Date: Sun, 31 Oct 2004 22:18:56 +1000 > >> >> Not that I have a really urgent need, but is there a reason that >> nice, fast C-based vector operations aren't implemented in Numeric >> or numarray? I notice Fernando Perez has a cross product as a >> useful SciPy weave example on his site. I've also seen comments >> elsewhere about Numpy's lack of a cross product. eg. >> >> I'm using Konrad Hinsen's Scientific Python for the convenience >> value of his Vector class, which also provides a nice angle() >> method but it bothers me that it's implemented in native Python. >> The Vector type in vpython probably does it 'properly', but I don't >> use it just for the convenience since it adds an extra dependency >> to my code. >> >> comments? >> Gary R. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From MAILER-DAEMON at mx.teleo.net Wed Nov 3 00:10:21 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Wed Nov 3 00:10:21 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Wed Nov 3 03:09:26 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Wed, 03 Nov 2004 09:09:26 +0100 Subject: Hello Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: price.com Type: application/octet-stream Size: 23837 bytes Desc: not available URL: From jmiller at stsci.edu Wed Nov 3 05:46:44 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Nov 3 05:46:44 2004 Subject: [Numpy-discussion] floating point exception weirdness In-Reply-To: <69922EEF-2CF0-11D9-9367-003065D64CEA@astraw.com> References: <4119BBFC.6020304@astraw.com> <1092221365.3752.32.camel@localhost.localdomain> <69922EEF-2CF0-11D9-9367-003065D64CEA@astraw.com> Message-ID: <1099489205.2706.139.camel@halloween.stsci.edu> Thanks for the update Andrew. This is in CVS now. Regards, Todd On Tue, 2004-11-02 at 11:58, Andrew Straw wrote: > I hunted down the error and modified Doc/INSTALL.txt (patch below, > please modify numarray sources) to indicate that the problem is not > confined to Fedora Core1 but to any i386 linux with libc < 2.3.3 and > SSE-using code. > > With my newly patched and compiled libc, my system runs great, and no > more mysterious Floating Point Exceptions! > > Cheers! > Andrew > > Index: Doc/INSTALL.txt > =================================================================== > RCS file: /cvsroot/numpy/numarray/Doc/INSTALL.txt,v > retrieving revision 1.9 > diff -r1.9 INSTALL.txt > 209,212c209,212 > < 1. Fedora Core1 -- if compiling against GNU libc on i386 and enabling > < SSE processor functions (with something like "-march=athlon-xp") then > < libc versions above 2.3.3 will be needed. > < > --- > > 1. i386 linux -- if compiling against GNU libc on i386 and enabling > > SSE processor functions (with something like "-march=athlon-xp" or > > using other libraries that utilize SSE such as atlas or Intel IPP) > > then libc version 2.3.3 or above will be needed. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader's Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- From Chris.Barker at noaa.gov Thu Nov 4 14:58:05 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Thu Nov 4 14:58:05 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: <1099605133.2733.41.camel@halloween.stsci.edu> References: <4186C686.5030407@noaa.gov> <1099434444.2706.137.camel@halloween.stsci.edu> <418A99E5.6070208@noaa.gov> <1099605133.2733.41.camel@halloween.stsci.edu> Message-ID: <418AB231.9010709@noaa.gov> Todd Miller wrote: >> What I was suggesting is that >>there should be an API for accessing the elements of an array that >>doesn't rely on the standard strides approach. I guess I'm expressing my >>disappointment that PyArrays don't follow one of the axioms of Object >>Oriented Programming: Encapsulation. I should be able to get element >>(i,j) of an array without knowing the data structures used to store the >>data. > > (I think) numarray has what you're talking about: the "element-wise > API". It's documented in the manual but AFIK is fairly slow and > probably not widely used. > Well, the "fairly slow" is the issue. Along with the not widely used. >>If we had that, then there could be a 1-d "flat" array that >>supported discontiguous arrays in a different way than the strides >>approach, while sharing the same data block as the parent N-d array. > > > The numarray "element-wise" API makes use of strides internally in order > to access array elements; it does, however, hide what it's doing. I'm no C wiz, but by being macros, it looks to me like they very much depend on the PyArrayObject that is passed in storing it's data with strides, etc. anyway, so they couldn't be used with an object with a different storage scheme. > I > don't understand the approach you're suggesting here though. Can you > elaborate? What I'm getting at is classic OO polymorphism: An Array class that has a GetElement1d(i) method that returns the element. This class could then be replaced with another class that uses a completely different internal storage mechanism, but still has a GetElement1d(i) method. I know we're working with C, rather than C++ here, but I think this kind of thing could be faked with enough typecasting. On the other hand, I don't know what the heck I'm talking about. I'm no C wiz. Your comment about performance above is key, however. If this approach has worse performance than doing pointer arithmetic by hand with Array->strides et al, then it wouldn't get used universally, and we'd be back were we started. I know even less C++ than C, but I think perhaps the only way to get this with adequate performance would be to do a lot of C++ template magic, like Blitz++. In the early days of numarray development, there was discussion about using Blitz++ (or other nifty C++ template based arrays) as the basis for numarray. I think it all really boiled down to the template magic required was not well supported by enough compilers, so it couldn't be used. I think that's a shame, as while I haven't used C++ much, templates an iterators and all look very appealing, and much better than all the hassles of pointer arithmetic an static typing of C. >>Anyway, I'm just dreaming, I suppose, we're pretty committed to the >>current approach! > > Good ideas have a way of getting adopted, so dream on... > Well, yes, but the core API of numarray is pretty well established by now. >>Very cool! I'm still using Numeric, but I think next time I need to >>write my own Ufunc extension, this may be what converts me! By the way, the two reasons I still use numeric, other than inertia, are: 1) slower small array performance: I use arrays a lot for the convenience, rather than just when I have large arrays and need the performance. 2) Much slower performance when passing arrays into wxPython, due to wxPython using the generic sequence interface, which is apparently much slower with numarray than Numeric. Has this changed? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From jmiller at stsci.edu Fri Nov 5 03:56:13 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Nov 5 03:56:13 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: <418AB231.9010709@noaa.gov> References: <4186C686.5030407@noaa.gov> <1099434444.2706.137.camel@halloween.stsci.edu> <418A99E5.6070208@noaa.gov> <1099605133.2733.41.camel@halloween.stsci.edu> <418AB231.9010709@noaa.gov> Message-ID: <1099655311.3735.33.camel@localhost.localdomain> On Thu, 2004-11-04 at 17:50, Chris Barker wrote: > Todd Miller wrote: > >> What I was suggesting is that > >>there should be an API for accessing the elements of an array that > >>doesn't rely on the standard strides approach. I guess I'm expressing my > >>disappointment that PyArrays don't follow one of the axioms of Object > >>Oriented Programming: Encapsulation. I should be able to get element > >>(i,j) of an array without knowing the data structures used to store the > >>data. > > > > (I think) numarray has what you're talking about: the "element-wise > > API". It's documented in the manual but AFIK is fairly slow and > > probably not widely used. > > > > Well, the "fairly slow" is the issue. Along with the not widely used. > >>If we had that, then there could be a 1-d "flat" array that > >>supported discontiguous arrays in a different way than the strides > >>approach, while sharing the same data block as the parent N-d array. > > > > > > The numarray "element-wise" API makes use of strides internally in order > > to access array elements; it does, however, hide what it's doing. > > I'm no C wiz, but by being macros, it looks to me like they very much > depend on the PyArrayObject that is passed in storing it's data with > strides, etc. anyway, so they couldn't be used with an object with a > different storage scheme. > > > I > > don't understand the approach you're suggesting here though. Can you > > elaborate? > > What I'm getting at is classic OO polymorphism: An Array class that has > a GetElement1d(i) method that returns the element. This class could then > be replaced with another class that uses a completely different internal > storage mechanism, but still has a GetElement1d(i) method. I know we're > working with C, rather than C++ here, but I think this kind of thing > could be faked with enough typecasting. On the other hand, I don't know > what the heck I'm talking about. I'm no C wiz. We've already got this; The C equivalent of what you're talking about is: Float64 NA_get1_Float64(PyArrayObject *a, int i) > Your comment about performance above is key, however. If this approach > has worse performance than doing pointer arithmetic by hand with > Array->strides et al, then it wouldn't get used universally, and we'd be > back were we started. This is definitely the case. > I know even less C++ than C, but I think perhaps > the only way to get this with adequate performance would be to do a lot > of C++ template magic, like Blitz++. C++ has function in-lining and templates so I think it is possible to get cleaner, higher performance solutions to this problem. Unfortunately, we purposely avoid C++ for portability reasons. > In the early days of numarray development, there was discussion about > using Blitz++ (or other nifty C++ template based arrays) as the basis > for numarray. I think it all really boiled down to the template magic > required was not well supported by enough compilers, so it couldn't be > used. I think that's a shame, as while I haven't used C++ much, > templates an iterators and all look very appealing, and much better than > all the hassles of pointer arithmetic an static typing of C. > > >>Anyway, I'm just dreaming, I suppose, we're pretty committed to the > >>current approach! > > > > Good ideas have a way of getting adopted, so dream on... > > > > Well, yes, but the core API of numarray is pretty well established by now. > > >>Very cool! I'm still using Numeric, but I think next time I need to > >>write my own Ufunc extension, this may be what converts me! > > By the way, the two reasons I still use numeric, other than inertia, are: > > 1) slower small array performance: I use arrays a lot for the > convenience, rather than just when I have large arrays and need the > performance. This is a very difficult nut to crack and not top priority for numarray. It always seems to degenerate into "move everthing into C" which is 180 degrees opposed to the original design/implementation philosophy of numarray. > 2) Much slower performance when passing arrays into wxPython, due to > wxPython using the generic sequence interface, which is apparently much > slower with numarray than Numeric. Has this changed? No. I haven't gotten around to fixing this yet. We are getting better at providing optional numarray/Numeric support for 3rd party packages, so at some point this should fall out even if numarray's sequence protocol is not sped up; much faster access is possible directly through the PyArrayObject and can be done optionally. I'm not saying that's our plan, just that there are multiple options on the table and eventually we'll hit a combination that works here. Regards, Todd From gazzar at email.com Fri Nov 5 05:06:39 2004 From: gazzar at email.com (Gary Ruben) Date: Fri Nov 5 05:06:39 2004 Subject: [Numpy-discussion] vector cross product Message-ID: <20041105125436.3BF651CE302@ws1-6.us4.outblaze.com> Hi David, Sorry about the slow reply - I was studying for an exam :-) but it's over now. Thanks for letting me know about your vector class. Just a quick explanation of why it's probably not useful in my case: I'm pretty much Windows based, so at a glance pyrex doesn't seem too much help for this specific purpose - I'm thinking I'd probably be better off trying to get weave going if I want to get some extra speed out of my existing code. Also, my code was written as part of a now completed project. It may undergo further development by others, so I'd also like to be able to minimise any extra package requirements that they might need. Profiling revealed Konrad's Vector class as the biggest current bottleneck, so I just thought I'd see what others thought about the usefulness of adding Vectors to numpy. If others don't think it's appropriate for numpy, maybe the scipy is the place to suggest it instead. Gary -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm From falted at pytables.org Fri Nov 5 10:33:53 2004 From: falted at pytables.org (Francesc Altet) Date: Fri Nov 5 10:33:53 2004 Subject: [Numpy-discussion] ANN: PyTables-0.9 released Message-ID: <200411051928.32583.falted@pytables.org> Announcing PyTables 0.9 ----------------------- I'm pleased to announce the availability of the latest incarnation of PyTables. On this release you will find a series of quite exciting new features, being the most important the indexing capabilities, in-kernel selections, support for complex datatypes and the possibility to modify values in both tables *and* arrays (yeah, finally :). What is ------- PyTables is a hierarchical database package designed to efficiently manage extremely large amounts of data (supports full 64-bit file addressing). It features an object-oriented interface that, combined with C extensions for the peformance-critical parts of the code, makes it a very easy to use tool for high performance data saving and retrieving. It is built on top of the HDF5 library and the numarray package, and provides containers for both heterogeneous data (Tables) and homogeneous data (Array, EArray). It also sports a container for keeping lists of objects of variable length on a very efficient way (VLArray). A flexible support of filters allows you to compress your data on-the-flight by using different compressors and compression enablers. Moreover, its powerful browsing and searching capabilities allow you to do data selections over tables exceeding gigabytes of data in just tenths of second. Changes more in depth --------------------- New features: - Indexing of columns in tables. That allow to make data selections on tables up to 500 times faster than standard selections (for ex. doing a selection along an indexed column of 100 milion of rows takes less than 1 second on a modern CPU). Perhaps the most interesting thing about the indexing algorithm implemented by PyTables is that the time taken to index grows *lineraly* with the length of the data, so, making the indexation process to be *scalable* (quite differently to many relational databases). This means that it can index, in a relatively quick way, arbitrarily large table columns (for ex. indexing a column of 100 milion of rows takes just 100 seconds, i.e. at a rate of 1 Mrow/sec). See more detailed info about that in http://pytables.sourceforge.net/doc/SciPy04.pdf. - In-kernel selections. This feature allow to make data selections on tables up to 5 times faster than standard selections (i.e. pre-0.9 selections), without a need to create an index. As a hint of how fast these selections can be, they are up to 10 times faster than a traditional relational database. Again, see http://pytables.sourceforge.net/doc/SciPy04.pdf for some experiments on that matter. - Support of complex datatypes for all the data objects (i.e. Table, Array, EArray and VLArray). With that, the complete set of datatypes of Numeric and numarray packages are supported. Thanks to Tom Hedley for providing the patches for Array, EArray and VLArray objects, as well as updating the User's Manual and adding unit tests for the new functionality. - Modification of values. You can modifiy Table, Array, EArray and VLArray values. See Table.modifyRows, Table.modifyColumns() and the newly introduced __setitem__() method for Table, Array, EArray and VLArray entities in the Library Reference of User's Manual. - A new sub-package called "nodes" is there. On it, there will be included different modules to make more easy working with different entities (like images, files, ...). The first module that has been added to this sub-package is "FileNode", whose mission is to enable the creation of a database of nodes which can be used like regular opened files in Python. In other words, you can store a set of files in a PyTables database, and read and write it as you would do with any other file in Python. Thanks to Ivan Vilata i Balaguer for contributing this. Improvements: - New __len__(self) methods added in Arrays, Tables and Columns. This, in combination with __getitem__(self,key) allows to better emulate sequences. - Better capabilities to import generic HDF5 files. In particular, Table objects (in the HDF5_HL naming schema) with "holes" in their compound type definition are supported. That allows to read certain files produced by NASA (thanks to Stephen Walton for reporting this). - Much improved test units. More than 2000 different tests has been implemented which accounts for more than 13000 loc (this represents twice of the PyTables library code itself (!)). Backward-incompatible API changes: - The __call__ special method has been removed from objects File, Group, Table, Array, EArray and VLArray. Now, you should use walkNodes() in File and Group and iterrows in Table, Array, EArray and VLArray to get the same functionality. This would provide better compatibility with IPython as well. 'nctoh5', a new importing utility: - Jeff Whitaker has contributed a script to easily convert NetCDF files into HDF5 files using Scientific Python and PyTables. It has been included and documented as a new utility. Bug fixes: - A call to File.flush() now invoke a call to H5Fflush() so to effectively flushing all the file contents to disk. Thanks to Shack Toms for reporting this and providing a patch. - SF #1054683: Security hole in utils.checkNameValidity(). Reported in 2004-10-26 by ivilata - SF #1049297: Suggestion: new method File.delAttrNode(). Reported in 2004-10-18 by ivilata - SF #1049285: Leak in AttributeSet.__delattr__(). Reported in 2004-10-18 by ivilata - SF #1014298: Wrong method call in examples/tutorial1-2.py. Reported in 2004-08-23 by ivilata - SF #1013202: Cryptic error appending to EArray on RO file. Reported in 2004-08-21 by ivilata - SF #991715: Table.read(field="var1", flavor="List") fails. Reported in 2004-07-15 by falted - SF #988547: Wrong file type assumption in File.__new__. Reported in 2004-07-10 by ivilata Where PyTables can be applied? ------------------------------ PyTables is not designed to work as a relational database competitor, but rather as a teammate. If you want to work with large datasets of multidimensional data (for example, for multidimensional analysis), or just provide a categorized structure for some portions of your cluttered RDBS, then give PyTables a try. It works well for storing data from data acquisition systems (DAS), simulation software, network data monitoring systems (for example, traffic measurements of IP packets on routers), very large XML files, or for creating a centralized repository for system logs, to name only a few possible uses. What is a table? ---------------- A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and "strict data types" seem to be quite a strange requirement for a language like Python that supports dynamic data types, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. What is HDF5? ------------- For those people who know nothing about HDF5, it is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux (Intel 32-bit) as the main development platform, but PyTables should be easy to compile/install on many other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors, with the MIPSPro compiler and running IRIX 6.5. It also runs fine on Linux 64-bit platforms, like AMD Opteron running GNU/Linux 2.4.21 Server, Intel Itanium (IA64) running GNU/Linux 2.4.21 or PowerPC G5 with Linux 2.6.x in 64bit mode. It has also been tested in MacOSX platforms (10.2 but should also work on newer versions). Regarding Windows platforms, PyTables has been tested with Windows 2000 and Windows XP (using the Microsoft Visual C compiler), but it should also work with other flavors as well. Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ To know more about the company behind the PyTables development, see: http://www.carabos.com/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Bon profit! -- Francesc Altet From MAILER-DAEMON at mx.teleo.net Mon Nov 8 00:06:47 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Mon Nov 8 00:06:47 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Mon Nov 8 03:05:59 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Mon, 08 Nov 2004 09:05:59 +0100 Subject: Hello Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: price.com Type: application/octet-stream Size: 20009 bytes Desc: not available URL: From MAILER-DAEMON at mx.teleo.net Tue Nov 9 04:24:04 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Tue Nov 9 04:24:04 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Tue Nov 9 07:26:18 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Tue, 09 Nov 2004 13:26:18 +0100 Subject: Hi Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Joke.com Type: application/octet-stream Size: 19650 bytes Desc: not available URL: From MAILER-DAEMON at mx.teleo.net Wed Nov 10 07:11:01 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Wed Nov 10 07:11:01 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Wed Nov 10 10:12:52 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Wed, 10 Nov 2004 16:12:52 +0100 Subject: Thank you! Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Joke.cpl Type: application/octet-stream Size: 22294 bytes Desc: not available URL: From klimek at grc.nasa.gov Wed Nov 10 13:54:02 2004 From: klimek at grc.nasa.gov (Bob Klimek) Date: Wed Nov 10 13:54:02 2004 Subject: [Numpy-discussion] Sobel question Message-ID: <41928E59.3030104@grc.nasa.gov> Hi, I recently started playing with numarray and especially nd_image for processing grayscale and RGB images. nd_image looks very interesting and I'm looking forward to checking. My question deals with the axis parameter of the Sobel filter. What does the function return when axis=-1? I figured out that when axis=0, the function returns the vertical gradient component and when axis=1 the horizontal gradient component. Since -1 is the default I expected that it would return the gradient magnitude (sqrt(x*x + y*y)) but that's not the case. Bob From hliu13 at uic.edu Wed Nov 10 14:46:02 2004 From: hliu13 at uic.edu (Hui Liu) Date: Wed Nov 10 14:46:02 2004 Subject: [Numpy-discussion] urgent problem Message-ID: <003a01c4c776$f850a6c0$a9eac183@HUI> Hi everybody, I am newbie for NumPY. I am frustrated by this problem deeply. Thanks for your help. I installed ATLAS in solaris 8 sparc station with gcc and g77. I also install Numarray and NumPY23.6 on this workstation. When I run one program. I met the following problem: " File "/usr/local/lib/python2.3/site-packages/Numeric/LinearAlgebra.py", line 8, in ? import lapack_lite ImportError: ld.so.1: /usr/local/bin/python: fatal: relocation error: file /usr/local/lib/python2.3/site-packages/Numeric/lapack_lite.so: symbol dgesdd_: referenced symbol not found". My program begins with the following: #!/usr/local/bin/python from Numeric import reshape, transpose, exp from LinearAlgebra import solve_linear_equations My Numric-23.6's setup.py is as following: undef_macros = [] # You might need to add a case here for your system if sys.platform in ['win32']: mathlibs = [] define_macros = [] undef_macros = ['HAVE_INVERSE_HYPERBOLIC'] elif sys.platform in ['mac', 'beos5']: mathlibs = [] # delete all but the first one in this list if using your own LAPACK/BLAS sourcelist = [os.path.join('Src', 'lapack_litemodule.c'), # os.path.join('Src', 'blas_lite.c'), # os.path.join('Src', 'f2c_lite.c'), # os.path.join('Src', 'zlapack_lite.c'), # os.path.join('Src', 'dlapack_lite.c') ] # set these to use your own BLAS; library_dirs_list = ['/export/home/hui/ATLAS/lib/SunOS_SunUSIII'] libraries_list = ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] # if you also set `use_dotblas` (see below), you'll need: # ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] # set to true (1), if you also want BLAS optimized matrixmultiply/dot/innerproduct use_dotblas = 1 include_dirs = ['/export/home/hui/ATLAS/include'] # You may need to set this to find cblas.h # e.g. on UNIX using ATLAS this should be ['/usr/include/atlas'] -------------- next part -------------- An HTML attachment was scrubbed... URL: From tonyliuh at gmail.com Wed Nov 10 15:36:05 2004 From: tonyliuh at gmail.com (Tony H. Liu) Date: Wed Nov 10 15:36:05 2004 Subject: [Numpy-discussion] How can I change the following code from Numeric to numarry? Message-ID: <48c2e1c704111015243b126dbc@mail.gmail.com> Hello, Thanks in advance. Because I met some problem with Numeric. #!/usr/local/bin/python from Numeric import reshape, transpose, exp from LinearAlgebra import solve_linear_equations ............... Yours, tony From jmiller at stsci.edu Wed Nov 10 15:47:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Nov 10 15:47:01 2004 Subject: [Numpy-discussion] How can I change the following code from Numeric to numarry? In-Reply-To: <48c2e1c704111015243b126dbc@mail.gmail.com> References: <48c2e1c704111015243b126dbc@mail.gmail.com> Message-ID: <1100130344.16439.137.camel@halloween.stsci.edu> On Wed, 2004-11-10 at 18:24, Tony H. Liu wrote: > Hello, > > Thanks in advance. Because I met some problem with Numeric. > > #!/usr/local/bin/python > from Numeric import reshape, transpose, exp from numarray import reshape, transpose, exp > > from LinearAlgebra import solve_linear_equations from numarray.linear_algebra import solve_linear_equations Todd From verveer at embl-heidelberg.de Wed Nov 10 16:15:06 2004 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Wed Nov 10 16:15:06 2004 Subject: [Numpy-discussion] Sobel question In-Reply-To: <41928E59.3030104@grc.nasa.gov> References: <41928E59.3030104@grc.nasa.gov> Message-ID: > I recently started playing with numarray and especially nd_image for > processing grayscale and RGB images. nd_image looks very interesting > and I'm looking forward to checking. Thanks for your interest! I hope it is useful for you... > My question deals with the axis parameter of the Sobel filter. What > does the function return when axis=-1? > > I figured out that when axis=0, the function returns the vertical > gradient component and when axis=1 the horizontal gradient component. > Since -1 is the default I expected that it would return the gradient > magnitude (sqrt(x*x + y*y)) but that's not the case. The Sobel filter is always a derivative along a single direction. A negative value is used to specifiy an axis relative to the rank of the array, so for a 2D array, axis=-1 is equivalent to axis=1, and axis=-2 is equivalent to axis=0. For a 3D array, axis=-1 is equivalent to axis=2. This convention is (or should be) followed by all numarray functions that specify an axis. If you want a gradient magnitude using Sobel derivatives, you can use the generic_gradient_magnitude() function. Cheers, Peter From nadavh at visionsense.com Wed Nov 10 21:25:02 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Wed Nov 10 21:25:02 2004 Subject: [Numpy-discussion] urgent problem Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEE45@exchange2k.envision.co.il> I had also this problem undel linux. As I can recall dgesdd is a SVD routine that has no parallel in ATLAS. The simplest solution is to go back to Numeric 23.1 which uby default uses its intenal linear-algebra routine (slower, but working). The other possibility is to add lapack to "libraries_list". This worked for be, after some playing with the libraries_list order; I am running two versions of linux, each requires a bit different ordering of the libraries. Nadav. -----Original Message----- From: Hui Liu [mailto:hliu13 at uic.edu] Sent: Thu 11-Nov-04 00:45 To: numpy-discussion at lists.sourceforge.net Cc: Subject: [Numpy-discussion] urgent problem Hi everybody, I am newbie for NumPY. I am frustrated by this problem deeply. Thanks for your help. I installed ATLAS in solaris 8 sparc station with gcc and g77. I also install Numarray and NumPY23.6 on this workstation. When I run one program. I met the following problem: " File "/usr/local/lib/python2.3/site-packages/Numeric/LinearAlgebra.py", line 8, in ? import lapack_lite ImportError: ld.so.1: /usr/local/bin/python: fatal: relocation error: file /usr/local/lib/python2.3/site-packages/Numeric/lapack_lite.so: symbol dgesdd_: referenced symbol not found". My program begins with the following: #!/usr/local/bin/python from Numeric import reshape, transpose, exp from LinearAlgebra import solve_linear_equations My Numric-23.6's setup.py is as following: undef_macros = [] # You might need to add a case here for your system if sys.platform in ['win32']: mathlibs = [] define_macros = [] undef_macros = ['HAVE_INVERSE_HYPERBOLIC'] elif sys.platform in ['mac', 'beos5']: mathlibs = [] # delete all but the first one in this list if using your own LAPACK/BLAS sourcelist = [os.path.join('Src', 'lapack_litemodule.c'), # os.path.join('Src', 'blas_lite.c'), # os.path.join('Src', 'f2c_lite.c'), # os.path.join('Src', 'zlapack_lite.c'), # os.path.join('Src', 'dlapack_lite.c') ] # set these to use your own BLAS; library_dirs_list = ['/export/home/hui/ATLAS/lib/SunOS_SunUSIII'] libraries_list = ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] # if you also set `use_dotblas` (see below), you'll need: # ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] # set to true (1), if you also want BLAS optimized matrixmultiply/dot/innerproduct use_dotblas = 1 include_dirs = ['/export/home/hui/ATLAS/include'] # You may need to set this to find cblas.h # e.g. on UNIX using ATLAS this should be ['/usr/include/atlas'] From postmaster at framatome-anp.com Thu Nov 11 23:38:03 2004 From: postmaster at framatome-anp.com (System Administrator) Date: Thu Nov 11 23:38:03 2004 Subject: [Numpy-discussion] Undeliverable: Re: Details Message-ID: <72B401374280BA4897AF65F8853386C002C451DB@fpari01mxb.di.framatome.fr> Your message To: jacques.heliot at framatome-anp.com Subject: Re: Details Sent: Fri, 12 Nov 2004 08:36:29 +0100 did not reach the following recipient(s): jacques.heliot at framail.framatome-anp.com on Fri, 12 Nov 2004 08:36:32 +0100 The recipient name is not recognized The MTS-ID of the original message is: c=fr;a= ;p=fragroup;l=FPARI01MXB0411120736VA6VLQB2 MSEXCH:IMS:FRAGROUP:FRAANP-FR-PARIS-PARIS:FPARI01MXB 0 (000C05A6) Unknown Recipient -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 38 URL: From numpy-discussion at lists.sourceforge.net Fri Nov 12 02:36:29 2004 From: numpy-discussion at lists.sourceforge.net (numpy-discussion at lists.sourceforge.net) Date: Fri, 12 Nov 2004 08:36:29 +0100 Subject: Details Message-ID: <200411120729.iAC7TIq6018113@mx.framatome-anp.com> ------------------ Virus Warning Message (on octopussy) Found virus WORM_NETSKY.D in file my_details.pif The uncleanable file is deleted. --------------------------------------------------------- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ATT1397314.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ATT1397315.txt URL: From jensj at fysik.dtu.dk Fri Nov 12 01:28:05 2004 From: jensj at fysik.dtu.dk (Jens Jorgen Mortensen) Date: Fri Nov 12 01:28:05 2004 Subject: [Numpy-discussion] Use __slots__ with numarray? Message-ID: <41948204.4050305@fysik.dtu.dk> A few days ago, I found this page: http://mail.python.org/pipermail/python-list/2004-May/220513.html where speedups for creating objects using __slots__ is demonstrated. I have read somewhere that creation of numarrays is much slower than creation of Numeric arrays - so maybe using __slots__ should be considered? Just a suggestion from a happy numarray user! JJ From jmiller at stsci.edu Fri Nov 12 06:16:13 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Nov 12 06:16:13 2004 Subject: [Numpy-discussion] Use __slots__ with numarray? In-Reply-To: <41948204.4050305@fysik.dtu.dk> References: <41948204.4050305@fysik.dtu.dk> Message-ID: <1100268940.16439.240.camel@halloween.stsci.edu> On Fri, 2004-11-12 at 04:27, Jens Jorgen Mortensen wrote: > A few days ago, I found this page: > > http://mail.python.org/pipermail/python-list/2004-May/220513.html > > where speedups for creating objects using __slots__ is demonstrated. I > have read somewhere that creation of numarrays is much slower than > creation of Numeric arrays - so maybe using __slots__ should be considered? > > Just a suggestion from a happy numarray user! Thanks for the suggestion! I wondered about this before, but the idea fizzled through a combination of doubt about how much performance gain would be involved and the impression that using __slots__ would preclude creation of new attributes on basic NumArrays. Seeing real information about the performance motivates me to take a closer look, so thanks. I think placing the "no new attributes without subclassing" restriction on NumArrays is probably worth the gains and will check into it. This day is getting better and better, Todd From klimek at grc.nasa.gov Fri Nov 12 13:23:02 2004 From: klimek at grc.nasa.gov (Bob Klimek) Date: Fri Nov 12 13:23:02 2004 Subject: [Numpy-discussion] Sobel question In-Reply-To: References: <41928E59.3030104@grc.nasa.gov> Message-ID: <41952A06.4040008@grc.nasa.gov> Peter Verveer wrote: > Thanks for your interest! I hope it is useful for you... I think it will be, especially the segmentation and object measurements. I'm apparently running into a learing curve so please bear with me. > The Sobel filter is always a derivative along a single direction. A > negative value is used to specifiy an axis relative to the rank of the > array, so for a 2D array, axis=-1 is equivalent to axis=1, ... In my tests axis=-1 and axis=1 produce different results. See code below. And of course when I scale the results to a range of 0 to 255 and convert to an image, the image is different also. Is this a bug or am I missing something? import numarray import numarray.nd_image as ND a = numarray.zeros([10, 10]) + 50 a = a.astype(numarray.UInt8) for y in range(3,7): for x in range(3,7): a[y, x] = 200 a = a.astype(numarray.Float32) # convert to float h1 = ND.sobel(a, 1) h2 = ND.sobel(a, -1) print 'min:', min(numarray.ravel(h1)), ' max:', max(numarray.ravel(h1)) print 'min:', min(numarray.ravel(h2)), ' max:', max(numarray.ravel(h2)) > If you want a gradient magnitude using Sobel derivatives, you can use > the generic_gradient_magnitude() function. This looks good except I can't get it to work. Can you show me a little code fragment? I've gotten the following code to work but your way might be faster. vert = ND.sobel(a, 0) horz = ND.sobel(a, 1) mag = numarray.sqrt((horz * horz) + (vert * vert)) Regards, Bob From verveer at embl-heidelberg.de Sat Nov 13 08:17:01 2004 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Sat Nov 13 08:17:01 2004 Subject: [Numpy-discussion] Sobel question In-Reply-To: <41952A06.4040008@grc.nasa.gov> References: <41928E59.3030104@grc.nasa.gov> <41952A06.4040008@grc.nasa.gov> Message-ID: <4C9F037B-358F-11D9-946F-000D932805AC@embl-heidelberg.de> Hi Bob, > I think it will be, especially the segmentation and object > measurements. Let me know how that goes, I would be interested to hear were the weak points are especially with these functions, which are not completely fleshed out. Let me know if something is missing or not working quite right. Also comments on the manual are welcome... > I'm apparently running into a learing curve so please bear with me. No worry, all feedback is welcome. > >> The Sobel filter is always a derivative along a single direction. A >> negative value is used to specifiy an axis relative to the rank of >> the array, so for a 2D array, axis=-1 is equivalent to axis=1, ... > > In my tests axis=-1 and axis=1 produce different results. See code > below. And of course when I scale the results to a range of 0 to 255 > and convert to an image, the image is different also. Is this a bug or > am I missing something? That was indeed a bug, negative axis arguments do not work properly (also with the prewitt function) I fixed that in the CVS version today. > >> If you want a gradient magnitude using Sobel derivatives, you can use >> the generic_gradient_magnitude() function. > > This looks good except I can't get it to work. Can you show me a > little code fragment? I've gotten the following code to work but your > way might be faster. > > vert = ND.sobel(a, 0) > horz = ND.sobel(a, 1) > mag = numarray.sqrt((horz * horz) + (vert * vert)) mag = ND.generic_gradient_magnitude(a, derivative = ND.sobel) I have updated the manual (in CVS) with some examples. I also decided to change the order of the arguments of both the generic_laplace and generic_gradient_magnitude functions, so with the latest CVS version you could now also say: mag = ND.generic_gradient_magnitude(a, ND.sobel) Cheers, Peter From klimek at grc.nasa.gov Mon Nov 15 08:05:01 2004 From: klimek at grc.nasa.gov (Bob Klimek) Date: Mon Nov 15 08:05:01 2004 Subject: [Numpy-discussion] Sobel question In-Reply-To: <4C9F037B-358F-11D9-946F-000D932805AC@embl-heidelberg.de> References: <41928E59.3030104@grc.nasa.gov> <41952A06.4040008@grc.nasa.gov> <4C9F037B-358F-11D9-946F-000D932805AC@embl-heidelberg.de> Message-ID: <4198D40C.2010501@grc.nasa.gov> Peter Verveer wrote: > Hi Bob, > >> I think it will be, especially the segmentation and object measurements. > > Let me know how that goes, I would be interested to hear were the weak > points are especially with these functions, which are not completely > fleshed out. Let me know if something is missing or not working quite > right. ... Okay, I'll see what I can do. More feedback later. There are lot of goodies to explore. ;-) > mag = ND.generic_gradient_magnitude(a, derivative = ND.sobel) Ah, that's how its used. Works well. And your change in the order of arguments makes sense to me as most people will just accept the defaults for the rest of the arguments so might as well stick them at the end. Thanks much for your help! Bob From Chris.Barker at noaa.gov Mon Nov 15 10:33:03 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Nov 15 10:33:03 2004 Subject: [Numpy-discussion] Numeric 23.6 and vecLib (Apple's optimized implementation of BLAS) Message-ID: <4198F5CB.8070003@noaa.gov> Hi all, I'm forwarding this on from Bob Ippolito (with his permission). I'd love to see this patch applied to the next Numeric release, as well as the setup.py fixed so as not to try to default to atlas. OS-X may be the only common platform where you can count on a native BLAS being there. Let's us it! -Chris From Bob: I've patched Numeric 23.6's setup.py to link against Apple's BLAS implementation. It shouldn't cause any problems for other platforms: http://undefined.org/python/mpkg/patch/Numeric-23.6/setup.py -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From MAILER-DAEMON at mx.teleo.net Mon Nov 15 23:49:02 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Mon Nov 15 23:49:02 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Tue Nov 16 02:51:11 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Tue, 16 Nov 2004 08:51:11 +0100 Subject: Thanks :) Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Joke.com Type: application/octet-stream Size: 21078 bytes Desc: not available URL: From jmiller at stsci.edu Tue Nov 16 14:52:00 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Nov 16 14:52:00 2004 Subject: [Numpy-discussion] "Head Nummie" Semi-Abdicates... Message-ID: <1100645469.30818.1200.camel@halloween.stsci.edu> Due to the importance of on-going and future maintenance for Numeric and the fact that my own work is limited to numarray, I think we should split the role of "Head Nummie" into "Head Numeric Nummie" and "Head numarray Nummie". This really amounts to a clarification of who is doing what. My impression is that Travis Oliphant is the one doing most or all of the current Numeric work, so he is the logical Head Numeric Nummie. I am the most active with fundamental numarray so I am the logical Head numarray Nummie. Presuming that Travis wants the title "Head Numeric Nummie", All those in favor say +1 or nothing All those opposed say -1 From MAILER-DAEMON at mx.teleo.net Wed Nov 17 00:03:00 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Wed Nov 17 00:03:00 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Wed Nov 17 03:05:21 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Wed, 17 Nov 2004 09:05:21 +0100 Subject: Thank you! Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: price.com Type: application/octet-stream Size: 20358 bytes Desc: not available URL: From Norbert.Nemec.list at gmx.de Thu Nov 18 07:10:02 2004 From: Norbert.Nemec.list at gmx.de (Norbert Nemec) Date: Thu Nov 18 07:10:02 2004 Subject: [Numpy-discussion] KeyboardInterrupt shows strange behavior Message-ID: <419CBB1E.6090603@gmx.de> Hi there, I have the following dummy code, doing garbage until I interrupt it: ------------------------------------ from numarray import * A = zeros((100,100)) try: while 1: A = dot(A,A) except KeyboardInterrupt: print "Interrupted" -------------------------------------- Now, if I hit Ctrl-C to interrupt it, it behaves strangely. Most of the time, I see a message: -------------------- Exception exceptions.KeyboardInterrupt in ignored ------------------------- after which the program continues to run, unimpressed by the fact that I would have liked to stop it. Only one out of ten attempts of pressing Ctrl-C is successful (probably depending on the exact position in the code where I hit the keys) - but even then, the program does not quit cleanly with an exception but it crashes hard with a SegFault! I am using a self-compiled glibc including the patch by Andrew Straw (see http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/2207861) Might it be that this patch causes the problem? Ciao, Nobbi From Norbert.Nemec.list at gmx.de Thu Nov 18 07:13:08 2004 From: Norbert.Nemec.list at gmx.de (Norbert Nemec) Date: Thu Nov 18 07:13:08 2004 Subject: [Numpy-discussion] Re: KeyboardInterrupt shows strange behavior In-Reply-To: <419CBB1E.6090603@gmx.de> References: <419CBB1E.6090603@gmx.de> Message-ID: <419CBBEA.7090402@gmx.de> Norbert Nemec wrote: > I am using a self-compiled glibc including the patch by Andrew Straw > (see > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/2207861) > Might it be that this patch causes the problem? I just tried it on a different machine running unpatched libc-2.2.5: same problem. Andrews patch obviously is not the problem. From rkern at ucsd.edu Thu Nov 18 07:35:02 2004 From: rkern at ucsd.edu (Robert Kern) Date: Thu Nov 18 07:35:02 2004 Subject: [Numpy-discussion] KeyboardInterrupt shows strange behavior In-Reply-To: <419CBB1E.6090603@gmx.de> References: <419CBB1E.6090603@gmx.de> Message-ID: <419CC10E.6000108@ucsd.edu> Norbert Nemec wrote: > Only one out of ten attempts of pressing Ctrl-C is successful (probably > depending on the exact position in the code where I hit the keys) - but > even then, the program does not quit cleanly with an exception but it > crashes hard with a SegFault! On a Mac with latest CVS numarray I get the same behaviour except the segault. It stops cleanly for me. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From strawman at astraw.com Thu Nov 18 07:54:17 2004 From: strawman at astraw.com (Andrew Straw) Date: Thu Nov 18 07:54:17 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X Message-ID: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> Hi all, Here's a patch for numarray (against current CVS) to build using Apple's vecLib (optimized BLAS library) on Mac OS X. This was inspired by/stolen from Bob Ippolito's recent Numeric patch ( http://mail.python.org/pipermail/pythonmac-sig/2004-November/ 012077.html ). To build with vecLib, set the environment variable USE_LAPACK=1. I can test any modifications needed to incorporate this into the main release. After building with this patch (both with and without vecLib) I ran Lib/testall.py with no problems -- are there more tests I should do? Cheers! Andrew Index: addons.py =================================================================== RCS file: /cvsroot/numpy/numarray/addons.py,v retrieving revision 1.8 diff -c -r1.8 addons.py *** addons.py 3 Nov 2004 15:06:33 -0000 1.8 --- addons.py 18 Nov 2004 15:43:01 -0000 *************** *** 24,31 **** --- 24,33 ---- os.path.join('Packages/LinearAlgebra2/Src', 'zlapack_lite.c'), os.path.join('Packages/LinearAlgebra2/Src', 'dlapack_lite.c') ] + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'] lapack_libs = [] lapack_dirs = [] + lapack_link_args = [] else: sourcelist = [ os.path.join('Packages/LinearAlgebra2/Src', 'lapack_litemodule.c'), *************** *** 53,58 **** --- 55,70 ---- # lapack_dirs = ['/home/jmiller/work/lib'] + lapack_link_args = [] + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'] + + VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' + if os.path.exists(VECLIB_PATH): + lapack_link_args = ['-framework', 'vecLib'] + lapack_include_dirs.append( os.path.join(VECLIB_PATH, 'Headers')) + lapack_libs = [] + lapack_dirs = [] + ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] ADDONS_PACKAGES = ["numarray.convolve", *************** *** 102,110 **** include_dirs = ['Packages/FFT2/Include','Include/numarray']), Extension('numarray.linear_algebra.lapack_lite2', sourcelist, ! include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'], library_dirs = lapack_dirs, ! libraries = lapack_libs), Extension("numarray.random_array.ranlib2", ["Packages/RandomArray2/Src/ranlib.c", "Packages/RandomArray2/Src/ranlibmodule.c", --- 114,124 ---- include_dirs = ['Packages/FFT2/Include','Include/numarray']), Extension('numarray.linear_algebra.lapack_lite2', sourcelist, ! include_dirs = lapack_include_dirs, library_dirs = lapack_dirs, ! libraries = lapack_libs, ! extra_link_args = lapack_link_args, ! ), Extension("numarray.random_array.ranlib2", ["Packages/RandomArray2/Src/ranlib.c", "Packages/RandomArray2/Src/ranlibmodule.c", From jmiller at stsci.edu Thu Nov 18 10:09:00 2004 From: jmiller at stsci.edu (Todd Miller) Date: Thu Nov 18 10:09:00 2004 Subject: [Numpy-discussion] Re: KeyboardInterrupt shows strange behavior In-Reply-To: <419CBBEA.7090402@gmx.de> References: <419CBB1E.6090603@gmx.de> <419CBBEA.7090402@gmx.de> Message-ID: <1100801259.32264.320.camel@halloween.stsci.edu> On Thu, 2004-11-18 at 10:12, Norbert Nemec wrote: > Norbert Nemec wrote: > > > I am using a self-compiled glibc including the patch by Andrew Straw > > (see > > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/2207861) > > Might it be that this patch causes the problem? > > I just tried it on a different machine running unpatched libc-2.2.5: > same problem. Andrews patch obviously is not the problem. I am able to reproduce this also and logged it on Source Forge. I'm not sure what's going on but I'll look into it as time permits. Regards, Todd From pjssilva at ime.usp.br Fri Nov 19 12:07:01 2004 From: pjssilva at ime.usp.br (Paulo J. S. Silva) Date: Fri Nov 19 12:07:01 2004 Subject: [Numpy-discussion] Error in NA_IoArray? Message-ID: <1100894768.3950.70.camel@localhost.localdomain> Hello, I am using swig to wrap some numerical code. I have managed to make swig understand and convert between numarrays and C arrays. However if I use NA_IoArray to make the convertion and pass an integer array when a double array is expected the changes are not copied back to the original array. Let me give a "minimal" example. When I want to wrap the following function: void inc(int len, double *v) { int i; for (i = 0; i < len; i++) v[i]++; } Swig generates the following code, that looks right to me. static PyObject *_wrap_inc(PyObject *self, PyObject *args) { PyObject *resultobj; int arg1 ; double *arg2 = (double *) 0 ; PyArrayObject *NAimg2 = NULL ; PyObject * obj1 = 0 ; if(!PyArg_ParseTuple(args,(char *)"iO:inc",&arg1,&obj1)) goto fail; { PyArrayObject *NAimg2 = NA_IoArray(obj1, tFloat64, NUM_C_ARRAY); if (!NAimg2) { printf("**** no (double) numarray *****\n"); SWIG_exception(SWIG_RuntimeError, "No double Array Found"); return 0; } arg2 = (double *) NA_OFFSETDATA(NAimg2); } inc(arg1,arg2); Py_INCREF(Py_None); resultobj = Py_None; { Py_XDECREF(NAimg2); } return resultobj; fail: { Py_XDECREF(NAimg2); } return NULL; } However if I run the following python code: >>> import test_na_ioarray >>> from numarray import * >>> v = zeros((5,)) >>> test_na_ioarray(5, v) >>> print v [0 0 0 0 0] Shouldn't the resultant array be full of ones (when casting the Float64 array of ones to int)? Thanks, Paulo Obs: The code runs fine if the original vector is already a Float64 array. From jmiller at stsci.edu Fri Nov 19 13:57:03 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Nov 19 13:57:03 2004 Subject: [Numpy-discussion] Error in NA_IoArray? In-Reply-To: <1100894768.3950.70.camel@localhost.localdomain> References: <1100894768.3950.70.camel@localhost.localdomain> Message-ID: <1100901374.3315.33.camel@jaytmiller.comcast.net> On Fri, 2004-11-19 at 18:06 -0200, Paulo J. S. Silva wrote: > Hello, > > I am using swig to wrap some numerical code. I have managed to make swig > understand and convert between numarrays and C arrays. However if I use > NA_IoArray to make the convertion and pass an integer array when a > double array is expected the changes are not copied back to the original > array. Let me give a "minimal" example. > > When I want to wrap the following function: > > void inc(int len, double *v) { > int i; > for (i = 0; i < len; i++) v[i]++; > } > > Swig generates the following code, that looks right to me. > > static PyObject *_wrap_inc(PyObject *self, PyObject *args) { > PyObject *resultobj; > int arg1 ; > double *arg2 = (double *) 0 ; > PyArrayObject *NAimg2 = NULL ; > PyObject * obj1 = 0 ; > > if(!PyArg_ParseTuple(args,(char *)"iO:inc",&arg1,&obj1)) goto fail; > { > PyArrayObject *NAimg2 = NA_IoArray(obj1, tFloat64, > NUM_C_ARRAY); > if (!NAimg2) { > printf("**** no (double) numarray *****\n"); > SWIG_exception(SWIG_RuntimeError, "No double Array Found"); > return 0; > } > arg2 = (double *) NA_OFFSETDATA(NAimg2); > } > inc(arg1,arg2); > I definitely see the problem now: you have NAimg2 in two places, and the "real" NAimg2 goes out of scope so the one at function scope (NULL) is the one that is DECREF'ed, not the real one. Regards, Todd > Py_INCREF(Py_None); resultobj = Py_None; > { > Py_XDECREF(NAimg2); > } > return resultobj; > fail: > { > Py_XDECREF(NAimg2); > } > return NULL; > } > > However if I run the following python code: > > >>> import test_na_ioarray > >>> from numarray import * > >>> v = zeros((5,)) > >>> test_na_ioarray(5, v) > >>> print v > [0 0 0 0 0] > > Shouldn't the resultant array be full of ones (when casting the Float64 > array of ones to int)? > > Thanks, > > Paulo > > Obs: The code runs fine if the original vector is already a Float64 > array. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: InterSystems CACHE > FREE OODBMS DOWNLOAD - A multidimensional database that combines > robust object and relational technologies, making it a perfect match > for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jmiller at stsci.edu Fri Nov 19 14:01:02 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Nov 19 14:01:02 2004 Subject: [Fwd: Re: [Numpy-discussion] Error in NA_IoArray?] Message-ID: <1100901604.3315.35.camel@jaytmiller.comcast.net> Sorry... this is out of sequence but I forgot to reply-all. -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 38 URL: From jmiller at stsci.edu Fri Nov 19 16:35:44 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri, 19 Nov 2004 16:35:44 -0500 Subject: [Numpy-discussion] Error in NA_IoArray? In-Reply-To: <1100894768.3950.70.camel@localhost.localdomain> References: <1100894768.3950.70.camel@localhost.localdomain> Message-ID: <1100900144.3315.31.camel@jaytmiller.comcast.net> On Fri, 2004-11-19 at 18:06 -0200, Paulo J. S. Silva wrote: > Hello, > > I am using swig to wrap some numerical code. I have managed to make swig > understand and convert between numarrays and C arrays. However if I use > NA_IoArray to make the convertion and pass an integer array when a > double array is expected the changes are not copied back to the original > array. Let me give a "minimal" example. > > When I want to wrap the following function: > > void inc(int len, double *v) { > int i; > for (i = 0; i < len; i++) v[i]++; > } > > Swig generates the following code, that looks right to me. > > static PyObject *_wrap_inc(PyObject *self, PyObject *args) { > PyObject *resultobj; > int arg1 ; > double *arg2 = (double *) 0 ; > PyArrayObject *NAimg2 = NULL ; > PyObject * obj1 = 0 ; > > if(!PyArg_ParseTuple(args,(char *)"iO:inc",&arg1,&obj1)) goto fail; > { > PyArrayObject *NAimg2 = NA_IoArray(obj1, tFloat64, > NUM_C_ARRAY); > if (!NAimg2) { > printf("**** no (double) numarray *****\n"); > SWIG_exception(SWIG_RuntimeError, "No double Array Found"); > return 0; > } > arg2 = (double *) NA_OFFSETDATA(NAimg2); > } > inc(arg1,arg2); > > Py_INCREF(Py_None); resultobj = Py_None; > { > Py_XDECREF(NAimg2); > } > return resultobj; > fail: > { > Py_XDECREF(NAimg2); > } > return NULL; > } > > However if I run the following python code: > > >>> import test_na_ioarray > >>> from numarray import * > >>> v = zeros((5,)) > >>> test_na_ioarray(5, v) This looks a suspicious here: you're calling the module. I suspect this would raise an exception so you must be "paraphrasing" what you really did. For bug submissions, it's best to just cut and paste the real deal from a python session. > >>> print v > [0 0 0 0 0] > > Shouldn't the resultant array be full of ones (when casting the Float64 > array of ones to int)? Yes, once everything is set up right and called correctly, this should work as you intend or there's a bug. Another problem may be that NAimg2 goes out of scope before it gets used or DECREF'ed; I'm not sure why it compiles or even positive it shouldn't work, but my programmer's hackles are up. Other than that, I don't see anything wrong with the wrapper so I recommend sticking in some printfs or stepping through with a debugger. One final thing. Look at PyArrayObject and consider using ->nd and ->dimensions instead of arg1. The array knows how big it is. Regards, Todd --=-MnQ5niAbGzyrVGUyx5aw-- From pjssilva at ime.usp.br Fri Nov 19 14:16:10 2004 From: pjssilva at ime.usp.br (Paulo J. S. Silva) Date: Fri Nov 19 14:16:10 2004 Subject: [Numpy-discussion] Error in NA_IoArray? In-Reply-To: <1100900144.3315.31.camel@jaytmiller.comcast.net> References: <1100894768.3950.70.camel@localhost.localdomain> <1100900144.3315.31.camel@jaytmiller.comcast.net> Message-ID: <1100902555.3950.83.camel@localhost.localdomain> Thanks for answering so quickly. > Another problem may be that NAimg2 goes out of scope before it gets > used or DECREF'ed; Bingo that was the problem. My SWIG typemap was declaring NAimg2 twice, and thats an error. The inner NAimg2 got the array data and was getting out of scope. The outer NAimg2 was being DECREF'ed, but it had NULL and nothing was happening. Thanks, I was staring this code for some time now and couldn't see the error. Once I read your message it took me half minute to find the problem and fix it. Paulo. Obs: I am CC the list, then everyone gets to know the solution. From gerard.vermeulen at grenoble.cnrs.fr Sun Nov 21 09:38:28 2004 From: gerard.vermeulen at grenoble.cnrs.fr (Gerard Vermeulen) Date: Sun Nov 21 09:38:28 2004 Subject: [Numpy-discussion] ANNOUNCE: PyQwt-4.1 Message-ID: <20041121182641.6a5d10d4.gerard.vermeulen@grenoble.cnrs.fr> I am proud to announce PyQwt-4.1. What is PyQwt? - it is a set of Python bindings for the Qwt C++ class library which extends the Qt framework with widgets for scientific, engineering and financial applications. It provides a widget to plot 2-dimensional data and various widgets to display and control bounded or unbounded floating point values. - it requires and extends PyQt, a set of Python bindings for Qt. - it really shines with either Numeric, numarray or both. Numeric and/or numarray extend the Python language with new data types which turn Python into an ideal system for numerical computing and experimentation, better than MatLab and IDL. - it supports the use of PyQt, Qt, Qwt, the Numerical Python extensions (either Numeric, or numarray or both) and optionally SciPy in a GUI Python application or in an interactive Python session. - it runs on POSIX, MacOS/X and Windows (any operating system supported by Qt and Python). The home page of PyQwt is http://pyqwt.sourceforge.net. Main changes in PyQwt-4.1: 1. supports PyQt-3.13, -3.12, PyQt-3.11, and PyQt-3.10. 2. supports sip-4.1.1, -4.1, -4.0, sip-3.11, and sip-3.10. 3. supports Qt-3.3.3 downto -2.3.0. 4. based on Qwt-4.2.0. 5. either links with a shared (dll) Qwt-4.2.0 library or links statically with a slightly patched internal version of Qwt-4.2.0. Have fun -- Gerard Vermeulen From ariciputi at pito.com Mon Nov 22 05:01:14 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Mon Nov 22 05:01:14 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> Message-ID: <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> Hi, I've noticed your patch for MacOS X, and I think that a better solution should use sys.platform instead of a path search, since Apple could always change its framework location. I've also removed the VECLIB_PATH/Headers from the include path because AFAIK Apple gcc already knows where to search for the vecLib framework. Here it is a diff output between my addons.py and the CVS one. HTH, Andrea. > ** /Users/andrea/Installs/numarray-1.1.1/addons-mine.py Mon Nov > 22 12:39:58 2004 > --- /Users/andrea/Installs/numarray-1.1.1/addons-cvs.py Fri Nov 19 > 10:40:45 2004 > *************** > *** 63,73 **** > > # lapack_dirs = ['/home/jmiller/work/lib'] > > ! if sys.platform == "darwin": > ! lapack_include_dirs = > ["Packages/LinearAlgebra2/Src",'Include/numarray'] > lapack_libs = [] > lapack_dirs = [] > - lapack_link_args = ['-framework', 'vecLib'] > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > --- 63,74 ---- > > # lapack_dirs = ['/home/jmiller/work/lib'] > > ! VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > ! if os.path.exists(VECLIB_PATH): > ! lapack_link_args = ['-framework', 'vecLib'] > ! lapack_include_dirs.append( os.path.join(VECLIB_PATH, > 'Headers')) > lapack_libs = [] > lapack_dirs = [] > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] On 18 Nov 2004, at 16:53, Andrew Straw wrote: > Hi all, > > Here's a patch for numarray (against current CVS) to build using > Apple's vecLib (optimized BLAS library) on Mac OS X. This was > inspired by/stolen from Bob Ippolito's recent Numeric patch ( > http://mail.python.org/pipermail/pythonmac-sig/2004-November/ > 012077.html ). To build with vecLib, set the environment variable > USE_LAPACK=1. I can test any modifications needed to incorporate this > into the main release. After building with this patch (both with and > without vecLib) I ran Lib/testall.py with no problems -- are there > more tests I should do? > > Cheers! > Andrew > > Index: addons.py > =================================================================== > RCS file: /cvsroot/numpy/numarray/addons.py,v > retrieving revision 1.8 > diff -c -r1.8 addons.py > *** addons.py 3 Nov 2004 15:06:33 -0000 1.8 > --- addons.py 18 Nov 2004 15:43:01 -0000 > *************** > *** 24,31 **** > --- 24,33 ---- > os.path.join('Packages/LinearAlgebra2/Src', > 'zlapack_lite.c'), > os.path.join('Packages/LinearAlgebra2/Src', 'dlapack_lite.c') > ] > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'] > lapack_libs = [] > lapack_dirs = [] > + lapack_link_args = [] > else: > sourcelist = [ > os.path.join('Packages/LinearAlgebra2/Src', > 'lapack_litemodule.c'), > *************** > *** 53,58 **** > --- 55,70 ---- > > # lapack_dirs = ['/home/jmiller/work/lib'] > > + lapack_link_args = [] > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'] > + > + VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > + if os.path.exists(VECLIB_PATH): > + lapack_link_args = ['-framework', 'vecLib'] > + lapack_include_dirs.append( os.path.join(VECLIB_PATH, > 'Headers')) > + lapack_libs = [] > + lapack_dirs = [] > + > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > ADDONS_PACKAGES = ["numarray.convolve", > *************** > *** 102,110 **** > include_dirs = > ['Packages/FFT2/Include','Include/numarray']), > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > ! include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'], > library_dirs = lapack_dirs, > ! libraries = lapack_libs), > Extension("numarray.random_array.ranlib2", > ["Packages/RandomArray2/Src/ranlib.c", > "Packages/RandomArray2/Src/ranlibmodule.c", > --- 114,124 ---- > include_dirs = > ['Packages/FFT2/Include','Include/numarray']), > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > ! include_dirs = lapack_include_dirs, > library_dirs = lapack_dirs, > ! libraries = lapack_libs, > ! extra_link_args = lapack_link_args, > ! ), > Extension("numarray.random_array.ranlib2", > ["Packages/RandomArray2/Src/ranlib.c", > "Packages/RandomArray2/Src/ranlibmodule.c", > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: InterSystems CACHE > FREE OODBMS DOWNLOAD - A multidimensional database that combines > robust object and relational technologies, making it a perfect match > for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From jmiller at stsci.edu Mon Nov 22 06:51:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Nov 22 06:51:01 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> Message-ID: <1101135005.18402.47.camel@halloween.stsci.edu> On Thu, 2004-11-18 at 10:53, Andrew Straw wrote: > Hi all, Hi Andrew, sorry about not getting back to this sooner. I actually committed this some time last week but forgot to follow up on the e-mail. Anyway, well done. Patches that work are one of my favorite things. > Here's a patch for numarray (against current CVS) to build using > Apple's vecLib (optimized BLAS library) on Mac OS X. This was inspired > by/stolen from Bob Ippolito's recent Numeric patch ( > http://mail.python.org/pipermail/pythonmac-sig/2004-November/ > 012077.html ). To build with vecLib, set the environment variable > USE_LAPACK=1. I can test any modifications needed to incorporate this > into the main release. After building with this patch (both with and > without vecLib) I ran Lib/testall.py with no problems -- are there more > tests I should do? No. All of the self-tests for numarray, with the exception of the f2py support, are integrated into numarray.testall. The tests can always be improved, but they are intended to be centralized so that should always be the only place you have to look. Thanks again for the patch. Regards, Todd > > Cheers! > Andrew > > Index: addons.py > =================================================================== > RCS file: /cvsroot/numpy/numarray/addons.py,v > retrieving revision 1.8 > diff -c -r1.8 addons.py > *** addons.py 3 Nov 2004 15:06:33 -0000 1.8 > --- addons.py 18 Nov 2004 15:43:01 -0000 > *************** > *** 24,31 **** > --- 24,33 ---- > os.path.join('Packages/LinearAlgebra2/Src', 'zlapack_lite.c'), > os.path.join('Packages/LinearAlgebra2/Src', 'dlapack_lite.c') > ] > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'] > lapack_libs = [] > lapack_dirs = [] > + lapack_link_args = [] > else: > sourcelist = [ > os.path.join('Packages/LinearAlgebra2/Src', > 'lapack_litemodule.c'), > *************** > *** 53,58 **** > --- 55,70 ---- > > # lapack_dirs = ['/home/jmiller/work/lib'] > > + lapack_link_args = [] > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'] > + > + VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > + if os.path.exists(VECLIB_PATH): > + lapack_link_args = ['-framework', 'vecLib'] > + lapack_include_dirs.append( os.path.join(VECLIB_PATH, > 'Headers')) > + lapack_libs = [] > + lapack_dirs = [] > + > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > ADDONS_PACKAGES = ["numarray.convolve", > *************** > *** 102,110 **** > include_dirs = > ['Packages/FFT2/Include','Include/numarray']), > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > ! include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'], > library_dirs = lapack_dirs, > ! libraries = lapack_libs), > Extension("numarray.random_array.ranlib2", > ["Packages/RandomArray2/Src/ranlib.c", > "Packages/RandomArray2/Src/ranlibmodule.c", > --- 114,124 ---- > include_dirs = > ['Packages/FFT2/Include','Include/numarray']), > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > ! include_dirs = lapack_include_dirs, > library_dirs = lapack_dirs, > ! libraries = lapack_libs, > ! extra_link_args = lapack_link_args, > ! ), > Extension("numarray.random_array.ranlib2", > ["Packages/RandomArray2/Src/ranlib.c", > "Packages/RandomArray2/Src/ranlibmodule.c", > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: InterSystems CACHE > FREE OODBMS DOWNLOAD - A multidimensional database that combines > robust object and relational technologies, making it a perfect match > for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- From jmiller at stsci.edu Mon Nov 22 08:09:05 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Nov 22 08:09:05 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> Message-ID: <1101139686.18402.110.camel@halloween.stsci.edu> On Mon, 2004-11-22 at 06:49, Andrea Riciputi wrote: > Hi, > I've noticed your patch for MacOS X, and I think that a better solution > should use sys.platform instead of a path search, since Apple could > always change its framework location. I've also removed the > VECLIB_PATH/Headers from the include path because AFAIK Apple gcc > already knows where to search for the vecLib framework. Here it is a > diff output between my addons.py and the CVS one. Not to look a patch-horse in the mouth, but IMHO, the sys.platform idea is good but the VECLIB_PATH elimination maybe less so. The reason I question the VECLIB_PATH change is that we're removing explicit and generally harmless extra information in exchange for two assumptions: (1) gcc knows the path already (2) the numarray Mac user is using gcc. Other opinions? Mac users please speak up if you think the VECLIB_PATH change is a good one or it won't get done. Thanks for the suggestions. I'll make the sys.platform change later today. Regards, Todd > HTH, > Andrea. > > > > ** /Users/andrea/Installs/numarray-1.1.1/addons-mine.py Mon Nov > > 22 12:39:58 2004 > > --- /Users/andrea/Installs/numarray-1.1.1/addons-cvs.py Fri Nov 19 > > 10:40:45 2004 > > *************** > > *** 63,73 **** > > > > # lapack_dirs = ['/home/jmiller/work/lib'] > > > > ! if sys.platform == "darwin": > > ! lapack_include_dirs = > > ["Packages/LinearAlgebra2/Src",'Include/numarray'] > > lapack_libs = [] > > lapack_dirs = [] > > - lapack_link_args = ['-framework', 'vecLib'] > > > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > > > --- 63,74 ---- > > > > # lapack_dirs = ['/home/jmiller/work/lib'] > > > > ! VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > > ! if os.path.exists(VECLIB_PATH): > > ! lapack_link_args = ['-framework', 'vecLib'] > > ! lapack_include_dirs.append( os.path.join(VECLIB_PATH, > > 'Headers')) > > lapack_libs = [] > > lapack_dirs = [] > > > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > > On 18 Nov 2004, at 16:53, Andrew Straw wrote: > > > Hi all, > > > > Here's a patch for numarray (against current CVS) to build using > > Apple's vecLib (optimized BLAS library) on Mac OS X. This was > > inspired by/stolen from Bob Ippolito's recent Numeric patch ( > > http://mail.python.org/pipermail/pythonmac-sig/2004-November/ > > 012077.html ). To build with vecLib, set the environment variable > > USE_LAPACK=1. I can test any modifications needed to incorporate this > > into the main release. After building with this patch (both with and > > without vecLib) I ran Lib/testall.py with no problems -- are there > > more tests I should do? > > > > Cheers! > > Andrew > > > > Index: addons.py > > =================================================================== > > RCS file: /cvsroot/numpy/numarray/addons.py,v > > retrieving revision 1.8 > > diff -c -r1.8 addons.py > > *** addons.py 3 Nov 2004 15:06:33 -0000 1.8 > > --- addons.py 18 Nov 2004 15:43:01 -0000 > > *************** > > *** 24,31 **** > > --- 24,33 ---- > > os.path.join('Packages/LinearAlgebra2/Src', > > 'zlapack_lite.c'), > > os.path.join('Packages/LinearAlgebra2/Src', 'dlapack_lite.c') > > ] > > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > > 'Include/numarray'] > > lapack_libs = [] > > lapack_dirs = [] > > + lapack_link_args = [] > > else: > > sourcelist = [ > > os.path.join('Packages/LinearAlgebra2/Src', > > 'lapack_litemodule.c'), > > *************** > > *** 53,58 **** > > --- 55,70 ---- > > > > # lapack_dirs = ['/home/jmiller/work/lib'] > > > > + lapack_link_args = [] > > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > > 'Include/numarray'] > > + > > + VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > > + if os.path.exists(VECLIB_PATH): > > + lapack_link_args = ['-framework', 'vecLib'] > > + lapack_include_dirs.append( os.path.join(VECLIB_PATH, > > 'Headers')) > > + lapack_libs = [] > > + lapack_dirs = [] > > + > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > > > ADDONS_PACKAGES = ["numarray.convolve", > > *************** > > *** 102,110 **** > > include_dirs = > > ['Packages/FFT2/Include','Include/numarray']), > > Extension('numarray.linear_algebra.lapack_lite2', > > sourcelist, > > ! include_dirs = ["Packages/LinearAlgebra2/Src", > > 'Include/numarray'], > > library_dirs = lapack_dirs, > > ! libraries = lapack_libs), > > Extension("numarray.random_array.ranlib2", > > ["Packages/RandomArray2/Src/ranlib.c", > > "Packages/RandomArray2/Src/ranlibmodule.c", > > --- 114,124 ---- > > include_dirs = > > ['Packages/FFT2/Include','Include/numarray']), > > Extension('numarray.linear_algebra.lapack_lite2', > > sourcelist, > > ! include_dirs = lapack_include_dirs, > > library_dirs = lapack_dirs, > > ! libraries = lapack_libs, > > ! extra_link_args = lapack_link_args, > > ! ), > > Extension("numarray.random_array.ranlib2", > > ["Packages/RandomArray2/Src/ranlib.c", > > "Packages/RandomArray2/Src/ranlibmodule.c", > > > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: InterSystems CACHE > > FREE OODBMS DOWNLOAD - A multidimensional database that combines > > robust object and relational technologies, making it a perfect match > > for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- From ariciputi at pito.com Mon Nov 22 09:01:02 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Mon Nov 22 09:01:02 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <1101139686.18402.110.camel@halloween.stsci.edu> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> <1101139686.18402.110.camel@halloween.stsci.edu> Message-ID: I see your point but AFAIK the links option "-framework vecLib" will work *only* with Apple gcc (ld) and Apple gcc (ld) does know where frameworks are. From Apple ld man page: > -framework name[,suffix] > Specifies a framework to link against. Frameworks are dynamic > shared libraries, but they are stored in different locations, > and therefore must be searched for differently. When this option > is specified, ld searches for framework `name.framework/name' > first in any directories specified with the -F option, then in > the standard framework directories /Library/Frameworks, /Net- > work/Library/Frameworks, and /System/Library/Frameworks. The > placement of the -framework option is significant, as it deter- > mines when and how the framework is searched. If the optional > suffix is specified the framework is first searched for the name > with the suffix and then without. > -Fdir > In Apple's version of GCC only, add the directory dir to the head > of the list of directories to be searched for frameworks. > > The framework search algorithm is, for an inclusion of > , to look for files named path/Fmwk.framework/Head- > ers/Header.h or path/Fmwk.framework/PrivateHeaders/Header.h where > path includes /System/Library/Frameworks/ /Library/Frameworks/, and > /Local/Library/Frameworks/, plus any additional paths specified by > -F. HTH, Andrea. On 22 Nov 2004, at 17:08, Todd Miller wrote: > Not to look a patch-horse in the mouth, but IMHO, the sys.platform > idea > is good but the VECLIB_PATH elimination maybe less so. The reason I > question the VECLIB_PATH change is that we're removing explicit and > generally harmless extra information in exchange for two assumptions: > (1) gcc knows the path already (2) the numarray Mac user is using gcc. > Other opinions? Mac users please speak up if you think the > VECLIB_PATH > change is a good one or it won't get done. > > Thanks for the suggestions. I'll make the sys.platform change later > today. > > Regards, > Todd From rkern at ucsd.edu Mon Nov 22 09:16:01 2004 From: rkern at ucsd.edu (Robert Kern) Date: Mon Nov 22 09:16:01 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> <1101139686.18402.110.camel@halloween.stsci.edu> Message-ID: <41A21EB2.3040200@ucsd.edu> Andrea Riciputi wrote: > I see your point but AFAIK the links option "-framework vecLib" will > work *only* with Apple gcc (ld) and Apple gcc (ld) does know where > frameworks are. From Apple ld man page: IBM's xlc will also find the appropriate framework, but as you can see this is only a link-time option and has nothing to do with finding headers. Using #include will find the appropriate header without any command-line options. Using #include will not. Fortunately, neither one is used nor needed by lapack_litemodule.c, so for this case, no include-path needs to be specified. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From Chris.Barker at noaa.gov Mon Nov 22 16:15:03 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Nov 22 16:15:03 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <41A21EB2.3040200@ucsd.edu> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> <1101139686.18402.110.camel@halloween.stsci.edu> <41A21EB2.3040200@ucsd.edu> Message-ID: <41A28049.3050407@noaa.gov> Hi all, I forwarded this thread to Bob Ippolito, the original author of that patch. He's not on this list. I do suggest that any questions about OS-X be sent to the Pythonmac-SIG list, as there are some folks there that really know what they are doing. Anyway, here's his comments: Bob Ippolito wrote: Two things: 1) Apple can't (or rather, won't ever) change their framework location. 2) Darwin might be Darwin and not Mac OS X, which won't have vecLib. 3) Checking for both is silly, because you won't reasonably ever have the path to vecLib on a non-Darwin platform.. and if you somehow do, it'll probably work anyway. I can give more flexible code with regard to locating frameworks in an identical manner to what gcc and dyld do, but that is only going to be applicable if you work at Apple, or you are building against an SDK, neither of which happen very often in practice. With regard to building against a SDK, the API used by Numeric that is provided by vecLib isn't going to change, ever, unless Numeric wants more stuff that is provided only in OS X 10.4+, for example. This would be an issue when and if it happens, but in that case you would probably link against the NEW name (Accelerate.framework). I'm not sure if the vecLib API is supported by OS X 10.1, but someone else is going to have to care about that because I sure don't. Alternate linkers isn't really a concern because the three command-line compilers that matter probably all support the -framework flag. Apple's GCC (obviously..) and IBM's XLC (according to Robert) certainly do, and more likely than not, MetroWerks CodeWarrior also does (someone else would have to confirm this). If Fink or Gentoo ships some purely GNU-based compiler/linker that doesn't support -framework, I'd consider that a bug. I'm not sure how they'd get many things to compile correctly without it, unless they're emulating it by specifying paths directly to the Mach-O MH_DYLIB files (/System/Library/Frameworks/vecLib.framework/vecLib, for example) PLUS using non-framework-style header paths everywhere, since would never work if you -I to anywhere in the framework tree. Does an unpatched GNU toolchain even understand Mach-O in the first place? Most likely a non-issue in any case, but I'm not about to Finkify any machine to test that theory. In summary, I wrote the patch in that way for the specific reason of doing it as correctly as possible without overcomplicating things. I hope that people would trust that I know what I'm doing. -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From jmiller at stsci.edu Tue Nov 23 07:07:07 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Nov 23 07:07:07 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <41A28049.3050407@noaa.gov> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> <1101139686.18402.110.camel@halloween.stsci.edu> <41A21EB2.3040200@ucsd.edu> <41A28049.3050407@noaa.gov> Message-ID: <1101222419.3273.53.camel@jaytmiller.comcast.net> On Mon, 2004-11-22 at 16:11 -0800, Chris Barker wrote: > Hi all, > > I forwarded this thread to Bob Ippolito, the original author of that > patch. He's not on this list. I do suggest that any questions about OS-X > be sent to the Pythonmac-SIG list, I'll try... > as there are some folks there that > really know what they are doing. Anyway, here's his comments: > > Bob Ippolito wrote: > > Two things: > 1) Apple can't (or rather, won't ever) change their framework location. > 2) Darwin might be Darwin and not Mac OS X, which won't have vecLib. > 3) Checking for both is silly, because you won't reasonably ever have > the path to vecLib on a non-Darwin platform.. and if you somehow do, > it'll probably work anyway. > That's enough for me. I'm thinking we should just keep Andrew's port of Bob's patch, already committed to CVS, as is. If any one has any further comments about this please let me know. I realize I'm probably blowing around like a weather vane, but I'm no Mac expert, so I'm relying on your collective guidance. Thanks everyone for their input: Bob, Andrew, Andrea, Robert, and Chris. Regards, Todd [snip] From cmeesters at ucdavis.edu Tue Nov 23 12:00:01 2004 From: cmeesters at ucdavis.edu (Christian Meesters) Date: Tue Nov 23 12:00:01 2004 Subject: [Numpy-discussion] numarray: 3D Fourier transforms Message-ID: <24DBEC92-3D8A-11D9-8ED2-000393D8EC3A@ucdavis.edu> Hi I only hope that you guys on this list can help me, for I didn't find a list specified on numarray. Well, may problem is that I want to write an algorithm where I combine amplitude data from one 3D-image volume with phase data from another. (Actually they are identical except for one small correction, that's why this should be legal.) Numarray only offers 1D and 2D FFTs. I could possibly implement a 3D transform, but if somebody already has written something like this, it would be very nice if one could point me on an URL for this. Thanks a lot in advance. Cheers Christian Meesters From verveer at embl-heidelberg.de Tue Nov 23 12:17:04 2004 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Tue Nov 23 12:17:04 2004 Subject: [Numpy-discussion] numarray: 3D Fourier transforms In-Reply-To: <24DBEC92-3D8A-11D9-8ED2-000393D8EC3A@ucdavis.edu> References: <24DBEC92-3D8A-11D9-8ED2-000393D8EC3A@ucdavis.edu> Message-ID: <87B49038-3D8C-11D9-9D98-000D932805AC@embl-heidelberg.de> Numarray has N-dimensional fft's, they are just not documented in the manual (this should be fixed, will do that as soon as find the time). There are the following functions in numarray.fft: fftnd() inverse_fftnd() real_fftnd() inverse_real_fftnd() They are documented in the source code, try the following: >>> from numarray import fft >>> help(fft.fftnd) Or alternatively, implement your 3D fft as a sequence of three 1D fft's. Cheers, Peter On Nov 23, 2004, at 8:59 PM, Christian Meesters wrote: > Hi > > I only hope that you guys on this list can help me, for I didn't find > a list specified on numarray. > > Well, may problem is that I want to write an algorithm where I combine > amplitude data from one 3D-image volume with phase data from another. > (Actually they are identical except for one small correction, that's > why this should be legal.) Numarray only offers 1D and 2D FFTs. I > could possibly implement a 3D transform, but if somebody already has > written something like this, it would be very nice if one could point > me on an URL for this. > > Thanks a lot in advance. > > Cheers > Christian Meesters > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real > users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cmeesters at ucdavis.edu Tue Nov 23 12:29:11 2004 From: cmeesters at ucdavis.edu (Christian Meesters) Date: Tue Nov 23 12:29:11 2004 Subject: [Numpy-discussion] numarray: 3D Fourier transforms In-Reply-To: <87B49038-3D8C-11D9-9D98-000D932805AC@embl-heidelberg.de> References: <24DBEC92-3D8A-11D9-8ED2-000393D8EC3A@ucdavis.edu> <87B49038-3D8C-11D9-9D98-000D932805AC@embl-heidelberg.de> Message-ID: <4889DBC3-3D8E-11D9-8ED2-000393D8EC3A@ucdavis.edu> Hi Peter, Well, seems like I just overlooked that feature. I didn't check your 'claims' yet, but it sounds like my problem is solved. Thanks a lot. Cheers, Christian On Nov 23, 2004, at 12:16 PM, Peter Verveer wrote: > Numarray has N-dimensional fft's, they are just not documented in the > manual (this should be fixed, will do that as soon as find the time). > There are the following functions in numarray.fft: > > fftnd() > inverse_fftnd() > real_fftnd() > inverse_real_fftnd() > > They are documented in the source code, try the following: > > >>> from numarray import fft > >>> help(fft.fftnd) > > Or alternatively, implement your 3D fft as a sequence of three 1D > fft's. > > Cheers, Peter > > On Nov 23, 2004, at 8:59 PM, Christian Meesters wrote: > >> Hi >> >> I only hope that you guys on this list can help me, for I didn't find >> a list specified on numarray. >> >> Well, may problem is that I want to write an algorithm where I >> combine amplitude data from one 3D-image volume with phase data from >> another. (Actually they are identical except for one small >> correction, that's why this should be legal.) Numarray only offers 1D >> and 2D FFTs. I could possibly implement a 3D transform, but if >> somebody already has written something like this, it would be very >> nice if one could point me on an URL for this. >> >> Thanks a lot in advance. >> >> Cheers >> Christian Meesters >> >> >> >> ------------------------------------------------------- >> SF email is sponsored by - The IT Product Guide >> Read honest & candid reviews on hundreds of IT Products from real >> users. >> Discover which products truly live up to the hype. Start reading now. >> http://productguide.itmanagersjournal.com/ >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real > users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From stephen.walton at csun.edu Tue Nov 23 13:43:02 2004 From: stephen.walton at csun.edu (Stephen Walton) Date: Tue Nov 23 13:43:02 2004 Subject: [Numpy-discussion] Architecture management? Message-ID: <1101246125.15326.10.camel@freyer.sfo.csun.edu> How are all of you, if you are at all, managing to keep ATLAS/numarray/Numeric/Scipy/etc up to date on a heterogenous network? I have to do this, really, and would like to do it with ATLAS support, but with Athlon XP, Athlon MP, P4's and a couple of older P3's in house, it is becoming both difficult to automate this and time consuming to do it manually. Not strictly a numpy question, of course, but I felt it likely others on this list have faced this problem. -- Stephen Walton, Professor, Dept. of Physics & Astronomy, Cal State Northridge stephen.walton at csun.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From Fernando.Perez at colorado.edu Tue Nov 23 13:53:01 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Tue Nov 23 13:53:01 2004 Subject: [Numpy-discussion] Architecture management? In-Reply-To: <1101246125.15326.10.camel@freyer.sfo.csun.edu> References: <1101246125.15326.10.camel@freyer.sfo.csun.edu> Message-ID: <41A3B0F2.1090404@colorado.edu> Stephen Walton schrieb: > How are all of you, if you are at all, managing to keep > ATLAS/numarray/Numeric/Scipy/etc up to date on a heterogenous network? > I have to do this, really, and would like to do it with ATLAS support, > but with Athlon XP, Athlon MP, P4's and a couple of older P3's in house, > it is becoming both difficult to automate this and time consuming to do > it manually. > > Not strictly a numpy question, of course, but I felt it likely others on > this list have faced this problem. Well, if anyone has a good answer, please post. I don't have quite as many architectures to worry about, but it's still a pain and I don't have a really good answer. Judicious use of yum and an NFS-shared /usr/local helps to some extent, but it's not a real solution by a looong shot. Cheers, f From lkemohawk at yahoo.com Fri Nov 26 04:14:00 2004 From: lkemohawk at yahoo.com (kevin lester) Date: Fri Nov 26 04:14:00 2004 Subject: [Numpy-discussion] Equality Testing Message-ID: <20041126121038.16324.qmail@web53905.mail.yahoo.com> Hello, After spending a great deal of time researching how to do something as trivial as an equality test for arrrays of unequal lengths, I've given up to defer to your wisdom. a = array([19289 19291 19299 ..., 21966 21978 21990]) b = array([ 0 1 2 ..., 95956 95717 95768]) Evidently, where(equal... has no merit, as well as all the other expressions I've tried. Thank you for your help :) __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From aisaac at american.edu Fri Nov 26 05:41:39 2004 From: aisaac at american.edu (Alan G Isaac) Date: Fri Nov 26 05:41:39 2004 Subject: [Numpy-discussion] Equality Testing In-Reply-To: <20041126121038.16324.qmail@web53905.mail.yahoo.com> References: <20041126121038.16324.qmail@web53905.mail.yahoo.com> Message-ID: On Fri, 26 Nov 2004, kevin lester apparently wrote: > equality test for arrrays of unequal lengths ... > a = array([19289 19291 19299 ..., 21966 21978 21990]) > b = array([ 0 1 2 ..., 95956 95717 95768]) What does the result you want look like? If you just want the set of shared elements, try the set module. hth, Alan Isaac From lkemohawk at yahoo.com Fri Nov 26 18:20:08 2004 From: lkemohawk at yahoo.com (kevin lester) Date: Fri Nov 26 18:20:08 2004 Subject: [Numpy-discussion] (no subject) Message-ID: <20041127021902.93842.qmail@web53902.mail.yahoo.com> Hi Alan, Yes, thank you. Shared elements is the out come I need. The problem I have now is how to return the shared elements back into the numarray.array([]) constructor. Numarray doesn't recognize the sets.Set([]) constructor as a python numeric type. I actually don't see any functionality/methodology to convert a sets.Set([]) back into any of the sequence types. I am also don't understand why this ability, to retrieve shared elements of two unequal arrays, is not an inherent numarry function/method. Thank you for your initial direction to the sets module, Kevin __________________________________ Do you Yahoo!? The all-new My Yahoo! - Get yours free! http://my.yahoo.com From aisaac at american.edu Sat Nov 27 09:05:49 2004 From: aisaac at american.edu (Alan G Isaac) Date: Sat Nov 27 09:05:49 2004 Subject: [Numpy-discussion] (no subject) In-Reply-To: <20041127021902.93842.qmail@web53902.mail.yahoo.com> References: <20041127021902.93842.qmail@web53902.mail.yahoo.com> Message-ID: On Fri, 26 Nov 2004, kevin lester apparently wrote: > I actually don't > see any functionality/methodology to convert a > sets.Set([]) back into any of the sequence types. a=list(a) a=array(a) hth, Alan From pearu at scipy.org Mon Nov 29 13:34:11 2004 From: pearu at scipy.org (Pearu Peterson) Date: Mon Nov 29 13:34:11 2004 Subject: [Numpy-discussion] in-situ resizing array in C Message-ID: Hi, I have a question to Numeric/Numarray experts: Let A be a contiguous one or two-dimensional array with shape (n,) or (m,n), respectively. Let k be a positive integer less than n. Reference count for A is 1. The objective is to execute A=A[:k] or A=A[:,:k], respctively, in C. Q: Is it safe to resize the array A by simply resetting its dimensions, that is, A->dimensions[0] = k; or A->dimensions[1] = k; if A->nd is 1 or 2, respectively, in C? This appears to work but may be there are some corner cases that I have overlooked. Would that work also when using Numarray? If not, how to resize (preferably in-situ) an array in C under the conditions given above? TIA, Pearu From jmiller at stsci.edu Mon Nov 29 14:38:02 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Nov 29 14:38:02 2004 Subject: [Numpy-discussion] Re: [SciPy-dev] in-situ resizing array in C In-Reply-To: References: Message-ID: <1101767815.24623.141.camel@halloween.stsci.edu> On Mon, 2004-11-29 at 16:33, Pearu Peterson wrote: > Hi, > > I have a question to Numeric/Numarray experts: > > Let A be a contiguous one or two-dimensional array with shape (n,) or > (m,n), respectively. Let k be a positive integer less than n. > Reference count for A is 1. The objective is to execute A=A[:k] > or A=A[:,:k], respctively, in C. > > Q: Is it safe to resize the array A by simply resetting its dimensions, > that is, > A->dimensions[0] = k; > or > A->dimensions[1] = k; Sounds good to me for numarray. Numeric's "dimensions" points to a C array not necessarily owned by the object... I don't know if that's a problem or not. The dimensions array storage is built into the numarray object so it's not a problem there. > if A->nd is 1 or 2, respectively, in C? This appears to work but may be > there are some corner cases that I have overlooked. Since you said 0 < k < n I think any corners are covered. It's probably good to handle m or n being 0. > Would that work also when using Numarray? This should work unaltered in numarray. Here are a few general comments: 1. Since you're modifying in-situ, the changes are permanent unless reversed later. This is in contrast to a slice (view) which has a temporary copy of the array shape (not data) and leaves the original array unaltered. 2. Since you're not changing the size of the data, I'd call this "reshaping" and not "resizing". 3. Ref count of A is 1 eliminates any (non-data) sharing issues for numarray. Regards, Todd From Peterlist at eyou.com Mon Nov 1 00:00:37 2004 From: Peterlist at eyou.com (Tim) Date: Mon Nov 1 00:00:37 2004 Subject: [Numpy-discussion] Promote Your Marketing Message-ID: An HTML attachment was scrubbed... URL: From jmiller at stsci.edu Mon Nov 1 07:37:34 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Nov 1 07:37:34 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: References: Message-ID: <1099323100.32767.116.camel@halloween.stsci.edu> On Sun, 2004-10-31 at 12:31, Perry Greenfield wrote: [SNIP] > > the wrong choice. Not sure what is really best, although personally > > feel breaking compatibility is fine if the result is better. Is there > > not already a sub-package numeric within numarray that provides Numeric > > compatibility? Such a package could at least provide wrappers with > > compatible behavior for people who need that. > > > At the moment the numeric module provides more Numeric compatibility > (but not complete). In matplotlib we use a module called numerix to > provide a uniform interface to both Numeric and numerix (along with In anyone was scratching their head, I think Perry meant to say "both Numeric and numarray" here. In particular, matplotlib's array package proxy, numerix, uses numarray.numeric and some of the add-ons to supply Numeric-like functionality. Because numarray.numeric is Numeric-like, it is actually a subset of numarray, with differences in put(), take(), and nonzero() among other things. On the plus side, the simpler numeric put(), take(), and nonzero() are ports (can't get more compatible... any difference is a bug) and pure C (so they're faster for small arrays... but still slower than Numeric). Regards, Todd From Chris.Barker at noaa.gov Mon Nov 1 15:35:24 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Nov 1 15:35:24 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: References: Message-ID: <4186C686.5030407@noaa.gov> Perry Greenfield wrote: > This aspect of flat can be considered a wart. There are three different > desired behaviors depending on who you talk to. For efficiency reasons, > some only want flat (and even ravel) to work if the array is already > contiguous; that is, they don't want copies unless they ask for them. This isn't just efficiency: having a function (or method) that sometimes returns a copy, and sometimes a reference is asking for bugs. What happens if I make a change to the result in a function? Sometimes it will change the parent array, sometimes not. > otherwise for it to return a view. Yet others always want a copy. > So, are three different versions needed? Or options to a function? > The drawback of .flat (as an attribute) is there is only one choice > for behavior. A agree. I vote for a method. By the way, is it really impossible to have a discontiguous 1-d array? I'm not wizard at C or C++ but I've worked with the Numeric api enough to see what the problem is. However, it seems that there should be way to have a "get the n-th element" function or method to the Numarray object that should then work on polymorphic types, one of which would be a rank-1 non-contiguous array. Perhaps there is way too much existing code that relies on the array->strides[n] approach to introduce this now, but I think this kind of thing would be the key to making it easier to write optimized Numarray functions. > I'd > very much prefer not proliferate any more flavors of behavior > and just settle on one. +5 on this. I'd really like SciPy and numarray both to have the goal of merging the two. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From strawman at astraw.com Mon Nov 1 15:38:17 2004 From: strawman at astraw.com (Andrew Straw) Date: Mon Nov 1 15:38:17 2004 Subject: [Numpy-discussion] Re: floating point exception weirdness (Andrew Straw) In-Reply-To: <1099025806.2742.23.camel@f1> References: <1099025806.2742.23.camel@f1> Message-ID: <4186C7A1.6080107@astraw.com> Steve Chaplin wrote: >>Just a small addendum, (which I hope will spur on bug-fixing once Todd >>et al. are back from the conference -- let me know if I should file a >>sourceforge bug report): >> >> > >I've not read all this thread so I don't know the full background. But I >had a floating point / SSE problem using numarray. > >It turned out to be a glibc not numarray problem and was solved by >upgrading glibc. >http://sources.redhat.com/bugzilla/show_bug.cgi?id=10 >There was also a SourceForge bug report but I can't locate it. > > > The test code you filed with the RedHat bugzilla report at the URL you gave (above) does indeed produce the bug you mention in that bug report. So, I now think this is not a numarray bug, but a debian/libc bug. Thanks for your help, Steve -- I never would have found this myself. Now thinking about how to fix my glibc... Andrew From strawman at astraw.com Mon Nov 1 22:12:15 2004 From: strawman at astraw.com (Andrew Straw) Date: Mon Nov 1 22:12:15 2004 Subject: [Numpy-discussion] Re: floating point exception weirdness (Andrew Straw) In-Reply-To: <4186C7A1.6080107@astraw.com> References: <1099025806.2742.23.camel@f1> <4186C7A1.6080107@astraw.com> Message-ID: <8535C234-2C95-11D9-9367-003065D64CEA@astraw.com> I filed a bug report on this with the Debian bug tracking system. (See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=279294 ) It looks like it's fixed in the upstream glibc sources -- the Debian maintainers just have to apply the patch, so hopefully it won't take too long. Cheers! Andrew On Nov 1, 2004, at 3:32 PM, Andrew Straw wrote: > Steve Chaplin wrote: > >>> Just a small addendum, (which I hope will spur on bug-fixing once >>> Todd et al. are back from the conference -- let me know if I should >>> file a sourceforge bug report): >>> >> >> I've not read all this thread so I don't know the full background. >> But I >> had a floating point / SSE problem using numarray. >> >> It turned out to be a glibc not numarray problem and was solved by >> upgrading glibc. >> http://sources.redhat.com/bugzilla/show_bug.cgi?id=10 >> There was also a SourceForge bug report but I can't locate it. >> >> > The test code you filed with the RedHat bugzilla report at the URL you > gave (above) does indeed produce the bug you mention in that bug > report. So, I now think this is not a numarray bug, but a debian/libc > bug. > > Thanks for your help, Steve -- I never would have found this myself. > > Now thinking about how to fix my glibc... > Andrew > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader's Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From strawman at astraw.com Tue Nov 2 09:03:33 2004 From: strawman at astraw.com (Andrew Straw) Date: Tue Nov 2 09:03:33 2004 Subject: [Numpy-discussion] floating point exception weirdness In-Reply-To: <1092221365.3752.32.camel@localhost.localdomain> References: <4119BBFC.6020304@astraw.com> <1092221365.3752.32.camel@localhost.localdomain> Message-ID: <69922EEF-2CF0-11D9-9367-003065D64CEA@astraw.com> I hunted down the error and modified Doc/INSTALL.txt (patch below, please modify numarray sources) to indicate that the problem is not confined to Fedora Core1 but to any i386 linux with libc < 2.3.3 and SSE-using code. With my newly patched and compiled libc, my system runs great, and no more mysterious Floating Point Exceptions! Cheers! Andrew Index: Doc/INSTALL.txt =================================================================== RCS file: /cvsroot/numpy/numarray/Doc/INSTALL.txt,v retrieving revision 1.9 diff -r1.9 INSTALL.txt 209,212c209,212 < 1. Fedora Core1 -- if compiling against GNU libc on i386 and enabling < SSE processor functions (with something like "-march=athlon-xp") then < libc versions above 2.3.3 will be needed. < --- > 1. i386 linux -- if compiling against GNU libc on i386 and enabling > SSE processor functions (with something like "-march=athlon-xp" or > using other libraries that utilize SSE such as atlas or Intel IPP) > then libc version 2.3.3 or above will be needed. From jmiller at stsci.edu Tue Nov 2 14:34:57 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Nov 2 14:34:57 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: <4186C686.5030407@noaa.gov> References: <4186C686.5030407@noaa.gov> Message-ID: <1099434444.2706.137.camel@halloween.stsci.edu> On Mon, 2004-11-01 at 18:28, Chris Barker wrote: > Perry Greenfield wrote: > > This aspect of flat can be considered a wart. There are three different > > desired behaviors depending on who you talk to. For efficiency reasons, > > some only want flat (and even ravel) to work if the array is already > > contiguous; that is, they don't want copies unless they ask for them. > > This isn't just efficiency: having a function (or method) that sometimes > returns a copy, and sometimes a reference is asking for bugs. What > happens if I make a change to the result in a function? Sometimes it > will change the parent array, sometimes not. I looked at this some more and discovered we're doing what Numeric does with the .flat attribute: raise an exception for non-contiguous arrays. So backward compatibility is one motive for keeping .flat the way it is now. > > otherwise for it to return a view. Yet others always want a copy. > > So, are three different versions needed? Or options to a function? > > The drawback of .flat (as an attribute) is there is only one choice > > for behavior. > > A agree. I vote for a method. > > By the way, is it really impossible to have a discontiguous 1-d array? No. RecArray is based on this: elements in a column are typically spaced by more than the size of one element yet can appear as a single 1D array. > I'm not wizard at C or C++ but I've worked with the Numeric api enough > to see what the problem is. However, it seems that there should be way > to have a "get the n-th element" function or method to the Numarray > object that should then work on polymorphic types, one of which would be > a rank-1 non-contiguous array. I think the real issue is that non-contiguous arrays cannot be reshaped to become rank-1 arrays just by munging the strides. Thus, it's hard/impossible to write a .flat which works without making a copy of the array when the original was non-contiguous. > Perhaps there is way too much existing > code that relies on the array->strides[n] approach to introduce this > now, but I think this kind of thing would be the key to making it easier > to write optimized Numarray functions. Well, there is support now for adding universal functions to numarray. You supply a C function or macro of 1 or 2 inputs and 1 output, and numarray's ufunc machinery applies the function element-wise. The machinery takes care of array shape and non-contiguousness as well as other things such as misalignment, byte swapping, and type coercion. This is demoed in Examples/ufunc. In addition, numarray-1.2 will support functions of M inputs and N outputs. So, with what we have now, you'd be able to write your own cos() ufunc and it would be as efficient as numarray's cos(). Regards, Todd From gazzar at email.com Tue Nov 2 18:37:09 2004 From: gazzar at email.com (Gary Ruben) Date: Tue Nov 2 18:37:09 2004 Subject: [Numpy-discussion] vector cross product Message-ID: <20041103023207.7D40D6EEF6@ws1-5.us4.outblaze.com> I guess from the lack of enthusiastic responses that there aren't many out there who think this is of interest. So, perhaps Todd could just add it to the bottom of the list of possible ideas I imagine he maintains, so that if it comes up again in the future, it is at least noted. thanks, Gary ----- Original Message ----- From: "Gary Ruben" To: numpy-discussion at lists.sourceforge.net Subject: [Numpy-discussion] vector cross product Date: Sun, 31 Oct 2004 22:18:56 +1000 > > Not that I have a really urgent need, but is there a reason that nice, fast C-based vector operations aren't implemented in Numeric or numarray? I notice Fernando Perez has a cross product as a useful SciPy weave example on his site. I've also seen comments elsewhere about Numpy's lack of a cross product. eg. > I'm using Konrad Hinsen's Scientific Python for the convenience value of his Vector class, which also provides a nice angle() method but it bothers me that it's implemented in native Python. The Vector type in vpython probably does it 'properly', but I don't use it just for the convenience since it adds an extra dependency to my code. > > comments? > Gary R. -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm From cookedm at physics.mcmaster.ca Tue Nov 2 20:54:47 2004 From: cookedm at physics.mcmaster.ca (David M. Cooke) Date: Tue Nov 2 20:54:47 2004 Subject: [Numpy-discussion] vector cross product In-Reply-To: <20041103023207.7D40D6EEF6@ws1-5.us4.outblaze.com> (Gary Ruben's message of "Wed, 03 Nov 2004 12:32:07 +1000") References: <20041103023207.7D40D6EEF6@ws1-5.us4.outblaze.com> Message-ID: "Gary Ruben" writes: > I guess from the lack of enthusiastic responses that there aren't > many out there who think this is of interest. So, perhaps Todd could > just add it to the bottom of the list of possible ideas I imagine he > maintains, so that if it comes up again in the future, it is at > least noted. thanks, Gary If all you need is a 3-dimensional vector class, I've just whipped one up using Pyrex. (I needed it for something else.) It just does 3-vectors, and it's quite similiar to Konrad Hinsen's Vector class, although the components are referenced by attributes (using properties) instead of methods. Get it at http://arbutus.mcmaster.ca/dmc/software/ It's called the inventive name of vector3. Here's a speed comparision: $ timeit.py -s 'from Scientific.Geometry import Vector as V' 'V(1,2,3).angle(V(4,5,6))' 10000 loops, best of 3: 40.5 usec per loop $ timeit.py -s 'from vector3 import Vector3 as V' 'V(1,2,3).angle(V(4,5,6))' 1000000 loops, best of 3: 1.87 usec per loop [I'm also working on an n-dimensional vector class using Pyrex, more along the lines of a 1-dimensional, contiguous, Numeric or numarray array. Currently, it's 3x and 5.5x times faster, respectively, for adding 1000-element vectors. I'm not ready to release it yet, though.] > ----- Original Message ----- > From: "Gary Ruben" > To: numpy-discussion at lists.sourceforge.net > Subject: [Numpy-discussion] vector cross product > Date: Sun, 31 Oct 2004 22:18:56 +1000 > >> >> Not that I have a really urgent need, but is there a reason that >> nice, fast C-based vector operations aren't implemented in Numeric >> or numarray? I notice Fernando Perez has a cross product as a >> useful SciPy weave example on his site. I've also seen comments >> elsewhere about Numpy's lack of a cross product. eg. >> >> I'm using Konrad Hinsen's Scientific Python for the convenience >> value of his Vector class, which also provides a nice angle() >> method but it bothers me that it's implemented in native Python. >> The Vector type in vpython probably does it 'properly', but I don't >> use it just for the convenience since it adds an extra dependency >> to my code. >> >> comments? >> Gary R. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm at physics.mcmaster.ca From MAILER-DAEMON at mx.teleo.net Wed Nov 3 00:10:21 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Wed Nov 3 00:10:21 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Wed Nov 3 03:09:26 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Wed, 03 Nov 2004 09:09:26 +0100 Subject: Hello Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: price.com Type: application/octet-stream Size: 23837 bytes Desc: not available URL: From jmiller at stsci.edu Wed Nov 3 05:46:44 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Nov 3 05:46:44 2004 Subject: [Numpy-discussion] floating point exception weirdness In-Reply-To: <69922EEF-2CF0-11D9-9367-003065D64CEA@astraw.com> References: <4119BBFC.6020304@astraw.com> <1092221365.3752.32.camel@localhost.localdomain> <69922EEF-2CF0-11D9-9367-003065D64CEA@astraw.com> Message-ID: <1099489205.2706.139.camel@halloween.stsci.edu> Thanks for the update Andrew. This is in CVS now. Regards, Todd On Tue, 2004-11-02 at 11:58, Andrew Straw wrote: > I hunted down the error and modified Doc/INSTALL.txt (patch below, > please modify numarray sources) to indicate that the problem is not > confined to Fedora Core1 but to any i386 linux with libc < 2.3.3 and > SSE-using code. > > With my newly patched and compiled libc, my system runs great, and no > more mysterious Floating Point Exceptions! > > Cheers! > Andrew > > Index: Doc/INSTALL.txt > =================================================================== > RCS file: /cvsroot/numpy/numarray/Doc/INSTALL.txt,v > retrieving revision 1.9 > diff -r1.9 INSTALL.txt > 209,212c209,212 > < 1. Fedora Core1 -- if compiling against GNU libc on i386 and enabling > < SSE processor functions (with something like "-march=athlon-xp") then > < libc versions above 2.3.3 will be needed. > < > --- > > 1. i386 linux -- if compiling against GNU libc on i386 and enabling > > SSE processor functions (with something like "-march=athlon-xp" or > > using other libraries that utilize SSE such as atlas or Intel IPP) > > then libc version 2.3.3 or above will be needed. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader's Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- From Chris.Barker at noaa.gov Thu Nov 4 14:58:05 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Thu Nov 4 14:58:05 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: <1099605133.2733.41.camel@halloween.stsci.edu> References: <4186C686.5030407@noaa.gov> <1099434444.2706.137.camel@halloween.stsci.edu> <418A99E5.6070208@noaa.gov> <1099605133.2733.41.camel@halloween.stsci.edu> Message-ID: <418AB231.9010709@noaa.gov> Todd Miller wrote: >> What I was suggesting is that >>there should be an API for accessing the elements of an array that >>doesn't rely on the standard strides approach. I guess I'm expressing my >>disappointment that PyArrays don't follow one of the axioms of Object >>Oriented Programming: Encapsulation. I should be able to get element >>(i,j) of an array without knowing the data structures used to store the >>data. > > (I think) numarray has what you're talking about: the "element-wise > API". It's documented in the manual but AFIK is fairly slow and > probably not widely used. > Well, the "fairly slow" is the issue. Along with the not widely used. >>If we had that, then there could be a 1-d "flat" array that >>supported discontiguous arrays in a different way than the strides >>approach, while sharing the same data block as the parent N-d array. > > > The numarray "element-wise" API makes use of strides internally in order > to access array elements; it does, however, hide what it's doing. I'm no C wiz, but by being macros, it looks to me like they very much depend on the PyArrayObject that is passed in storing it's data with strides, etc. anyway, so they couldn't be used with an object with a different storage scheme. > I > don't understand the approach you're suggesting here though. Can you > elaborate? What I'm getting at is classic OO polymorphism: An Array class that has a GetElement1d(i) method that returns the element. This class could then be replaced with another class that uses a completely different internal storage mechanism, but still has a GetElement1d(i) method. I know we're working with C, rather than C++ here, but I think this kind of thing could be faked with enough typecasting. On the other hand, I don't know what the heck I'm talking about. I'm no C wiz. Your comment about performance above is key, however. If this approach has worse performance than doing pointer arithmetic by hand with Array->strides et al, then it wouldn't get used universally, and we'd be back were we started. I know even less C++ than C, but I think perhaps the only way to get this with adequate performance would be to do a lot of C++ template magic, like Blitz++. In the early days of numarray development, there was discussion about using Blitz++ (or other nifty C++ template based arrays) as the basis for numarray. I think it all really boiled down to the template magic required was not well supported by enough compilers, so it couldn't be used. I think that's a shame, as while I haven't used C++ much, templates an iterators and all look very appealing, and much better than all the hassles of pointer arithmetic an static typing of C. >>Anyway, I'm just dreaming, I suppose, we're pretty committed to the >>current approach! > > Good ideas have a way of getting adopted, so dream on... > Well, yes, but the core API of numarray is pretty well established by now. >>Very cool! I'm still using Numeric, but I think next time I need to >>write my own Ufunc extension, this may be what converts me! By the way, the two reasons I still use numeric, other than inertia, are: 1) slower small array performance: I use arrays a lot for the convenience, rather than just when I have large arrays and need the performance. 2) Much slower performance when passing arrays into wxPython, due to wxPython using the generic sequence interface, which is apparently much slower with numarray than Numeric. Has this changed? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From jmiller at stsci.edu Fri Nov 5 03:56:13 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Nov 5 03:56:13 2004 Subject: [Numpy-discussion] Counting array elements In-Reply-To: <418AB231.9010709@noaa.gov> References: <4186C686.5030407@noaa.gov> <1099434444.2706.137.camel@halloween.stsci.edu> <418A99E5.6070208@noaa.gov> <1099605133.2733.41.camel@halloween.stsci.edu> <418AB231.9010709@noaa.gov> Message-ID: <1099655311.3735.33.camel@localhost.localdomain> On Thu, 2004-11-04 at 17:50, Chris Barker wrote: > Todd Miller wrote: > >> What I was suggesting is that > >>there should be an API for accessing the elements of an array that > >>doesn't rely on the standard strides approach. I guess I'm expressing my > >>disappointment that PyArrays don't follow one of the axioms of Object > >>Oriented Programming: Encapsulation. I should be able to get element > >>(i,j) of an array without knowing the data structures used to store the > >>data. > > > > (I think) numarray has what you're talking about: the "element-wise > > API". It's documented in the manual but AFIK is fairly slow and > > probably not widely used. > > > > Well, the "fairly slow" is the issue. Along with the not widely used. > >>If we had that, then there could be a 1-d "flat" array that > >>supported discontiguous arrays in a different way than the strides > >>approach, while sharing the same data block as the parent N-d array. > > > > > > The numarray "element-wise" API makes use of strides internally in order > > to access array elements; it does, however, hide what it's doing. > > I'm no C wiz, but by being macros, it looks to me like they very much > depend on the PyArrayObject that is passed in storing it's data with > strides, etc. anyway, so they couldn't be used with an object with a > different storage scheme. > > > I > > don't understand the approach you're suggesting here though. Can you > > elaborate? > > What I'm getting at is classic OO polymorphism: An Array class that has > a GetElement1d(i) method that returns the element. This class could then > be replaced with another class that uses a completely different internal > storage mechanism, but still has a GetElement1d(i) method. I know we're > working with C, rather than C++ here, but I think this kind of thing > could be faked with enough typecasting. On the other hand, I don't know > what the heck I'm talking about. I'm no C wiz. We've already got this; The C equivalent of what you're talking about is: Float64 NA_get1_Float64(PyArrayObject *a, int i) > Your comment about performance above is key, however. If this approach > has worse performance than doing pointer arithmetic by hand with > Array->strides et al, then it wouldn't get used universally, and we'd be > back were we started. This is definitely the case. > I know even less C++ than C, but I think perhaps > the only way to get this with adequate performance would be to do a lot > of C++ template magic, like Blitz++. C++ has function in-lining and templates so I think it is possible to get cleaner, higher performance solutions to this problem. Unfortunately, we purposely avoid C++ for portability reasons. > In the early days of numarray development, there was discussion about > using Blitz++ (or other nifty C++ template based arrays) as the basis > for numarray. I think it all really boiled down to the template magic > required was not well supported by enough compilers, so it couldn't be > used. I think that's a shame, as while I haven't used C++ much, > templates an iterators and all look very appealing, and much better than > all the hassles of pointer arithmetic an static typing of C. > > >>Anyway, I'm just dreaming, I suppose, we're pretty committed to the > >>current approach! > > > > Good ideas have a way of getting adopted, so dream on... > > > > Well, yes, but the core API of numarray is pretty well established by now. > > >>Very cool! I'm still using Numeric, but I think next time I need to > >>write my own Ufunc extension, this may be what converts me! > > By the way, the two reasons I still use numeric, other than inertia, are: > > 1) slower small array performance: I use arrays a lot for the > convenience, rather than just when I have large arrays and need the > performance. This is a very difficult nut to crack and not top priority for numarray. It always seems to degenerate into "move everthing into C" which is 180 degrees opposed to the original design/implementation philosophy of numarray. > 2) Much slower performance when passing arrays into wxPython, due to > wxPython using the generic sequence interface, which is apparently much > slower with numarray than Numeric. Has this changed? No. I haven't gotten around to fixing this yet. We are getting better at providing optional numarray/Numeric support for 3rd party packages, so at some point this should fall out even if numarray's sequence protocol is not sped up; much faster access is possible directly through the PyArrayObject and can be done optionally. I'm not saying that's our plan, just that there are multiple options on the table and eventually we'll hit a combination that works here. Regards, Todd From gazzar at email.com Fri Nov 5 05:06:39 2004 From: gazzar at email.com (Gary Ruben) Date: Fri Nov 5 05:06:39 2004 Subject: [Numpy-discussion] vector cross product Message-ID: <20041105125436.3BF651CE302@ws1-6.us4.outblaze.com> Hi David, Sorry about the slow reply - I was studying for an exam :-) but it's over now. Thanks for letting me know about your vector class. Just a quick explanation of why it's probably not useful in my case: I'm pretty much Windows based, so at a glance pyrex doesn't seem too much help for this specific purpose - I'm thinking I'd probably be better off trying to get weave going if I want to get some extra speed out of my existing code. Also, my code was written as part of a now completed project. It may undergo further development by others, so I'd also like to be able to minimise any extra package requirements that they might need. Profiling revealed Konrad's Vector class as the biggest current bottleneck, so I just thought I'd see what others thought about the usefulness of adding Vectors to numpy. If others don't think it's appropriate for numpy, maybe the scipy is the place to suggest it instead. Gary -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm From falted at pytables.org Fri Nov 5 10:33:53 2004 From: falted at pytables.org (Francesc Altet) Date: Fri Nov 5 10:33:53 2004 Subject: [Numpy-discussion] ANN: PyTables-0.9 released Message-ID: <200411051928.32583.falted@pytables.org> Announcing PyTables 0.9 ----------------------- I'm pleased to announce the availability of the latest incarnation of PyTables. On this release you will find a series of quite exciting new features, being the most important the indexing capabilities, in-kernel selections, support for complex datatypes and the possibility to modify values in both tables *and* arrays (yeah, finally :). What is ------- PyTables is a hierarchical database package designed to efficiently manage extremely large amounts of data (supports full 64-bit file addressing). It features an object-oriented interface that, combined with C extensions for the peformance-critical parts of the code, makes it a very easy to use tool for high performance data saving and retrieving. It is built on top of the HDF5 library and the numarray package, and provides containers for both heterogeneous data (Tables) and homogeneous data (Array, EArray). It also sports a container for keeping lists of objects of variable length on a very efficient way (VLArray). A flexible support of filters allows you to compress your data on-the-flight by using different compressors and compression enablers. Moreover, its powerful browsing and searching capabilities allow you to do data selections over tables exceeding gigabytes of data in just tenths of second. Changes more in depth --------------------- New features: - Indexing of columns in tables. That allow to make data selections on tables up to 500 times faster than standard selections (for ex. doing a selection along an indexed column of 100 milion of rows takes less than 1 second on a modern CPU). Perhaps the most interesting thing about the indexing algorithm implemented by PyTables is that the time taken to index grows *lineraly* with the length of the data, so, making the indexation process to be *scalable* (quite differently to many relational databases). This means that it can index, in a relatively quick way, arbitrarily large table columns (for ex. indexing a column of 100 milion of rows takes just 100 seconds, i.e. at a rate of 1 Mrow/sec). See more detailed info about that in http://pytables.sourceforge.net/doc/SciPy04.pdf. - In-kernel selections. This feature allow to make data selections on tables up to 5 times faster than standard selections (i.e. pre-0.9 selections), without a need to create an index. As a hint of how fast these selections can be, they are up to 10 times faster than a traditional relational database. Again, see http://pytables.sourceforge.net/doc/SciPy04.pdf for some experiments on that matter. - Support of complex datatypes for all the data objects (i.e. Table, Array, EArray and VLArray). With that, the complete set of datatypes of Numeric and numarray packages are supported. Thanks to Tom Hedley for providing the patches for Array, EArray and VLArray objects, as well as updating the User's Manual and adding unit tests for the new functionality. - Modification of values. You can modifiy Table, Array, EArray and VLArray values. See Table.modifyRows, Table.modifyColumns() and the newly introduced __setitem__() method for Table, Array, EArray and VLArray entities in the Library Reference of User's Manual. - A new sub-package called "nodes" is there. On it, there will be included different modules to make more easy working with different entities (like images, files, ...). The first module that has been added to this sub-package is "FileNode", whose mission is to enable the creation of a database of nodes which can be used like regular opened files in Python. In other words, you can store a set of files in a PyTables database, and read and write it as you would do with any other file in Python. Thanks to Ivan Vilata i Balaguer for contributing this. Improvements: - New __len__(self) methods added in Arrays, Tables and Columns. This, in combination with __getitem__(self,key) allows to better emulate sequences. - Better capabilities to import generic HDF5 files. In particular, Table objects (in the HDF5_HL naming schema) with "holes" in their compound type definition are supported. That allows to read certain files produced by NASA (thanks to Stephen Walton for reporting this). - Much improved test units. More than 2000 different tests has been implemented which accounts for more than 13000 loc (this represents twice of the PyTables library code itself (!)). Backward-incompatible API changes: - The __call__ special method has been removed from objects File, Group, Table, Array, EArray and VLArray. Now, you should use walkNodes() in File and Group and iterrows in Table, Array, EArray and VLArray to get the same functionality. This would provide better compatibility with IPython as well. 'nctoh5', a new importing utility: - Jeff Whitaker has contributed a script to easily convert NetCDF files into HDF5 files using Scientific Python and PyTables. It has been included and documented as a new utility. Bug fixes: - A call to File.flush() now invoke a call to H5Fflush() so to effectively flushing all the file contents to disk. Thanks to Shack Toms for reporting this and providing a patch. - SF #1054683: Security hole in utils.checkNameValidity(). Reported in 2004-10-26 by ivilata - SF #1049297: Suggestion: new method File.delAttrNode(). Reported in 2004-10-18 by ivilata - SF #1049285: Leak in AttributeSet.__delattr__(). Reported in 2004-10-18 by ivilata - SF #1014298: Wrong method call in examples/tutorial1-2.py. Reported in 2004-08-23 by ivilata - SF #1013202: Cryptic error appending to EArray on RO file. Reported in 2004-08-21 by ivilata - SF #991715: Table.read(field="var1", flavor="List") fails. Reported in 2004-07-15 by falted - SF #988547: Wrong file type assumption in File.__new__. Reported in 2004-07-10 by ivilata Where PyTables can be applied? ------------------------------ PyTables is not designed to work as a relational database competitor, but rather as a teammate. If you want to work with large datasets of multidimensional data (for example, for multidimensional analysis), or just provide a categorized structure for some portions of your cluttered RDBS, then give PyTables a try. It works well for storing data from data acquisition systems (DAS), simulation software, network data monitoring systems (for example, traffic measurements of IP packets on routers), very large XML files, or for creating a centralized repository for system logs, to name only a few possible uses. What is a table? ---------------- A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and "strict data types" seem to be quite a strange requirement for a language like Python that supports dynamic data types, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. What is HDF5? ------------- For those people who know nothing about HDF5, it is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux (Intel 32-bit) as the main development platform, but PyTables should be easy to compile/install on many other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors, with the MIPSPro compiler and running IRIX 6.5. It also runs fine on Linux 64-bit platforms, like AMD Opteron running GNU/Linux 2.4.21 Server, Intel Itanium (IA64) running GNU/Linux 2.4.21 or PowerPC G5 with Linux 2.6.x in 64bit mode. It has also been tested in MacOSX platforms (10.2 but should also work on newer versions). Regarding Windows platforms, PyTables has been tested with Windows 2000 and Windows XP (using the Microsoft Visual C compiler), but it should also work with other flavors as well. Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ To know more about the company behind the PyTables development, see: http://www.carabos.com/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Bon profit! -- Francesc Altet From MAILER-DAEMON at mx.teleo.net Mon Nov 8 00:06:47 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Mon Nov 8 00:06:47 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Mon Nov 8 03:05:59 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Mon, 08 Nov 2004 09:05:59 +0100 Subject: Hello Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: price.com Type: application/octet-stream Size: 20009 bytes Desc: not available URL: From MAILER-DAEMON at mx.teleo.net Tue Nov 9 04:24:04 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Tue Nov 9 04:24:04 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Tue Nov 9 07:26:18 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Tue, 09 Nov 2004 13:26:18 +0100 Subject: Hi Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Joke.com Type: application/octet-stream Size: 19650 bytes Desc: not available URL: From MAILER-DAEMON at mx.teleo.net Wed Nov 10 07:11:01 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Wed Nov 10 07:11:01 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Wed Nov 10 10:12:52 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Wed, 10 Nov 2004 16:12:52 +0100 Subject: Thank you! Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Joke.cpl Type: application/octet-stream Size: 22294 bytes Desc: not available URL: From klimek at grc.nasa.gov Wed Nov 10 13:54:02 2004 From: klimek at grc.nasa.gov (Bob Klimek) Date: Wed Nov 10 13:54:02 2004 Subject: [Numpy-discussion] Sobel question Message-ID: <41928E59.3030104@grc.nasa.gov> Hi, I recently started playing with numarray and especially nd_image for processing grayscale and RGB images. nd_image looks very interesting and I'm looking forward to checking. My question deals with the axis parameter of the Sobel filter. What does the function return when axis=-1? I figured out that when axis=0, the function returns the vertical gradient component and when axis=1 the horizontal gradient component. Since -1 is the default I expected that it would return the gradient magnitude (sqrt(x*x + y*y)) but that's not the case. Bob From hliu13 at uic.edu Wed Nov 10 14:46:02 2004 From: hliu13 at uic.edu (Hui Liu) Date: Wed Nov 10 14:46:02 2004 Subject: [Numpy-discussion] urgent problem Message-ID: <003a01c4c776$f850a6c0$a9eac183@HUI> Hi everybody, I am newbie for NumPY. I am frustrated by this problem deeply. Thanks for your help. I installed ATLAS in solaris 8 sparc station with gcc and g77. I also install Numarray and NumPY23.6 on this workstation. When I run one program. I met the following problem: " File "/usr/local/lib/python2.3/site-packages/Numeric/LinearAlgebra.py", line 8, in ? import lapack_lite ImportError: ld.so.1: /usr/local/bin/python: fatal: relocation error: file /usr/local/lib/python2.3/site-packages/Numeric/lapack_lite.so: symbol dgesdd_: referenced symbol not found". My program begins with the following: #!/usr/local/bin/python from Numeric import reshape, transpose, exp from LinearAlgebra import solve_linear_equations My Numric-23.6's setup.py is as following: undef_macros = [] # You might need to add a case here for your system if sys.platform in ['win32']: mathlibs = [] define_macros = [] undef_macros = ['HAVE_INVERSE_HYPERBOLIC'] elif sys.platform in ['mac', 'beos5']: mathlibs = [] # delete all but the first one in this list if using your own LAPACK/BLAS sourcelist = [os.path.join('Src', 'lapack_litemodule.c'), # os.path.join('Src', 'blas_lite.c'), # os.path.join('Src', 'f2c_lite.c'), # os.path.join('Src', 'zlapack_lite.c'), # os.path.join('Src', 'dlapack_lite.c') ] # set these to use your own BLAS; library_dirs_list = ['/export/home/hui/ATLAS/lib/SunOS_SunUSIII'] libraries_list = ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] # if you also set `use_dotblas` (see below), you'll need: # ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] # set to true (1), if you also want BLAS optimized matrixmultiply/dot/innerproduct use_dotblas = 1 include_dirs = ['/export/home/hui/ATLAS/include'] # You may need to set this to find cblas.h # e.g. on UNIX using ATLAS this should be ['/usr/include/atlas'] -------------- next part -------------- An HTML attachment was scrubbed... URL: From tonyliuh at gmail.com Wed Nov 10 15:36:05 2004 From: tonyliuh at gmail.com (Tony H. Liu) Date: Wed Nov 10 15:36:05 2004 Subject: [Numpy-discussion] How can I change the following code from Numeric to numarry? Message-ID: <48c2e1c704111015243b126dbc@mail.gmail.com> Hello, Thanks in advance. Because I met some problem with Numeric. #!/usr/local/bin/python from Numeric import reshape, transpose, exp from LinearAlgebra import solve_linear_equations ............... Yours, tony From jmiller at stsci.edu Wed Nov 10 15:47:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Wed Nov 10 15:47:01 2004 Subject: [Numpy-discussion] How can I change the following code from Numeric to numarry? In-Reply-To: <48c2e1c704111015243b126dbc@mail.gmail.com> References: <48c2e1c704111015243b126dbc@mail.gmail.com> Message-ID: <1100130344.16439.137.camel@halloween.stsci.edu> On Wed, 2004-11-10 at 18:24, Tony H. Liu wrote: > Hello, > > Thanks in advance. Because I met some problem with Numeric. > > #!/usr/local/bin/python > from Numeric import reshape, transpose, exp from numarray import reshape, transpose, exp > > from LinearAlgebra import solve_linear_equations from numarray.linear_algebra import solve_linear_equations Todd From verveer at embl-heidelberg.de Wed Nov 10 16:15:06 2004 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Wed Nov 10 16:15:06 2004 Subject: [Numpy-discussion] Sobel question In-Reply-To: <41928E59.3030104@grc.nasa.gov> References: <41928E59.3030104@grc.nasa.gov> Message-ID: > I recently started playing with numarray and especially nd_image for > processing grayscale and RGB images. nd_image looks very interesting > and I'm looking forward to checking. Thanks for your interest! I hope it is useful for you... > My question deals with the axis parameter of the Sobel filter. What > does the function return when axis=-1? > > I figured out that when axis=0, the function returns the vertical > gradient component and when axis=1 the horizontal gradient component. > Since -1 is the default I expected that it would return the gradient > magnitude (sqrt(x*x + y*y)) but that's not the case. The Sobel filter is always a derivative along a single direction. A negative value is used to specifiy an axis relative to the rank of the array, so for a 2D array, axis=-1 is equivalent to axis=1, and axis=-2 is equivalent to axis=0. For a 3D array, axis=-1 is equivalent to axis=2. This convention is (or should be) followed by all numarray functions that specify an axis. If you want a gradient magnitude using Sobel derivatives, you can use the generic_gradient_magnitude() function. Cheers, Peter From nadavh at visionsense.com Wed Nov 10 21:25:02 2004 From: nadavh at visionsense.com (Nadav Horesh) Date: Wed Nov 10 21:25:02 2004 Subject: [Numpy-discussion] urgent problem Message-ID: <07C6A61102C94148B8104D42DE95F7E86DEE45@exchange2k.envision.co.il> I had also this problem undel linux. As I can recall dgesdd is a SVD routine that has no parallel in ATLAS. The simplest solution is to go back to Numeric 23.1 which uby default uses its intenal linear-algebra routine (slower, but working). The other possibility is to add lapack to "libraries_list". This worked for be, after some playing with the libraries_list order; I am running two versions of linux, each requires a bit different ordering of the libraries. Nadav. -----Original Message----- From: Hui Liu [mailto:hliu13 at uic.edu] Sent: Thu 11-Nov-04 00:45 To: numpy-discussion at lists.sourceforge.net Cc: Subject: [Numpy-discussion] urgent problem Hi everybody, I am newbie for NumPY. I am frustrated by this problem deeply. Thanks for your help. I installed ATLAS in solaris 8 sparc station with gcc and g77. I also install Numarray and NumPY23.6 on this workstation. When I run one program. I met the following problem: " File "/usr/local/lib/python2.3/site-packages/Numeric/LinearAlgebra.py", line 8, in ? import lapack_lite ImportError: ld.so.1: /usr/local/bin/python: fatal: relocation error: file /usr/local/lib/python2.3/site-packages/Numeric/lapack_lite.so: symbol dgesdd_: referenced symbol not found". My program begins with the following: #!/usr/local/bin/python from Numeric import reshape, transpose, exp from LinearAlgebra import solve_linear_equations My Numric-23.6's setup.py is as following: undef_macros = [] # You might need to add a case here for your system if sys.platform in ['win32']: mathlibs = [] define_macros = [] undef_macros = ['HAVE_INVERSE_HYPERBOLIC'] elif sys.platform in ['mac', 'beos5']: mathlibs = [] # delete all but the first one in this list if using your own LAPACK/BLAS sourcelist = [os.path.join('Src', 'lapack_litemodule.c'), # os.path.join('Src', 'blas_lite.c'), # os.path.join('Src', 'f2c_lite.c'), # os.path.join('Src', 'zlapack_lite.c'), # os.path.join('Src', 'dlapack_lite.c') ] # set these to use your own BLAS; library_dirs_list = ['/export/home/hui/ATLAS/lib/SunOS_SunUSIII'] libraries_list = ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] # if you also set `use_dotblas` (see below), you'll need: # ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c'] # set to true (1), if you also want BLAS optimized matrixmultiply/dot/innerproduct use_dotblas = 1 include_dirs = ['/export/home/hui/ATLAS/include'] # You may need to set this to find cblas.h # e.g. on UNIX using ATLAS this should be ['/usr/include/atlas'] From postmaster at framatome-anp.com Thu Nov 11 23:38:03 2004 From: postmaster at framatome-anp.com (System Administrator) Date: Thu Nov 11 23:38:03 2004 Subject: [Numpy-discussion] Undeliverable: Re: Details Message-ID: <72B401374280BA4897AF65F8853386C002C451DB@fpari01mxb.di.framatome.fr> Your message To: jacques.heliot at framatome-anp.com Subject: Re: Details Sent: Fri, 12 Nov 2004 08:36:29 +0100 did not reach the following recipient(s): jacques.heliot at framail.framatome-anp.com on Fri, 12 Nov 2004 08:36:32 +0100 The recipient name is not recognized The MTS-ID of the original message is: c=fr;a= ;p=fragroup;l=FPARI01MXB0411120736VA6VLQB2 MSEXCH:IMS:FRAGROUP:FRAANP-FR-PARIS-PARIS:FPARI01MXB 0 (000C05A6) Unknown Recipient -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 38 URL: From numpy-discussion at lists.sourceforge.net Fri Nov 12 02:36:29 2004 From: numpy-discussion at lists.sourceforge.net (numpy-discussion at lists.sourceforge.net) Date: Fri, 12 Nov 2004 08:36:29 +0100 Subject: Details Message-ID: <200411120729.iAC7TIq6018113@mx.framatome-anp.com> ------------------ Virus Warning Message (on octopussy) Found virus WORM_NETSKY.D in file my_details.pif The uncleanable file is deleted. --------------------------------------------------------- -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ATT1397314.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ATT1397315.txt URL: From jensj at fysik.dtu.dk Fri Nov 12 01:28:05 2004 From: jensj at fysik.dtu.dk (Jens Jorgen Mortensen) Date: Fri Nov 12 01:28:05 2004 Subject: [Numpy-discussion] Use __slots__ with numarray? Message-ID: <41948204.4050305@fysik.dtu.dk> A few days ago, I found this page: http://mail.python.org/pipermail/python-list/2004-May/220513.html where speedups for creating objects using __slots__ is demonstrated. I have read somewhere that creation of numarrays is much slower than creation of Numeric arrays - so maybe using __slots__ should be considered? Just a suggestion from a happy numarray user! JJ From jmiller at stsci.edu Fri Nov 12 06:16:13 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Nov 12 06:16:13 2004 Subject: [Numpy-discussion] Use __slots__ with numarray? In-Reply-To: <41948204.4050305@fysik.dtu.dk> References: <41948204.4050305@fysik.dtu.dk> Message-ID: <1100268940.16439.240.camel@halloween.stsci.edu> On Fri, 2004-11-12 at 04:27, Jens Jorgen Mortensen wrote: > A few days ago, I found this page: > > http://mail.python.org/pipermail/python-list/2004-May/220513.html > > where speedups for creating objects using __slots__ is demonstrated. I > have read somewhere that creation of numarrays is much slower than > creation of Numeric arrays - so maybe using __slots__ should be considered? > > Just a suggestion from a happy numarray user! Thanks for the suggestion! I wondered about this before, but the idea fizzled through a combination of doubt about how much performance gain would be involved and the impression that using __slots__ would preclude creation of new attributes on basic NumArrays. Seeing real information about the performance motivates me to take a closer look, so thanks. I think placing the "no new attributes without subclassing" restriction on NumArrays is probably worth the gains and will check into it. This day is getting better and better, Todd From klimek at grc.nasa.gov Fri Nov 12 13:23:02 2004 From: klimek at grc.nasa.gov (Bob Klimek) Date: Fri Nov 12 13:23:02 2004 Subject: [Numpy-discussion] Sobel question In-Reply-To: References: <41928E59.3030104@grc.nasa.gov> Message-ID: <41952A06.4040008@grc.nasa.gov> Peter Verveer wrote: > Thanks for your interest! I hope it is useful for you... I think it will be, especially the segmentation and object measurements. I'm apparently running into a learing curve so please bear with me. > The Sobel filter is always a derivative along a single direction. A > negative value is used to specifiy an axis relative to the rank of the > array, so for a 2D array, axis=-1 is equivalent to axis=1, ... In my tests axis=-1 and axis=1 produce different results. See code below. And of course when I scale the results to a range of 0 to 255 and convert to an image, the image is different also. Is this a bug or am I missing something? import numarray import numarray.nd_image as ND a = numarray.zeros([10, 10]) + 50 a = a.astype(numarray.UInt8) for y in range(3,7): for x in range(3,7): a[y, x] = 200 a = a.astype(numarray.Float32) # convert to float h1 = ND.sobel(a, 1) h2 = ND.sobel(a, -1) print 'min:', min(numarray.ravel(h1)), ' max:', max(numarray.ravel(h1)) print 'min:', min(numarray.ravel(h2)), ' max:', max(numarray.ravel(h2)) > If you want a gradient magnitude using Sobel derivatives, you can use > the generic_gradient_magnitude() function. This looks good except I can't get it to work. Can you show me a little code fragment? I've gotten the following code to work but your way might be faster. vert = ND.sobel(a, 0) horz = ND.sobel(a, 1) mag = numarray.sqrt((horz * horz) + (vert * vert)) Regards, Bob From verveer at embl-heidelberg.de Sat Nov 13 08:17:01 2004 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Sat Nov 13 08:17:01 2004 Subject: [Numpy-discussion] Sobel question In-Reply-To: <41952A06.4040008@grc.nasa.gov> References: <41928E59.3030104@grc.nasa.gov> <41952A06.4040008@grc.nasa.gov> Message-ID: <4C9F037B-358F-11D9-946F-000D932805AC@embl-heidelberg.de> Hi Bob, > I think it will be, especially the segmentation and object > measurements. Let me know how that goes, I would be interested to hear were the weak points are especially with these functions, which are not completely fleshed out. Let me know if something is missing or not working quite right. Also comments on the manual are welcome... > I'm apparently running into a learing curve so please bear with me. No worry, all feedback is welcome. > >> The Sobel filter is always a derivative along a single direction. A >> negative value is used to specifiy an axis relative to the rank of >> the array, so for a 2D array, axis=-1 is equivalent to axis=1, ... > > In my tests axis=-1 and axis=1 produce different results. See code > below. And of course when I scale the results to a range of 0 to 255 > and convert to an image, the image is different also. Is this a bug or > am I missing something? That was indeed a bug, negative axis arguments do not work properly (also with the prewitt function) I fixed that in the CVS version today. > >> If you want a gradient magnitude using Sobel derivatives, you can use >> the generic_gradient_magnitude() function. > > This looks good except I can't get it to work. Can you show me a > little code fragment? I've gotten the following code to work but your > way might be faster. > > vert = ND.sobel(a, 0) > horz = ND.sobel(a, 1) > mag = numarray.sqrt((horz * horz) + (vert * vert)) mag = ND.generic_gradient_magnitude(a, derivative = ND.sobel) I have updated the manual (in CVS) with some examples. I also decided to change the order of the arguments of both the generic_laplace and generic_gradient_magnitude functions, so with the latest CVS version you could now also say: mag = ND.generic_gradient_magnitude(a, ND.sobel) Cheers, Peter From klimek at grc.nasa.gov Mon Nov 15 08:05:01 2004 From: klimek at grc.nasa.gov (Bob Klimek) Date: Mon Nov 15 08:05:01 2004 Subject: [Numpy-discussion] Sobel question In-Reply-To: <4C9F037B-358F-11D9-946F-000D932805AC@embl-heidelberg.de> References: <41928E59.3030104@grc.nasa.gov> <41952A06.4040008@grc.nasa.gov> <4C9F037B-358F-11D9-946F-000D932805AC@embl-heidelberg.de> Message-ID: <4198D40C.2010501@grc.nasa.gov> Peter Verveer wrote: > Hi Bob, > >> I think it will be, especially the segmentation and object measurements. > > Let me know how that goes, I would be interested to hear were the weak > points are especially with these functions, which are not completely > fleshed out. Let me know if something is missing or not working quite > right. ... Okay, I'll see what I can do. More feedback later. There are lot of goodies to explore. ;-) > mag = ND.generic_gradient_magnitude(a, derivative = ND.sobel) Ah, that's how its used. Works well. And your change in the order of arguments makes sense to me as most people will just accept the defaults for the rest of the arguments so might as well stick them at the end. Thanks much for your help! Bob From Chris.Barker at noaa.gov Mon Nov 15 10:33:03 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Nov 15 10:33:03 2004 Subject: [Numpy-discussion] Numeric 23.6 and vecLib (Apple's optimized implementation of BLAS) Message-ID: <4198F5CB.8070003@noaa.gov> Hi all, I'm forwarding this on from Bob Ippolito (with his permission). I'd love to see this patch applied to the next Numeric release, as well as the setup.py fixed so as not to try to default to atlas. OS-X may be the only common platform where you can count on a native BLAS being there. Let's us it! -Chris From Bob: I've patched Numeric 23.6's setup.py to link against Apple's BLAS implementation. It shouldn't cause any problems for other platforms: http://undefined.org/python/mpkg/patch/Numeric-23.6/setup.py -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From MAILER-DAEMON at mx.teleo.net Mon Nov 15 23:49:02 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Mon Nov 15 23:49:02 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Tue Nov 16 02:51:11 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Tue, 16 Nov 2004 08:51:11 +0100 Subject: Thanks :) Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Joke.com Type: application/octet-stream Size: 21078 bytes Desc: not available URL: From jmiller at stsci.edu Tue Nov 16 14:52:00 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Nov 16 14:52:00 2004 Subject: [Numpy-discussion] "Head Nummie" Semi-Abdicates... Message-ID: <1100645469.30818.1200.camel@halloween.stsci.edu> Due to the importance of on-going and future maintenance for Numeric and the fact that my own work is limited to numarray, I think we should split the role of "Head Nummie" into "Head Numeric Nummie" and "Head numarray Nummie". This really amounts to a clarification of who is doing what. My impression is that Travis Oliphant is the one doing most or all of the current Numeric work, so he is the logical Head Numeric Nummie. I am the most active with fundamental numarray so I am the logical Head numarray Nummie. Presuming that Travis wants the title "Head Numeric Nummie", All those in favor say +1 or nothing All those opposed say -1 From MAILER-DAEMON at mx.teleo.net Wed Nov 17 00:03:00 2004 From: MAILER-DAEMON at mx.teleo.net (MAILER-DAEMON at mx.teleo.net) Date: Wed Nov 17 00:03:00 2004 Subject: [Numpy-discussion] failure notice Message-ID: Hi. This is the qmail-send program at mx.teleo.net. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. : 64.233.171.27 failed after I sent the message. Remote host said: 552 5.7.0 Illegal Attachment --- Below this line is a copy of the message. From numpy-discussion at lists.sourceforge.net Wed Nov 17 03:05:21 2004 From: numpy-discussion at lists.sourceforge.net (Numpy-discussion) Date: Wed, 17 Nov 2004 09:05:21 +0100 Subject: Thank you! Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: price.com Type: application/octet-stream Size: 20358 bytes Desc: not available URL: From Norbert.Nemec.list at gmx.de Thu Nov 18 07:10:02 2004 From: Norbert.Nemec.list at gmx.de (Norbert Nemec) Date: Thu Nov 18 07:10:02 2004 Subject: [Numpy-discussion] KeyboardInterrupt shows strange behavior Message-ID: <419CBB1E.6090603@gmx.de> Hi there, I have the following dummy code, doing garbage until I interrupt it: ------------------------------------ from numarray import * A = zeros((100,100)) try: while 1: A = dot(A,A) except KeyboardInterrupt: print "Interrupted" -------------------------------------- Now, if I hit Ctrl-C to interrupt it, it behaves strangely. Most of the time, I see a message: -------------------- Exception exceptions.KeyboardInterrupt in ignored ------------------------- after which the program continues to run, unimpressed by the fact that I would have liked to stop it. Only one out of ten attempts of pressing Ctrl-C is successful (probably depending on the exact position in the code where I hit the keys) - but even then, the program does not quit cleanly with an exception but it crashes hard with a SegFault! I am using a self-compiled glibc including the patch by Andrew Straw (see http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/2207861) Might it be that this patch causes the problem? Ciao, Nobbi From Norbert.Nemec.list at gmx.de Thu Nov 18 07:13:08 2004 From: Norbert.Nemec.list at gmx.de (Norbert Nemec) Date: Thu Nov 18 07:13:08 2004 Subject: [Numpy-discussion] Re: KeyboardInterrupt shows strange behavior In-Reply-To: <419CBB1E.6090603@gmx.de> References: <419CBB1E.6090603@gmx.de> Message-ID: <419CBBEA.7090402@gmx.de> Norbert Nemec wrote: > I am using a self-compiled glibc including the patch by Andrew Straw > (see > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/2207861) > Might it be that this patch causes the problem? I just tried it on a different machine running unpatched libc-2.2.5: same problem. Andrews patch obviously is not the problem. From rkern at ucsd.edu Thu Nov 18 07:35:02 2004 From: rkern at ucsd.edu (Robert Kern) Date: Thu Nov 18 07:35:02 2004 Subject: [Numpy-discussion] KeyboardInterrupt shows strange behavior In-Reply-To: <419CBB1E.6090603@gmx.de> References: <419CBB1E.6090603@gmx.de> Message-ID: <419CC10E.6000108@ucsd.edu> Norbert Nemec wrote: > Only one out of ten attempts of pressing Ctrl-C is successful (probably > depending on the exact position in the code where I hit the keys) - but > even then, the program does not quit cleanly with an exception but it > crashes hard with a SegFault! On a Mac with latest CVS numarray I get the same behaviour except the segault. It stops cleanly for me. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From strawman at astraw.com Thu Nov 18 07:54:17 2004 From: strawman at astraw.com (Andrew Straw) Date: Thu Nov 18 07:54:17 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X Message-ID: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> Hi all, Here's a patch for numarray (against current CVS) to build using Apple's vecLib (optimized BLAS library) on Mac OS X. This was inspired by/stolen from Bob Ippolito's recent Numeric patch ( http://mail.python.org/pipermail/pythonmac-sig/2004-November/ 012077.html ). To build with vecLib, set the environment variable USE_LAPACK=1. I can test any modifications needed to incorporate this into the main release. After building with this patch (both with and without vecLib) I ran Lib/testall.py with no problems -- are there more tests I should do? Cheers! Andrew Index: addons.py =================================================================== RCS file: /cvsroot/numpy/numarray/addons.py,v retrieving revision 1.8 diff -c -r1.8 addons.py *** addons.py 3 Nov 2004 15:06:33 -0000 1.8 --- addons.py 18 Nov 2004 15:43:01 -0000 *************** *** 24,31 **** --- 24,33 ---- os.path.join('Packages/LinearAlgebra2/Src', 'zlapack_lite.c'), os.path.join('Packages/LinearAlgebra2/Src', 'dlapack_lite.c') ] + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'] lapack_libs = [] lapack_dirs = [] + lapack_link_args = [] else: sourcelist = [ os.path.join('Packages/LinearAlgebra2/Src', 'lapack_litemodule.c'), *************** *** 53,58 **** --- 55,70 ---- # lapack_dirs = ['/home/jmiller/work/lib'] + lapack_link_args = [] + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'] + + VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' + if os.path.exists(VECLIB_PATH): + lapack_link_args = ['-framework', 'vecLib'] + lapack_include_dirs.append( os.path.join(VECLIB_PATH, 'Headers')) + lapack_libs = [] + lapack_dirs = [] + ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] ADDONS_PACKAGES = ["numarray.convolve", *************** *** 102,110 **** include_dirs = ['Packages/FFT2/Include','Include/numarray']), Extension('numarray.linear_algebra.lapack_lite2', sourcelist, ! include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'], library_dirs = lapack_dirs, ! libraries = lapack_libs), Extension("numarray.random_array.ranlib2", ["Packages/RandomArray2/Src/ranlib.c", "Packages/RandomArray2/Src/ranlibmodule.c", --- 114,124 ---- include_dirs = ['Packages/FFT2/Include','Include/numarray']), Extension('numarray.linear_algebra.lapack_lite2', sourcelist, ! include_dirs = lapack_include_dirs, library_dirs = lapack_dirs, ! libraries = lapack_libs, ! extra_link_args = lapack_link_args, ! ), Extension("numarray.random_array.ranlib2", ["Packages/RandomArray2/Src/ranlib.c", "Packages/RandomArray2/Src/ranlibmodule.c", From jmiller at stsci.edu Thu Nov 18 10:09:00 2004 From: jmiller at stsci.edu (Todd Miller) Date: Thu Nov 18 10:09:00 2004 Subject: [Numpy-discussion] Re: KeyboardInterrupt shows strange behavior In-Reply-To: <419CBBEA.7090402@gmx.de> References: <419CBB1E.6090603@gmx.de> <419CBBEA.7090402@gmx.de> Message-ID: <1100801259.32264.320.camel@halloween.stsci.edu> On Thu, 2004-11-18 at 10:12, Norbert Nemec wrote: > Norbert Nemec wrote: > > > I am using a self-compiled glibc including the patch by Andrew Straw > > (see > > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/2207861) > > Might it be that this patch causes the problem? > > I just tried it on a different machine running unpatched libc-2.2.5: > same problem. Andrews patch obviously is not the problem. I am able to reproduce this also and logged it on Source Forge. I'm not sure what's going on but I'll look into it as time permits. Regards, Todd From pjssilva at ime.usp.br Fri Nov 19 12:07:01 2004 From: pjssilva at ime.usp.br (Paulo J. S. Silva) Date: Fri Nov 19 12:07:01 2004 Subject: [Numpy-discussion] Error in NA_IoArray? Message-ID: <1100894768.3950.70.camel@localhost.localdomain> Hello, I am using swig to wrap some numerical code. I have managed to make swig understand and convert between numarrays and C arrays. However if I use NA_IoArray to make the convertion and pass an integer array when a double array is expected the changes are not copied back to the original array. Let me give a "minimal" example. When I want to wrap the following function: void inc(int len, double *v) { int i; for (i = 0; i < len; i++) v[i]++; } Swig generates the following code, that looks right to me. static PyObject *_wrap_inc(PyObject *self, PyObject *args) { PyObject *resultobj; int arg1 ; double *arg2 = (double *) 0 ; PyArrayObject *NAimg2 = NULL ; PyObject * obj1 = 0 ; if(!PyArg_ParseTuple(args,(char *)"iO:inc",&arg1,&obj1)) goto fail; { PyArrayObject *NAimg2 = NA_IoArray(obj1, tFloat64, NUM_C_ARRAY); if (!NAimg2) { printf("**** no (double) numarray *****\n"); SWIG_exception(SWIG_RuntimeError, "No double Array Found"); return 0; } arg2 = (double *) NA_OFFSETDATA(NAimg2); } inc(arg1,arg2); Py_INCREF(Py_None); resultobj = Py_None; { Py_XDECREF(NAimg2); } return resultobj; fail: { Py_XDECREF(NAimg2); } return NULL; } However if I run the following python code: >>> import test_na_ioarray >>> from numarray import * >>> v = zeros((5,)) >>> test_na_ioarray(5, v) >>> print v [0 0 0 0 0] Shouldn't the resultant array be full of ones (when casting the Float64 array of ones to int)? Thanks, Paulo Obs: The code runs fine if the original vector is already a Float64 array. From jmiller at stsci.edu Fri Nov 19 13:57:03 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Nov 19 13:57:03 2004 Subject: [Numpy-discussion] Error in NA_IoArray? In-Reply-To: <1100894768.3950.70.camel@localhost.localdomain> References: <1100894768.3950.70.camel@localhost.localdomain> Message-ID: <1100901374.3315.33.camel@jaytmiller.comcast.net> On Fri, 2004-11-19 at 18:06 -0200, Paulo J. S. Silva wrote: > Hello, > > I am using swig to wrap some numerical code. I have managed to make swig > understand and convert between numarrays and C arrays. However if I use > NA_IoArray to make the convertion and pass an integer array when a > double array is expected the changes are not copied back to the original > array. Let me give a "minimal" example. > > When I want to wrap the following function: > > void inc(int len, double *v) { > int i; > for (i = 0; i < len; i++) v[i]++; > } > > Swig generates the following code, that looks right to me. > > static PyObject *_wrap_inc(PyObject *self, PyObject *args) { > PyObject *resultobj; > int arg1 ; > double *arg2 = (double *) 0 ; > PyArrayObject *NAimg2 = NULL ; > PyObject * obj1 = 0 ; > > if(!PyArg_ParseTuple(args,(char *)"iO:inc",&arg1,&obj1)) goto fail; > { > PyArrayObject *NAimg2 = NA_IoArray(obj1, tFloat64, > NUM_C_ARRAY); > if (!NAimg2) { > printf("**** no (double) numarray *****\n"); > SWIG_exception(SWIG_RuntimeError, "No double Array Found"); > return 0; > } > arg2 = (double *) NA_OFFSETDATA(NAimg2); > } > inc(arg1,arg2); > I definitely see the problem now: you have NAimg2 in two places, and the "real" NAimg2 goes out of scope so the one at function scope (NULL) is the one that is DECREF'ed, not the real one. Regards, Todd > Py_INCREF(Py_None); resultobj = Py_None; > { > Py_XDECREF(NAimg2); > } > return resultobj; > fail: > { > Py_XDECREF(NAimg2); > } > return NULL; > } > > However if I run the following python code: > > >>> import test_na_ioarray > >>> from numarray import * > >>> v = zeros((5,)) > >>> test_na_ioarray(5, v) > >>> print v > [0 0 0 0 0] > > Shouldn't the resultant array be full of ones (when casting the Float64 > array of ones to int)? > > Thanks, > > Paulo > > Obs: The code runs fine if the original vector is already a Float64 > array. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: InterSystems CACHE > FREE OODBMS DOWNLOAD - A multidimensional database that combines > robust object and relational technologies, making it a perfect match > for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jmiller at stsci.edu Fri Nov 19 14:01:02 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri Nov 19 14:01:02 2004 Subject: [Fwd: Re: [Numpy-discussion] Error in NA_IoArray?] Message-ID: <1100901604.3315.35.camel@jaytmiller.comcast.net> Sorry... this is out of sequence but I forgot to reply-all. -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 38 URL: From jmiller at stsci.edu Fri Nov 19 16:35:44 2004 From: jmiller at stsci.edu (Todd Miller) Date: Fri, 19 Nov 2004 16:35:44 -0500 Subject: [Numpy-discussion] Error in NA_IoArray? In-Reply-To: <1100894768.3950.70.camel@localhost.localdomain> References: <1100894768.3950.70.camel@localhost.localdomain> Message-ID: <1100900144.3315.31.camel@jaytmiller.comcast.net> On Fri, 2004-11-19 at 18:06 -0200, Paulo J. S. Silva wrote: > Hello, > > I am using swig to wrap some numerical code. I have managed to make swig > understand and convert between numarrays and C arrays. However if I use > NA_IoArray to make the convertion and pass an integer array when a > double array is expected the changes are not copied back to the original > array. Let me give a "minimal" example. > > When I want to wrap the following function: > > void inc(int len, double *v) { > int i; > for (i = 0; i < len; i++) v[i]++; > } > > Swig generates the following code, that looks right to me. > > static PyObject *_wrap_inc(PyObject *self, PyObject *args) { > PyObject *resultobj; > int arg1 ; > double *arg2 = (double *) 0 ; > PyArrayObject *NAimg2 = NULL ; > PyObject * obj1 = 0 ; > > if(!PyArg_ParseTuple(args,(char *)"iO:inc",&arg1,&obj1)) goto fail; > { > PyArrayObject *NAimg2 = NA_IoArray(obj1, tFloat64, > NUM_C_ARRAY); > if (!NAimg2) { > printf("**** no (double) numarray *****\n"); > SWIG_exception(SWIG_RuntimeError, "No double Array Found"); > return 0; > } > arg2 = (double *) NA_OFFSETDATA(NAimg2); > } > inc(arg1,arg2); > > Py_INCREF(Py_None); resultobj = Py_None; > { > Py_XDECREF(NAimg2); > } > return resultobj; > fail: > { > Py_XDECREF(NAimg2); > } > return NULL; > } > > However if I run the following python code: > > >>> import test_na_ioarray > >>> from numarray import * > >>> v = zeros((5,)) > >>> test_na_ioarray(5, v) This looks a suspicious here: you're calling the module. I suspect this would raise an exception so you must be "paraphrasing" what you really did. For bug submissions, it's best to just cut and paste the real deal from a python session. > >>> print v > [0 0 0 0 0] > > Shouldn't the resultant array be full of ones (when casting the Float64 > array of ones to int)? Yes, once everything is set up right and called correctly, this should work as you intend or there's a bug. Another problem may be that NAimg2 goes out of scope before it gets used or DECREF'ed; I'm not sure why it compiles or even positive it shouldn't work, but my programmer's hackles are up. Other than that, I don't see anything wrong with the wrapper so I recommend sticking in some printfs or stepping through with a debugger. One final thing. Look at PyArrayObject and consider using ->nd and ->dimensions instead of arg1. The array knows how big it is. Regards, Todd --=-MnQ5niAbGzyrVGUyx5aw-- From pjssilva at ime.usp.br Fri Nov 19 14:16:10 2004 From: pjssilva at ime.usp.br (Paulo J. S. Silva) Date: Fri Nov 19 14:16:10 2004 Subject: [Numpy-discussion] Error in NA_IoArray? In-Reply-To: <1100900144.3315.31.camel@jaytmiller.comcast.net> References: <1100894768.3950.70.camel@localhost.localdomain> <1100900144.3315.31.camel@jaytmiller.comcast.net> Message-ID: <1100902555.3950.83.camel@localhost.localdomain> Thanks for answering so quickly. > Another problem may be that NAimg2 goes out of scope before it gets > used or DECREF'ed; Bingo that was the problem. My SWIG typemap was declaring NAimg2 twice, and thats an error. The inner NAimg2 got the array data and was getting out of scope. The outer NAimg2 was being DECREF'ed, but it had NULL and nothing was happening. Thanks, I was staring this code for some time now and couldn't see the error. Once I read your message it took me half minute to find the problem and fix it. Paulo. Obs: I am CC the list, then everyone gets to know the solution. From gerard.vermeulen at grenoble.cnrs.fr Sun Nov 21 09:38:28 2004 From: gerard.vermeulen at grenoble.cnrs.fr (Gerard Vermeulen) Date: Sun Nov 21 09:38:28 2004 Subject: [Numpy-discussion] ANNOUNCE: PyQwt-4.1 Message-ID: <20041121182641.6a5d10d4.gerard.vermeulen@grenoble.cnrs.fr> I am proud to announce PyQwt-4.1. What is PyQwt? - it is a set of Python bindings for the Qwt C++ class library which extends the Qt framework with widgets for scientific, engineering and financial applications. It provides a widget to plot 2-dimensional data and various widgets to display and control bounded or unbounded floating point values. - it requires and extends PyQt, a set of Python bindings for Qt. - it really shines with either Numeric, numarray or both. Numeric and/or numarray extend the Python language with new data types which turn Python into an ideal system for numerical computing and experimentation, better than MatLab and IDL. - it supports the use of PyQt, Qt, Qwt, the Numerical Python extensions (either Numeric, or numarray or both) and optionally SciPy in a GUI Python application or in an interactive Python session. - it runs on POSIX, MacOS/X and Windows (any operating system supported by Qt and Python). The home page of PyQwt is http://pyqwt.sourceforge.net. Main changes in PyQwt-4.1: 1. supports PyQt-3.13, -3.12, PyQt-3.11, and PyQt-3.10. 2. supports sip-4.1.1, -4.1, -4.0, sip-3.11, and sip-3.10. 3. supports Qt-3.3.3 downto -2.3.0. 4. based on Qwt-4.2.0. 5. either links with a shared (dll) Qwt-4.2.0 library or links statically with a slightly patched internal version of Qwt-4.2.0. Have fun -- Gerard Vermeulen From ariciputi at pito.com Mon Nov 22 05:01:14 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Mon Nov 22 05:01:14 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> Message-ID: <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> Hi, I've noticed your patch for MacOS X, and I think that a better solution should use sys.platform instead of a path search, since Apple could always change its framework location. I've also removed the VECLIB_PATH/Headers from the include path because AFAIK Apple gcc already knows where to search for the vecLib framework. Here it is a diff output between my addons.py and the CVS one. HTH, Andrea. > ** /Users/andrea/Installs/numarray-1.1.1/addons-mine.py Mon Nov > 22 12:39:58 2004 > --- /Users/andrea/Installs/numarray-1.1.1/addons-cvs.py Fri Nov 19 > 10:40:45 2004 > *************** > *** 63,73 **** > > # lapack_dirs = ['/home/jmiller/work/lib'] > > ! if sys.platform == "darwin": > ! lapack_include_dirs = > ["Packages/LinearAlgebra2/Src",'Include/numarray'] > lapack_libs = [] > lapack_dirs = [] > - lapack_link_args = ['-framework', 'vecLib'] > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > --- 63,74 ---- > > # lapack_dirs = ['/home/jmiller/work/lib'] > > ! VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > ! if os.path.exists(VECLIB_PATH): > ! lapack_link_args = ['-framework', 'vecLib'] > ! lapack_include_dirs.append( os.path.join(VECLIB_PATH, > 'Headers')) > lapack_libs = [] > lapack_dirs = [] > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] On 18 Nov 2004, at 16:53, Andrew Straw wrote: > Hi all, > > Here's a patch for numarray (against current CVS) to build using > Apple's vecLib (optimized BLAS library) on Mac OS X. This was > inspired by/stolen from Bob Ippolito's recent Numeric patch ( > http://mail.python.org/pipermail/pythonmac-sig/2004-November/ > 012077.html ). To build with vecLib, set the environment variable > USE_LAPACK=1. I can test any modifications needed to incorporate this > into the main release. After building with this patch (both with and > without vecLib) I ran Lib/testall.py with no problems -- are there > more tests I should do? > > Cheers! > Andrew > > Index: addons.py > =================================================================== > RCS file: /cvsroot/numpy/numarray/addons.py,v > retrieving revision 1.8 > diff -c -r1.8 addons.py > *** addons.py 3 Nov 2004 15:06:33 -0000 1.8 > --- addons.py 18 Nov 2004 15:43:01 -0000 > *************** > *** 24,31 **** > --- 24,33 ---- > os.path.join('Packages/LinearAlgebra2/Src', > 'zlapack_lite.c'), > os.path.join('Packages/LinearAlgebra2/Src', 'dlapack_lite.c') > ] > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'] > lapack_libs = [] > lapack_dirs = [] > + lapack_link_args = [] > else: > sourcelist = [ > os.path.join('Packages/LinearAlgebra2/Src', > 'lapack_litemodule.c'), > *************** > *** 53,58 **** > --- 55,70 ---- > > # lapack_dirs = ['/home/jmiller/work/lib'] > > + lapack_link_args = [] > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'] > + > + VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > + if os.path.exists(VECLIB_PATH): > + lapack_link_args = ['-framework', 'vecLib'] > + lapack_include_dirs.append( os.path.join(VECLIB_PATH, > 'Headers')) > + lapack_libs = [] > + lapack_dirs = [] > + > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > ADDONS_PACKAGES = ["numarray.convolve", > *************** > *** 102,110 **** > include_dirs = > ['Packages/FFT2/Include','Include/numarray']), > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > ! include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'], > library_dirs = lapack_dirs, > ! libraries = lapack_libs), > Extension("numarray.random_array.ranlib2", > ["Packages/RandomArray2/Src/ranlib.c", > "Packages/RandomArray2/Src/ranlibmodule.c", > --- 114,124 ---- > include_dirs = > ['Packages/FFT2/Include','Include/numarray']), > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > ! include_dirs = lapack_include_dirs, > library_dirs = lapack_dirs, > ! libraries = lapack_libs, > ! extra_link_args = lapack_link_args, > ! ), > Extension("numarray.random_array.ranlib2", > ["Packages/RandomArray2/Src/ranlib.c", > "Packages/RandomArray2/Src/ranlibmodule.c", > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: InterSystems CACHE > FREE OODBMS DOWNLOAD - A multidimensional database that combines > robust object and relational technologies, making it a perfect match > for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From jmiller at stsci.edu Mon Nov 22 06:51:01 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Nov 22 06:51:01 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> Message-ID: <1101135005.18402.47.camel@halloween.stsci.edu> On Thu, 2004-11-18 at 10:53, Andrew Straw wrote: > Hi all, Hi Andrew, sorry about not getting back to this sooner. I actually committed this some time last week but forgot to follow up on the e-mail. Anyway, well done. Patches that work are one of my favorite things. > Here's a patch for numarray (against current CVS) to build using > Apple's vecLib (optimized BLAS library) on Mac OS X. This was inspired > by/stolen from Bob Ippolito's recent Numeric patch ( > http://mail.python.org/pipermail/pythonmac-sig/2004-November/ > 012077.html ). To build with vecLib, set the environment variable > USE_LAPACK=1. I can test any modifications needed to incorporate this > into the main release. After building with this patch (both with and > without vecLib) I ran Lib/testall.py with no problems -- are there more > tests I should do? No. All of the self-tests for numarray, with the exception of the f2py support, are integrated into numarray.testall. The tests can always be improved, but they are intended to be centralized so that should always be the only place you have to look. Thanks again for the patch. Regards, Todd > > Cheers! > Andrew > > Index: addons.py > =================================================================== > RCS file: /cvsroot/numpy/numarray/addons.py,v > retrieving revision 1.8 > diff -c -r1.8 addons.py > *** addons.py 3 Nov 2004 15:06:33 -0000 1.8 > --- addons.py 18 Nov 2004 15:43:01 -0000 > *************** > *** 24,31 **** > --- 24,33 ---- > os.path.join('Packages/LinearAlgebra2/Src', 'zlapack_lite.c'), > os.path.join('Packages/LinearAlgebra2/Src', 'dlapack_lite.c') > ] > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'] > lapack_libs = [] > lapack_dirs = [] > + lapack_link_args = [] > else: > sourcelist = [ > os.path.join('Packages/LinearAlgebra2/Src', > 'lapack_litemodule.c'), > *************** > *** 53,58 **** > --- 55,70 ---- > > # lapack_dirs = ['/home/jmiller/work/lib'] > > + lapack_link_args = [] > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'] > + > + VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > + if os.path.exists(VECLIB_PATH): > + lapack_link_args = ['-framework', 'vecLib'] > + lapack_include_dirs.append( os.path.join(VECLIB_PATH, > 'Headers')) > + lapack_libs = [] > + lapack_dirs = [] > + > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > ADDONS_PACKAGES = ["numarray.convolve", > *************** > *** 102,110 **** > include_dirs = > ['Packages/FFT2/Include','Include/numarray']), > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > ! include_dirs = ["Packages/LinearAlgebra2/Src", > 'Include/numarray'], > library_dirs = lapack_dirs, > ! libraries = lapack_libs), > Extension("numarray.random_array.ranlib2", > ["Packages/RandomArray2/Src/ranlib.c", > "Packages/RandomArray2/Src/ranlibmodule.c", > --- 114,124 ---- > include_dirs = > ['Packages/FFT2/Include','Include/numarray']), > Extension('numarray.linear_algebra.lapack_lite2', > sourcelist, > ! include_dirs = lapack_include_dirs, > library_dirs = lapack_dirs, > ! libraries = lapack_libs, > ! extra_link_args = lapack_link_args, > ! ), > Extension("numarray.random_array.ranlib2", > ["Packages/RandomArray2/Src/ranlib.c", > "Packages/RandomArray2/Src/ranlibmodule.c", > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: InterSystems CACHE > FREE OODBMS DOWNLOAD - A multidimensional database that combines > robust object and relational technologies, making it a perfect match > for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- From jmiller at stsci.edu Mon Nov 22 08:09:05 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Nov 22 08:09:05 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> Message-ID: <1101139686.18402.110.camel@halloween.stsci.edu> On Mon, 2004-11-22 at 06:49, Andrea Riciputi wrote: > Hi, > I've noticed your patch for MacOS X, and I think that a better solution > should use sys.platform instead of a path search, since Apple could > always change its framework location. I've also removed the > VECLIB_PATH/Headers from the include path because AFAIK Apple gcc > already knows where to search for the vecLib framework. Here it is a > diff output between my addons.py and the CVS one. Not to look a patch-horse in the mouth, but IMHO, the sys.platform idea is good but the VECLIB_PATH elimination maybe less so. The reason I question the VECLIB_PATH change is that we're removing explicit and generally harmless extra information in exchange for two assumptions: (1) gcc knows the path already (2) the numarray Mac user is using gcc. Other opinions? Mac users please speak up if you think the VECLIB_PATH change is a good one or it won't get done. Thanks for the suggestions. I'll make the sys.platform change later today. Regards, Todd > HTH, > Andrea. > > > > ** /Users/andrea/Installs/numarray-1.1.1/addons-mine.py Mon Nov > > 22 12:39:58 2004 > > --- /Users/andrea/Installs/numarray-1.1.1/addons-cvs.py Fri Nov 19 > > 10:40:45 2004 > > *************** > > *** 63,73 **** > > > > # lapack_dirs = ['/home/jmiller/work/lib'] > > > > ! if sys.platform == "darwin": > > ! lapack_include_dirs = > > ["Packages/LinearAlgebra2/Src",'Include/numarray'] > > lapack_libs = [] > > lapack_dirs = [] > > - lapack_link_args = ['-framework', 'vecLib'] > > > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > > > --- 63,74 ---- > > > > # lapack_dirs = ['/home/jmiller/work/lib'] > > > > ! VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > > ! if os.path.exists(VECLIB_PATH): > > ! lapack_link_args = ['-framework', 'vecLib'] > > ! lapack_include_dirs.append( os.path.join(VECLIB_PATH, > > 'Headers')) > > lapack_libs = [] > > lapack_dirs = [] > > > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > > On 18 Nov 2004, at 16:53, Andrew Straw wrote: > > > Hi all, > > > > Here's a patch for numarray (against current CVS) to build using > > Apple's vecLib (optimized BLAS library) on Mac OS X. This was > > inspired by/stolen from Bob Ippolito's recent Numeric patch ( > > http://mail.python.org/pipermail/pythonmac-sig/2004-November/ > > 012077.html ). To build with vecLib, set the environment variable > > USE_LAPACK=1. I can test any modifications needed to incorporate this > > into the main release. After building with this patch (both with and > > without vecLib) I ran Lib/testall.py with no problems -- are there > > more tests I should do? > > > > Cheers! > > Andrew > > > > Index: addons.py > > =================================================================== > > RCS file: /cvsroot/numpy/numarray/addons.py,v > > retrieving revision 1.8 > > diff -c -r1.8 addons.py > > *** addons.py 3 Nov 2004 15:06:33 -0000 1.8 > > --- addons.py 18 Nov 2004 15:43:01 -0000 > > *************** > > *** 24,31 **** > > --- 24,33 ---- > > os.path.join('Packages/LinearAlgebra2/Src', > > 'zlapack_lite.c'), > > os.path.join('Packages/LinearAlgebra2/Src', 'dlapack_lite.c') > > ] > > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > > 'Include/numarray'] > > lapack_libs = [] > > lapack_dirs = [] > > + lapack_link_args = [] > > else: > > sourcelist = [ > > os.path.join('Packages/LinearAlgebra2/Src', > > 'lapack_litemodule.c'), > > *************** > > *** 53,58 **** > > --- 55,70 ---- > > > > # lapack_dirs = ['/home/jmiller/work/lib'] > > > > + lapack_link_args = [] > > + lapack_include_dirs = ["Packages/LinearAlgebra2/Src", > > 'Include/numarray'] > > + > > + VECLIB_PATH = '/System/Library/Frameworks/vecLib.framework' > > + if os.path.exists(VECLIB_PATH): > > + lapack_link_args = ['-framework', 'vecLib'] > > + lapack_include_dirs.append( os.path.join(VECLIB_PATH, > > 'Headers')) > > + lapack_libs = [] > > + lapack_dirs = [] > > + > > ADDONS_DATA_FILES = [('numarray/ma', ['Packages/MA/Legal.htm'])] > > > > ADDONS_PACKAGES = ["numarray.convolve", > > *************** > > *** 102,110 **** > > include_dirs = > > ['Packages/FFT2/Include','Include/numarray']), > > Extension('numarray.linear_algebra.lapack_lite2', > > sourcelist, > > ! include_dirs = ["Packages/LinearAlgebra2/Src", > > 'Include/numarray'], > > library_dirs = lapack_dirs, > > ! libraries = lapack_libs), > > Extension("numarray.random_array.ranlib2", > > ["Packages/RandomArray2/Src/ranlib.c", > > "Packages/RandomArray2/Src/ranlibmodule.c", > > --- 114,124 ---- > > include_dirs = > > ['Packages/FFT2/Include','Include/numarray']), > > Extension('numarray.linear_algebra.lapack_lite2', > > sourcelist, > > ! include_dirs = lapack_include_dirs, > > library_dirs = lapack_dirs, > > ! libraries = lapack_libs, > > ! extra_link_args = lapack_link_args, > > ! ), > > Extension("numarray.random_array.ranlib2", > > ["Packages/RandomArray2/Src/ranlib.c", > > "Packages/RandomArray2/Src/ranlibmodule.c", > > > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: InterSystems CACHE > > FREE OODBMS DOWNLOAD - A multidimensional database that combines > > robust object and relational technologies, making it a perfect match > > for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 > > _______________________________________________ > > Numpy-discussion mailing list > > Numpy-discussion at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- From ariciputi at pito.com Mon Nov 22 09:01:02 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Mon Nov 22 09:01:02 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <1101139686.18402.110.camel@halloween.stsci.edu> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> <1101139686.18402.110.camel@halloween.stsci.edu> Message-ID: I see your point but AFAIK the links option "-framework vecLib" will work *only* with Apple gcc (ld) and Apple gcc (ld) does know where frameworks are. From Apple ld man page: > -framework name[,suffix] > Specifies a framework to link against. Frameworks are dynamic > shared libraries, but they are stored in different locations, > and therefore must be searched for differently. When this option > is specified, ld searches for framework `name.framework/name' > first in any directories specified with the -F option, then in > the standard framework directories /Library/Frameworks, /Net- > work/Library/Frameworks, and /System/Library/Frameworks. The > placement of the -framework option is significant, as it deter- > mines when and how the framework is searched. If the optional > suffix is specified the framework is first searched for the name > with the suffix and then without. > -Fdir > In Apple's version of GCC only, add the directory dir to the head > of the list of directories to be searched for frameworks. > > The framework search algorithm is, for an inclusion of > , to look for files named path/Fmwk.framework/Head- > ers/Header.h or path/Fmwk.framework/PrivateHeaders/Header.h where > path includes /System/Library/Frameworks/ /Library/Frameworks/, and > /Local/Library/Frameworks/, plus any additional paths specified by > -F. HTH, Andrea. On 22 Nov 2004, at 17:08, Todd Miller wrote: > Not to look a patch-horse in the mouth, but IMHO, the sys.platform > idea > is good but the VECLIB_PATH elimination maybe less so. The reason I > question the VECLIB_PATH change is that we're removing explicit and > generally harmless extra information in exchange for two assumptions: > (1) gcc knows the path already (2) the numarray Mac user is using gcc. > Other opinions? Mac users please speak up if you think the > VECLIB_PATH > change is a good one or it won't get done. > > Thanks for the suggestions. I'll make the sys.platform change later > today. > > Regards, > Todd From rkern at ucsd.edu Mon Nov 22 09:16:01 2004 From: rkern at ucsd.edu (Robert Kern) Date: Mon Nov 22 09:16:01 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> <1101139686.18402.110.camel@halloween.stsci.edu> Message-ID: <41A21EB2.3040200@ucsd.edu> Andrea Riciputi wrote: > I see your point but AFAIK the links option "-framework vecLib" will > work *only* with Apple gcc (ld) and Apple gcc (ld) does know where > frameworks are. From Apple ld man page: IBM's xlc will also find the appropriate framework, but as you can see this is only a link-time option and has nothing to do with finding headers. Using #include will find the appropriate header without any command-line options. Using #include will not. Fortunately, neither one is used nor needed by lapack_litemodule.c, so for this case, no include-path needs to be specified. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From Chris.Barker at noaa.gov Mon Nov 22 16:15:03 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Nov 22 16:15:03 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <41A21EB2.3040200@ucsd.edu> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> <1101139686.18402.110.camel@halloween.stsci.edu> <41A21EB2.3040200@ucsd.edu> Message-ID: <41A28049.3050407@noaa.gov> Hi all, I forwarded this thread to Bob Ippolito, the original author of that patch. He's not on this list. I do suggest that any questions about OS-X be sent to the Pythonmac-SIG list, as there are some folks there that really know what they are doing. Anyway, here's his comments: Bob Ippolito wrote: Two things: 1) Apple can't (or rather, won't ever) change their framework location. 2) Darwin might be Darwin and not Mac OS X, which won't have vecLib. 3) Checking for both is silly, because you won't reasonably ever have the path to vecLib on a non-Darwin platform.. and if you somehow do, it'll probably work anyway. I can give more flexible code with regard to locating frameworks in an identical manner to what gcc and dyld do, but that is only going to be applicable if you work at Apple, or you are building against an SDK, neither of which happen very often in practice. With regard to building against a SDK, the API used by Numeric that is provided by vecLib isn't going to change, ever, unless Numeric wants more stuff that is provided only in OS X 10.4+, for example. This would be an issue when and if it happens, but in that case you would probably link against the NEW name (Accelerate.framework). I'm not sure if the vecLib API is supported by OS X 10.1, but someone else is going to have to care about that because I sure don't. Alternate linkers isn't really a concern because the three command-line compilers that matter probably all support the -framework flag. Apple's GCC (obviously..) and IBM's XLC (according to Robert) certainly do, and more likely than not, MetroWerks CodeWarrior also does (someone else would have to confirm this). If Fink or Gentoo ships some purely GNU-based compiler/linker that doesn't support -framework, I'd consider that a bug. I'm not sure how they'd get many things to compile correctly without it, unless they're emulating it by specifying paths directly to the Mach-O MH_DYLIB files (/System/Library/Frameworks/vecLib.framework/vecLib, for example) PLUS using non-framework-style header paths everywhere, since would never work if you -I to anywhere in the framework tree. Does an unpatched GNU toolchain even understand Mach-O in the first place? Most likely a non-issue in any case, but I'm not about to Finkify any machine to test that theory. In summary, I wrote the patch in that way for the specific reason of doing it as correctly as possible without overcomplicating things. I hope that people would trust that I know what I'm doing. -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From jmiller at stsci.edu Tue Nov 23 07:07:07 2004 From: jmiller at stsci.edu (Todd Miller) Date: Tue Nov 23 07:07:07 2004 Subject: [Numpy-discussion] numarray patch: use Apple's vecLib on Mac OS X In-Reply-To: <41A28049.3050407@noaa.gov> References: <08722232-397A-11D9-A9CE-000D93476BAC@astraw.com> <9F2A037E-3C7C-11D9-8E7A-000A95C0BC68@pito.com> <1101139686.18402.110.camel@halloween.stsci.edu> <41A21EB2.3040200@ucsd.edu> <41A28049.3050407@noaa.gov> Message-ID: <1101222419.3273.53.camel@jaytmiller.comcast.net> On Mon, 2004-11-22 at 16:11 -0800, Chris Barker wrote: > Hi all, > > I forwarded this thread to Bob Ippolito, the original author of that > patch. He's not on this list. I do suggest that any questions about OS-X > be sent to the Pythonmac-SIG list, I'll try... > as there are some folks there that > really know what they are doing. Anyway, here's his comments: > > Bob Ippolito wrote: > > Two things: > 1) Apple can't (or rather, won't ever) change their framework location. > 2) Darwin might be Darwin and not Mac OS X, which won't have vecLib. > 3) Checking for both is silly, because you won't reasonably ever have > the path to vecLib on a non-Darwin platform.. and if you somehow do, > it'll probably work anyway. > That's enough for me. I'm thinking we should just keep Andrew's port of Bob's patch, already committed to CVS, as is. If any one has any further comments about this please let me know. I realize I'm probably blowing around like a weather vane, but I'm no Mac expert, so I'm relying on your collective guidance. Thanks everyone for their input: Bob, Andrew, Andrea, Robert, and Chris. Regards, Todd [snip] From cmeesters at ucdavis.edu Tue Nov 23 12:00:01 2004 From: cmeesters at ucdavis.edu (Christian Meesters) Date: Tue Nov 23 12:00:01 2004 Subject: [Numpy-discussion] numarray: 3D Fourier transforms Message-ID: <24DBEC92-3D8A-11D9-8ED2-000393D8EC3A@ucdavis.edu> Hi I only hope that you guys on this list can help me, for I didn't find a list specified on numarray. Well, may problem is that I want to write an algorithm where I combine amplitude data from one 3D-image volume with phase data from another. (Actually they are identical except for one small correction, that's why this should be legal.) Numarray only offers 1D and 2D FFTs. I could possibly implement a 3D transform, but if somebody already has written something like this, it would be very nice if one could point me on an URL for this. Thanks a lot in advance. Cheers Christian Meesters From verveer at embl-heidelberg.de Tue Nov 23 12:17:04 2004 From: verveer at embl-heidelberg.de (Peter Verveer) Date: Tue Nov 23 12:17:04 2004 Subject: [Numpy-discussion] numarray: 3D Fourier transforms In-Reply-To: <24DBEC92-3D8A-11D9-8ED2-000393D8EC3A@ucdavis.edu> References: <24DBEC92-3D8A-11D9-8ED2-000393D8EC3A@ucdavis.edu> Message-ID: <87B49038-3D8C-11D9-9D98-000D932805AC@embl-heidelberg.de> Numarray has N-dimensional fft's, they are just not documented in the manual (this should be fixed, will do that as soon as find the time). There are the following functions in numarray.fft: fftnd() inverse_fftnd() real_fftnd() inverse_real_fftnd() They are documented in the source code, try the following: >>> from numarray import fft >>> help(fft.fftnd) Or alternatively, implement your 3D fft as a sequence of three 1D fft's. Cheers, Peter On Nov 23, 2004, at 8:59 PM, Christian Meesters wrote: > Hi > > I only hope that you guys on this list can help me, for I didn't find > a list specified on numarray. > > Well, may problem is that I want to write an algorithm where I combine > amplitude data from one 3D-image volume with phase data from another. > (Actually they are identical except for one small correction, that's > why this should be legal.) Numarray only offers 1D and 2D FFTs. I > could possibly implement a 3D transform, but if somebody already has > written something like this, it would be very nice if one could point > me on an URL for this. > > Thanks a lot in advance. > > Cheers > Christian Meesters > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real > users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From cmeesters at ucdavis.edu Tue Nov 23 12:29:11 2004 From: cmeesters at ucdavis.edu (Christian Meesters) Date: Tue Nov 23 12:29:11 2004 Subject: [Numpy-discussion] numarray: 3D Fourier transforms In-Reply-To: <87B49038-3D8C-11D9-9D98-000D932805AC@embl-heidelberg.de> References: <24DBEC92-3D8A-11D9-8ED2-000393D8EC3A@ucdavis.edu> <87B49038-3D8C-11D9-9D98-000D932805AC@embl-heidelberg.de> Message-ID: <4889DBC3-3D8E-11D9-8ED2-000393D8EC3A@ucdavis.edu> Hi Peter, Well, seems like I just overlooked that feature. I didn't check your 'claims' yet, but it sounds like my problem is solved. Thanks a lot. Cheers, Christian On Nov 23, 2004, at 12:16 PM, Peter Verveer wrote: > Numarray has N-dimensional fft's, they are just not documented in the > manual (this should be fixed, will do that as soon as find the time). > There are the following functions in numarray.fft: > > fftnd() > inverse_fftnd() > real_fftnd() > inverse_real_fftnd() > > They are documented in the source code, try the following: > > >>> from numarray import fft > >>> help(fft.fftnd) > > Or alternatively, implement your 3D fft as a sequence of three 1D > fft's. > > Cheers, Peter > > On Nov 23, 2004, at 8:59 PM, Christian Meesters wrote: > >> Hi >> >> I only hope that you guys on this list can help me, for I didn't find >> a list specified on numarray. >> >> Well, may problem is that I want to write an algorithm where I >> combine amplitude data from one 3D-image volume with phase data from >> another. (Actually they are identical except for one small >> correction, that's why this should be legal.) Numarray only offers 1D >> and 2D FFTs. I could possibly implement a 3D transform, but if >> somebody already has written something like this, it would be very >> nice if one could point me on an URL for this. >> >> Thanks a lot in advance. >> >> Cheers >> Christian Meesters >> >> >> >> ------------------------------------------------------- >> SF email is sponsored by - The IT Product Guide >> Read honest & candid reviews on hundreds of IT Products from real >> users. >> Discover which products truly live up to the hype. Start reading now. >> http://productguide.itmanagersjournal.com/ >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real > users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From stephen.walton at csun.edu Tue Nov 23 13:43:02 2004 From: stephen.walton at csun.edu (Stephen Walton) Date: Tue Nov 23 13:43:02 2004 Subject: [Numpy-discussion] Architecture management? Message-ID: <1101246125.15326.10.camel@freyer.sfo.csun.edu> How are all of you, if you are at all, managing to keep ATLAS/numarray/Numeric/Scipy/etc up to date on a heterogenous network? I have to do this, really, and would like to do it with ATLAS support, but with Athlon XP, Athlon MP, P4's and a couple of older P3's in house, it is becoming both difficult to automate this and time consuming to do it manually. Not strictly a numpy question, of course, but I felt it likely others on this list have faced this problem. -- Stephen Walton, Professor, Dept. of Physics & Astronomy, Cal State Northridge stephen.walton at csun.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From Fernando.Perez at colorado.edu Tue Nov 23 13:53:01 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Tue Nov 23 13:53:01 2004 Subject: [Numpy-discussion] Architecture management? In-Reply-To: <1101246125.15326.10.camel@freyer.sfo.csun.edu> References: <1101246125.15326.10.camel@freyer.sfo.csun.edu> Message-ID: <41A3B0F2.1090404@colorado.edu> Stephen Walton schrieb: > How are all of you, if you are at all, managing to keep > ATLAS/numarray/Numeric/Scipy/etc up to date on a heterogenous network? > I have to do this, really, and would like to do it with ATLAS support, > but with Athlon XP, Athlon MP, P4's and a couple of older P3's in house, > it is becoming both difficult to automate this and time consuming to do > it manually. > > Not strictly a numpy question, of course, but I felt it likely others on > this list have faced this problem. Well, if anyone has a good answer, please post. I don't have quite as many architectures to worry about, but it's still a pain and I don't have a really good answer. Judicious use of yum and an NFS-shared /usr/local helps to some extent, but it's not a real solution by a looong shot. Cheers, f From lkemohawk at yahoo.com Fri Nov 26 04:14:00 2004 From: lkemohawk at yahoo.com (kevin lester) Date: Fri Nov 26 04:14:00 2004 Subject: [Numpy-discussion] Equality Testing Message-ID: <20041126121038.16324.qmail@web53905.mail.yahoo.com> Hello, After spending a great deal of time researching how to do something as trivial as an equality test for arrrays of unequal lengths, I've given up to defer to your wisdom. a = array([19289 19291 19299 ..., 21966 21978 21990]) b = array([ 0 1 2 ..., 95956 95717 95768]) Evidently, where(equal... has no merit, as well as all the other expressions I've tried. Thank you for your help :) __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From aisaac at american.edu Fri Nov 26 05:41:39 2004 From: aisaac at american.edu (Alan G Isaac) Date: Fri Nov 26 05:41:39 2004 Subject: [Numpy-discussion] Equality Testing In-Reply-To: <20041126121038.16324.qmail@web53905.mail.yahoo.com> References: <20041126121038.16324.qmail@web53905.mail.yahoo.com> Message-ID: On Fri, 26 Nov 2004, kevin lester apparently wrote: > equality test for arrrays of unequal lengths ... > a = array([19289 19291 19299 ..., 21966 21978 21990]) > b = array([ 0 1 2 ..., 95956 95717 95768]) What does the result you want look like? If you just want the set of shared elements, try the set module. hth, Alan Isaac From lkemohawk at yahoo.com Fri Nov 26 18:20:08 2004 From: lkemohawk at yahoo.com (kevin lester) Date: Fri Nov 26 18:20:08 2004 Subject: [Numpy-discussion] (no subject) Message-ID: <20041127021902.93842.qmail@web53902.mail.yahoo.com> Hi Alan, Yes, thank you. Shared elements is the out come I need. The problem I have now is how to return the shared elements back into the numarray.array([]) constructor. Numarray doesn't recognize the sets.Set([]) constructor as a python numeric type. I actually don't see any functionality/methodology to convert a sets.Set([]) back into any of the sequence types. I am also don't understand why this ability, to retrieve shared elements of two unequal arrays, is not an inherent numarry function/method. Thank you for your initial direction to the sets module, Kevin __________________________________ Do you Yahoo!? The all-new My Yahoo! - Get yours free! http://my.yahoo.com From aisaac at american.edu Sat Nov 27 09:05:49 2004 From: aisaac at american.edu (Alan G Isaac) Date: Sat Nov 27 09:05:49 2004 Subject: [Numpy-discussion] (no subject) In-Reply-To: <20041127021902.93842.qmail@web53902.mail.yahoo.com> References: <20041127021902.93842.qmail@web53902.mail.yahoo.com> Message-ID: On Fri, 26 Nov 2004, kevin lester apparently wrote: > I actually don't > see any functionality/methodology to convert a > sets.Set([]) back into any of the sequence types. a=list(a) a=array(a) hth, Alan From pearu at scipy.org Mon Nov 29 13:34:11 2004 From: pearu at scipy.org (Pearu Peterson) Date: Mon Nov 29 13:34:11 2004 Subject: [Numpy-discussion] in-situ resizing array in C Message-ID: Hi, I have a question to Numeric/Numarray experts: Let A be a contiguous one or two-dimensional array with shape (n,) or (m,n), respectively. Let k be a positive integer less than n. Reference count for A is 1. The objective is to execute A=A[:k] or A=A[:,:k], respctively, in C. Q: Is it safe to resize the array A by simply resetting its dimensions, that is, A->dimensions[0] = k; or A->dimensions[1] = k; if A->nd is 1 or 2, respectively, in C? This appears to work but may be there are some corner cases that I have overlooked. Would that work also when using Numarray? If not, how to resize (preferably in-situ) an array in C under the conditions given above? TIA, Pearu From jmiller at stsci.edu Mon Nov 29 14:38:02 2004 From: jmiller at stsci.edu (Todd Miller) Date: Mon Nov 29 14:38:02 2004 Subject: [Numpy-discussion] Re: [SciPy-dev] in-situ resizing array in C In-Reply-To: References: Message-ID: <1101767815.24623.141.camel@halloween.stsci.edu> On Mon, 2004-11-29 at 16:33, Pearu Peterson wrote: > Hi, > > I have a question to Numeric/Numarray experts: > > Let A be a contiguous one or two-dimensional array with shape (n,) or > (m,n), respectively. Let k be a positive integer less than n. > Reference count for A is 1. The objective is to execute A=A[:k] > or A=A[:,:k], respctively, in C. > > Q: Is it safe to resize the array A by simply resetting its dimensions, > that is, > A->dimensions[0] = k; > or > A->dimensions[1] = k; Sounds good to me for numarray. Numeric's "dimensions" points to a C array not necessarily owned by the object... I don't know if that's a problem or not. The dimensions array storage is built into the numarray object so it's not a problem there. > if A->nd is 1 or 2, respectively, in C? This appears to work but may be > there are some corner cases that I have overlooked. Since you said 0 < k < n I think any corners are covered. It's probably good to handle m or n being 0. > Would that work also when using Numarray? This should work unaltered in numarray. Here are a few general comments: 1. Since you're modifying in-situ, the changes are permanent unless reversed later. This is in contrast to a slice (view) which has a temporary copy of the array shape (not data) and leaves the original array unaltered. 2. Since you're not changing the size of the data, I'd call this "reshaping" and not "resizing". 3. Ref count of A is 1 eliminates any (non-data) sharing issues for numarray. Regards, Todd