From bgranger at scu.edu Sun Oct 2 11:03:07 2005 From: bgranger at scu.edu (Brian Granger) Date: Sun Oct 2 11:03:07 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Release of SciPy Core 0.4 (Beta) Message-ID: Which version of gccc are you using? As I understand it, g77 3.4 is not compatible with gcc 4.0 (the default on OS X). Switching to gcc 3.4 might help: gcc_select 3.3 I will soon try scipy_core on Tiger with Python 2.4.1. Brian Brian Granger, Ph.D. Assistant Professor of Physics Santa Clara University bgranger at scu.edu Phone: 408-551-1891 Fax: 408-554-6965 >>> managan at llnl.gov 09/30/05 11:21 AM >>> OK, I am giving this a go. I seem to be not setting a flag about some math functions availability. Here are the details (aorry about the length of it but I snipped a lot!): OSX 10.3.9, Python 2.4.1 Installed g77v3.4-bin.tar.gz from http://hpc.sourceforge.net/ as described into /usr/local/... Installed fftw-3.0.1-fma.tar.gz from http://www.fftw.org/download.html into ~/Documents/local/... where I have other libraries I have installed. Got scipy_core-0.4.1.tar.gz from http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=6222, wanted to try out the new replacement for Numeric tar -zxvf scipy_core-0.4.1.tar.gz cd scipy_core-0.4.1 python setup.py build Assuming default configuration (scipy/distutils/command/{setup_command,setup}.py was not found) Appending scipy.distutils.command configuration to scipy.distutils Assuming default configuration (scipy/distutils/fcompiler/{setup_fcompiler,setup}.py was not found) ... Appending scipy.base configuration to scipy blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] Appending scipy.lib configuration to scipy Assuming default configuration (scipy/fftpack/{setup_fftpack,setup}.py was not found) ... scipy_core version 0.4.1 running build running config_fc running build_src building extension "scipy.base.multiarray" sources creating build creating build/src creating build/src/scipy creating build/src/scipy/base Generating build/src/scipy/base/config.h customize NAGFCompiler customize AbsoftFCompiler customize IbmFCompiler Could not locate executable f77 customize GnuFCompiler customize GnuFCompiler ... compiling C sources gcc options: '-fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes' creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base/src compile options: '-Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c' gcc: build/src/scipy/base/src/umathmodule.c In file included from build/src/scipy/base/src/umathmodule.c:8: scipy/base/include/scipy/arrayobject.h:84: warning: redefinition of `ushort' /usr/include/sys/types.h:82: warning: `ushort' previously declared here scipy/base/include/scipy/arrayobject.h:85: warning: redefinition of `uint' /usr/include/sys/types.h:83: warning: `uint' previously declared here build/src/scipy/base/src/umathmodule.c: In function `nc_floor_quotl': build/src/scipy/base/src/umathmodule.c:1257: warning: implicit declaration of function `floorl' build/src/scipy/base/src/umathmodule.c: In function `nc_sqrtl': build/src/scipy/base/src/umathmodule.c:1269: warning: implicit declaration of function `sqrtl' ... build/src/scipy/base/__umath_generated.c:171: error: initializer element is not constant build/src/scipy/base/__umath_generated.c:171: error: (near initialization for `arccosh_data[2]') error: Command "gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dy namic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/s rc/scipy/base -Iscipy/base/src -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c b uild/src/scipy/base/src/umathmodule.c -o build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base /src/umathmodule.o" failed with exit status 1 At 12:59 AM -0600 9/30/05, Travis Oliphant wrote: >This is to announce the release of SciPy Core 0.4.X (beta) > >It is available at sourceforge which you can access by going to > >http://numeric.scipy.org > >Thanks to all who helped me with this release, especially > >Robert Kern >Pearu Peterson > >Now, let's start getting this stable.... > >-Travis Oliphant > > > >_______________________________________________ >SciPy-user mailing list >SciPy-user at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-user -- *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- Rob Managan email managan at llnl.gov LLNL phone: 925-423-0903 P.O. Box 808, L-095 FAX: 925-422-3389 Livermore, CA 94551-0808 _______________________________________________ SciPy-user mailing list SciPy-user at scipy.net http://www.scipy.net/mailman/listinfo/scipy-user This message scanned for viruses and SPAM by GWGuardian at SCU (MGW1) From gerard.vermeulen at grenoble.cnrs.fr Sun Oct 2 12:41:08 2005 From: gerard.vermeulen at grenoble.cnrs.fr (Gerard Vermeulen) Date: Sun Oct 2 12:41:08 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <433CE24A.6040509@ee.byu.edu> References: <433CE24A.6040509@ee.byu.edu> Message-ID: <20051002213837.70eec02c.gerard.vermeulen@grenoble.cnrs.fr> On Fri, 30 Sep 2005 00:59:22 -0600 Travis Oliphant wrote: > > This is to announce the release of SciPy Core 0.4.X (beta) > > It is available at sourceforge which you can access by going to > > http://numeric.scipy.org > > Thanks to all who helped me with this release, especially > > Robert Kern > Pearu Peterson > > Now, let's start getting this stable.... > > -Travis Oliphant > I have found two problems: (1) scipy_core-0.4.1 does not compile on this system without ATLAS, because there are missing files: ... creating build/temp.linux-i686-2.4/lapack_lite compile options: '-Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c' gcc: scipy/corelib/lapack_lite/lapack_litemodule.c gcc: lapack_lite/dlapack_lite.c gcc: lapack_lite/dlapack_lite.c: No such file or directory gcc: no input files gcc: lapack_lite/dlapack_lite.c: No such file or directory gcc: no input files error: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -fomit-frame-pointer -pipe -march=i586 -mtune=pentiumpro-g -fPIC -Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c lapack_lite/dlapack_lite.c -o build/temp.linux-i686-2.4/lapack_lite/dlapack_lite.o" failed with exit status 1 [packer at titan scipy_core-0.4.1]$ (2) when building scipy_core from SVN the header files do not get installed. The install section from my RPM SPEC file reads: %install rm -rf %{buildroot} python setup.py install --root=%{buildroot} # The API headers do not get installed mkdir -p %{buildroot}/%{_includedir}/python%{pyver}/scipy/ for header in scipy/base/include/scipy/*object.h ; do install -m 644 $header %{buildroot}/%{_includedir}/python%{pyver}/scipy/ done # The generated API headers and config.h do not get installed either for header in build/src/scipy/base/*.h ; do install -m 644 $header %{buildroot}/%{_includedir}/python%{pyver}/scipy/ done # Mandrake 10.2 needs this: %multiarch_includes %{buildroot}/%{_includedir}/python%{pyver}/scipy/config.h %clean #rm -rf %{buildroot} Regards -- Gerard From managan at llnl.gov Sun Oct 2 13:46:48 2005 From: managan at llnl.gov (Rob Managan) Date: Sun Oct 2 13:46:48 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Release of SciPy Core 0.4 (Beta) In-Reply-To: <433CE24A.6040509@ee.byu.edu> References: <433CE24A.6040509@ee.byu.edu> Message-ID: OK, I am giving this a go. I seem to be not setting a flag about some math functions availability. Here are the details (aorry about the length of it but I snipped a lot!): OSX 10.3.9, Python 2.4.1 Installed g77v3.4-bin.tar.gz from http://hpc.sourceforge.net/ as described into /usr/local/... Installed fftw-3.0.1-fma.tar.gz from http://www.fftw.org/download.html into ~/Documents/local/... where I have other libraries I have installed. Got scipy_core-0.4.1.tar.gz from http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=6222, wanted to try out the new replacement for Numeric tar -zxvf scipy_core-0.4.1.tar.gz cd scipy_core-0.4.1 python setup.py build Assuming default configuration (scipy/distutils/command/{setup_command,setup}.py was not found) Appending scipy.distutils.command configuration to scipy.distutils Assuming default configuration (scipy/distutils/fcompiler/{setup_fcompiler,setup}.py was not found) ... Appending scipy.base configuration to scipy blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] Appending scipy.lib configuration to scipy Assuming default configuration (scipy/fftpack/{setup_fftpack,setup}.py was not found) ... scipy_core version 0.4.1 running build running config_fc running build_src building extension "scipy.base.multiarray" sources creating build creating build/src creating build/src/scipy creating build/src/scipy/base Generating build/src/scipy/base/config.h customize NAGFCompiler customize AbsoftFCompiler customize IbmFCompiler Could not locate executable f77 customize GnuFCompiler customize GnuFCompiler ... compiling C sources gcc options: '-fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes' creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base/src compile options: '-Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c' gcc: build/src/scipy/base/src/umathmodule.c In file included from build/src/scipy/base/src/umathmodule.c:8: scipy/base/include/scipy/arrayobject.h:84: warning: redefinition of `ushort' /usr/include/sys/types.h:82: warning: `ushort' previously declared here scipy/base/include/scipy/arrayobject.h:85: warning: redefinition of `uint' /usr/include/sys/types.h:83: warning: `uint' previously declared here build/src/scipy/base/src/umathmodule.c: In function `nc_floor_quotl': build/src/scipy/base/src/umathmodule.c:1257: warning: implicit declaration of function `floorl' build/src/scipy/base/src/umathmodule.c: In function `nc_sqrtl': build/src/scipy/base/src/umathmodule.c:1269: warning: implicit declaration of function `sqrtl' ... build/src/scipy/base/__umath_generated.c:171: error: initializer element is not constant build/src/scipy/base/__umath_generated.c:171: error: (near initialization for `arccosh_data[2]') error: Command "gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dy namic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/s rc/scipy/base -Iscipy/base/src -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c b uild/src/scipy/base/src/umathmodule.c -o build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base /src/umathmodule.o" failed with exit status 1 At 12:59 AM -0600 9/30/05, Travis Oliphant wrote: >This is to announce the release of SciPy Core 0.4.X (beta) > >It is available at sourceforge which you can access by going to > >http://numeric.scipy.org > >Thanks to all who helped me with this release, especially > >Robert Kern >Pearu Peterson > >Now, let's start getting this stable.... > >-Travis Oliphant > > > >_______________________________________________ >SciPy-user mailing list >SciPy-user at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-user -- *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- Rob Managan email managan at llnl.gov LLNL phone: 925-423-0903 P.O. Box 808, L-095 FAX: 925-422-3389 Livermore, CA 94551-0808 From efiring at hawaii.edu Sun Oct 2 18:17:42 2005 From: efiring at hawaii.edu (Eric Firing) Date: Sun Oct 2 18:17:42 2005 Subject: [Numpy-discussion] Re: Release of SciPy Core 0.4 (Beta) In-Reply-To: <20051001031446.28F4210589F@sc8-sf-spam1.sourceforge.net> References: <20051001031446.28F4210589F@sc8-sf-spam1.sourceforge.net> Message-ID: <433F279D.8070109@hawaii.edu> Travis, > This is to announce the release of SciPy Core 0.4.X (beta) Wonderful progress--I can't thank you enough for the work you are doing on this. I initially tried to build from the tarball, but ran into missing pieces: compile options: '-Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c' gcc: lapack_lite/blas_lite.c gcc: lapack_lite/blas_lite.c: No such file or directory gcc: no input files I then installed from the rpm (on a Mandriva 2005 LE system) with no problems, and proceeded to poke around. It looks great! > Now, let's start getting this stable.... Bug: in function_base.py, there are four places where "nx" should be "_nx"; see attached diff. Also attached is a diff for arraymethods, to fix what look to me like erroneous docstrings for the reshape and resize methods. The present docstrings indicate that reshape makes the change in place, and resize returns a new array, but in both cases the opposite is true--reshape returns a reshaped copy, and resize returns None. Eric -------------- next part -------------- A non-text attachment was scrubbed... Name: function_base.diff Type: text/x-patch Size: 991 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: arraymethods.diff Type: text/x-patch Size: 1370 bytes Desc: not available URL: From tchur at optusnet.com.au Sun Oct 2 21:52:41 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Sun Oct 2 21:52:41 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <433CE24A.6040509@ee.byu.edu> References: <433CE24A.6040509@ee.byu.edu> Message-ID: <4340B872.8020102@optusnet.com.au> Travis Oliphant wrote: > > This is to announce the release of SciPy Core 0.4.X (beta) > > It is available at sourceforge which you can access by going to > > http://numeric.scipy.org > > Thanks to all who helped me with this release, especially > > Robert Kern > Pearu Peterson > > Now, let's start getting this stable.... Great work, and we are much relieved that NumPy has a firm future (albeit under a different name). However, can I ask if there are plans to add Masked Arrays to SciPy Core? If not, we'll have to stick with NumPy - missing values are ubiquitous in the biological sciences and any package which can't handle them isn't, to put it bluntly, of much use. We are happy to help test MA for SciPy Core, and our project (www.netepi.org) has over a thousand unit tests which give the basics of NumPy and MA a good work-out - so if/when we convert to SciPy Core the unit tests will be helpful. However we are not familiar enough with how SciPy Core differs from NumPy to help with the porting, I'm afraid. Tim C From tchur at optusnet.com.au Sun Oct 2 22:53:11 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Sun Oct 2 22:53:11 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340BE90.6080302@pfdubois.com> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> Message-ID: <4340C6FA.1050307@optusnet.com.au> Paul F. Dubois wrote: > I was going to see if I could port MA. It is pure Python so shouldn't be > too hard. That would eb a most welcome development. If you are in the mood for some minor extensions to MA (you probably aren't, but just on the off-chance...), then support for multiple masking values would be great. In other words, instead of the mask just taking the values zero or one, it can also take 2, 3, etc (within the constraints of the Int8 data type), where any non-zero value means masked. That would allow the reason why a value in an array is masked to be neatly encoded, not just the fact that it is masked (missing). For example, in (human) survey data, a scalar value (eg age at menopasue) may be missing because the respondent did not meet the precondition(s) for the question (eg sex not female), or because they have not yet undergone menopause, or because they refuse to answer, or because they don't know or don't recall. That's at least 4 different reasons why the value may be missing (masked). But don't ask me the the mask value should be when you multiply a masked-because-not-asked value by a masked-because-refused-to-answer value. Perhaps a reasonable rule would be max(mask0,mask1)? Tim C > > Tim Churches wrote: > >> Travis Oliphant wrote: >> >>> This is to announce the release of SciPy Core 0.4.X (beta) >>> >>> It is available at sourceforge which you can access by going to >>> >>> http://numeric.scipy.org >>> >>> Thanks to all who helped me with this release, especially >>> >>> Robert Kern >>> Pearu Peterson >>> >>> Now, let's start getting this stable.... >> >> >> >> Great work, and we are much relieved that NumPy has a firm future >> (albeit under a different name). >> >> However, can I ask if there are plans to add Masked Arrays to SciPy >> Core? If not, we'll have to stick with NumPy - missing values are >> ubiquitous in the biological sciences and any package which can't handle >> them isn't, to put it bluntly, of much use. We are happy to help test MA >> for SciPy Core, and our project (www.netepi.org) has over a thousand >> unit tests which give the basics of NumPy and MA a good work-out - so >> if/when we convert to SciPy Core the unit tests will be helpful. However >> we are not familiar enough with how SciPy Core differs from NumPy to >> help with the porting, I'm afraid. >> >> Tim C >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > From paul at pfdubois.com Sun Oct 2 23:23:34 2005 From: paul at pfdubois.com (Paul F. Dubois) Date: Sun Oct 2 23:23:34 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340B872.8020102@optusnet.com.au> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> Message-ID: <4340BE90.6080302@pfdubois.com> I was going to see if I could port MA. It is pure Python so shouldn't be too hard. Tim Churches wrote: > Travis Oliphant wrote: > >>This is to announce the release of SciPy Core 0.4.X (beta) >> >>It is available at sourceforge which you can access by going to >> >>http://numeric.scipy.org >> >>Thanks to all who helped me with this release, especially >> >>Robert Kern >>Pearu Peterson >> >>Now, let's start getting this stable.... > > > Great work, and we are much relieved that NumPy has a firm future > (albeit under a different name). > > However, can I ask if there are plans to add Masked Arrays to SciPy > Core? If not, we'll have to stick with NumPy - missing values are > ubiquitous in the biological sciences and any package which can't handle > them isn't, to put it bluntly, of much use. We are happy to help test MA > for SciPy Core, and our project (www.netepi.org) has over a thousand > unit tests which give the basics of NumPy and MA a good work-out - so > if/when we convert to SciPy Core the unit tests will be helpful. However > we are not familiar enough with how SciPy Core differs from NumPy to > help with the porting, I'm afraid. > > Tim C > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From stephen.walton at csun.edu Mon Oct 3 08:25:09 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Mon Oct 3 08:25:09 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340C6FA.1050307@optusnet.com.au> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> <4340C6FA.1050307@optusnet.com.au> Message-ID: <43414CD7.8020507@csun.edu> Tim Churches wrote: >That would eb a most welcome development. If you are in the mood for >some minor extensions to MA (you probably aren't, but just on the >off-chance...), then support for multiple masking values would be great. > > This sounds like an overly complicated addition to MA. If I may be so bold, it seems to me that what you might want is your own object which maintains parallel arrays of values and "mask reasons," if you will, and generates a mask array accordingly From Chris.Barker at noaa.gov Mon Oct 3 09:55:40 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 3 09:55:40 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340BE90.6080302@pfdubois.com> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> Message-ID: <434161F0.80408@noaa.gov> > Tim Churches wrote: >> missing values are >> ubiquitous in the biological sciences and any package which can't handle >> them isn't, to put it bluntly, of much use. MA is great, but I wonder if many of the simple "missing value" use cases could be covered by robust handling of NaNs. Which brings up the question: How does scipy_core handle NaN and the other IEEE special values? This was a major weakness in Numeric for me. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From efiring at hawaii.edu Mon Oct 3 12:49:57 2005 From: efiring at hawaii.edu (Eric Firing) Date: Mon Oct 3 12:49:57 2005 Subject: [Numpy-discussion] ma is in scipy_core Message-ID: <43418AA8.2040809@hawaii.edu> Paul, Tim, Masked arrays *are* in scipy_core. import scipy.base.ma as ma In addition, a quick look indicates that NaN-handling is in good shape. Eric From stephen.walton at csun.edu Mon Oct 3 13:05:28 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Mon Oct 3 13:05:28 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340B872.8020102@optusnet.com.au> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> Message-ID: <43418E5A.5040105@csun.edu> Tim Churches wrote: >However, can I ask if there are plans to add Masked Arrays to SciPy >Core? > Eric Firing pointed out on the matplotlib mailing list that they're already there: import scipy.base.ma as ma From Chris.Barker at noaa.gov Mon Oct 3 13:49:50 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 3 13:49:50 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <43417A87.9000103@ee.byu.edu> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> <434161F0.80408@noaa.gov> <43417A87.9000103@ee.byu.edu> Message-ID: <43419904.7040509@noaa.gov> Travis Oliphant wrote: > Chris Barker wrote: >> Which brings up the question: How does scipy_core handle NaN and the >> other IEEE special values? This was a major weakness in Numeric for me. > > They can exist in your arrays and isnan, isinf, isposinf, isneginf > detects them. Wonderful! I'm really looking forward to this being stable. thanks, Travis! -CHB -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From oliphant at ee.byu.edu Mon Oct 3 14:05:01 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon Oct 3 14:05:01 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <43418E5A.5040105@csun.edu> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <43418E5A.5040105@csun.edu> Message-ID: <43419C54.6030003@ee.byu.edu> Stephen Walton wrote: > Tim Churches wrote: > >> However, can I ask if there are plans to add Masked Arrays to SciPy >> Core? >> > Eric Firing pointed out on the matplotlib mailing list that they're > already there: > > import scipy.base.ma as ma from scipy import ma also works (everything under scipy.base is also under scipy alone). -Travis From tchur at optusnet.com.au Mon Oct 3 14:09:14 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 14:09:14 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <43414CD7.8020507@csun.edu> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> <4340C6FA.1050307@optusnet.com.au> <43414CD7.8020507@csun.edu> Message-ID: <43419D88.3020006@optusnet.com.au> Stephen Walton wrote: > Tim Churches wrote: > >> That would eb a most welcome development. If you are in the mood for >> some minor extensions to MA (you probably aren't, but just on the >> off-chance...), then support for multiple masking values would be great. >> >> > This sounds like an overly complicated addition to MA. If I may be so > bold, it seems to me that what you might want is your own object which > maintains parallel arrays of values and "mask reasons," if you will, and > generates a mask array accordingly Yes, but the wish is that NumPy, or rather, SciPy Core, treats any mask value other than zero the same - that is, that the mask value doesn't have to be one. Teh machinery to attach meaning to mask values is indeed situation-specific and shouldn't be part of SciPy Core (what is the handy abbreviation of SciPy Core, BTW?). Tim C From tchur at optusnet.com.au Mon Oct 3 14:18:29 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 14:18:29 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <434161F0.80408@noaa.gov> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> <434161F0.80408@noaa.gov> Message-ID: <43419FE3.2070109@optusnet.com.au> Chris Barker wrote: >> Tim Churches wrote: >> >>> missing values are >>> ubiquitous in the biological sciences and any package which can't handle >>> them isn't, to put it bluntly, of much use. > > > MA is great, but I wonder if many of the simple "missing value" use > cases could be covered by robust handling of NaNs. Most. > Which brings up the question: How does scipy_core handle NaN and the > other IEEE special values? This was a major weakness in Numeric for me. MA is indeed very flexible, well-designed and easy to use, but its weakness is that it is slow - necessarily so due to its "add-on" design - every operation is at least twice as slow as the equivalent oepration on a Numeric array. The mask arrays also eat some additional memory, which is sometimes an issue (untill everyone moves to 64 bit systems). Tim C From tchur at optusnet.com.au Mon Oct 3 14:36:36 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 14:36:36 2005 Subject: [Numpy-discussion] ma is in scipy_core In-Reply-To: <43418AA8.2040809@hawaii.edu> References: <43418AA8.2040809@hawaii.edu> Message-ID: <4341A410.5010504@optusnet.com.au> Eric Firing wrote: > Paul, Tim, > > Masked arrays *are* in scipy_core. > > import scipy.base.ma as ma OK, thanks. In the absence of documentation, I just looked for an MA subdirectory, couldn't find one and assumed that it wasn't (yet) supported. > In addition, a quick look indicates that NaN-handling is in good shape. How about other IEEE special values? Tim C From efiring at hawaii.edu Mon Oct 3 15:50:11 2005 From: efiring at hawaii.edu (Eric Firing) Date: Mon Oct 3 15:50:11 2005 Subject: [Numpy-discussion] ma is in scipy_core In-Reply-To: <4341A410.5010504@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> Message-ID: <4341B549.3000107@hawaii.edu> Tim Churches wrote: > Eric Firing wrote: > >>Paul, Tim, >> >>Masked arrays *are* in scipy_core. >> >>import scipy.base.ma as ma > > > OK, thanks. In the absence of documentation, I just looked for an MA > subdirectory, couldn't find one and assumed that it wasn't (yet) supported. > > >>In addition, a quick look indicates that NaN-handling is in good shape. > > > How about other IEEE special values? > > Tim C Tim, It has inf: >>> import scipy.base as nx >>> nx.inf inf >>> nx.isposinf(nx.inf) True >>> nx.isposinf(-nx.inf) False >>> nx.isneginf(-nx.inf) True >>> I don't know about signed floating point zero; I have never used such a beast, and am aware of it only because it is in numarray's ieeespecial module. See http://numeric.scipy.org/new_features.html; it includes: Errors are handled through the IEEE floating point status flags and there is flexibility on a per function / module / builtin level for handling these errors. I haven't looked into how this is done. Eric From efiring at hawaii.edu Mon Oct 3 15:56:19 2005 From: efiring at hawaii.edu (Eric Firing) Date: Mon Oct 3 15:56:19 2005 Subject: [Numpy-discussion] ma is in scipy_core In-Reply-To: <4341A410.5010504@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> Message-ID: <4341B659.7070503@hawaii.edu> > OK, thanks. In the absence of documentation, I just looked for an MA > subdirectory, couldn't find one and assumed that it wasn't (yet) supported. Tim, Documentation is coming along, but being made available in an unusual way: http://www.tramy.us/ Eric From tchur at optusnet.com.au Mon Oct 3 16:43:52 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 16:43:52 2005 Subject: [Numpy-discussion] ma is in scipy_core In-Reply-To: <4341B659.7070503@hawaii.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> Message-ID: <4341C1C8.3000506@optusnet.com.au> Eric Firing wrote: > >> OK, thanks. In the absence of documentation, I just looked for an MA >> subdirectory, couldn't find one and assumed that it wasn't (yet) >> supported. > > > Tim, > > Documentation is coming along, but being made available in an unusual > way: http://www.tramy.us/ Yes. I don't mind paying about AUD$50 for a copy of the SciPy Core documentation in order to expedite its production, but the Total Price and Total Time parameters as they currently stand are ridiculous: $300k represents about 7 or 8 thousand copies - and although there may be that many users of NumPy worldwide, realistically only a proportion of them will migrate to SciPy Core and of those, only a proportion will buy a copy of the documentation. Most likely, one copy of the documentation will be purchased to be shared between several (or many) users in a workgroup within an institution. Thus, the $300k mark is unlikely to be reached and it will be 7 years before the documentation is freely available. Given that the lifecycle of projects like NumPy or SciPy Core is of that same order of magnitude, it effectively means that the documentation for SciPy Core will not be free for most or all of its lifespan. That is a pretty severe limitation and one which end users should carefully consider before converting to SciPy Core. I would say that perhaps $30k or one year (after completion of the documentation) would be more reasonable criteria for making the documentation freely available (but then I am not writing it). Tim C From oliphant at ee.byu.edu Mon Oct 3 17:27:02 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon Oct 3 17:27:02 2005 Subject: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4341C1C8.3000506@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> Message-ID: <4341CB9E.4050306@ee.byu.edu> Tim Churches wrote: >Eric Firing wrote: > > >>>OK, thanks. In the absence of documentation, I just looked for an MA >>>subdirectory, couldn't find one and assumed that it wasn't (yet) >>>supported. >>> >>> >>Tim, >> >>Documentation is coming along, but being made available in an unusual >>way: http://www.tramy.us/ >> >> > >copy of the documentation. Most likely, one copy of the documentation >will be purchased to be shared between several (or many) users in a >workgroup within an institution. > Note that it is expressly against the agreement for one copy to be shared between multiple users at the same institution. I hope this is clear.... Of course you can let somebody else look at it a couple of times, but if they will use it regularly, they need to get their own copy. Prices are always a matter of supply and demand. The whole point of the system is to allow the price system to help coordinate what people think is valuable and what developers spend time doing. What you see currently is the maximum price (and time) I could possibly set as per the agreement with Trelgol. These things can always come down, however, as time proceeds, and the market responds. Now, obviously the cost of the documentation includes something of the cost of producing the actual code. Of course, you may disagree, but I did choose the numbers based on a little bit of market research. I don't think that 7000 copies of the documentation or 7 years is all that ridiculous given that there have been over 12000 downloads of the Numeric 24.0b2 source code since April and Numeric has been in stable use for nearly 10 years. If scipy does it's job correctly, then a user-base needing documentation of 7000 is rather low-balling it I would say. I want scipy to surpass the number of users of Numeric. I'm trying to make scipy core so that everybody can convert to it, eventually. The old Numeric manual still provides documentation, and the source is still available. I think you are still getting a great deal. Unless there is another majore re-write, the documentation will be updated as it goes (and you get the updates). >I would say that perhaps $30k or one year (after completion of the >documentation) would be more reasonable criteria for making the >documentation freely available (but then I am not writing it). > > Well, given the time I had to spend on this, that is quite a bit less than the market will bear for my services elsewhere. I suppose if I were rich, I could donate more of my time. But, I'm not.... I'm really not trying to make people upset. I'm really a huge fan of open information, and would love to see the documentation completely free. It's just that I cannot afford to create it for nothing. I have lots of demands on my time. Spending it doing scientific python has to be justified, somehow. I did not start the creation of a hybrid Numeric / numarray with the hope of making any money. I started it because there was a need. I thought I would get more help with its implementation. But, given the reality of people's scarce time (they need to make money...), nobody was able to help me. Out of this, the documentation idea was born to try and help fund development of scipy core. I hope people can understand that the reality of scarcity dictates that we coordinate efforts through some mechanism. The price mechanism has been the most succesful large-scale mechanism yet developed. I am interested in feedback. If you don't buy the book because you think I'm asking too much money, then let me know, as Tim has done. You can email me directly, as well. Best regards, -Travis From tchur at optusnet.com.au Mon Oct 3 17:59:25 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 17:59:25 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <4341CB9E.4050306@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> Message-ID: <4341D32A.5000000@optusnet.com.au> Travis Oliphant wrote: > Tim Churches wrote: > >> Eric Firing wrote: >> >> >>>> OK, thanks. In the absence of documentation, I just looked for an MA >>>> subdirectory, couldn't find one and assumed that it wasn't (yet) >>>> supported. >>>> >>> >>> Tim, >>> >>> Documentation is coming along, but being made available in an unusual >>> way: http://www.tramy.us/ >>> >> >> >> copy of the documentation. Most likely, one copy of the documentation >> will be purchased to be shared between several (or many) users in a >> workgroup within an institution. > > > Note that it is expressly against the agreement for one copy to be > shared between multiple users at the same institution. I hope this is > clear.... Of course you can let somebody else look at it a couple of > times, but if they will use it regularly, they need to get their own copy. > Prices are always a matter of supply and demand. The whole point of > the system is to allow the price system to help coordinate what people > think is valuable and what developers spend time doing. What you see > currently is the maximum price (and time) I could possibly set as per > the agreement with Trelgol. These things can always come down, > however, as time proceeds, and the market responds. The only agreement I can find is these words on the above site: "When you receive a copy of content under an MBTDR, you implicitly agree to make only back-up copies of the content and to not make a copy for others during the restriction period. If additional copies of the content are needed (for multiple users at a company, for example) you must purchase them. You may make exactly one "printed" or hard-copy for each copy of the content you purchase. For books, Trelgol may make available volume printing options at cost for those who have purchased a copy of the electronic content." Travis, are you saying that this agreement only allows a single person to read the single printed copy? If so, I think you need a formally worded legal license to make that stick - certainly Australian copyright law (nor copyright law in other countries, I suspect) does not provide any support for such severe restrictions in the use of a printed document. Under copyright law, you may not make unauthorised copies of a printed document, but you can certainly lend that copy to others, or sell or give it to them, or share it as a bench manual. > Now, obviously the cost of the documentation includes something of the > cost of producing the actual code. Of course, you may disagree, but I > did choose the numbers based on a little bit of market research. I > don't think that 7000 copies of the documentation or 7 years is all that > ridiculous given that there have been over 12000 downloads of the > Numeric 24.0b2 source code since April and Numeric has been in stable > use for nearly 10 years. Many of those downloads are for "t[y|i]re-kicking" - potential users determining whether it meets their needs. If those potential users have to pay for documentation up-front, then a large proportion will go elsewhere. The rest are probably existing NumPy users upgrading (or testing the upgrade waters). The latter group may well pay for documentation, but how large is that group? For example, how many people are subscribed to the numpy-discussion mailing list? > If scipy does it's job correctly, then a user-base needing > documentation of 7000 is rather low-balling it I would say. I want > scipy to surpass the number of users of Numeric. I'm trying to make > scipy core so that everybody can convert to it, eventually. The old > Numeric manual still provides documentation, and the source is still > available. I think you are still getting a great deal. Unless there > is another majore re-write, the documentation will be updated as it goes > (and you get the updates). Yes, maybe all that is needed is a (free) SciPy Core update of the existing (free) NumPy documentation. The (non-free) SciPy Core book which Travis is writing can complement that, just as the dozens of commercial Python books complement the free Python documentation. But imagine if everyone had to pay $50 to access the basic Python documentation - the number of Python users worldwide would be very much smaller than it is is now, I dare say. >> I would say that perhaps $30k or one year (after completion of the >> documentation) would be more reasonable criteria for making the >> documentation freely available (but then I am not writing it). >> >> > Well, given the time I had to spend on this, that is quite a bit less > than the market will bear for my services elsewhere. I suppose if I > were rich, I could donate more of my time. But, I'm not.... Travis, although many of us are grateful for your efforts on SciPy Core, no-one made you do it. If you wanted to earn (more) money by doing other things, you should have done them. > I'm really not trying to make people upset. I'm really a huge fan of > open information, and would love to see the documentation completely > free. It's just that I cannot afford to create it for nothing. I have > lots of demands on my time. Spending it doing scientific python has to > be justified, somehow. I did not start the creation of a hybrid > Numeric / numarray with the hope of making any money. I started it > because there was a need. I thought I would get more help with its > implementation. But, given the reality of people's scarce time (they > need to make money...), nobody was able to help me. Out of this, the > documentation idea was born to try and help fund development of scipy core. > I hope people can understand that the reality of scarcity dictates that > we coordinate efforts through some mechanism. The price mechanism has > been the most succesful large-scale mechanism yet developed. > > I am interested in feedback. If you don't buy the book because you > think I'm asking too much money, then let me know, as Tim has done. You > can email me directly, as well. I think there needs to be some community debate about this. Is there sufficient interest for people other than Travis to start with the Numeric documentation and update it as necessary to become a free SciPy Core documentation? The NumPy documentation is available in HTML format as the basis of this - perhaps the original source (LaTeX?) for the Numeric docs is also available? Tim C From oliphant at ee.byu.edu Mon Oct 3 20:55:20 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon Oct 3 20:55:20 2005 Subject: [Numpy-discussion] Contributions to development is one way to get documentation In-Reply-To: <4341F42F.2020802@pfdubois.com> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341F42F.2020802@pfdubois.com> Message-ID: <4341FCD0.5020405@ee.byu.edu> I have not made this explicitly clear, but if somebody cannot afford the documentation, I will give it to them in return for help with development (significant bug-fixes, optimizations, improved docstrings, new features etc.). Any copy I give away, counts against the total price as well... I've already given away several copies. If you feel like you should receive one, and I've forgotten about you, let me know. Best, -Travis From tchur at optusnet.com.au Mon Oct 3 21:29:57 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 21:29:57 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <4341F777.3030404@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> Message-ID: <434204B8.4050308@optusnet.com.au> Travis Oliphant wrote: > Tim Churches wrote: > >> Travis Oliphant wrote: >> >> >>> Tim Churches wrote: >>> >>> >> >> Travis, are you saying that this agreement only allows a single person >> to read the single printed copy? If so, I think you need a formally >> worded legal license to make that stick - certainly Australian copyright >> law (nor copyright law in other countries, I suspect) does not provide >> any support for such severe restrictions in the use of a printed >> document. Under copyright law, you may not make unauthorised copies of a >> printed document, but you can certainly lend that copy to others, or >> sell or give it to them, or share it as a bench manual. >> >> > I'm just stating what I think is fair. I'm not going to try and use the > power of the Australian state or any other to try and enforce it. My point was that you won't get any help from the State if what you think is fair extends to restricting who can read a single printed copy of your documentation. You will need to resort to tort law to enforce such restrictions, and thus you really need a formal documentation license, if that is your aim. >> Many of those downloads are for "t[y|i]re-kicking" - potential users >> determining whether it meets their needs. If those potential users have >> to pay for documentation up-front, then a large proportion will go >> elsewhere. The rest are probably existing NumPy users upgrading (or >> testing the upgrade waters). The latter group may well pay for >> documentation, but how large is that group? For example, how many people >> are subscribed to the numpy-discussion mailing list? >> >> > Don't know. But, there is enough free documentation to get started, I > think. Lack of documentation has hampered more people than cost of > existing documentation, I think. I have to disagree on this point. Granted the existing documentation for NumPy is not fantastic - some aspects of it are positively obscure - but overall it is better than good-enough, we have found, and was not a barrier to our decision to use NumPy. Having to cough up $50 to even peek at the documentation would be a far, far greater barrier to uptake than the occasional obscure wording of some of the function descriptions, IMHO. >> Travis, although many of us are grateful for your efforts on SciPy Core, >> no-one made you do it. If you wanted to earn (more) money by doing other >> things, you should have done them. >> > I'm really thinking about the future here. If I can succesfully raise > money to support work on this using a simple time-delay documentation > encouragement, then perhaps others will do the same, and we can all have > more. Waiting for people to "donate" their time to develop scientific > python is rather slow.... I've been freely donating time for a long > time, and there are few who help. So, we'll try a slightly different > route and see where that goes. Yes, fair enough. My point was that 7 years is far too long a time delay. Given that SciPy Core will probably take another year to become rock-solid, I would happily buy several copies of your documentation at $50 per copy if I knew that in a year the documentation could be freely distributed to all God's children. But in 7 years? Nope, too long to wait, not interested in supporting that. >> I think there needs to be some community debate about this. Is there >> sufficient interest for people other than Travis to start with the >> Numeric documentation and update it as necessary to become a free SciPy >> Core documentation? The NumPy documentation is available in HTML format >> as the basis of this - perhaps the original source (LaTeX?) for the >> Numeric docs is also available? > > I don't know why you would want to undermine my efforts in this way by > duplicating effort. Travis, I don't want to undermine your efforts, but you must understand that one of the attractions of NumPy and thus Scipy Core is that they are free, open source software, which implies that there exists adequate free, open source documentation for them, or if such free, open source documentation does not exist, then taht users are collectively at liberty to create it. That's a basic FOSS tenet. What you are are proposing is the creation of proprietary documentation for SciPy Core. No-one objects to this, but only if it is supplementary to free, open source "core" documentation, not instead of it. > Perhaps, instead you could have people donate $$ > instead of time to releasing the documentation. I give away copies of > the documentation to people who participate in the development process > all the time (and that comes off the total price --- though I haven't > advertised this as of yet). So, why don't you encourage people who > don't have the money to contribute to the project instead. I understand that open source software and its documentation don't just appear out of thin air, and I fully support the idea of funded or collectively-commissioned open source development. But only if the results of that funded or commissioned development is, in fact, free and open sourced. By preventing free distribution of the documentation you are creating for 7 years, the end product cannot even remotely be described as open source, thus personally I am disinterested in funding it. If the delay until open sourcing was just one year, then that's a different matter, but you do not seem to be proposing that. Hence I think the NumPy/SciPy Core user community should establish a SciPy Core documentation project, with the aim of updating and enhancing the existing NumPy documentation so that it covers the new and changed features of SciPy Core, and making that documentation available on teh same free, open source basis as NumPy/SciPy Core itself. To that end, are the original LaTeX or whatever sources for the currenet NumPy documenattion available somewhere (apart from the HTML and PDF versions, that is). Tim C From Chris.Barker at noaa.gov Mon Oct 3 22:12:48 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 3 22:12:48 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <434204B8.4050308@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> Message-ID: <43420ECD.5070806@noaa.gov> Tim Churches wrote: > I think there needs to be some community debate about this. Well, no, there doesn't. Travis is doing two things: 1) Spending a great deal of his time producing some wonderful open source software. 2) Embarking on a small business venture, trying to sell books. There is no need for community debate about an individual's business venture. It might benefit Travis to do a bit more market research, but as was pointed out, there's no reason he couldn't lower those limits and open source the book sooner if he chooses to. Travis Oliphant wrote: > I am interested in feedback. If you don't buy the book because you > think I'm asking too much money, then let me know. Since you ask, I think $39.99 is a bit steep for an electronic copy. As a quick comparison, I just bought the new wxWidgets book, documenting another open source project. You can get a printed copy from Amazon for $32.99. I also do think Tim has a point. Usually the open source model is that the basic reference docs are freely available, and there are nice user and newbie friendly books for sale. It will be much harder for scipy.base to catch on if there are no freely available docs. However, having looked at the little bit of the book that is now available, the quality and detail are far beyond what one usually finds in open-source references (with the possible exception of the python references). It certainly looks better than the old NumPy docs, which were still adequate. I have no doubt that the NumPy community will produce some free "getting started" docs anyway, so we can all be happy. Maybe, as Tim suggests, we could fill a Wiki with the existing Numpy docs, and all start editing away! -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From tchur at optusnet.com.au Mon Oct 3 22:28:18 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 22:28:18 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <43420ECD.5070806@noaa.gov> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420ECD.5070806@noaa.gov> Message-ID: <43421271.2010705@optusnet.com.au> Chris Barker wrote: > Tim Churches wrote: > >> I think there needs to be some community debate about this. > > Well, no, there doesn't. Travis is doing two things: > > 1) Spending a great deal of his time producing some wonderful open > source software. > > 2) Embarking on a small business venture, trying to sell books. > > There is no need for community debate about an individual's business > venture. It might benefit Travis to do a bit more market research, but > as was pointed out, there's no reason he couldn't lower those limits and > open source the book sooner if he chooses to. Chris, I think you misunderstand what I meant - whic is clarified in another message I just posted a minute ago. I meant that there needs to be community debate over whether there is a requirement for a project to create free, open source documentation for SciPy Core, not over whether Travis does or doesn't have the right to, as you put it "...embark on a small business venture, trying to sell books.". I completely agree that he has every legal and moral right in the world to do that. > Travis Oliphant wrote: > >> I am interested in feedback. If you don't buy the book because you >> think I'm asking too much money, then let me know. > > Since you ask, I think $39.99 is a bit steep for an electronic copy. As > a quick comparison, I just bought the new wxWidgets book, documenting > another open source project. You can get a printed copy from Amazon for > $32.99. > > I also do think Tim has a point. Usually the open source model is that > the basic reference docs are freely available, and there are nice user > and newbie friendly books for sale. It will be much harder for > scipy.base to catch on if there are no freely available docs. > > However, having looked at the little bit of the book that is now > available, the quality and detail are far beyond what one usually finds > in open-source references (with the possible exception of the python > references). It certainly looks better than the old NumPy docs, which > were still adequate. > > I have no doubt that the NumPy community will produce some free "getting > started" docs anyway, so we can all be happy. Maybe, as Tim suggests, we > could fill a Wiki with the existing Numpy docs, and all start editing away! Yup, that's all I had in mind. If, in fact we are allowed to edit the existing NumPy docs. If not, then a wiki to produce an addendum or set of annotations to them (like a Biblical concordance, perhaps) is the best that can be done. Tim C From tchur at optusnet.com.au Mon Oct 3 22:34:52 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 22:34:52 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <43420C28.5010701@pfdubois.com> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> Message-ID: <434213F0.30105@optusnet.com.au> Paul F. Dubois wrote: > The original sources were in Framemaker. I am not positive where they > are. In any case they are copyrighted by the Regents of the University > of California. I am not a lawyer and don't know what the consequences of > that are. LLNL granted free distribution of the printed document with > the Numeric source code but I don't know what their position would be on > using their copyrighted text in a new document or on giving away the > sources. OK, thanks Paul. That may have implications for you, Travis, if you are planning to base your SciPy Core book on the existing NumPy documentation. Given the circumstance which Paul describes, perhaps the only way forward is to create a freely available addendum to the freely available, existing NumPy documentation which describes how and where SciPy Core differs or extends NumPy? Or to write entirely new documentation from scratch. > I believe that under U.S. copyright law, a person may lend their copy of > a book to anyone but not reproduce it and give them a copy. Yes, likewise here in Oz, and in most countries. Of course, the publisher of a book can impose additional restrictions to which the purchaser of the book must agree, such as 'You may not show or lend your copy of this book to your colleagues, or to anyone else.' However, who in their right mind would buy a book which was sold with such restrictions attached to it? > Given the way open source capitalism works, I would not be surprised if > someone produces a 'quick reference guide' that they give away, or > writes a book on 'Using scipy.core in biology' that they sell for $88. > Let a thousand flowers bloom. Yes, I agree entirely. Travis, you are at perfect liberty to create commercial documentation for SciPy Core, but please don't object if others try to organise to create free open source documentation as well. Tim C > Tim Churches wrote: > >> Travis Oliphant wrote: >> >>> Tim Churches wrote: >>> >>> >>>> Travis Oliphant wrote: >>>> >>>> >>>> >>>>> Tim Churches wrote: >>>>> >>>>> >>>> >>>> >>>> Travis, are you saying that this agreement only allows a single person >>>> to read the single printed copy? If so, I think you need a formally >>>> worded legal license to make that stick - certainly Australian >>>> copyright >>>> law (nor copyright law in other countries, I suspect) does not provide >>>> any support for such severe restrictions in the use of a printed >>>> document. Under copyright law, you may not make unauthorised copies >>>> of a >>>> printed document, but you can certainly lend that copy to others, or >>>> sell or give it to them, or share it as a bench manual. >>>> >>>> >>> >>> I'm just stating what I think is fair. I'm not going to try and use the >>> power of the Australian state or any other to try and enforce it. >> >> >> >> My point was that you won't get any help from the State if what you >> think is fair extends to restricting who can read a single printed copy >> of your documentation. You will need to resort to tort law to enforce >> such restrictions, and thus you really need a formal documentation >> license, if that is your aim. >> >> >>>> Many of those downloads are for "t[y|i]re-kicking" - potential users >>>> determining whether it meets their needs. If those potential users have >>>> to pay for documentation up-front, then a large proportion will go >>>> elsewhere. The rest are probably existing NumPy users upgrading (or >>>> testing the upgrade waters). The latter group may well pay for >>>> documentation, but how large is that group? For example, how many >>>> people >>>> are subscribed to the numpy-discussion mailing list? >>>> >>>> >>> >>> Don't know. But, there is enough free documentation to get started, I >>> think. Lack of documentation has hampered more people than cost of >>> existing documentation, I think. >> >> >> >> I have to disagree on this point. Granted the existing documentation for >> NumPy is not fantastic - some aspects of it are positively obscure - but >> overall it is better than good-enough, we have found, and was not a >> barrier to our decision to use NumPy. Having to cough up $50 to even >> peek at the documentation would be a far, far greater barrier to uptake >> than the occasional obscure wording of some of the function >> descriptions, IMHO. >> >> >>>> Travis, although many of us are grateful for your efforts on SciPy >>>> Core, >>>> no-one made you do it. If you wanted to earn (more) money by doing >>>> other >>>> things, you should have done them. >>>> >>> >>> I'm really thinking about the future here. If I can succesfully raise >>> money to support work on this using a simple time-delay documentation >>> encouragement, then perhaps others will do the same, and we can all have >>> more. Waiting for people to "donate" their time to develop scientific >>> python is rather slow.... I've been freely donating time for a long >>> time, and there are few who help. So, we'll try a slightly different >>> route and see where that goes. >> >> >> >> Yes, fair enough. My point was that 7 years is far too long a time >> delay. Given that SciPy Core will probably take another year to become >> rock-solid, I would happily buy several copies of your documentation at >> $50 per copy if I knew that in a year the documentation could be freely >> distributed to all God's children. But in 7 years? Nope, too long to >> wait, not interested in supporting that. >> >> >>>> I think there needs to be some community debate about this. Is there >>>> sufficient interest for people other than Travis to start with the >>>> Numeric documentation and update it as necessary to become a free SciPy >>>> Core documentation? The NumPy documentation is available in HTML format >>>> as the basis of this - perhaps the original source (LaTeX?) for the >>>> Numeric docs is also available? >>> >>> >>> I don't know why you would want to undermine my efforts in this way by >>> duplicating effort. >> >> >> >> Travis, I don't want to undermine your efforts, but you must understand >> that one of the attractions of NumPy and thus Scipy Core is that they >> are free, open source software, which implies that there exists adequate >> free, open source documentation for them, or if such free, open source >> documentation does not exist, then taht users are collectively at >> liberty to create it. That's a basic FOSS tenet. What you are are >> proposing is the creation of proprietary documentation for SciPy Core. >> No-one objects to this, but only if it is supplementary to free, open >> source "core" documentation, not instead of it. >> >> >>> Perhaps, instead you could have people donate $$ >>> instead of time to releasing the documentation. I give away copies of >>> the documentation to people who participate in the development process >>> all the time (and that comes off the total price --- though I haven't >>> advertised this as of yet). So, why don't you encourage people who >>> don't have the money to contribute to the project instead. >> >> >> >> I understand that open source software and its documentation don't just >> appear out of thin air, and I fully support the idea of funded or >> collectively-commissioned open source development. But only if the >> results of that funded or commissioned development is, in fact, free and >> open sourced. By preventing free distribution of the documentation you >> are creating for 7 years, the end product cannot even remotely be >> described as open source, thus personally I am disinterested in funding >> it. If the delay until open sourcing was just one year, then that's a >> different matter, but you do not seem to be proposing that. >> >> Hence I think the NumPy/SciPy Core user community should establish a >> SciPy Core documentation project, with the aim of updating and enhancing >> the existing NumPy documentation so that it covers the new and changed >> features of SciPy Core, and making that documentation available on teh >> same free, open source basis as NumPy/SciPy Core itself. >> >> To that end, are the original LaTeX or whatever sources for the currenet >> NumPy documenattion available somewhere (apart from the HTML and PDF >> versions, that is). >> >> Tim C >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > From perry at stsci.edu Tue Oct 4 02:18:07 2005 From: perry at stsci.edu (Perry Greenfield) Date: Tue Oct 4 02:18:07 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <434213F0.30105@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> <434213F0.30105@optusnet.com.au> Message-ID: First of all, I certainly don't want to prevent Travis from recovering something for all the effort he has put into scipy_core. Nor do I wish to discourage anyone that wishes to provide alternative free documentation if they so choose. But I will note some facts about numarray documentation below that may prove useful to anyone considering the latter. On Oct 4, 2005, at 1:32 AM, Tim Churches wrote: > Paul F. Dubois wrote: >> The original sources were in Framemaker. I am not positive where they >> are. In any case they are copyrighted by the Regents of the University >> of California. I am not a lawyer and don't know what the consequences >> of >> that are. LLNL granted free distribution of the printed document with >> the Numeric source code but I don't know what their position would be >> on >> using their copyrighted text in a new document or on giving away the >> sources. The numarray User's Manual was derived from the Numpy manual that LLNL originally sponsored David Ascher to write (if that is incorrect I'm sure Paul will correct me). We dutifully propagated the legal notice in the original to the numarray version. Although IANAL I'll note that the text seems to permit changes by others, but it also seems mis-worded as it refers to software, not documentation regarding the rights. What that really means in the end I'm not sure. In any case, Jochen Kupper (I hope I have that right) converted the format from Framemaker to the Python document latex style. The source for the numarray version is currently on sourceforge (under the numarray/doc directory). I'll also note much of the capabilities in scipy_core are very similar to those of numarray. There are differences, though I don't believe it would take a great deal of work to udpate the numarray version to reflect these (e.g. the changes in the types system, how rank-0 issues, the C-API, object array details and the names of the standard packages within scipy_core.) > OK, thanks Paul. That may have implications for you, Travis, if you are > planning to base your SciPy Core book on the existing NumPy > documentation. > From what I've seen of it, I don't believe it is based at all on the original manual or any derivative, but I'll leave that for Travis to comment further on. Perry From aisaac at american.edu Tue Oct 4 03:01:45 2005 From: aisaac at american.edu (Alan G Isaac) Date: Tue Oct 4 03:01:45 2005 Subject: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4341CB9E.4050306@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> Message-ID: On Mon, 03 Oct 2005, Travis Oliphant apparently wrote: > I hope people can understand that the reality of scarcity > dictates that we coordinate efforts through some > mechanism. The price mechanism has been the most > succesful large-scale mechanism yet developed. > I am interested in feedback. If you don't buy the book > because you think I'm asking too much money, then let me > know, as Tim has done. I found this an interesting approach to supporting the project. I plan to buy the book when it is released. Hmm, why wait? I should put my money where my mouth is. Just a moment ... ok, done. I view the book as a *complement* to other documentation that will appear and as a way to support the project. I agree with Tim that freely accessible online documentation will and must become available as well. As Chris notes, some of this can happen on the Wiki. I also plan to ask our library to purchase the book, but I am concerned that your statement that multiple users each need their own copy might mean a library purchase is forbidden. I assume it did not mean that, and that you just meant that making multiple copies is restricted. (Our library supports electronic book check out.) Ruling out library purchases would, I think, be a costly mistake for many reasons, which I can list if you are interested. Finally, I agree with Tim that seven years is too long and at the price I'd hope for a paperback copy. I think a better strategy would be two years copy protection, with an updated edition every two years. (But then I am not writing the code!) The basic concept is really nice, as long as it does not make it harder for you to - fully document your code, - smile on the free documentation that emerges, and - keep your sunny disposition. Cheers, Alan Isaac From jonas at cortical.mit.edu Tue Oct 4 04:50:54 2005 From: jonas at cortical.mit.edu (Eric Jonas) Date: Tue Oct 4 04:50:54 2005 Subject: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> Message-ID: <20051004114825.GX5015@convolution.mit.edu> I wanted to echo isaac's point: > I also plan to ask our library to purchase the book, but > I am concerned that your statement that multiple users each > need their own copy might mean a library purchase is > forbidden. I assume it did not mean that, and that you > just meant that making multiple copies is restricted. (Our > library supports electronic book check out.) Ruling out > library purchases would, I think, be a costly mistake for > many reasons, which I can list if you are interested. I couldn't agree more, although I'm not quite sure how a license could be worded such that it would allow a library copy and prevent a lab bench copy, which I think was Travis' intent. That said, have you considered selling "site licenses" of a sort? I know my lab would pay a few hundred to get a PDF that we could just stick in our fileserver and use in perpetuity. I know that right now there's nothing -preventing- us from buying just one copy and doing that (other than pesky copyright law), but we'd like to support the project. ...Eric From rays at blue-cove.com Tue Oct 4 08:08:51 2005 From: rays at blue-cove.com (RayS) Date: Tue Oct 4 08:08:51 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation Message-ID: <6.2.3.4.2.20051004070429.02bca890@pop-server.san.rr.com> As a lurker and neophyte user of Numeric/numarray, I inject my perspective... >the existing documentation for >NumPy is not fantastic - some aspects of it are positively >obscure - but overall it is better than good-enough I disagree here - I have found the docs to be a real stumbling block to learning and putting new (to me) methods to use. There is often a very high level of C and/or specific math familiarity assumed; I spend a large amount of time in those instances Googling for example code (usually not much Numpy code is just sitting, posted online) and Usenet examples and discussion. I then spent time at the prompt testing up my own examples. The printed Python books are usually very good resources, unfortunately, there is too much competition for general, intro Python and not enough depth into specific areas. This is the classic Publishers Dilemma. I am buying the new wx book, myself. A similar, in-depth book on Numpy does interest me at $29-$39, and Ch2 seems to be nicely done. Hopefully the new scipy_core will have the low overhead of Numeric for small arrays. Does it have easy access to the pointer to the array memory, as in numarray? (for use with ctypes.memmove()) That said, I still like more examples. People learn in different ways. I still maintain some PHP web sites, and the best (some say only ;-) ) thing is their style of online docs with user comments, ex.: array-rand: http://www.php.net/manual/en/function.array-rand.php I you need to do a particular Thing, it is very likely to already be _in_ the docs. I think this is a better approach than Wikis, and is easier and more likely for users to contribute, like this: http://www.php.net/manual/add-note.php?sect=index&redirect=http://www.php.net/manual/en/index.php (Notice also: "Please note that periodically, the developers may go through the notes and incorporate the information in them into the documentation. This means that any note submitted here becomes the property of the PHP Documentation Group.") Ray Schumacher hobby: http://rjs.org/astro/1004x/Python/ From meng at are.berkeley.edu Tue Oct 4 12:01:53 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Tue Oct 4 12:01:53 2005 Subject: [Numpy-discussion] how to use lu decomposition in lapack library? Message-ID: <5.1.1.5.2.20051004113223.0390ae40@are.berkeley.edu> Hi there- Can someone help me on this? Thanks! Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng at are.berkeley.edu From luszczek at cs.utk.edu Tue Oct 4 12:16:40 2005 From: luszczek at cs.utk.edu (Piotr Luszczek) Date: Tue Oct 4 12:16:40 2005 Subject: [Numpy-discussion] how to use lu decomposition in lapack library? In-Reply-To: <5.1.1.5.2.20051004113223.0390ae40@are.berkeley.edu> References: <5.1.1.5.2.20051004113223.0390ae40@are.berkeley.edu> Message-ID: <4342D48B.7070904@cs.utk.edu> You would have to be more specific. If you just want to solve a system of linear equations with matrix 'a' and right-hand-side 'b': import numarray.linear_algebra as LA x = LA.solve_linear_equations(a, b) Unlike LAPACK, the above will leave your 'a' untouched. So, if you have another right hand side 'b1' and the same matrix 'a' you'll have to pay the cost of factorization all over again. There is a way around it but I don't know what you really need. Piotr meng at are.berkeley.edu wrote: > Hi there- > > Can someone help me on this? Thanks! > > Best, > Xiangyi > > Xiangyi Meng > Department of Agricultural and Resource Economics > Room 303, Giannini Hall #3310 > University of California, Berkeley > Berkeley, CA 94720-3310 > Tel: (510) 643-4124 > Fax: (510) 643-8911 > Email: meng at are.berkeley.edu > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From luszczek at cs.utk.edu Tue Oct 4 12:41:04 2005 From: luszczek at cs.utk.edu (Piotr Luszczek) Date: Tue Oct 4 12:41:04 2005 Subject: [Numpy-discussion] how to use lu decomposition in lapack library? Message-ID: <4342DAB4.40703@cs.utk.edu> Then you have to dig in deeper: import numarray import numarray.linear_algebra import numarray.linear_algebra.lapack_lite2 as lapack_lite piv = numarray.zeros((n,), "l") outcome = lapack_lite.dgesv(n, nrhs, a, lda, piv, b, ldb, 0) outcome['info'] is the 'info' parameter set by LAPACK. You have to make sure that 'a' and 'b' are in Fortran order because you're calling Fortran code. Here is a sample code: >>> a=numarray.array([[1,2,3],[4,5,6]], 'd') >>> a.is_fortran_contiguous() 0 >>> a.is_c_array() True >>> a.is_f_array() 0 >>> a.transpose() >>> a.is_fortran_contiguous() 1 >>> a.is_c_array() False >>> a.is_f_array() True Call to 'transpose()' made it a Fortran array but it transposed the array too. Piotr meng at are.berkeley.edu wrote: > Thanks, Piotr! > > What I actually want is the lower (L) and upper triangular matrix (U) > from matrix 'a'. How to get it? > > Xiangyi > > At 15:14 2005-10-4 -0400, you wrote: > >> You would have to be more specific. If you just want to >> solve a system of linear equations with matrix 'a' and >> right-hand-side 'b': >> >> import numarray.linear_algebra as LA >> >> x = LA.solve_linear_equations(a, b) >> >> Unlike LAPACK, the above will leave your 'a' untouched. >> So, if you have another right hand side 'b1' and the >> same matrix 'a' you'll have to pay the cost of factorization >> all over again. >> >> There is a way around it but I don't know what you really need. >> >> Piotr >> >> meng at are.berkeley.edu wrote: >> >>> Hi there- >>> Can someone help me on this? Thanks! >>> Best, >>> Xiangyi >>> Xiangyi Meng >>> Department of Agricultural and Resource Economics >>> Room 303, Giannini Hall #3310 >>> University of California, Berkeley >>> Berkeley, CA 94720-3310 >>> Tel: (510) 643-4124 >>> Fax: (510) 643-8911 >>> Email: meng at are.berkeley.edu >>> >>> >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: >>> Power Architecture Resource Center: Free content, downloads, >>> discussions, >>> and more. http://solutions.newsforge.com/ibmarch.tmpl >>> _______________________________________________ >>> Numpy-discussion mailing list >>> Numpy-discussion at lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > Best, > Xiangyi > > Xiangyi Meng > Department of Agricultural and Resource Economics > Room 303, Giannini Hall #3310 > University of California, Berkeley > Berkeley, CA 94720-3310 > Tel: (510) 643-4124 > Fax: (510) 643-8911 > Email: meng at are.berkeley.edu > From tchur at optusnet.com.au Tue Oct 4 14:56:32 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Tue Oct 4 14:56:32 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <87wtkt2x36.fsf@peds-pc311.bsd.uchicago.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> <434213F0.30105@optusnet.com.au> <87wtkt2x36.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <4342FA08.805@optusnet.com.au> John Hunter wrote: >>>>>>"Tim" == Tim Churches writes: > > > Tim> Yes, I agree entirely. Travis, you are at perfect liberty to > Tim> create commercial documentation for SciPy Core, but please > Tim> don't object if others try to organise to create free open > Tim> source documentation as well. > > Object? I'll bet dollars to doughnuts that Travis would be delighted > to see this, just as he would have been at any time over the last > several years. And be careful: Perry still owes me a doughnut. Travis did indeed object, which surprised me. Verbatim: Tim C: >> I think there needs to be some community debate about this. Is there >> sufficient interest for people other than Travis to start with the >> Numeric documentation and update it as necessary to become a free >> SciPy Core documentation? The NumPy documentation is available in >> HTML format as the basis of this - perhaps the original source >> (LaTeX?) for the Numeric docs is also available? Travis: > I don't know why you would want to undermine my efforts in this way > by duplicating effort. Perhaps, instead you could have people donate > $$ instead of time to releasing the documentation. I give away > copies of the documentation to people who participate in the > development process all the time (and that comes off the total price > --- though I haven't advertised this as of yet). So, why don't you > encourage people who don't have the money to contribute to the project > instead. So, Travis seems to be proposing that he write his documentation book, which will be available only on a commercial basis (or by barter for development work) and which cannot be shared as a lab bench manual, and that any effort to create freely available documentation for scipy.core undermines his efforts. Now, we are **extremely** grateful to Travis for rescuing NumPy from a slow death by decay, and for saving us the pain of having to convert all our code to numarray (whereas the conversion to scipy.core should be much less painful). But the foregoing sets off alarm bells ringing in my head... I really think it would be useful if Travis makes a clear and unambiguous statement about where he stands on the free, open source nature of both the scipy.core code AND on documentation for it, just so we all know where we respectively stand. I have stated my position (not that it really matters), but I will re-iterate: a) scipy.core deserves a basic set of documentation (similar to the current NumPy documentation) which is available to everyone on exactly the same basis as the scipy.core code itself - that is, freely available at no cost. b) It is not Travis's duty to create such freely-available documentation. c) Others at at liberty to create such freely-available documentation for scipy.core if they so chose. d) The documentation which Travis is proposing to write will not be made freely available until targets of $300k in sales or 7 years have been reached, and this effectively renders such documentation proprietary. e) I think that the creation of proprietary supplementary documentation for scipy.core, as Travis is doing, is a good idea and I will personally buy several copies provided that the work is licensed in a reasonable manner (eg sharing of single physical copies as a bench manual is permitted). Tim C From oliphant at ee.byu.edu Tue Oct 4 15:10:37 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Oct 4 15:10:37 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Release of SciPy Core 0.4 (Beta) In-Reply-To: References: <433CE24A.6040509@ee.byu.edu> <4341F044.8030900@shrogers.com> <4341FBD0.8090107@ee.byu.edu> Message-ID: <4342FD6B.20009@ee.byu.edu> Rob Managan wrote: > Does the new scipy_core support the Numeric function > PyArray_FromDimsAndData? > > I use that to create a front end to a stand alone program that > generates a lot of arrays that I want to be able to query. Not the > best approach maybe but it works! > Yes, All of the old Numeric C-API is available (I believe...). Direct access to descr->one and descr->zero does not work anymore, though (it's replaced with a function call). If you look at the source, you will see that these older calls are all special cases of the call to PyArray_New() -Travis From oliphant at ee.byu.edu Tue Oct 4 15:25:50 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Oct 4 15:25:50 2005 Subject: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> Message-ID: <434300FB.7070606@ee.byu.edu> Alan G Isaac wrote: >I also plan to ask our library to purchase the book, but >I am concerned that your statement that multiple users each >need their own copy might mean a library purchase is >forbidden. I assume it did not mean that, and that you >just meant that making multiple copies is restricted. (Our >library supports electronic book check out.) Ruling out >library purchases would, I think, be a costly mistake for >many reasons, which I can list if you are interested. > > A library purchase is fine. If that how a single copy is shared. I'll make that more clear. But, really, if multiple users need to use it at the same time, then the library should purchase several copies. >Finally, I agree with Tim that seven years is too long and >at the price I'd hope for a paperback copy. I think >a better strategy would be two years copy protection, with >an updated edition every two years. (But then I am not >writing the code!) T > Thanks for the feedback. I'm experimenting with the right combination of total price and total time so feedback is welcomed. I want to encourage people who can really afford the book to spend the money on it. What is the "right" time/$$ combination that will encourage this. I'm willing to shorten the time and come down on the total price. They are set so I cannot increase them. But, there is no problem with lowering them. I could also support the idea of a cheaper total price 1st edition with a need to spend for the 2nd edition again. Thanks for the feedback. >he basic concept is really nice, as >long as it does not make it harder for you to >- fully document your code, >- smile on the free documentation that emerges, and >- keep your sunny disposition. > > Don't worry, I'm not banking my future on this little experiment, so I won't worry about what other people do. In fact, as John Hunter inferred, I would be thrilled by more contributions however they come. I just want to see more people use Python for scientific computing, and am trying some things out to help it along. My only question about writing "free" documentation, is that it just seems rather wasteful to spend time writing free documentation when you can set free the documentation by spending a little money instead. If you think I'm charging too much (either in $$ or time-delay), then continue to give me feedback. I am interested in what works. -Travis From aisaac at american.edu Tue Oct 4 15:40:31 2005 From: aisaac at american.edu (Alan G Isaac) Date: Tue Oct 4 15:40:31 2005 Subject: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <434300FB.7070606@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> Message-ID: On Tue, 04 Oct 2005, Travis Oliphant apparently wrote: > A library purchase is fine. If that how a single copy is shared. I'll > make that more clear. But, really, if multiple users need to use it at > the same time, then the library should purchase several copies. Our library supports single copy checkout. I think this currently is a pretty standard library function these days. > My only question about writing "free" documentation, is that it just > seems rather wasteful to spend time writing free documentation when you > can set free the documentation by spending a little money instead. I've done my part. ;-) Cheers, Alan Isaac From Fernando.Perez at colorado.edu Tue Oct 4 15:54:25 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Tue Oct 4 15:54:25 2005 Subject: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <434300FB.7070606@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> Message-ID: <434307CC.7020303@colorado.edu> Travis Oliphant wrote: > Alan G Isaac wrote: > > >>I also plan to ask our library to purchase the book, but >>I am concerned that your statement that multiple users each >>need their own copy might mean a library purchase is >>forbidden. I assume it did not mean that, and that you >>just meant that making multiple copies is restricted. (Our >>library supports electronic book check out.) Ruling out >>library purchases would, I think, be a costly mistake for >>many reasons, which I can list if you are interested. >> >> > > A library purchase is fine. If that how a single copy is shared. I'll > make that more clear. But, really, if multiple users need to use it at > the same time, then the library should purchase several copies. Travis, I think that what confused some people (and which I believe was not your original intent) was the impression that you meant to have terms on the printed version of the book which were more restrictive than those of traditional paper books. With a single physical copy of a paper book, the rules are pretty simple and constrained by the laws of nature (one non-quantum object can't really be in more than one place at the same time). Lending, borrowing, library use, 'lab bench' use, etc, are all accepted practices because while one person is using the book, nobody else has access to it. Since your book is originally provided electronically, there is the technical possiblity to make multiple physical copies, which I believe is what you wish to prevent (and something I'm not arguing with). So perhaps a clarification along the lines of the following could help (this is my wording, of course, so you should say what _you_ want, not what I get from trying to read your mind :) 'a single printed copy can be made from the electronic version, which is subject to the same restrictions imposed on paper books (lending is OK but not wholesale photocopying for redistribution, for example)' This would put at ease a lot of people who normally buy a book in a lab or research group with the natural assumption that anyone in that lab can go to the shelf and read it. Obviously if it's a book with very frequent use, traditional book purchasers buy multiple copies. With your book, the exact same thing would be expected: just because they have the PDF doesn't mean they can print 10 copies of it for the whole lab. Or at least that's my understanding. Best regards, Fernando. From oliphant at ee.byu.edu Tue Oct 4 16:10:08 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Oct 4 16:10:08 2005 Subject: [SciPy-user] Re: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <434307CC.7020303@colorado.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> Message-ID: <43430B62.9090908@ee.byu.edu> Fernando Perez wrote: > Travis Oliphant wrote: > >> Alan G Isaac wrote: >> >> >>> I also plan to ask our library to purchase the book, but I am >>> concerned that your statement that multiple users each need their >>> own copy might mean a library purchase is forbidden. I assume it >>> did not mean that, and that you just meant that making multiple >>> copies is restricted. (Our library supports electronic book check >>> out.) Ruling out library purchases would, I think, be a costly >>> mistake for many reasons, which I can list if you are interested. >>> >> >> A library purchase is fine. If that how a single copy is shared. >> I'll make that more clear. But, really, if multiple users need to >> use it at the same time, then the library should purchase several >> copies. > > > 'a single printed copy can be made from the electronic version, which > is subject to the same restrictions imposed on paper books (lending is > OK but not wholesale photocopying for redistribution, for example)' > > This would put at ease a lot of people who normally buy a book in a > lab or research group with the natural assumption that anyone in that > lab can go to the shelf and read it. Obviously if it's a book with > very frequent use, traditional book purchasers buy multiple copies. > With your book, the exact same thing would be expected: just because > they have the PDF doesn't mean they can print 10 copies of it for the > whole lab. Or at least that's my understanding. > Thanks, I like this wording. It is exactly what I meant. I also think except for a library-checkout system (where only one digital copy is in circulation), somebody should not be able to buy an e-copy and then make electronic copies for everybody in their organization. That's really quite counter productive. If you want to share your e-copy with someone for a while (or give it away) fine... I'm really just asking that you treat the e-copy something like a physical book. I'll make some more details concerning the intent, available. Thanks for everybody's help. -Travis From cjw at sympatico.ca Wed Oct 5 16:57:17 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Wed Oct 5 16:57:17 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> <434213F0.30105@optusnet.com.au> Message-ID: <434467D9.3040707@sympatico.ca> Perry Greenfield wrote: > First of all, I certainly don't want to prevent Travis from recovering > something for all the effort he has put into scipy_core. Nor do I wish > to discourage anyone that wishes to provide alternative free > documentation > if they so choose. But I will note some facts about numarray > documentation > below that may prove useful to anyone considering the latter. > > On Oct 4, 2005, at 1:32 AM, Tim Churches wrote: > >> Paul F. Dubois wrote: >> >>> The original sources were in Framemaker. I am not positive where they >>> are. In any case they are copyrighted by the Regents of the University >>> of California. I am not a lawyer and don't know what the >>> consequences of >>> that are. LLNL granted free distribution of the printed document with >>> the Numeric source code but I don't know what their position would >>> be on >>> using their copyrighted text in a new document or on giving away the >>> sources. >> > > The numarray User's Manual was derived from the Numpy manual that LLNL > originally sponsored David Ascher to write (if that is incorrect I'm sure > Paul will correct me). We dutifully propagated the legal notice in the > original to the numarray version. Although IANAL I'll note that the > text seems to permit changes by others, but it also seems mis-worded as > it refers to software, not documentation regarding the rights. What that > really means in the end I'm not sure. > > In any case, Jochen Kupper (I hope I have that right) converted the > format > from Framemaker to the Python document latex style. The source for the > numarray version is currently on sourceforge (under the numarray/doc > directory). I'll also note much of the capabilities in scipy_core are > very similar to those of numarray. There are differences, though I don't > believe it would take a great deal of work to udpate the numarray version > to reflect these (e.g. the changes in the types system, how rank-0 > issues, the C-API, object array details and the names of the standard > packages within scipy_core.) > >> OK, thanks Paul. That may have implications for you, Travis, if you are >> planning to base your SciPy Core book on the existing NumPy >> documentation. >> > From what I've seen of it, I don't believe it is based at all on the > original manual or any derivative, but I'll leave that for Travis to > comment further on. > > Perry > > Perry, It would be helpful if you gave some indication of your organization's intention with respect to numarray. Will it be maintained, as in the past, fully in the public domain and without charge for either the code or the API information needed to make use of the code? New versions have been produced every few months, will this continue? Colin W. From faltet at carabos.com Thu Oct 6 02:53:59 2005 From: faltet at carabos.com (Francesc Altet) Date: Thu Oct 6 02:53:59 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <434467D9.3040707@sympatico.ca> References: <43418AA8.2040809@hawaii.edu> <434467D9.3040707@sympatico.ca> Message-ID: <200510061152.55708.faltet@carabos.com> Exactly, I'm wondering the same questions. Provided that numarray is an excellent and mature piece of code, I'd like to hear a declaration of principles on that subject. Regards, A Dijous 06 Octubre 2005 01:55, Colin J. Williams va escriure: > Perry, > > It would be helpful if you gave some indication of your organization's > intention with respect to numarray. > > Will it be maintained, as in the past, fully in the public domain and > without charge for either the code or the API information needed to make > use of the code? > > New versions have been produced every few months, will this continue? > > Colin W. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From cjw at sympatico.ca Thu Oct 6 04:53:56 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu Oct 6 04:53:56 2005 Subject: [Numpy-discussion] pysci base formerly Numeric3 Message-ID: <43451032.10702@sympatico.ca> I have just downloaded and installed the Windows version. The install appeared to go smoothly but test_basic.py gave some errors, is this what one should expect? --------------------------------------------------------------------------------------------------------------------- >pythonw -u "test_basic.py" !! No test file 'test_scimath.py' found for !! No test file 'test_umath.py' found for Found 23 tests for scipy.base.function_base !! No test file 'test_machar.py' found for !! No test file 'test_ma.py' found for !! No test file 'test_numerictypes.py' found for !! No test file 'test_getlimits.py' found for Found 9 tests for scipy.base.twodim_base !! No test file 'test__compiled_base.py' found for !! No test file 'test_info_scipy_base.py' found for !! No test file 'test_base.py' found for !! No test file 'test_convertcode.py' found for !! No test file 'test_multiarray.py' found for !! No test file 'test_matrix.py' found for !! No test file 'test_oldnumeric.py' found for Found 44 tests for scipy.base.shape_base !! No test file 'test_numeric.py' found for !! No test file 'test_polynomial.py' found for Found 42 tests for scipy.base.type_check !! No test file 'test_arrayprint.py' found for Found 4 tests for scipy.base.index_tricks Found 0 tests for __main__ ................................E.................................................E........E.E.E...........EEEEEE.E..E.... ====================================================================== ERROR: check_simple (scipy.base.shape_base.test_shape_base.test_apply_along_axis) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_shape_base.py", line 14, in check_simple assert_array_equal(apply_along_axis(len,0,a),len(a)*ones(shape(a)[1])) File "C:\Python24\Lib\site-packages\scipy\base\shape_base.py", line 31, in apply_along_axis res = func1d(arr[tuple(i)],*args) IndexError: each subindex must be either a slice, an integer, Ellipsis, or newaxis ====================================================================== ERROR: check_complex1 (scipy.base.type_check.test_type_check.test_isfinite) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 162, in check_complex1 assert_all(isfinite(array(1+1j)/0.) == 0) ZeroDivisionError: complex division ====================================================================== ERROR: check_neginf_scalar (scipy.base.type_check.test_type_check.test_isinf) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 176, in check_neginf_scalar assert_all(isinf(array(-1.)/0.) == 1) ZeroDivisionError: float division ====================================================================== ERROR: check_posinf_scalar (scipy.base.type_check.test_type_check.test_isinf) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 172, in check_posinf_scalar assert_all(isinf(array(1.,)/0.) == 1) ZeroDivisionError: float division ====================================================================== ERROR: check_complex1 (scipy.base.type_check.test_type_check.test_isnan) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 142, in check_complex1 assert_all(isnan(array(0+0j)/0.) == 1) ZeroDivisionError: complex division ====================================================================== ERROR: check_default_1 (scipy.base.type_check.test_type_check.test_mintypecode) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 18, in check_default_1 assert_equal(mintypecode(itype),'d') File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 30, in mintypecode typecodes = [(type(t) is type('') and t) or asarray(t).dtypechar\ UnboundLocalError: local variable 'typecodes' referenced before assignment ====================================================================== ERROR: check_default_2 (scipy.base.type_check.test_type_check.test_mintypecode) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 26, in check_default_2 assert_equal(mintypecode(itype+'f'),'f') File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 30, in mintypecode typecodes = [(type(t) is type('') and t) or asarray(t).dtypechar\ UnboundLocalError: local variable 'typecodes' referenced before assignment ====================================================================== ERROR: check_default_3 (scipy.base.type_check.test_type_check.test_mintypecode) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 50, in check_default_3 assert_equal(mintypecode('fdF'),'D') File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 30, in mintypecode typecodes = [(type(t) is type('') and t) or asarray(t).dtypechar\ UnboundLocalError: local variable 'typecodes' referenced before assignment ====================================================================== ERROR: check_complex_bad (scipy.base.type_check.test_type_check.test_nan_to_num) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 211, in check_complex_bad v += array(0+1.j)/0. ZeroDivisionError: complex division ====================================================================== ERROR: check_complex_bad2 (scipy.base.type_check.test_type_check.test_nan_to_num) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 217, in check_complex_bad2 v += array(-1+1.j)/0. ZeroDivisionError: complex division ====================================================================== ERROR: check_complex_good (scipy.base.type_check.test_type_check.test_nan_to_num) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 207, in check_complex_good vals = nan_to_num(1+1j) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 99, in nan_to_num y = nan_to_num(x.real) + 1j * nan_to_num(x.imag) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 106, in nan_to_num y[are_nan] = 0 TypeError: object does not support item assignment ====================================================================== ERROR: check_integer (scipy.base.type_check.test_type_check.test_nan_to_num) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 204, in check_integer vals = nan_to_num(1) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 105, in nan_to_num maxf, minf = _getmaxmin(y.dtype) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 86, in _getmaxmin f = getlimits.finfo(t) File "C:\Python24\Lib\site-packages\scipy\base\getlimits.py", line 29, in __init__ raise ValueError, "data type not inexact" ValueError: data type not inexact ====================================================================== ERROR: check_basic (scipy.base.type_check.test_type_check.test_real_if_close) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 230, in check_basic b = real_if_close(a+1e-15j) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 120, in real_if_close tol = f.epsilon * tol AttributeError: 'finfo' object has no attribute 'epsilon' ---------------------------------------------------------------------- Ran 122 tests in 0.521s FAILED (errors=13) >Exit code: -1073741819 ------------------------------------------------------------------------------------------------------------------- Colin W. From matthew.brett at gmail.com Thu Oct 6 05:31:25 2005 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu Oct 6 05:31:25 2005 Subject: [SciPy-user] Re: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <43430B62.9090908@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> Message-ID: <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> Hi Travis, So, can I summarize for my own benefit? I think we all agree that software without documentation is not usable, and therefore that free software requires that the basic documentation also be free. Obviously if the _basic_ documentation is not free, this will dramatically reduce software uptake. And I think we all also agree that there is no problem of any sort with not-free documentation that expands in a useful way on the basic documentation. So, just to clarify - does Fenando win his bet (was it dollars or doughnuts)? Are you happy for the community to write and release free basic documentation for scipy.base? Thanks a lot, Matthew From cjw at sympatico.ca Thu Oct 6 06:22:25 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu Oct 6 06:22:25 2005 Subject: [Numpy-discussion] A possible problem with PythonWin and Numeric3 Message-ID: <434524A0.9020400@sympatico.ca> PythonWin 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32. Portions Copyright 1994-2004 Mark Hammond (mhammond at skippinet.com.au) - see 'Help/About PythonWin' for further copyright information. >>> import scipy <== Executing this causes a crash. >>> import scipy.base <== This has the same effect Colin W. From faltet at carabos.com Thu Oct 6 07:59:19 2005 From: faltet at carabos.com (Francesc Altet) Date: Thu Oct 6 07:59:19 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <200510061152.55708.faltet@carabos.com> References: <43418AA8.2040809@hawaii.edu> <434467D9.3040707@sympatico.ca> <200510061152.55708.faltet@carabos.com> Message-ID: <200510061657.09349.faltet@carabos.com> A Dijous 06 Octubre 2005 11:52, Francesc Altet va escriure: > Exactly, I'm wondering the same questions. Provided that numarray is > an excellent and mature piece of code, I'd like to hear a declaration > of principles on that subject. Er, I think I'd better said "declaration of intentions", sorry. Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From oliphant at ee.byu.edu Thu Oct 6 08:24:10 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu Oct 6 08:24:10 2005 Subject: [SciPy-user] Re: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> Message-ID: <43454135.503@ee.byu.edu> Matthew Brett wrote: >Hi Travis, > >So, can I summarize for my own benefit? > >I think we all agree that software without documentation is not >usable, and therefore that free software requires that the basic >documentation also be free. > >Obviously if the _basic_ documentation is not free, this will >dramatically reduce software uptake. > >And I think we all also agree that there is no problem of any sort >with not-free documentation that expands in a useful way on the basic >documentation. > >So, just to clarify - does Fenando win his bet (was it dollars or >doughnuts)? Are you happy for the community to write and release free >basic documentation for scipy.base? > > Yes, yes. Fernando knows me pretty well in that regard. I'll probably contribute to it with useful selections from the book (once I have the book finished). It really motivates me to produce more (of everything) when I see people actually purchasing the documentation. And I certainly don't want "lack of free documentation" to hamper adoption of scipy core. So, it benefits everybody to have free version available. It would be great if somebody could at least post the pydoc output for the core. That is a good start. I'll add basic C-API calls to free documentation and we'll be ready to go. -Travis From oliphant at ee.byu.edu Thu Oct 6 08:31:38 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu Oct 6 08:31:38 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Fwd: Scipy_core and numarray In-Reply-To: <36BF0393-F2E6-46FA-8A2A-978B1D51535C@gmail.com> References: <17221.11905.227711.3197@imperial.ac.uk> <36BF0393-F2E6-46FA-8A2A-978B1D51535C@gmail.com> Message-ID: <434542BE.3010005@ee.byu.edu> Andrew Jaffe wrote: >Hi All, > >I've tried to post via the newsgroup interface, but my messages have >never appeared, so here's another try!... > >I'm currently a numarray user from the astrophysics community and I >suppose I want to understand a bit more about the relationship between >the two projects. Do the two projects exist side-by-side? Will numarray >be superseded by scipy_core? I like the speed advantage of scipy_core, >but I there may be features of numarray I don't want to give up (or >legacy code I need to interfact with). > > I'm interested in the "features" you don't want to give up. SciPy core should have all features of numarray. Also, scipy core, numarray, and Numeric can all live together. If you are using sufficient versions of each, you can exchange data via the array interface. The ONLY reason I started scipy core was to unify Numeric and numarray users. My selling documentation, should not be construed as anything else but trying to help people who can't find time to contribute in other ways to contribute to that goal. If I was "just trying to make money," I would be doing something far different than this... > >Along the latter lines, I have a more practical question: one >numarray capability that I used a lot was the ability to create >uninitialized arrays, such as > > arr = numarray.array(shape=(3,5), type=Float64) > > It's actually there in Numeric (since 23.7 I think). Numeric.empty((3,5),'d') and also in (new) scipy scipy.empty((3,5),scipy.float64) In (new) scipy you can also create new arrays from buffers very easily. -Travis From jswhit at fastmail.fm Thu Oct 6 08:43:52 2005 From: jswhit at fastmail.fm (Jeff Whitaker) Date: Thu Oct 6 08:43:52 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Fwd: Scipy_core and numarray In-Reply-To: <434542BE.3010005@ee.byu.edu> References: <17221.11905.227711.3197@imperial.ac.uk> <36BF0393-F2E6-46FA-8A2A-978B1D51535C@gmail.com> <434542BE.3010005@ee.byu.edu> Message-ID: <434545C5.80709@fastmail.fm> Travis Oliphant wrote: > Andrew Jaffe wrote: > >> Hi All, >> >> I've tried to post via the newsgroup interface, but my messages have >> never appeared, so here's another try!... >> >> I'm currently a numarray user from the astrophysics community and I >> suppose I want to understand a bit more about the relationship between >> the two projects. Do the two projects exist side-by-side? Will numarray >> be superseded by scipy_core? I like the speed advantage of scipy_core, >> but I there may be features of numarray I don't want to give up (or >> legacy code I need to interfact with). >> >> > I'm interested in the "features" you don't want to give up. SciPy > core should have all features of numarray. > Also, scipy core, numarray, and Numeric can all live together. If you > are using sufficient versions of each, you can exchange data via the > array interface. Will scipy_core have nd_image and memory mapping? -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/CDC R/CDC1 Email : Jeffrey.S.Whitaker at noaa.gov 325 Broadway Office : Skaggs Research Cntr 1D-124 Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg From matthew.brett at gmail.com Thu Oct 6 11:16:15 2005 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu Oct 6 11:16:15 2005 Subject: [SciPy-user] Re: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <43454135.503@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> Message-ID: <1e2af89e0510061115l57351184id6ebba394b4b5b26@mail.gmail.com> Hi, > >So, just to clarify - does Fenando win his bet (was it dollars or > >doughnuts)? Are you happy for the community to write and release free > >basic documentation for scipy.base? > > Yes, yes. Fernando knows me pretty well in that regard. I'll probably > contribute to it with useful selections from the book (once I have the > book finished). It really motivates me to produce more (of > everything) when I see people actually purchasing the documentation. Excellent - thanks very much for that. When my credit cards have recovered from their address changes I will be early in line! Best, Matthew From cjw at sympatico.ca Thu Oct 6 11:57:37 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu Oct 6 11:57:37 2005 Subject: [Numpy-discussion] A comparative test Message-ID: <43457338.2010503@sympatico.ca> # NumericNumarrayTest.py ''' A test, provided by Francesc Altet http://sourceforge.net/mailarchive/forum.php?thread_id=6396059&forum_id=4890 ''' from timeit import Timer setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" print 'Timer-Numeric23.8:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) # setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" print 'Timer-numarray1.3.3:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) # scipy/Numeric3 added setup = "import scipy.base; a = scipy.base.arange(2000);a.shape=(1000,2)" print 'Timer-Numeric3:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) RESULTS: >pythonw -u "NumericNumarrayTest.py" Timer-Numeric23.8: 0.179712784725 Timer-numarray1.3.3: 0.21674933546 Timer-Numeric3: 0.253077136899 >Exit code: 0 Colin W. From faltet at carabos.com Thu Oct 6 12:43:46 2005 From: faltet at carabos.com (Francesc Altet) Date: Thu Oct 6 12:43:46 2005 Subject: [Numpy-discussion] A comparative test In-Reply-To: <43457338.2010503@sympatico.ca> References: <43457338.2010503@sympatico.ca> Message-ID: <200510062142.56692.faltet@carabos.com> A Dijous 06 Octubre 2005 20:55, Colin J. Williams va escriure: > RESULTS: > >pythonw -u "NumericNumarrayTest.py" > > Timer-Numeric23.8: 0.179712784725 > Timer-numarray1.3.3: 0.21674933546 > Timer-Numeric3: 0.253077136899 Is that really true? From my original post: >>> from timeit import Timer >>> setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) 0.12782907485961914 >>> setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) 1.2013700008392334 My post was from January, so, in the interim numarray has improved *a lot* it's object creation time. In that case, why the numarray team hasn't publisized that more? Well, I think I should be missing something. Time for nap. Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From rlw at stsci.edu Thu Oct 6 13:16:34 2005 From: rlw at stsci.edu (Rick White) Date: Thu Oct 6 13:16:34 2005 Subject: [Numpy-discussion] A comparative test In-Reply-To: <200510062142.56692.faltet@carabos.com> References: <43457338.2010503@sympatico.ca> <200510062142.56692.faltet@carabos.com> Message-ID: On Thu, 6 Oct 2005, Francesc Altet wrote: > A Dijous 06 Octubre 2005 20:55, Colin J. Williams va escriure: > > RESULTS: > > >pythonw -u "NumericNumarrayTest.py" > > > > Timer-Numeric23.8: 0.179712784725 > > Timer-numarray1.3.3: 0.21674933546 > > Timer-Numeric3: 0.253077136899 > > Is that really true? From my original post: > > >>> from timeit import Timer > >>> setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" > >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) > 0.12782907485961914 > >>> setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" > >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) > 1.2013700008392334 > > My post was from January, so, in the interim numarray has improved *a > lot* it's object creation time. In that case, why the numarray team > hasn't publisized that more? Well, I think I should be missing > something. Time for nap. Well, this seems pretty clear: On Thu, 14 Apr 2005, Todd Miller wrote: > Subject: [Numpy-discussion] ANN: numarray-1.3.0 > > I. ENHANCEMENTS > > 2. Removal of dictionary update from array view creation improves > performance of view/slice/subarray creation. This should e.g. improve > the performance of wxPython sequence protocol access to Nx2 arrays. You might wonder why everyone didn't notice when their numarray programs started to run blazingly fast after 1.3.0. My (jaded) view is that it is very rare to have an application where the overall execution time is dominated by the time to create small arrays. When you're in that limit, there are a million other overheads that are also slowing you down. In the vast majority of applications you could reduce the array creation time to zero and probably wouldn't notice the difference without running timing tests. IMHO the numarray developers put their emphasis in the right place by focusing on large array performance and improved functionality, and all the noise around small array indexing performance was just a red herring that convinced some folks not to try numarray because they heard it was slow. I hope Travis doesn't devote a lot of effort to this optimization right now. I'd be much more interested in seeing a large array benchmark. Rick From Fernando.Perez at colorado.edu Thu Oct 6 13:28:36 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Thu Oct 6 13:28:36 2005 Subject: [Numpy-discussion] A comparative test In-Reply-To: References: <43457338.2010503@sympatico.ca> <200510062142.56692.faltet@carabos.com> Message-ID: <4345889B.6050909@colorado.edu> Rick White wrote: > IMHO the numarray developers put their emphasis in the right place > by focusing on large array performance and improved functionality, > and all the noise around small array indexing performance was just > a red herring that convinced some folks not to try numarray because > they heard it was slow. I hope Travis doesn't devote a lot of > effort to this optimization right now. I'd be much more interested > in seeing a large array benchmark. Except for codes like our PDE solvers, which need to create 10s of millions of small arrays very, very fast. So no, it's not a red herring [1]. Just because you don't need something doesn't mean there's no valid need for it. I am not disputing in any way the value of the many, many improvements made by numarray, nor the importance of fixing large array performance for many applications (such as I imagine is the typical workflow of astronomical data analysis). The fact that Travis tried to incorporate all of numarrays' improvements into the new library is a recognition of the value of all this work. But calling the small array performance problem a 'red herring' is inaccurate and unfair, at best. Regards, f [1] Yes, I could preallocate pools of memory for this, manage them manually, etc. But then I wouldn't be writing my code in python, would I? The whole point of using python is to have my cake and eat it too: I want high-level code that gives me near-hardware max speeds. With carefully tuned (python) code, judiciously sprinkled minimal amounts of C/C++/Fortran, and a LOT of thinking about algorithm design, I get that. From oliphant at ee.byu.edu Thu Oct 6 15:15:40 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu Oct 6 15:15:40 2005 Subject: [Numpy-discussion] Simple optimization In-Reply-To: <43457338.2010503@sympatico.ca> References: <43457338.2010503@sympatico.ca> Message-ID: <4345A1C4.9080409@ee.byu.edu> Colin J. Williams wrote: > # NumericNumarrayTest.py > ''' A test, provided by Francesc Altet > > http://sourceforge.net/mailarchive/forum.php?thread_id=6396059&forum_id=4890 > > ''' > from timeit import Timer > setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" > print 'Timer-Numeric23.8:', Timer("for i in range(len(a)): row=a[i]", > setup).timeit(number=100) > > # > setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" > print 'Timer-numarray1.3.3:', Timer("for i in range(len(a)): > row=a[i]", setup).timeit(number=100) > > # scipy/Numeric3 added > setup = "import scipy.base; a = scipy.base.arange(2000);a.shape=(1000,2)" > print 'Timer-Numeric3:', Timer("for i in range(len(a)): row=a[i]", > setup).timeit(number=100) > > RESULTS: > >pythonw -u "NumericNumarrayTest.py" > Timer-Numeric23.8: 0.179712784725 > Timer-numarray1.3.3: 0.21674933546 > Timer-Numeric3: 0.253077136899 After a simple optimization in the array_subscript function (to check for scalar selection up front): new results are (for 1000 runs on my system): Numeric: 0.56431102752685547 numarray: 1.0897960662841797 scipy: 0.57508182525634766 (it's now executing nearly the identical code as Numeric) Any more optimization ideas? -Travis From Fernando.Perez at colorado.edu Thu Oct 6 15:26:03 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Thu Oct 6 15:26:03 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <43454135.503@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> Message-ID: <4345A449.8040303@colorado.edu> Travis Oliphant wrote: > Yes, yes. Fernando knows me pretty well in that regard. I'll probably > contribute to it with useful selections from the book (once I have the > book finished). It really motivates me to produce more (of > everything) when I see people actually purchasing the documentation. To reiterate before my reputation is tarnished forever: the bet was with John, who is the doughnuts-eating one. Being a man of taste, I eat bagels :) [and I wasn't present when the bet was made] With that critical point out of the way, on to minor details. > It would be great if somebody could at least post the pydoc output for > the core. That is a good start. I'll add basic C-API calls to free > documentation and we'll be ready to go. While I fully respect the opinion of those concerned about Travis' decision, I have to say that I am personally not so worried. At scipy'05, when Travis announced the book idea, I asked a very simple question: 'are the docstrings complete?'. His answer was 'yes'. There was a good reason for my asking this question: I personally find that when coding, the most productive and efficient to learn a library is by typing import foo foo. and then foo.bar? or foo.bar?? if I want to see the source (for python-implemented things). I understand that some people prefer to read manuals/books, and there are topics beyond the single-function API that are best discussed in a book format. But given the instantaneous response of the above, I would rather use this than wade through an index in most cases. In fact, I'd like docstrings to provide (simple) examples in them, and one thing on my radar now that ipython is growing GUI/XML legs is to add support for latex in docstrings. I think that a library where all individual methods/functions have full, accurate and example-loaded docstrings, and where class declarations and top-level module docstrings provide the higher-level overview, is extremely usable as is. Especially when you can also keep pydoc running in html server mode (pydoc -p) to browse at your leisure, and trivially dump that HTML info to disk for static reference if needed. If this is done, I think that the space for a 'real book' is better defined, providing less of a detail reference and more of a teaching coverage. I sense that this is Travis' approach, and I personally have no beef with it whatsoever. If he had deliberately stripped or crippled all docstrings from new_scipy to artificially create a need for his book, that would be a different story. But he certainly didn't do such a thing. I do think that there's a lot of room for higher-level books on python for scientific computing. Langtangen's is already out there, and Perry's excellent tutorial is already free for all to use (while slightly focused on astronomy, I find it a very useful document). I guess what I'm trying to say is that the _library_ documentation can be considered to be a (good) set of docstrings. As long as scipy ships with such a thing (which can be improved with the easiest of all patches by users, since no code is touched), I am not so worried that it will be perceived as lacking real docs. Please note that I am not discussing the price/time constraints of his licensing, which is strictly his choice to make. Again, this is just my _personal_ opinion. And while I have great appreciation for Travis, he's not paying me to say any of this :) Regards, f From arnd.baecker at web.de Fri Oct 7 00:12:54 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Fri Oct 7 00:12:54 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4345A449.8040303@colorado.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> Message-ID: On Thu, 6 Oct 2005, Fernando Perez wrote: [...] > To reiterate before my reputation is tarnished forever: the bet was with John, > who is the doughnuts-eating one. Being a man of taste, I eat bagels :) [and I > wasn't present when the bet was made] > > With that critical point out of the way, on to minor details. Thanks for the clarification - this definitively made my day ;-) [...] > In fact, I'd like docstrings to provide (simple) examples in them, +(some large positive number ;-) > and one thing on my radar now that > ipython is growing GUI/XML legs is to add support for latex in docstrings. Are you thinking of restructured text http://docutils.sourceforge.net/ together with the mathhack http://docutils.sourceforge.net/sandbox/cben/rolehack/ or the tools used in the LaTeXwiki, http://mcelrath.org/Notes/LatexWiki ? Independent of this, I see two main points: - how should the examples be added - who should add the examples to the doc-strings? Getting "end-users" to help with this task sounds quite attractive to me. Would it make sense to revive the pynotes idea http://www.scipy.org/documentation/mailman?fn=scipy-dev/2004-November/002536.html which would make it possible to add remarks to any python command? However, I don't know if there is a good solution on how to submit these notes and how they finally should get merged into the doc-string (One possibility would be to combine this in some way with Travis' live-docs). It might be unavoidable that at the end some knowledgable person has to go over suggested additions and incorporate them on a case-by-case basis? Best, Arnd From Chris.Barker at noaa.gov Fri Oct 7 00:14:45 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Fri Oct 7 00:14:45 2005 Subject: [Numpy-discussion] A comparative test In-Reply-To: References: <43457338.2010503@sympatico.ca> <200510062142.56692.faltet@carabos.com> Message-ID: <43461FF1.4010300@noaa.gov> Rick White wrote: > You might wonder why everyone didn't notice when their numarray > programs started to run blazingly fast after 1.3.0. Maybe because those of us for whom small array performance is important don't use numarray? I know I haven't tested my wxPython FloatCanvas with numarray since I discovered painfully slow performance a while back. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From perry at stsci.edu Fri Oct 7 05:50:21 2005 From: perry at stsci.edu (Perry Greenfield) Date: Fri Oct 7 05:50:21 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <200510061152.55708.faltet@carabos.com> Message-ID: Sorry for the tardy response. Transatlantic flights don't yet have convenient internet service. I've been meaning to put out a message of this sort soon anyway. This is few days earlier than I was planning, but close enough. Our basic position has been and continues to be that if scipy_core provides all the capabilities that we need, we will switch to it. We have been involved in installing it on the platforms we need it for (namely Solaris, which has proven difficult to build on) and testing it (which is ongoing). So far things look pretty promising and barring any unforseen problems, we will likely begin porting recarray to it next week. As Travis has already mentioned, the whole point of scipy_core was to unify the two existing projects, not introduce a 3rd. I have been involved all along in design and interface discussions with Travis on this project. We will maintain numarray until the point at which we switch all of our code to scipy_core (i.e., all libraries and applications that depend on it). This may take several months for some (likely the most work is for C extensions that use the numarray C-API). When this is complete (including testing), we may support numarray for a while in the sense of answering questions, but it would not likely see any more new releases after that point. We do not have the resources to support two different numeric packages indefinitely. We will report on our progress in this switchover (most likely our libraries and applications will be set up to work with both in the interim). While I appreciate the pain that this may cause those who have written C-extensions for numarray, it's hard to deny that unification will bring great benefits. It's as much pain for us as anyone. It should not be as painful to port python code since scipy_core should support all capabilities. But some changes will be needed since things are not always done the same way (e.g., dtype for type). Perry Francesc Altet wrote: > > Exactly, I'm wondering the same questions. Provided that numarray is > an excellent and mature piece of code, I'd like to hear a declaration > of principles on that subject. > > Regards, > > A Dijous 06 Octubre 2005 01:55, Colin J. Williams va escriure: > > Perry, > > > > It would be helpful if you gave some indication of your organization's > > intention with respect to numarray. > > > > Will it be maintained, as in the past, fully in the public domain and > > without charge for either the code or the API information needed to make > > use of the code? > > > > New versions have been produced every few months, will this continue? > > > > Colin W. > > > From usr01873 at prolocation.net Fri Oct 7 05:55:22 2005 From: usr01873 at prolocation.net (usr01873 at prolocation.net) Date: Fri Oct 7 05:55:22 2005 Subject: [Numpy-discussion] wrapping question Message-ID: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> Hi all, I'm using the numarray package. I would like to use the numarray C API to create a new array object and fill it with data without copying. Also, I would like numarray to take care of freeing the data once the numarray object is destructed. I've managed to find some related posts in the mailing list archive, but it is really unclear which of these posts are still accurate and what the current 'recommended' approach is. See: http://sourceforge.net/mailarchive/message.php?msg_id=11788304 http://sourceforge.net/mailarchive/message.php?msg_id=2494535 Currently, I first create a new PyArrayObject that is large enough to hold the data: array = NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims); then I pass array->data to a C function that reads the data from disk. Any comments on this approach? Also, I was wondering if numarray frees the data using this approach (because it also allocates the memory in the first place within the NA_vNewArray function.) Thanks, Joris van Zwieten From jmiller at stsci.edu Fri Oct 7 06:46:29 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 7 06:46:29 2005 Subject: [Numpy-discussion] wrapping question In-Reply-To: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> References: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> Message-ID: <43467B61.7080101@stsci.edu> usr01873 at prolocation.net wrote: >Hi all, > >I'm using the numarray package. I would like to use the numarray C API to >create a new array object and fill it with data without copying. Also, I > > The numarray C-API has a compatibility layer which supports most Numeric C-API functionality. So, using numarray, you can write most Numeric/Numeric3/scipy_core-like code in C. For historical reasons numarray also has a wider set of native APIs (i.e. NA_vNewArray()), but since numarray is likely to be replaced by scipy_core when it matures, it's wisest to use the compatibility API. >would like numarray to take care of freeing the data once the numarray >object is destructed. I've managed to find some related posts in the >mailing list archive, but it is really unclear which of these posts are >still accurate and what the current 'recommended' approach is. See: > >http://sourceforge.net/mailarchive/message.php?msg_id=11788304 >http://sourceforge.net/mailarchive/message.php?msg_id=2494535 > >Currently, I first create a new PyArrayObject that is large enough to hold >the data: >array = NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims); > >then I pass array->data to a C function that reads the data from disk. Any >comments on this approach? > That should work fine; 'array' owns the data and will free it when 'array' is destructed. A better approach, however, is to use numarray's PyArray_FromDims() which is the Numeric/Numeric3/scipy_core API. >Also, I was wondering if numarray frees the >data using this approach (because it also allocates the memory in the >first place within the NA_vNewArray function.) > > Yes, numarray will free the data allocated internally by NA_vNewArray(). numarray will also free the data allocated by PyArray_FromDims(). Regards, Todd From usr01873 at prolocation.net Fri Oct 7 07:27:28 2005 From: usr01873 at prolocation.net (usr01873 at prolocation.net) Date: Fri Oct 7 07:27:28 2005 Subject: [Numpy-discussion] wrapping question In-Reply-To: <43467B61.7080101@stsci.edu> References: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> <43467B61.7080101@stsci.edu> Message-ID: <52411.217.77.152.33.1128695128.squirrel@control.prolocation.net> > usr01873 at prolocation.net wrote: > >> Hi all, >> >> >> I'm using the numarray package. I would like to use the numarray C API >> to create a new array object and fill it with data without copying. >> Also, I >> >> >> > The numarray C-API has a compatibility layer which supports most Numeric > C-API functionality. So, using numarray, you can write most > Numeric/Numeric3/scipy_core-like code in C. For historical reasons > numarray also has a wider set of native APIs (i.e. NA_vNewArray()), but > since numarray is likely to be replaced by scipy_core when it matures, > it's wisest to use the compatibility API. > >> would like numarray to take care of freeing the data once the numarray >> object is destructed. I've managed to find some related posts in the >> mailing list archive, but it is really unclear which of these posts are >> still accurate and what the current 'recommended' approach is. See: >> >> http://sourceforge.net/mailarchive/message.php?msg_id=11788304 >> http://sourceforge.net/mailarchive/message.php?msg_id=2494535 >> >> >> Currently, I first create a new PyArrayObject that is large enough to >> hold the data: array = >> NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims); >> >> >> then I pass array->data to a C function that reads the data from disk. >> Any >> comments on this approach? >> > That should work fine; 'array' owns the data and will free it when > 'array' is destructed. > > > A better approach, however, is to use numarray's PyArray_FromDims() > which is the Numeric/Numeric3/scipy_core API. > >> Also, I was wondering if numarray frees the >> data using this approach (because it also allocates the memory in the >> first place within the NA_vNewArray function.) >> >> > Yes, numarray will free the data allocated internally by > NA_vNewArray(). numarray will also free the data allocated by > PyArray_FromDims(). Thanx for the reply! Just to be sure I understand this correctly: to conform to the Numeric/Numeric3/scipy api I could/should do the following: 1. allocate a new array using PyArray_FromDims() 2. pass the ->data pointer to a C routine that reads the data 3. use PyArray_Return to return the array to Python 4. No DECREF's are necessary? Also, PyArray_FromDims() will not copy the data, right? Thanks, Joris /* (From http://stsdas.stsci.edu/numarray/numarray-1.3.html/node58.html) PyObject* PyArray_FromDims(int nd, int *dims, int type) This function will allocate a new numarray. An array created with PyArray_FromDims can be used as a temporary or returned using PyArray_Return. Used as a temporary, calling Py_DECREF deallocates it. */ > > Regards, > Todd > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > From jmiller at stsci.edu Fri Oct 7 07:51:50 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 7 07:51:50 2005 Subject: [Numpy-discussion] wrapping question In-Reply-To: <52411.217.77.152.33.1128695128.squirrel@control.prolocation.net> References: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> <43467B61.7080101@stsci.edu> <52411.217.77.152.33.1128695128.squirrel@control.prolocation.net> Message-ID: <43468AED.1030403@stsci.edu> usr01873 at prolocation.net wrote: >>usr01873 at prolocation.net wrote: >> >> >> >>>Hi all, >>> >>> >>>I'm using the numarray package. I would like to use the numarray C API >>>to create a new array object and fill it with data without copying. >>>Also, I >>> >>> >>> >>> >>> >>The numarray C-API has a compatibility layer which supports most Numeric >>C-API functionality. So, using numarray, you can write most >>Numeric/Numeric3/scipy_core-like code in C. For historical reasons >>numarray also has a wider set of native APIs (i.e. NA_vNewArray()), but >>since numarray is likely to be replaced by scipy_core when it matures, >>it's wisest to use the compatibility API. >> >> >> >>>would like numarray to take care of freeing the data once the numarray >>>object is destructed. I've managed to find some related posts in the >>>mailing list archive, but it is really unclear which of these posts are >>> still accurate and what the current 'recommended' approach is. See: >>> >>>http://sourceforge.net/mailarchive/message.php?msg_id=11788304 >>>http://sourceforge.net/mailarchive/message.php?msg_id=2494535 >>> >>> >>>Currently, I first create a new PyArrayObject that is large enough to >>>hold the data: array = >>>NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims); >>> >>> >>>then I pass array->data to a C function that reads the data from disk. >>>Any >>>comments on this approach? >>> >>> >>> >>That should work fine; 'array' owns the data and will free it when >>'array' is destructed. >> >> >>A better approach, however, is to use numarray's PyArray_FromDims() >>which is the Numeric/Numeric3/scipy_core API. >> >> >> >>>Also, I was wondering if numarray frees the >>>data using this approach (because it also allocates the memory in the >>>first place within the NA_vNewArray function.) >>> >>> >>> >>> >>Yes, numarray will free the data allocated internally by >>NA_vNewArray(). numarray will also free the data allocated by >>PyArray_FromDims(). >> >> > >Thanx for the reply! Just to be sure I understand this correctly: to >conform to the Numeric/Numeric3/scipy api I could/should do the following: > >1. allocate a new array using PyArray_FromDims() >2. pass the ->data pointer to a C routine that reads the data >3. use PyArray_Return to return the array to Python >4. No DECREF's are necessary? > >Also, PyArray_FromDims() will not copy the data, right? >Thanks, > > 1. yes 2. yes 3. PyArray_Return is mostly important if you want rank-0 arrays converted into equivalent Python scalars. Otherwise I think it's not really needed and is commonly omitted. 4. When you call PyArray_FromDims(), the returned array has one reference. If the array is a temporary you want to deallocate at extension function exit, then DECREF; if you want to return the array, don't DECREF. Todd >Joris > >/* >(From http://stsdas.stsci.edu/numarray/numarray-1.3.html/node58.html) >PyObject* PyArray_FromDims(int nd, int *dims, int type) > This function will allocate a new numarray. > An array created with PyArray_FromDims can be used as a temporary or > returned using PyArray_Return. > > Used as a temporary, calling Py_DECREF deallocates it. >*/ > > > >>Regards, >>Todd >> >> >> >>------------------------------------------------------- >>This SF.Net email is sponsored by: >>Power Architecture Resource Center: Free content, downloads, discussions, >>and more. http://solutions.newsforge.com/ibmarch.tmpl >>_______________________________________________ >>Numpy-discussion mailing list >>Numpy-discussion at lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> >> >> > > > > >------------------------------------------------------- >This SF.Net email is sponsored by: >Power Architecture Resource Center: Free content, downloads, discussions, >and more. http://solutions.newsforge.com/ibmarch.tmpl >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion at lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > From faltet at carabos.com Fri Oct 7 08:07:40 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 7 08:07:40 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: References: Message-ID: <200510071704.55100.faltet@carabos.com> A Divendres 07 Octubre 2005 14:51, Perry Greenfield va escriure: > Our basic position has been and continues to be that if scipy_core provides > all the capabilities that we need, we will switch to it. We have been > involved in installing it on the platforms we need it for (namely Solaris, > which has proven difficult to build on) and testing it (which is ongoing). > So far things look pretty promising and barring any unforseen problems, we > will likely begin porting recarray to it next week. As Travis has already > mentioned, the whole point of scipy_core was to unify the two existing > projects, not introduce a 3rd. I have been involved all along in design and > interface discussions with Travis on this > project. Ok. That's clear enough and satifies my curiosity, thank you. I must confess, though, that I'm a bit disappointed becuase I was hoping that numarray, now that has arrived to maturity, was going to stay for long time (as any other good piece of software deserves). Anyway, as a long-time user of numarray I can't express enough my gratitude towards you and your team for suporting and enhancing a beast like numarray, that introduced a lot of new features that missed Numeric and that without it all of our current projects (and I guess many others around the world) simply would not have been possible. > We will maintain numarray until the point at which we switch all of our > code to scipy_core (i.e., all libraries and applications that depend on > it). This may take several months for some (likely the most work is for C > extensions that use the numarray C-API). Yeah, I do think so. However, what it worries me is, in my (limited) experience, that substituting large packages like Numeric or numarray, is not a trivial task at all (after all, how much time took until numarray was able to be a good substitute of Numeric?). So, I forsee a transitional time no shorter than a year before arriving to the high standards of quality that numarray has nowadays. This is why I'm not precisely eager to quickly switch to scipy_core until it has proven to be, at least, as solid as it is numarray. Having said that, we are in the mood of offering our implementation of a NestedRecArray class that allows nested fields in RecArray objects. This class has been included in PyTables for some months and has proven to be stable, comes with a nice set of unit tests, and is highly efficient (at least, as much as RecArray for most operations). Also, until ready-for-production scipy_core arrives, I'd ask you (and the maintainers of scipy_core) to provide an array protocol to share data as efficiently as possible between numarray and scipy_core, just to easy the transition between the packages. If you want, we will be happy to collaborate on that area as well. > While I appreciate the pain that this may cause those who have written > C-extensions for numarray, it's hard to deny that unification will bring > great benefits. It's as much pain for us as anyone. Sure, but migration work is not the biggest problem (hopefully) for us. As I said before, the key point to migrate to scipy_core is to make sure that is stable enough, featured enough, coner-cases-proof enough and well-documented enough as it is numarray now. Thanks again for all your work in numarray, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From oliphant at ee.byu.edu Fri Oct 7 10:00:51 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri Oct 7 10:00:51 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <200510071704.55100.faltet@carabos.com> References: <200510071704.55100.faltet@carabos.com> Message-ID: <4346A954.5000400@ee.byu.edu> Francesc Altet wrote: >Also, until ready-for-production scipy_core arrives, I'd ask you (and >the maintainers of scipy_core) to provide an array protocol to share >data as efficiently as possible between numarray and scipy_core, just >to easy the transition between the packages. If you want, we will be >happy to collaborate on that area as well. > > There is an array protocol in place already (see the Array Interface section on numeric.scipy.org). I think it is fully supported in numarray. Also, I have a PEP for placing a simple array object in Python that is nothing more than a shell for passing data around. If somebody would like to push that forward they are welcome. Right now what I have is available in an SVN server. http://svn.scipy.org/svn/PEP get it with svn co http://svn.scipy.org/svn/PEP PEP -Travis From faltet at carabos.com Fri Oct 7 10:07:06 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 7 10:07:06 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <4346A4ED.80404@sympatico.ca> References: <200510071704.55100.faltet@carabos.com> <4346A4ED.80404@sympatico.ca> Message-ID: <200510071905.25174.faltet@carabos.com> A Divendres 07 Octubre 2005 18:40, Colin J. Williams va escriure: > I share these feelings and would like to express my thanks to Todd > Miller and to Perry Greenfield for their help over the years. Of course, I did not explicitely mention Todd in my previous message, but I would like to note that he has always been extremely responsive to suggestions and very efficient in solving many, many issues that appeared in numarray through the years. A big thanks to you as well, Todd :-) -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From jmiller at stsci.edu Fri Oct 7 11:39:42 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 7 11:39:42 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <200510071905.25174.faltet@carabos.com> References: <200510071704.55100.faltet@carabos.com> <4346A4ED.80404@sympatico.ca> <200510071905.25174.faltet@carabos.com> Message-ID: <4346C097.3020703@stsci.edu> Francesc Altet wrote: >A Divendres 07 Octubre 2005 18:40, Colin J. Williams va escriure: > > >>I share these feelings and would like to express my thanks to Todd >>Miller and to Perry Greenfield for their help over the years. >> >> > >Of course, I did not explicitely mention Todd in my previous message, >but I would like to note that he has always been extremely responsive >to suggestions and very efficient in solving many, many issues that >appeared in numarray through the years. > >A big thanks to you as well, Todd :-) > > Ah... you're welcome! Working on numarray has been a great privilege. Regards, Todd From cjw at sympatico.ca Fri Oct 7 11:49:28 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri Oct 7 11:49:28 2005 Subject: [Numpy-discussion] Numeric3.0 - unable to retrieve data typecode Message-ID: <4346C2BC.8020109@sympatico.ca> # temp.py import scipy.base as num3 a= num3.array([[1, 2], [9, 8]]) print a.typecode C:\>python temp.py Traceback (most recent call last): File "temp.py", line 4, in ? print a.typecode AttributeError: 'scipy.ndarray' object has no attribute 'typecode' Is there any simple way to retrieve the dataType, as with numarray's numerictypes? Colin W. From cjw at sympatico.ca Fri Oct 7 13:20:24 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri Oct 7 13:20:24 2005 Subject: [Numpy-discussion] Problem with name numeric Message-ID: <4346D825.70309@sympatico.ca> With the following sequence I had expected "scipy.base.numeric" to be a module, instead it is reported as a type. Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy.base.numeric >>> dir() ['__builtins__', '__doc__', '__name__', 'scipy'] >>> scipy >>> scipy.base >>> scipy.base.numeric >>> I need to be able to access ndarray to create a subclass. Colin W. From mantqygk at donahue.com Fri Oct 7 18:34:22 2005 From: mantqygk at donahue.com (Fa Mantz) Date: Fri Oct 7 18:34:22 2005 Subject: [Numpy-discussion] Re: Meaghan exhibitionism Message-ID: <003001c5cba8$5ae79d80$9b1ea8c0@pouter> Hi Do you want t VE UP n your Med o SA TO 70% o dications? It's easy with us - The WebSite VCXAVL iagialanabiealiitr ra $is $xnum $a 134 (30 p.)169 (30 p.) 218 (180 p.) And much more , Good luck -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjw at sympatico.ca Sat Oct 8 07:31:49 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Sat Oct 8 07:31:49 2005 Subject: [Numpy-discussion] Problem with Windows version of Numeric3.0 Message-ID: <4347D83C.10002@sympatico.ca> Either of these import lines crashes in Pythonw, with Boa-constructor, SPE or PythonWin. # subclass.py import scipy.base as num3 import scipy.base.multiarray as M Colin W. From cjw at sympatico.ca Sat Oct 8 09:44:34 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Sat Oct 8 09:44:34 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray Message-ID: <4347F755.2080304@sympatico.ca> With a view to exploring Numeric3, below is a script to check the availability of doc strings. Some are not yet available. There is a method __class__ module but the purpose is not clear. Colin W. # checkDocs.py ''' To check the availabilty of doc strings for the ndarray class. Among the attributes of ndarray is: __class__ module(name[, doc]) Create a module object. The name must be a string; the optional doc argument can have any type. ndarray is referenced as a class and reported as "" but the common Python usage is see the last output line. ''' import scipy.base.multiarray as M print type(M.ndarray) print '>', type( M) print 'module doc:', M.__doc__ for i in dir(M.ndarray): print i, try: print eval('M.'+i+'.__doc__') except: print 'No doc' print '\n', M.ndarray.__doc__ print type(M.ndarray) # Check normal usage import timeit print timeit.Timer, type(timeit.Timer) From oliphant at ee.byu.edu Sat Oct 8 12:48:07 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat Oct 8 12:48:07 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <4347F755.2080304@sympatico.ca> References: <4347F755.2080304@sympatico.ca> Message-ID: <4348224A.3030006@ee.byu.edu> Colin J. Williams wrote: > With a view to exploring Numeric3, below is a script to check the > availability of doc strings. > > Some are not yet available. > I don't think your script will pick up the docs for the ndarray methods. You need to replace eval('M.'+i+'.__doc__') with eval('M.ndarray.'+i+'.__doc__') -Travis From oliphant at ee.byu.edu Sat Oct 8 12:55:20 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat Oct 8 12:55:20 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <4347F755.2080304@sympatico.ca> References: <4347F755.2080304@sympatico.ca> Message-ID: <4348240D.6010308@ee.byu.edu> Colin J. Williams wrote: > With a view to exploring Numeric3, below is a script to check the > availability of doc strings. > > Some are not yet available. > > There is a method __class__ module but the purpose is not clear. > > Colin W. > > # checkDocs.py > ''' To check the availabilty of doc strings for the ndarray class. > > Among the attributes of ndarray is: > __class__ module(name[, doc]) I'm not sure what you are talking about here? Where are you getting this? I don't know of any such attribute. There is a __class__ attribute. But I don't see a __class__ module(name[, doc]) attribute -Travis From oliphant at ee.byu.edu Sat Oct 8 21:50:10 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat Oct 8 21:50:10 2005 Subject: [Numpy-discussion] Another PEP needed for the Numeric community. Message-ID: <4348A153.5050405@ee.byu.edu> Full, 64-bit support in Python is problematic because of the buffer and sequence protocols which require int's in their interfaces. Right now, numarray support memory mapped arrays. SciPy can easily support memory mapped arrays by modifying the same module that numarray has forged (using the frombuffer function). However, the module will be limited on 64-bit systems because memory-map support in Python is limited to 32-bit files even on 64-bit systems. This was an intentional limitation of the memory map module in Python so that the sequence and buffer protocols could be supported. However, the choice to not even allow a 64-bit size memory map to be created seems wrong. There is no need to go through the sequence and buffer interface at all times. If the memory-map module in Python were re-written to inherit from a big-map object that did not export the buffer and sequence protocols (or did so in a limited fashion), then all that is needed is for the object to export it's data pointer and size. Then a function could be written to create an array in scipy that used the mmap's exported buffer as its data region. This would allow very big memory mapped arrays without waiting for Python to fix it's sequence and buffer protocols (which may not be fixed until Python 3.0). Thus, a PEP stating how the mmap module should change to support 64-bit systems is needed. Another possible project idea for any lurking people desiring to help. The other possibility is to just copy the nice muliplatform code in the Python mmap module and use an ndarray as the memory-mapped object. -Travis From cjw at sympatico.ca Sun Oct 9 10:54:43 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Sun Oct 9 10:54:43 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <4348240D.6010308@ee.byu.edu> References: <4347F755.2080304@sympatico.ca> <4348240D.6010308@ee.byu.edu> Message-ID: <43495948.9020402@sympatico.ca> Travis Oliphant wrote: > Colin J. Williams wrote: > >> With a view to exploring Numeric3, below is a script to check the >> availability of doc strings. >> >> Some are not yet available. >> >> There is a method __class__ module but the purpose is not clear. >> >> Colin W. >> >> # checkDocs.py >> ''' To check the availabilty of doc strings for the ndarray class. >> >> Among the attributes of ndarray is: >> __class__ module(name[, doc]) > > > > I'm not sure what you are talking about here? Where are you getting > this? > > > I don't know of any such attribute. There is a __class__ attribute. > But I don't see a > > __class__ module(name[, doc]) attribute > > -Travis > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > It might have been clearer if I had referred to the name "__class__" in the multiarray module. Please see the sequence below: >>> import scipy.base.multiarray as M >>> M.__class__ >>> I am surprised that this name points to a module. It usually is an attribute of a class instance which points to the class object of that instance. Colin W. From matthew.brett at gmail.com Sun Oct 9 13:07:46 2005 From: matthew.brett at gmail.com (Matthew Brett) Date: Sun Oct 9 13:07:46 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4345A449.8040303@colorado.edu> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> Message-ID: <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> Hi, > While I fully respect the opinion of those concerned about Travis' decision, I > have to say that I am personally not so worried. At scipy'05, when Travis > announced the book idea, I asked a very simple question: 'are the docstrings > complete?'. His answer was 'yes'. > > There was a good reason for my asking this question: I personally find that > when coding, the most productive and efficient to learn a library is by typing > > import foo > foo. > > and then > > foo.bar? Just a tiny addition. I know what you mean, but my experience is of someone recently trying to learn numarray / Numeric, and I suspect I am not alone in printing out the documentation and reading a moderate amount of it before I get started. Call me old-fashioned (it wouldn't be the first time!), See you, Matthew From Fernando.Perez at colorado.edu Sun Oct 9 16:19:30 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Sun Oct 9 16:19:30 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> Message-ID: <4349A524.40208@colorado.edu> Matthew Brett wrote: >>While I fully respect the opinion of those concerned about Travis' decision, I >>have to say that I am personally not so worried. At scipy'05, when Travis >>announced the book idea, I asked a very simple question: 'are the docstrings >>complete?'. His answer was 'yes'. >> >>There was a good reason for my asking this question: I personally find that >>when coding, the most productive and efficient to learn a library is by typing >> >>import foo >>foo. >> >>and then >> >>foo.bar? > > > Just a tiny addition. I know what you mean, but my experience is of > someone recently trying to learn numarray / Numeric, and I suspect I > am not alone in printing out the documentation and reading a moderate > amount of it before I get started. Call me old-fashioned (it > wouldn't be the first time!), Certainly. As I said, I do think there is room for 'book-style' (as opposed to API reference) books for scipy. Langtangen's ($$$) and Perry's (free) are two such existing offers, and now Travis' comes in as well (and I still believe there is room for more). My idea was of top-level (module) docstrings which would provide a reasonable overview, along with single-function ones providing not only API reference but also a few examples. Since pydoc -w can generate permanent HTML for browsing/printing out of any module (and docutils has even more sophisticated facilities), I think this provides acceptable coverage of the basic library. And it does give anyone who wants 'material to read on the bus' (which I often need myself) a reasonable solution, I think. I just wanted to clarify that a docstring-based set of docs is not limited either to interactive usage via ipython, nor to raw API information. It can both be printed for offline use, and can cover enough overview and examples to be genuinely useful standalone. Not a substitute for a full book, but not a crippled tool either, I think. Cheers, f From tchur at optusnet.com.au Sun Oct 9 16:56:38 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Sun Oct 9 16:56:38 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4349A524.40208@colorado.edu> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> <4349A524.40208@colorado.edu> Message-ID: <4349ADD4.7060701@optusnet.com.au> Fernando Perez wrote: > Certainly. As I said, I do think there is room for 'book-style' (as > opposed to API reference) books for scipy. Langtangen's ($$$) and > Perry's (free) are two such existing offers, and now Travis' comes in as > well (and I still believe there is room for more). > > My idea was of top-level (module) docstrings which would provide a > reasonable overview, along with single-function ones providing not only > API reference but also a few examples. Since pydoc -w can generate > permanent HTML for browsing/printing out of any module (and docutils has > even more sophisticated facilities), I think this provides acceptable > coverage of the basic library. And it does give anyone who wants > 'material to read on the bus' (which I often need myself) a reasonable > solution, I think. > > I just wanted to clarify that a docstring-based set of docs is not > limited either to interactive usage via ipython, nor to raw API > information. It can both be printed for offline use, and can cover > enough overview and examples to be genuinely useful standalone. Not a > substitute for a full book, but not a crippled tool either, I think. Sounds like the ideal solution to me. Now, I wonder if it is possible to resolve the legal situation over the existing NumPy documentation? By which I mean, I wonder if it is possible to just copy suitable parts of the NumPy docs into the scipy.core docstrings (improving on them if possible) - with attribution of course. Or is it necessary or safer to start from scratch, or at least to extensively paraphrase the NumPy docs? Finally, to whom should we send docstrings? To Travis? Tim C From oliphant at ee.byu.edu Sun Oct 9 17:07:21 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sun Oct 9 17:07:21 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <43495948.9020402@sympatico.ca> References: <4347F755.2080304@sympatico.ca> <4348240D.6010308@ee.byu.edu> <43495948.9020402@sympatico.ca> Message-ID: <4349B055.8060006@ee.byu.edu> >> I'm not sure what you are talking about here? Where are you getting >> this? >> >> >> I don't know of any such attribute. There is a __class__ attribute. >> But I don't see a >> >> __class__ module(name[, doc]) attribute >> >> -Travis >> > > Please see the sequence below: > > >>> import scipy.base.multiarray as M > >>> M.__class__ > > >>> > > I am surprised that this name points to a module. It usually is an > attribute of a class instance which > points to the class object of that instance. Why is this surprising? scipy.base.multiarray is a module, therefore it's "class" is type 'module'. This seems fine to me. At any rate, Python is assigning the __class__ attribute to the extension module multiarray, so it is what it is. -Travis From greg.ewing at canterbury.ac.nz Sun Oct 9 19:39:50 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun Oct 9 19:39:50 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <43495948.9020402@sympatico.ca> References: <4347F755.2080304@sympatico.ca> <4348240D.6010308@ee.byu.edu> <43495948.9020402@sympatico.ca> Message-ID: <4349D442.6050507@canterbury.ac.nz> Colin J. Williams wrote: > >>> import scipy.base.multiarray as M > >>> M.__class__ > > >>> > > I am surprised that this name points to a module. is NOT a module, it's the type (or class) to which all modules belong. This is to be expected if scipy.base.multiarray is itself a module. The reason it says and not is because it's a built-in type/class rather than one defined in Python. But ever since the type/class unification, there's very little difference in meaning between the terms 'type' and 'class'. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From oliphant at ee.byu.edu Sun Oct 9 21:28:33 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sun Oct 9 21:28:33 2005 Subject: [Numpy-discussion] Question about 64-bit integers being cast to double precision Message-ID: <4349EDA2.2090802@ee.byu.edu> What is the opinion of people here regarding the casting of 64-bit integers to double precision. In scipy (as in Numeric), there is the concept of "Casting safely" to a type. This concept is used when choosing a ufunc, for example. My understanding is that a 64-bit integer cannot be cast safely to a double-precision floating point number, because precision is lost in the conversion. However, at least a signed 64-bit integer can usually be cast safely to a long double precision floating point number. This is not too big a deal on 32-bit systems where people rarely request 64-bit integers. However, on some 64-bit systems (where the C long is 64-bit), Python's default integer is 64-bit. Therefore, simple expressions like sqrt(2) which require conversion to floating point will look for the first floating point number that it can convert a 64-bit integer to safely. This can only be a long double. The result is that on 64-bit systems, the long double type gets used a lot more. Is this acceptable? expected? What do those of you on 64-bit systems think? -Travis From Chris.Barker at noaa.gov Sun Oct 9 21:38:25 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Sun Oct 9 21:38:25 2005 Subject: [Numpy-discussion] Can you enumerarte an array? Message-ID: <4349F00C.7080605@noaa.gov> Hi all, A freind of mine that I jsut introduced to NumPy asked me about enumerating an array. What he'd like to see is something like: A = N.ones((3,4)) for indexes, item in enumeate(A): print indexes result in: (0,0) (0,1) (0,2) (1,0) . . . You get the idea. Can you do this with any of the three NumPys? If not consider this a feature request for scipy_base. -Chris From arnd.baecker at web.de Sun Oct 9 23:39:01 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Sun Oct 9 23:39:01 2005 Subject: [Numpy-discussion] Can you enumerarte an array? In-Reply-To: <4349F00C.7080605@noaa.gov> References: <4349F00C.7080605@noaa.gov> Message-ID: Hi Chris, On Sun, 9 Oct 2005, Chris Barker wrote: > Hi all, > > A freind of mine that I jsut introduced to NumPy asked me about > enumerating an array. What he'd like to see is something like: > > A = N.ones((3,4)) > > for indexes, item in enumeate(A): > print indexes > > result in: > > (0,0) > (0,1) > (0,2) > (1,0) > . > . > . > You get the idea. Can you do this with any of the three NumPys? Not that I know: import Numeric as N A = N.ones((3,4)) for indizes, item in enumerate(A): print indizes,item gives 0 [1 1 1 1] 1 [1 1 1 1] 2 [1 1 1 1] (the same applies for numarrary and the new scipy_core) It might be the best to define your own iterator (e.g. `enumerate_ind`) because the present behaviour can be useful as well. > If not > consider this a feature request for scipy_base. Best, Arnd From cjw at sympatico.ca Mon Oct 10 04:23:56 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Mon Oct 10 04:23:56 2005 Subject: [Numpy-discussion] Access to online documeation - was Purchasing Documentation In-Reply-To: <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> Message-ID: <434A4EF1.90004@sympatico.ca> Matthew Brett wrote: >Hi, > > > >>While I fully respect the opinion of those concerned about Travis' decision, I >>have to say that I am personally not so worried. At scipy'05, when Travis >>announced the book idea, I asked a very simple question: 'are the docstrings >>complete?'. His answer was 'yes'. >> >>There was a good reason for my asking this question: I personally find that >>when coding, the most productive and efficient to learn a library is by typing >> >>import foo >>foo. >> >>and then >> >>foo.bar? >> >> > >Just a tiny addition. I know what you mean, but my experience is of >someone recently trying to learn numarray / Numeric, and I suspect I >am not alone in printing out the documentation and reading a moderate >amount of it before I get started. Call me old-fashioned (it >wouldn't be the first time!), > >See you, > >Matthew > > > > Could someone elaborate on the remark below please?: There was a good reason for my asking this question: I personally find that when coding, the most productive and efficient to learn a library is by typing import foo foo. and then foo.bar? Is this with some particular IDE? Using Windows, I don't get this. This sort of help is available with PythonWin but Numeric3 and pythonw seem to be mutually incompatible at this stage. Colin W. From cjw at sympatico.ca Mon Oct 10 04:53:30 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Mon Oct 10 04:53:30 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <4349B055.8060006@ee.byu.edu> References: <4347F755.2080304@sympatico.ca> <4348240D.6010308@ee.byu.edu> <43495948.9020402@sympatico.ca> <4349B055.8060006@ee.byu.edu> Message-ID: <434A55B7.5010900@sympatico.ca> Travis Oliphant wrote: > >>> I'm not sure what you are talking about here? Where are you getting >>> this? >>> >>> >>> I don't know of any such attribute. There is a __class__ >>> attribute. But I don't see a >>> >>> __class__ module(name[, doc]) attribute >>> >>> -Travis >>> >> >> Please see the sequence below: >> >> >>> import scipy.base.multiarray as M >> >>> M.__class__ >> >> >>> >> >> I am surprised that this name points to a module. It usually is an >> attribute of a class instance which >> points to the class object of that instance. > > > Why is this surprising? scipy.base.multiarray is a module, therefore > it's "class" is type 'module'. This seems fine to me. At any rate, > Python is assigning the __class__ attribute to the extension module > multiarray, so it is what it is. > > -Travis > Yes, this is the Python practice: >>> import timeit >>> timeit.__class__ >>> import anydbm >>> anydbm.__class__ >>> Yes, timeit.__class__ points to the internal module creator function: >>> import new >>> new.module >>> new.module.__doc__ 'module(name[, doc])\n\nCreate a module object.\nThe name must be a string; the optional doc argumen t can have any type.' >>> new.module is timeit.__class__ True >>> With a little digging, this is logical, consistent and no longer surprising. Colin W. From kosh at skynet.be Mon Oct 10 10:26:00 2005 From: kosh at skynet.be (Kosta Gaitanis) Date: Mon Oct 10 10:26:00 2005 Subject: [Numpy-discussion] Problems compiling ufuncs example In-Reply-To: <20051010170845.6C9A7FFCB@sc8-sf-spam2.sourceforge.net> References: <20051010170845.6C9A7FFCB@sc8-sf-spam2.sourceforge.net> Message-ID: <434AA3D2.80105@skynet.be> Hi all, I'm trying to run the UFunc example in the Numarray manual (http://stsdas.stsci.edu/numarray/numarray-1.1.html/node32.html) but when I run the command python setup.py buil_ext I get the following output : running build_ext error: Python was built with version 7.1 of Visual Studio, and extensions need t o be built with the same version of the compiler, but it isn't installed so I installed Visual C++ Toolkit 2003 and then run the same command again and got the following output : running build_ext error: the .NET Framework SDK needs to be installed before building extensions f or Python so I installed .NET Framework SDK 1.1 and then tried again. I got back the first error... does anyone have an idea ? What am I doing wrong ? thanks in advance Kosta I'm using Python24 and NumArray 1.3.3 -- setup.py -- """ This setup demonstrates how to use the numarray code generation facilities to define additional ufuncs at the user level. Universal functions apply operators or C-functions elementwise to all of the elements of an array or pair of arrays. This setup generates code defining three universal functions which are installed as the numarray.foo package: Cos, bessel, and bar. The ufuncs are used like this: >>> import numarray as na >>> import numarray.foo as foo >>> a = na.array([1,2,3,4], type='Int32') >>> foo.Cos(a) array([ 0.54030228, -0.41614684, -0.9899925 , -0.65364361 ], type=Float32) Note that since a is an Int32 array, Cos(a) first does blockwise conversion of a to Float32 before calling the cosine library function. """ import distutils, os from distutils.core import setup from numarray.numarrayext import NumarrayExtension from numarray.codegenerator import UfuncModule, make_stub # ====================================================================== # # Generate the C-code for the numarray.foo._foo extension: # m = UfuncModule("_foo") # Inline the code for a couple C functions to be turned into ufuncs. # This method lets you define your function here in the setup.py. An # alternate approach would be to link with a libarary or additional C # module. m.add_code(""" static double c_bessel(double x, double y) { double x2=x*x, y2=y*y, diff=x2-y2; return diff*diff/(x2+y2); } static UInt8 c_bar(UInt32 x ) { return (x >> 24) & 0xFF; } """) # Method add_ufunc() defines the name of a ufunc, it's corresponding C # function which is applied element-wise to all elements of an array, # The arity of the ufunc (unary or binary only), and the I/O type # signatures which are directly supported by the ufunc. Binary Ufunc # "bessel" is implemented by the inline function "c_bessel" from the # add_code() call above. m.add_ufunc("bessel", "c_bessel", 2, [('Float32', 'Float32'), ('Float64', 'Float64')]) # Unary Ufunc "bar" only supports one builtin type signature: # UInt32-->UInt8. Other types are blockwise converted by the ufunc # machinery to UInt32 before "c_bar" is called. m.add_ufunc("bar", "c_bar", 1, [('UInt32','UInt8')]) # Unary Ufunc "cos" is implemented by the C standard library function # "cos". Given a Float32 array, it returns a Float32 array. Given a # Float64 array, it returns a Float64 array. For other arrays, # transparent conversion to one of the supported input types is performed # by the ufunc machinery. m.add_ufunc("cos", "cos", 1, [('Float32', 'Float32'), ('Float64', 'Float64')]) # The generate() method writes out the complete extension module to the # specified C file. m.generate("foo/Src/_foo.c") # ====================================================================== # Create foo's __init__.py for defining UFuncs corresponding to CFuncs # in _foo. make_stub() emits boiler-plate which makes your extension # a package. The __init__ file imports all the public symbols from # C extension _foo making them visible as numarray.foo. make_stub("foo/Lib/__init__", "_foo") # ====================================================================== # # Standard distutils setup for the generated code. # setup(name = "foo", version = "0.1", maintainer = "Todd Miller", maintainer_email = "jmiller at stsci.edu", description = "foo universal functions for numarray", url = "http://www.stsci.edu/projects/foo/", packages = ["numarray.foo"], package_dir = { "numarray.foo":"foo/Lib" }, ext_modules = [ NumarrayExtension( 'numarray.foo._foo', ['foo/Src/_foo.c'], # libraries = ['ttf'], # include_dirs = ['include'], # library_dirs = ['lib'] ) ] ) From efiring at hawaii.edu Mon Oct 10 10:34:09 2005 From: efiring at hawaii.edu (Eric Firing) Date: Mon Oct 10 10:34:09 2005 Subject: [Numpy-discussion] Access to online documeation - was Purchasing Documentation In-Reply-To: <434A4EF1.90004@sympatico.ca> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> <434A4EF1.90004@sympatico.ca> Message-ID: <434AA585.4090900@hawaii.edu> Colin, The tab completion works on Linux from the standard python prompt and from ipython; the "?" syntax is added by ipython (http://ipython.scipy.org/). Eric > Could someone elaborate on the remark below please?: > > There was a good reason for my asking this question: I personally find > that > when coding, the most productive and efficient to learn a library is by > typing > > import foo > foo. > > and then > > foo.bar? > > > Is this with some particular IDE? Using Windows, I don't get this. From Chris.Barker at noaa.gov Mon Oct 10 11:37:55 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 10 11:37:55 2005 Subject: [Numpy-discussion] Can you enumerarte an array? In-Reply-To: <434A952F.7030909@ee.byu.edu> References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> Message-ID: <434AB4E0.9040607@noaa.gov> Travis Oliphant wrote: > All flavors of Numeric would return > > 0 a[0] > 1 a[1] > . > . > . > n-1 a[n-1] > > where n=a.shape[0] > > in response to index, item in enumerate(A): Right, as expected, and as Anrd pointed out, this is useful behaviour. > In scipy you could also get > > 0 a.flat[0] > 1 a.flat[1] I do like the new flat! > To do something like he wants, I think we would have to construct a > different iterator then enumerate. Exactly,. I didn't mean to imply that that's what the built-in enumerate should do. > It wouldn't be that hard using the > a.flat iterator (it's just a remapping of the 1-d index). Cool. then again, consider this a feature request, for a nd_enumerate, or whatever you want to call it. I'm sorry that implimenting something like this myself is way out my depth! -Chris From jochen at fhi-berlin.mpg.de Mon Oct 10 11:58:41 2005 From: jochen at fhi-berlin.mpg.de (=?iso-8859-1?Q?Jochen_K=FCpper?=) Date: Mon Oct 10 11:58:41 2005 Subject: NumPies future (was: [Numpy-discussion] Re: Purchasing Documentation) In-Reply-To: (Perry Greenfield's message of "Fri, 7 Oct 2005 08:51:41 -0400") References: Message-ID: All, following all this up and down I decide to write a short email for two reasons: First of all I want to thank all developers of Numeric, numarray, and the yet to fly scipy_core, esp. Perry/Todd and Travis. I hope that this apparent joining of forces will really get Numeric Python a big step ahead. On the other hand I have personally stepped down from using Python for larger numerical calculation projects, because it was too much of an hassle to get sysops convinced to install it, and because overall the hassles to follow releases, adopting code and explaining colleagues was to big to make up for the ease of coding compared to C++. Yep, there we are. I am REALLY looking forward to the day when I can start a larger project in Python again. I wish and hope that you guys are paving the road for me;) [1] Using scipy_core I would also be happy to work on public documentation, as that is where everybody using scipy_core can easily contribute his share. Maybe it would be a nice team effort to get the numarray Python-style documentation cut down and updated to a good and current reference-manual, pretty much a nicer and cross-linked version of the doc-strings, maybe with more of the examples that were discussed here before. I am sure that Travis writes a good user-manual, and, voila, both would be in the right place! To all the guys actually doing the work: Thank you very much, nevertheless! Greetings, Jochen -- Einigkeit und Recht und Freiheit http://www.Jochen-Kuepper.de Libert?, ?galit?, Fraternit? GnuPG key: CC1B0B4D (Part 3 you find in my messages before fall 2003.) From oliphant at ee.byu.edu Mon Oct 10 14:40:50 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon Oct 10 14:40:50 2005 Subject: [Numpy-discussion] Re: NumPies future In-Reply-To: References: Message-ID: <434ADF7A.6070300@ee.byu.edu> Jochen K?pper wrote: >All, > >following all this up and down I decide to write a short email for two >reasons: > >First of all I want to thank all developers of Numeric, numarray, and >the yet to fly scipy_core, esp. Perry/Todd and Travis. I hope that >this apparent joining of forces will really get Numeric Python a big >step ahead. > >On the other hand I have personally stepped down from using Python for >larger numerical calculation projects, because it was too much of an >hassle to get sysops convinced to install it, and because overall the >hassles to follow releases, adopting code and explaining colleagues >was to big to make up for the ease of coding compared to C++. Yep, >there we are. I am REALLY looking forward to the day when I can start >a larger project in Python again. I wish and hope that you guys are >paving the road for me;) [1] > >Using scipy_core I would also be happy to work on public >documentation, as that is where everybody using scipy_core can easily >contribute his share. Maybe it would be a nice team effort to get the >numarray Python-style documentation cut down and updated to a good and >current reference-manual, pretty much a nicer and cross-linked version >of the doc-strings, maybe with more of the examples that were >discussed here before. I am sure that Travis writes a good >user-manual, and, voila, both would be in the right place! > > I actually do think that converting the numarray manual will be easier. Because it is closer in function to what current scipy is. Advanced indexing has all been explained (the PEP for SciPy explains what is in current SciPy a bit better --- especially when it comes to mixing slices and integer indexing arrays. So, I would start from there. Hopefully in response to Jochen's other question (regarding hassles of install) we can get to the point where scipy core is installed by default on everybody's system.... :-) No need to give up C++ coding either with weave and pyrex (and f2py which can wrap C-code as well). -Travis From rowen at cesmail.net Mon Oct 10 16:18:20 2005 From: rowen at cesmail.net (Russell E. Owen) Date: Mon Oct 10 16:18:20 2005 Subject: [Numpy-discussion] Error involving import_libnumarray in packaged program Message-ID: If I convert my python code to an application (Windows via py2exe or Mac via bundlebuilder) it fails with the following error: Fatal Python error: Call to API function without first calling import_libnumarray() in Src/_convmodule.c I can force *all* of numarray into the application, which avoids the issue. But that is overkill. Is there some simpler way to solve this, e.g. some an import or command in my code that will prevent this from happening? -- Russell From Chris.Barker at noaa.gov Mon Oct 10 22:21:46 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 10 22:21:46 2005 Subject: [Numpy-discussion] Re: NumPies future In-Reply-To: <434ADF7A.6070300@ee.byu.edu> References: <434ADF7A.6070300@ee.byu.edu> Message-ID: <434B4BF2.7010505@noaa.gov> Travis Oliphant wrote: > No need to give up C++ coding either with weave and pyrex And SWIG and Boost::Python, which appears to have NumPy support...Has anyone used that, and can tell me about how well it works? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception From jochen at fhi-berlin.mpg.de Tue Oct 11 00:18:38 2005 From: jochen at fhi-berlin.mpg.de (=?iso-8859-1?Q?Jochen_K=FCpper?=) Date: Tue Oct 11 00:18:38 2005 Subject: [Numpy-discussion] Re: NumPies future In-Reply-To: <434ADF7A.6070300@ee.byu.edu> (Travis Oliphant's message of "Mon, 10 Oct 2005 15:39:06 -0600") References: <434ADF7A.6070300@ee.byu.edu> Message-ID: <9eoe5wbjyb.fsf@gowron.rz-berlin.mpg.de> Travis Oliphant writes: > I actually do think that converting the numarray manual will be easier. easier than? > Because it is closer in function to what current scipy is. closer than what? > No need to give up C++ coding either with weave and pyrex (and f2py > which can wrap C-code as well). I don't really want to program C++ -- I want a simple environment that's available on the typical number cruncher... Therefore, having scipy on every system would be a solution;) Greetings, Jochen -- Einigkeit und Recht und Freiheit http://www.Jochen-Kuepper.de Libert?, ?galit?, Fraternit? GnuPG key: CC1B0B4D (Part 3 you find in my messages before fall 2003.) From faltet at carabos.com Tue Oct 11 00:32:19 2005 From: faltet at carabos.com (Francesc Altet) Date: Tue Oct 11 00:32:19 2005 Subject: [Numpy-discussion] Re: NumPies future In-Reply-To: <434ADF7A.6070300@ee.byu.edu> References: <434ADF7A.6070300@ee.byu.edu> Message-ID: <200510110931.09415.faltet@carabos.com> A Dilluns 10 Octubre 2005 23:39, Travis Oliphant va escriure: > No need to give up C++ coding either with weave and pyrex (and f2py > which can wrap C-code as well). No, pyrex is not ready for C++ yet (at least natively, i.e. without C wrappers), but it would eventually be, with Greg permission. -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From kosh at skynet.be Tue Oct 11 05:08:38 2005 From: kosh at skynet.be (Kosta Gaitanis) Date: Tue Oct 11 05:08:38 2005 Subject: [Numpy-discussion] Problems compiling ufuncs example (2) In-Reply-To: <434AA3D2.80105@skynet.be> References: <20051010170845.6C9A7FFCB@sc8-sf-spam2.sourceforge.net> <434AA3D2.80105@skynet.be> Message-ID: <434BAACC.4010302@skynet.be> Kosta Gaitanis wrote: Hi again, I downloaded mingw32 and this seemed to fix the compilator problem. However I still can't build the ufunc example ! This time I tried : python setup.py build_ext --compiler=mingw32 and get the following output: running build_ext building 'numarray.foo._foo' extension c:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -Iinclude -Iinclude/numarray -Ic :\python24\Include umarray -IC:\Python24\include -IC:\Python24\PC -c foo/Src/_foo.c -o build\temp.w in32-2.4\Release\foo\src\_foo.o foo/Src/_foo.c:8: warning: ignoring #pragma warning foo/Src/_foo.c: In function `init_funcDict': foo/Src/_foo.c:26: warning: unused variable `keytuple' foo/Src/_foo.c:26: warning: unused variable `cfunc' writing build\temp.win32-2.4\Release\foo\src\_foo.def c:\mingw\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.4\Release\foo\src \_foo.o build\temp.win32-2.4\Release\foo\src\_foo.def -LLib -LC:\Python24\libs - LC:\Python24\PCBuild -lpython24 -lmsvcr71 -o build\lib.win32-2.4\numarray\foo\_f oo.pyd build\temp.win32-2.4\Release\foo\src\_foo.o(.text+0x87):_foo.c: undefined refere nce to `_imp__PyCObject_Type' build\temp.win32-2.4\Release\foo\src\_foo.o(.text+0xa6):_foo.c: undefined refere nce to `_imp__PyExc_ImportError' collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 If I remove the comments from the line libraries = ['ttf'] (at the end of setup.py) and try again I get this error : running build_ext building 'numarray.foo._foo' extension c:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -Iinclude -Iinclude/numarray -Ic :\python24\Include umarray -IC:\Python24\include -IC:\Python24\PC -c foo/Src/_foo.c -o build\temp.w in32-2.4\Release\foo\src\_foo.o foo/Src/_foo.c:8: warning: ignoring #pragma warning foo/Src/_foo.c: In function `init_funcDict': foo/Src/_foo.c:26: warning: unused variable `keytuple' foo/Src/_foo.c:26: warning: unused variable `cfunc' writing build\temp.win32-2.4\Release\foo\src\_foo.def c:\mingw\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.4\Release\foo\src \_foo.o build\temp.win32-2.4\Release\foo\src\_foo.def -LLib -LC:\Python24\libs - LC:\Python24\PCBuild -lttf -lpython24 -lmsvcr71 -o build\lib.win32-2.4\numarray\ foo\_foo.pyd c:\mingw\bin\..\lib\gcc\mingw32\3.4.2\..\..\..\..\mingw32\bin\ld.exe: cannot fin d -lttf collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 Can someone tell me what I'm doing wrong please. Thanks Kosta > Hi all, > > I'm trying to run the UFunc example in the Numarray manual > (http://stsdas.stsci.edu/numarray/numarray-1.1.html/node32.html) > but when I run the command > > python setup.py buil_ext > > > I get the following output : > > running build_ext > error: Python was built with version 7.1 of Visual Studio, and > extensions need t > o be built with the same version of the compiler, but it isn't > installed > > > so I installed Visual C++ Toolkit 2003 and then run the same command > again and got the following output : > > running build_ext > error: the .NET Framework SDK needs to be installed before building > extensions f > or Python > > > so I installed .NET Framework SDK 1.1 and then tried again. > I got back the first error... > > does anyone have an idea ? What am I doing wrong ? > > thanks in advance > Kosta > > I'm using Python24 and NumArray 1.3.3 > > > -- setup.py -- > """ This setup demonstrates how to use the numarray code generation > facilities to define additional ufuncs at the user level. Universal > functions apply operators or C-functions elementwise to all of the > elements of an array or pair of arrays. This setup generates code > defining three universal functions which are installed as the > numarray.foo package: Cos, bessel, and bar. > > The ufuncs are used like this: > > >>> import numarray as na > >>> import numarray.foo as foo > >>> a = na.array([1,2,3,4], type='Int32') > >>> foo.Cos(a) > array([ 0.54030228, -0.41614684, -0.9899925 , -0.65364361 ], > type=Float32) > > Note that since a is an Int32 array, Cos(a) first does blockwise > conversion of a to Float32 before calling the cosine library function. > """ > > import distutils, os > from distutils.core import setup > from numarray.numarrayext import NumarrayExtension > from numarray.codegenerator import UfuncModule, make_stub > > # ====================================================================== > # > # Generate the C-code for the numarray.foo._foo extension: > # > > m = UfuncModule("_foo") > > # Inline the code for a couple C functions to be turned into ufuncs. > # This method lets you define your function here in the setup.py. An > # alternate approach would be to link with a libarary or additional C > # module. > m.add_code(""" > > static double c_bessel(double x, double y) > { > double x2=x*x, y2=y*y, diff=x2-y2; > return diff*diff/(x2+y2); > } > > static UInt8 c_bar(UInt32 x ) > { > return (x >> 24) & 0xFF; > } > > """) > > # Method add_ufunc() defines the name of a ufunc, it's corresponding C > # function which is applied element-wise to all elements of an array, > # The arity of the ufunc (unary or binary only), and the I/O type > # signatures which are directly supported by the ufunc. Binary Ufunc > # "bessel" is implemented by the inline function "c_bessel" from the > # add_code() call above. > > m.add_ufunc("bessel", "c_bessel", 2, [('Float32', 'Float32'), > ('Float64', 'Float64')]) > > # Unary Ufunc "bar" only supports one builtin type signature: > # UInt32-->UInt8. Other types are blockwise converted by the ufunc > # machinery to UInt32 before "c_bar" is called. > > m.add_ufunc("bar", "c_bar", 1, [('UInt32','UInt8')]) > > # Unary Ufunc "cos" is implemented by the C standard library function > # "cos". Given a Float32 array, it returns a Float32 array. Given a > # Float64 array, it returns a Float64 array. For other arrays, > # transparent conversion to one of the supported input types is performed > # by the ufunc machinery. > > m.add_ufunc("cos", "cos", 1, [('Float32', 'Float32'), > ('Float64', 'Float64')]) > > # The generate() method writes out the complete extension module to the > # specified C file. > m.generate("foo/Src/_foo.c") > > # ====================================================================== > # Create foo's __init__.py for defining UFuncs corresponding to CFuncs > # in _foo. make_stub() emits boiler-plate which makes your extension > # a package. The __init__ file imports all the public symbols from > # C extension _foo making them visible as numarray.foo. > > make_stub("foo/Lib/__init__", "_foo") > # ====================================================================== > # > # Standard distutils setup for the generated code. > # > setup(name = "foo", > version = "0.1", > maintainer = "Todd Miller", > maintainer_email = "jmiller at stsci.edu", > description = "foo universal functions for numarray", > url = "http://www.stsci.edu/projects/foo/", > packages = ["numarray.foo"], > package_dir = { "numarray.foo":"foo/Lib" }, > ext_modules = [ NumarrayExtension( 'numarray.foo._foo', > ['foo/Src/_foo.c'], > # libraries = ['ttf'], > # include_dirs = ['include'], > # library_dirs = ['lib'] > ) > ] > ) > > From jh at oobleck.astro.cornell.edu Tue Oct 11 07:34:46 2005 From: jh at oobleck.astro.cornell.edu (Joe Harrington) Date: Tue Oct 11 07:34:46 2005 Subject: [Numpy-discussion] ASP status, docs, and supporting development Message-ID: <200510111433.j9BEXo0p013993@oobleck.astro.cornell.edu> After SciPy '04, Perry, Janet, and a few others and I put together a roadmap and framework, called ASP, for people to contribute to all the parts of SciPy other than the software: docs, packaging, website, etc. There was much interest expressed, and a few people even signed on the electronic dotted line to be involved: http://www.scipy.org/wikis/accessible_scipy/AccessibleSciPy The roadmap is laid out in this thread, which is linked in the first paragraph of the page above: http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2004-October/002412.html The first thing we did was to gather all the external projects using SciPy that we could find, and make an index page. The community found them and Fernando Perez did the hard work (more than he bargained for) of collating everything into the index page: http://www.scipy.org/wikis/topical_software/TopicalSoftware That is now a live wiki page, so if your project isn't on there, please add it! We were gearing up for effort #2, a web-site overhaul, early this year, when Travis announced his intention to heal the rift between the small- and large-array communities. We held up our push on web development, which was a few days from kickoff, so that it wouldn't take volunteers from his more-crucial effort. We all know the story of last year: Travis worked like crazy, called for volunteers, got only a few, and finished the job anyway. Now he's publishing a book in hopes of supporting some of his work from the revenues. He's made it clear that he will not be offended by or opposed to free docs, and may even contribute to them. He's also still Head Nummie and is leading the hard work of testing and bug swatting. Of course, we all want him to continue, and most of us freely admit our getting more than we are giving, and our gratitude to Travis, Todd, Robert, and the other core developers. Meanwhile, we have the problem of needing some basic docs that are free, and there seems to be quite a bit of interest in the community for doing one or more free docs. This seems to have more community energy behind it now than a web overhaul, so let's do it. The wiki and procedures are all set up to do some docs, and have been for about a year now. If you're interested in doing some docs, either as the lead author of a doc, as a contributor, or as a reviewer, please sign up on http://www.scipy.org/wikis/accessible_scipy/AccessibleSciPy The goal of the signups page is to make it easy for people to find each other: for lead authors to find people who will help them, for the community to identify who is taking part in what efforts, for low-level-of-effort volunteers to become hooked up with bigger projects, etc. There should be dozens of names there, not just three! So: If you're interested in LEADING A DOC, please add your name to the page, make a page for your doc on the wiki, and hang it off the main page, as "Scipy Cookbook" has done (there's a help link at the top of the page with instructions). A project can be anything from writing a collaborative book from scratch, to writing a monograph, to editing and revising existing docs. Announce your project on scipy-dev. If you would like to do a little work but not take the lead on something, you can contribute to the Cookbook or sign up to be a reviewer or contributor, either on an existing doc or at large. Or, contact a doc lead directly and sign up under that project. Please read the roadmap for ideas of docs we thought the community needs. The roadmap document is meant to be amended. For example, is the idea of using the docstrings to make a full-blown reference manual a good idea? I think so, since it's a rather self-updating format, but it will require some substantial work to get them all fleshed out and up to par. Discuss plans, changes, and ongoing efforts for docs on the scipy-dev mailing. It would be nice to have each project have a home on scipy.org, and Plone has excellent workflow-management tools. But, it's ok to home a project on your own site and just put a link on scipy.org. Finally, PLEASE everyone buy Travis's book if you can! Wait for the price to go up. Buy copies for everyone in your lab, if you can afford that. Buy one for your grandmother. It looks like it will be a really nice book, but it's more than that. This is a way to support development, which everyone desperately needs but few have the time (and fewer the skill and experience) to do. If you're at a company that benefits from SciPy and that hasn't already contributed resources to the effort, please consider parting with a larger chunk of change, either by buying more copies or by hiring Travis or others directly to do ongoing maintenance and package development. --jh-- From jmiller at stsci.edu Tue Oct 11 09:13:15 2005 From: jmiller at stsci.edu (Todd Miller) Date: Tue Oct 11 09:13:15 2005 Subject: [Numpy-discussion] Error involving import_libnumarray in packaged program In-Reply-To: References: Message-ID: <434BE46E.6090000@stsci.edu> Russell E. Owen wrote: >If I convert my python code to an application (Windows via py2exe or Mac >via bundlebuilder) it fails with the following error: > >Fatal Python error: Call to API function without first calling >import_libnumarray() in Src/_convmodule.c > > This currently (1.3.3) happens when a numarray API function is called before the API is successfully initialized. Although the message was intended as an aid to extension writers, in this case numarray is failing to import altogether. At one point numarray had a fatal error for import failures but I removed it at someone's request. I've restored it because I think it's most commonly fatal anyway and removing the message just obfuscated the problem. The non-fatal behavior is now in the _import_libnumarray() macro. >I can force *all* of numarray into the application, which avoids the > > This is what you need to do. It should be possible to factor out (or not explicitly list) numarray's Packages, but core numarray is not meant to be distributed in pieces. The many type-specific extensions were only added to work around a compiler problem, not to lighten binary distributions. >issue. But that is overkill. Is there some simpler way to solve this, > > Unfortunately, I think it's just necessary to (a) include all of core numarray and (b) help out automated tools (which choke on circular dependencies) by explicitly listing numarray's core extensions. Regards, Todd From rowen at cesmail.net Tue Oct 11 13:33:26 2005 From: rowen at cesmail.net (Russell E. Owen) Date: Tue Oct 11 13:33:26 2005 Subject: [Numpy-discussion] Re: Error involving import_libnumarray in packaged program References: <434BE46E.6090000@stsci.edu> Message-ID: In article <434BE46E.6090000 at stsci.edu>, Todd Miller wrote: > Russell E. Owen wrote: > > >If I convert my python code to an application (Windows via py2exe or Mac > >via bundlebuilder) it fails with the following error: > > > >Fatal Python error: Call to API function without first calling > >import_libnumarray() in Src/_convmodule.c >... > >I can force *all* of numarray into the application, which avoids the >... > Unfortunately, I think it's just necessary to (a) include all of core > numarray and (b) help out automated tools (which choke on circular > dependencies) by explicitly listing numarray's core extensions. Is there some practical way to include all of core numarray (without getting the unused extensions)? Could "import numarray" itself do the importing of core numarray? Then automatic packaging tools would "just work". -- Russell From meng at are.berkeley.edu Tue Oct 11 14:14:22 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Tue Oct 11 14:14:22 2005 Subject: [Numpy-discussion] question on differential results in windows and debian Message-ID: <5.1.1.5.2.20051011140513.0393b080@are.berkeley.edu> Hi there- I have a matrix X, calculate cX=matrixmultiply(transpose(X), X) and finally invert it by icX = linear_algebra.inverse(cX) cX is kind of ill-conditioned, which caused the icX differs a lot between windows and debian, after a tiny difference on cX across platforms. Here are the results: In windows: >>> x.min() -0.51187150837988826 >>> x.max() 0.81517690875232773 >>> cX = matrixmultiply(transpose(x), x) >>> cX.min() -666.96857541904126 >>> cX.max() 1073.3945530725441 >>> import numarray.linear_algebra as la >>> icX = la.inverse(cX) >>> icX.min() -3287030277.3580685 >>> icX.max() 0.0 In debian: >>> x.min() -0.51187150837988826 >>> x.max() 0.81517690875232773 >>> cX = matrixmultiply(transpose(x), x) >>> cX.min() -666.96857541898294 >>> cX.max() 1073.3945530726264 >>> import numarray.linear_algebra as la >>> icX = la.inverse(cX) >>> icX.min() 0.0 >>> icX.max() 84778028554.337051 I don't understand why a multiplication of a matrix and its transpose will create a tiny difference across windows and debian, which could be disastrous when inverting it. Please help me. Thanks a lot for your time! BTW, I am using Python2.4 and numarray 1.3.2 in both machines. Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng at are.berkeley.edu From arnd.baecker at web.de Wed Oct 12 06:06:45 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Wed Oct 12 06:06:45 2005 Subject: [Numpy-discussion] Can you enumerarte an array? In-Reply-To: <434AB4E0.9040607@noaa.gov> References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> Message-ID: Hi Chris, On Mon, 10 Oct 2005, Chris Barker wrote: > Travis Oliphant wrote: > > All flavors of Numeric would return > > > > 0 a[0] > > 1 a[1] > > . > > . > > . > > n-1 a[n-1] > > > > where n=a.shape[0] > > > > in response to index, item in enumerate(A): > > Right, as expected, and as Anrd pointed out, this is useful behaviour. > > > In scipy you could also get > > > > 0 a.flat[0] > > 1 a.flat[1] > > I do like the new flat! > > > To do something like he wants, I think we would have to construct a > > different iterator then enumerate. > > Exactly,. I didn't mean to imply that that's what the built-in enumerate > should do. > > > It wouldn't be that hard using the > > a.flat iterator (it's just a remapping of the 1-d index). > > Cool. then again, consider this a feature request, for a nd_enumerate, > or whatever you want to call it. > > I'm sorry that implimenting something like this myself is way out my depth! Below is one way to do it. BUT: - it is the first iterator I ever wrote... - it is not using yield, which might be better style - Oh and indeed it does give nicer code (see example 1). - it does not use a.flat. (BTW: I found http://heather.cs.ucdavis.edu/~matloff/Python/PyIterGen.pdf helpful) Best, Arnd 1) Short variant ---------------- from Numeric import * def arr_enumerate(arr): for indy in xrange(arr.shape[1]): for indx in xrange(arr.shape[0]): yield (indy,indx),arr[indy,indx] if __name__=="__main__": x=reshape(arange(25),(5,5)) print x for ind,value in arr_enumerate(x): print ind,value 2) Longer variant (just left in for historical reasons ...;-) ------------------------------------------------------------- from Numeric import * class arr_enumerate: def __init__(self,arr): self.Nx,self.Ny=arr.shape # just assume 2D here self.arr=arr self.indx=0 self.indy=0 def next(self): indx=self.indx indy=self.indy self.indx+=1 if self.indx==self.Nx: self.indx=0 self.indy+=1 if self.indy==self.Ny and self.indx==1: raise StopIteration return (indy,indx),self.arr[indy,indx] def __iter__(self): return self if __name__=="__main__": x=reshape(arange(25),(5,5)) print x for ind,value in arr_enumerate(x): print ind,value ###################################### From Chris.Barker at noaa.gov Wed Oct 12 08:52:24 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Wed Oct 12 08:52:24 2005 Subject: [Numpy-discussion] Can you enumerate an array? In-Reply-To: References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> Message-ID: <434D3106.8090305@noaa.gov> Arnd, You've written some nice compact code, but it only works for rank-2 arrays. Did you see Travis' solution? I think he's put it (or a C version, I'm not sure about that) in scipy_base. However, not using flat is nice, as in Numeric and numarray is sometimes makes a copy, and sometimes doesn't. -Chris Travis Oliphant wrote: > class ndenumerate(enumerate): > def __init__(self, arr): > self.iter = enumerate(arr.flat) > self.ashape = arr.shape > self.nd = arr.ndim > self.tot = arr.size > self.factors = [None]*(self.nd-1) > val = self.ashape[-1] > for i in range(self.nd-1,0,-1): > self.factors[i-1] = val > val *= self.ashape[i-1] > > def __len__(self): > return self.tot > > def next(self): > res = self.iter.next() > indxs = [None]*self.nd > val = res[0] > for i in range(self.nd-1): > indxs[i] = val / self.factors[i] > val = val % self.factors[i] > indxs[self.nd-1] = val > return tuple(indxs), res[1] -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From arnd.baecker at web.de Wed Oct 12 10:29:29 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Wed Oct 12 10:29:29 2005 Subject: [Numpy-discussion] Can you enumerate an array? In-Reply-To: <434D3106.8090305@noaa.gov> References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> <434D3106.8090305@noaa.gov> Message-ID: Hi Chris, On Wed, 12 Oct 2005, Chris Barker wrote: > Arnd, > > You've written some nice compact code, but it only works for rank-2 > arrays. Yup - I was aware of this limitation ;-) > Did you see Travis' solution? I think he's put it (or a C > version, I'm not sure about that) in scipy_base. Somehow I missed that - (his speed of coding is obviously way too fast for me to follow even in terms of the routines being added ;-). > However, not using flat > is nice, as in Numeric and numarray is sometimes makes a copy, and > sometimes doesn't. Whatever is most suitable for the task, at least I learnt about about iteraters ;-). Best, Arnd From oliphant at ee.byu.edu Wed Oct 12 12:10:44 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 12:10:44 2005 Subject: ***[Possible UCE]*** Re: [Numpy-discussion] Can you enumerate an array? In-Reply-To: <434D3106.8090305@noaa.gov> References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> <434D3106.8090305@noaa.gov> Message-ID: <434D5F66.1070901@ee.byu.edu> Chris Barker wrote: > Arnd, > > You've written some nice compact code, but it only works for rank-2 > arrays. Did you see Travis' solution? I think he's put it (or a C > version, I'm not sure about that) in scipy_base. However, not using > flat is nice, as in Numeric and numarray is sometimes makes a copy, > and sometimes doesn't. Yes, for Numeric and numarray a.flat does that. But, for scipy core, a.flat never makes a copy. It just returns an iterator object (that can be indexed and has an .__array__() method). So using a.flat is not a problem. Right now the solution I have is in Python. -Travis From oliphant at ee.byu.edu Wed Oct 12 12:12:21 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 12:12:21 2005 Subject: [Numpy-discussion] Can you enumerate an array? In-Reply-To: References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> <434D3106.8090305@noaa.gov> Message-ID: <434D5FCE.9080202@ee.byu.edu> >Whatever is most suitable for the task, at least I learnt >about about iteraters ;-). > > A very worth pursuit. Iterators are a very nice abstraction. I used them extensively in writing the new scipy core. It made N-dimensional coding, broadcasting, and indexing much, much easier. -Travis From oliphant at ee.byu.edu Wed Oct 12 13:37:49 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 13:37:49 2005 Subject: [Numpy-discussion] Re: [SciPy-dev] matrix and ravel() In-Reply-To: References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> Message-ID: <434D73D9.5040802@ee.byu.edu> Alan G Isaac wrote: >>Travis Oliphant wrote: >> >> >>>Not really a side-effect. An intended feature. Matrices are supposed >>>to be matrices (i.e. 2-d arrays). They cannot be 1-d arrays no matter >>>how hard you try.... If you want to ravel a matrix and get a 1-d array >>>out, you are not using matrices right.... Use a 1-d array (i.e. the .A >>>attribute). >>> >>> >1. Better support for matrices is very welcome; I use them > a lot. >2. I expect ravel to return a 1-d array, not a matrix. My > background is a matrix programming language (GAUSS from > before its N-dimensional array support), and yet the > > > behavior of ravel never surprised me. > > I'm less concerned about this issue because it is so easy to get an array out of a matrix. >3. I expect a row and column vectorization functions (named > e.g., vecr and vecc) to do row and column vectorization. > I do not have a strong opinion on whether what is really > wanted is a vec function that takes an axis argument, > although this is probably right. In particular, both row > and column vectorization of matrices is common, and this > is just a choice of axis. > > These are m.T.ravel() and m.ravel() right now. To get a 1-d array it is m.A.ravel() >The same goes double for asarray: it will be a really >unwanted surprise if it does not return an array. > > I'm much more concerned about this issue. I would like to hear more people way in on what should be the behavior here. In particular, here are two proposals. 1) current behavior array (and asarray) --- returns the object itself if it is a sub-class of the array (or an array scalar). asndarray --- returns an actual nd array object 2) new behavior array (and asarray) always return an ndarray base-class object (or a big-nd array if the object is already one). asanyarray --- returns an nd-array or a sub-class if the object is already a sub-class. Please weigh in... --Travis >fwiw, >Alan Isaac > > >_______________________________________________ >Scipy-dev mailing list >Scipy-dev at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-dev > > From tchur at optusnet.com.au Wed Oct 12 13:45:50 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Wed Oct 12 13:45:50 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <43420C28.5010701@pfdubois.com> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> Message-ID: <4342107C.8070404@optusnet.com.au> Paul F. Dubois wrote: > The original sources were in Framemaker. I am not positive where they > are. In any case they are copyrighted by the Regents of the University > of California. I am not a lawyer and don't know what the consequences of > that are. LLNL granted free distribution of the printed document with > the Numeric source code but I don't know what their position would be on > using their copyrighted text in a new document or on giving away the > sources. OK, thanks Paul. That may have implications for you, Travis, if you are planning to base your SciPy Core book on the existing NumPy documentation. Given the circumstance which Paul describes, perhaps the only way forward is to create a freely available addendum to the freely available, existing NumPy documentation which describes how and where SciPy Core differs or extends NumPy? Or to write entirely new documentation from scratch. > I believe that under U.S. copyright law, a person may lend their copy of > a book to anyone but not reproduce it and give them a copy. Yes, likewise here in Oz, and in most countries. Of course, the publisher of a book can impose additional restrictions to which the purchaser of the book must agree, such as 'You may not show or lend your copy of this book to your colleagues, or to anyone else.' However, who in their right mind would buy a book which was sold with such restrictions attached to it? > Given the way open source capitalism works, I would not be surprised if > someone produces a 'quick reference guide' that they give away, or > writes a book on 'Using scipy.core in biology' that they sell for $88. > Let a thousand flowers bloom. Yes, I agree entirely. Travis, you are at perfect liberty to create commercial documentation for SciPy Core, but please don't object if others try to organise to create free open source documentation as well. Tim C > Tim Churches wrote: > >> Travis Oliphant wrote: >> >>> Tim Churches wrote: >>> >>> >>>> Travis Oliphant wrote: >>>> >>>> >>>> >>>>> Tim Churches wrote: >>>>> >>>>> >>>> >>>> >>>> Travis, are you saying that this agreement only allows a single person >>>> to read the single printed copy? If so, I think you need a formally >>>> worded legal license to make that stick - certainly Australian >>>> copyright >>>> law (nor copyright law in other countries, I suspect) does not provide >>>> any support for such severe restrictions in the use of a printed >>>> document. Under copyright law, you may not make unauthorised copies >>>> of a >>>> printed document, but you can certainly lend that copy to others, or >>>> sell or give it to them, or share it as a bench manual. >>>> >>>> >>> >>> I'm just stating what I think is fair. I'm not going to try and use the >>> power of the Australian state or any other to try and enforce it. >> >> >> >> My point was that you won't get any help from the State if what you >> think is fair extends to restricting who can read a single printed copy >> of your documentation. You will need to resort to tort law to enforce >> such restrictions, and thus you really need a formal documentation >> license, if that is your aim. >> >> >>>> Many of those downloads are for "t[y|i]re-kicking" - potential users >>>> determining whether it meets their needs. If those potential users have >>>> to pay for documentation up-front, then a large proportion will go >>>> elsewhere. The rest are probably existing NumPy users upgrading (or >>>> testing the upgrade waters). The latter group may well pay for >>>> documentation, but how large is that group? For example, how many >>>> people >>>> are subscribed to the numpy-discussion mailing list? >>>> >>>> >>> >>> Don't know. But, there is enough free documentation to get started, I >>> think. Lack of documentation has hampered more people than cost of >>> existing documentation, I think. >> >> >> >> I have to disagree on this point. Granted the existing documentation for >> NumPy is not fantastic - some aspects of it are positively obscure - but >> overall it is better than good-enough, we have found, and was not a >> barrier to our decision to use NumPy. Having to cough up $50 to even >> peek at the documentation would be a far, far greater barrier to uptake >> than the occasional obscure wording of some of the function >> descriptions, IMHO. >> >> >>>> Travis, although many of us are grateful for your efforts on SciPy >>>> Core, >>>> no-one made you do it. If you wanted to earn (more) money by doing >>>> other >>>> things, you should have done them. >>>> >>> >>> I'm really thinking about the future here. If I can succesfully raise >>> money to support work on this using a simple time-delay documentation >>> encouragement, then perhaps others will do the same, and we can all have >>> more. Waiting for people to "donate" their time to develop scientific >>> python is rather slow.... I've been freely donating time for a long >>> time, and there are few who help. So, we'll try a slightly different >>> route and see where that goes. >> >> >> >> Yes, fair enough. My point was that 7 years is far too long a time >> delay. Given that SciPy Core will probably take another year to become >> rock-solid, I would happily buy several copies of your documentation at >> $50 per copy if I knew that in a year the documentation could be freely >> distributed to all God's children. But in 7 years? Nope, too long to >> wait, not interested in supporting that. >> >> >>>> I think there needs to be some community debate about this. Is there >>>> sufficient interest for people other than Travis to start with the >>>> Numeric documentation and update it as necessary to become a free SciPy >>>> Core documentation? The NumPy documentation is available in HTML format >>>> as the basis of this - perhaps the original source (LaTeX?) for the >>>> Numeric docs is also available? >>> >>> >>> I don't know why you would want to undermine my efforts in this way by >>> duplicating effort. >> >> >> >> Travis, I don't want to undermine your efforts, but you must understand >> that one of the attractions of NumPy and thus Scipy Core is that they >> are free, open source software, which implies that there exists adequate >> free, open source documentation for them, or if such free, open source >> documentation does not exist, then taht users are collectively at >> liberty to create it. That's a basic FOSS tenet. What you are are >> proposing is the creation of proprietary documentation for SciPy Core. >> No-one objects to this, but only if it is supplementary to free, open >> source "core" documentation, not instead of it. >> >> >>> Perhaps, instead you could have people donate $$ >>> instead of time to releasing the documentation. I give away copies of >>> the documentation to people who participate in the development process >>> all the time (and that comes off the total price --- though I haven't >>> advertised this as of yet). So, why don't you encourage people who >>> don't have the money to contribute to the project instead. >> >> >> >> I understand that open source software and its documentation don't just >> appear out of thin air, and I fully support the idea of funded or >> collectively-commissioned open source development. But only if the >> results of that funded or commissioned development is, in fact, free and >> open sourced. By preventing free distribution of the documentation you >> are creating for 7 years, the end product cannot even remotely be >> described as open source, thus personally I am disinterested in funding >> it. If the delay until open sourcing was just one year, then that's a >> different matter, but you do not seem to be proposing that. >> >> Hence I think the NumPy/SciPy Core user community should establish a >> SciPy Core documentation project, with the aim of updating and enhancing >> the existing NumPy documentation so that it covers the new and changed >> features of SciPy Core, and making that documentation available on teh >> same free, open source basis as NumPy/SciPy Core itself. >> >> To that end, are the original LaTeX or whatever sources for the currenet >> NumPy documenattion available somewhere (apart from the HTML and PDF >> versions, that is). >> >> Tim C >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > From meng at are.berkeley.edu Wed Oct 12 13:58:26 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Wed Oct 12 13:58:26 2005 Subject: [Numpy-discussion] question on differential results in windows and debian Message-ID: <5.1.1.5.2.20051012135240.03920290@are.berkeley.edu> >Hi there- > >I have a matrix X, calculate cX=matrixmultiply(transpose(X), X) and >finally invert it by icX = linear_algebra.inverse(cX) >cX is kind of ill-conditioned, which caused the icX differs a lot between >windows and debian, after a tiny difference on cX across platforms. I >don't understand why a multiplication of a matrix and its transpose will >create a tiny difference across windows and debian, which could be >disastrous when inverting it. Here are the results: >In windows: > >>> cX = matrixmultiply(transpose(x), x) > >>> cX.min() >-666.96857541904126 > >>> cX.max() >1073.3945530725441 > >>> import numarray.linear_algebra as la > >>> icX = la.inverse(cX) > >>> icX.min() >-3287030277.3580685 > >>> icX.max() >0.0 > >In debian: > >>> cX = matrixmultiply(transpose(x), x) > >>> cX.min() >-666.96857541898294 > >>> cX.max() >1073.3945530726264 > >>> import numarray.linear_algebra as la > >>> icX = la.inverse(cX) > >>> icX.min() >0.0 > >>> icX.max() >84778028554.337051 > >Thanks a lot for your time! BTW, I am using Python2.4 and numarray 1.3.2 >in both machines. > >Best, >Xiangyi > >Xiangyi Meng >Department of Agricultural and Resource Economics >Room 303, Giannini Hall #3310 >University of California, Berkeley >Berkeley, CA 94720-3310 >Tel: (510) 643-4124 >Fax: (510) 643-8911 >Email: meng at are.berkeley.edu > Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng at are.berkeley.edu From oliphant at ee.byu.edu Wed Oct 12 14:00:15 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 14:00:15 2005 Subject: [Numpy-discussion] New 0.4.2 beta release of scipy core Message-ID: <434D791B.9060809@ee.byu.edu> I made another beta release of scipy core last night. There are windows binaries for Python 2.4 and Python 2.3. If you are already a user of scipy, the new __init__ file installed for newcore will break your current installation of scipy (but the problem with linalg, fftpack, and stats is no longer there). There have been many improvements: - bug fixes (including 64-bit fixes) - threading support fixes - optimizations - more random numbers (thanks Robert Kern). - more distutils fixes (thanks Pearu Peterson). More tests are welcome. We are moving towards a release (but still need to get Masked Arrays working and all of scipy to build on top of the new scipy core before a full release). -Travis From stephen.walton at csun.edu Wed Oct 12 15:24:30 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Wed Oct 12 15:24:30 2005 Subject: [Numpy-discussion] Numeric precision Message-ID: <434AB6F6.1040500@csun.edu> Over on the scipy-dev mailing list, Travis Oliphant raised a question which is of interest to the folks over here as well. To summarize very briefly, Travis wondered whether the rule of "safest conversion" might require that an integer to float computation conversion on a 64-bit platform would require promotion of, for example, sqrt(2) to a long double (128 bits). I suggested that it would make the most sense for an M-bit integer to be converted to an N-bit real, where N>=M, on all platforms. For example, sqrt(2) would become a 32-bit real if 2 was a 32-bit integer on the plaform. I am not sure if this is a change from current Numeric and numarray practice, but wanted to give a heads-up over here. You can read the entire original thread beginning at http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2005-October/003403.html From Fernando.Perez at colorado.edu Wed Oct 12 15:37:26 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Wed Oct 12 15:37:26 2005 Subject: [Numpy-discussion] Re: [SciPy-dev] New 0.4.2 beta release of scipy core In-Reply-To: <434D791B.9060809@ee.byu.edu> References: <434D791B.9060809@ee.byu.edu> Message-ID: <434D8FDC.2050306@colorado.edu> Travis Oliphant wrote: > I made another beta release of scipy core last night. There are > windows binaries for Python 2.4 and Python 2.3. If you are already a Sorry if I missed the blindingly obvious, but a URL for the tarballs (do they exist?) would be most appreciated at this point (I have a friend looking for it, and I'm embarrassed to say that I can't seem to find it). As a hook: this friend may test the build on Itanium boxes, so I'd like to respond to him soonish :) If it's just an SVN pull that's fine, just let me know and I'll pass the info along. Cheers, f From Chris.Barker at noaa.gov Wed Oct 12 18:15:58 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Wed Oct 12 18:15:58 2005 Subject: [Numpy-discussion] Re: [SciPy-dev] matrix and ravel() In-Reply-To: <434D73D9.5040802@ee.byu.edu> References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> <434D73D9.5040802@ee.byu.edu> Message-ID: <434DB553.9040303@noaa.gov> Travis Oliphant wrote: > In particular, here are two proposals. > 1) current behavior > array (and asarray) --- returns the object itself if it is a sub-class > of the array > (or an array scalar). > asndarray --- returns an actual nd array object -1. I expect as array to return an array. Period. That's why I use it. > 2) new behavior > > array (and asarray) always return an ndarray base-class object (or a > big-nd array if the object is already one). +1 ...I think. What is a big-nd array? > asanyarray --- returns an nd-array or a sub-class if the object is > already a sub-class. What about having each subclass define it's own function-- asmatrix, etc? Maybe that's not general enough, but I know I use asarray because I know I want a NumPy Array, and nothing else. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception From oliphant at ee.byu.edu Wed Oct 12 18:49:38 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 18:49:38 2005 Subject: [Numpy-discussion] Problem with name numeric In-Reply-To: <4346D825.70309@sympatico.ca> References: <4346D825.70309@sympatico.ca> Message-ID: <4346DADA.6050200@ee.byu.edu> Colin J. Williams wrote: > With the following sequence I had expected "scipy.base.numeric" to be > a module, instead it is reported as a type. Yes numeric is a type under scipy.base you just need scipy.ndarray The idea is not not import all the little submodules that make up scipy (that way we can change them around in the future should we need to). The only modules that should import numeric directly are those that are part of the scipy.base package itself. All you need to do is import scipy alone import scipy scipy.ndarray is what you want. -Travis From Chris.Barker at noaa.gov Wed Oct 12 21:47:11 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Wed Oct 12 21:47:11 2005 Subject: [Numpy-discussion] Re: asarray behaviour In-Reply-To: <434DB93C.9010303@ee.byu.edu> References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> <434D73D9.5040802@ee.byu.edu> <434DB553.9040303@noaa.gov> <434DB93C.9010303@ee.byu.edu> Message-ID: <434DE709.20208@noaa.gov> Travis Oliphant wrote: > Chris Barker wrote: >> Travis Oliphant wrote: >>> 2) new behavior >>> >>> array (and asarray) always return an ndarray base-class object (or a >>> big-nd array if the object is already one). >> >> +1 ...I think. What is a big-nd array? > > A big-nd arrary is the parent-class of the ndarray object. It doesn't > have the sequence protocols or the buffer protocols and should not > suffer from the 32-bit limitations of those protocols. An ndarray > inherits from the bigarray. Eventually, when Python cleans up its > 32-bit limitations, the bigarray will disappear. > >>> asanyarray --- returns an nd-array or a sub-class if the object is >>> already a sub-class. >> >> What about having each subclass define it's own function-- asmatrix, >> etc? Maybe that's not general enough, but I know I use asarray because >> I know I want a NumPy Array, and nothing else. > > I guess the problem is that we are not used to coding to interfaces. I'm not sure I get this. If we were coding to an interface, then I'd write my functions to jsut expect the type I want, and be done iowth it, I wouldn't need asarray(). > I'm going to make the change suggested by the second point, just because > I think it's more explicit and will make porting scipy a lot easier. um, I'm not sure I follow...what change are you making? > The fact that multiplication could be redefined by the matrix which > still passes as an array, means that lots of code can choke on matrices. Exactly. What I like about asarray is that I can write a function that really does require a NumPy array, and let the user code pass in anything that can be turned into one, without any performance penalty if a NumPy array is passed in. However, it's very important that I know EXACTLY what type asarray will return. The same thing applies to matrixes, etc. If I write a function that manipulates a matrix, Iwant to be darn sure that's what I've got...I'd need an asmatrix() function to give it to me. > Of course, this will have negative consequences. It will make matrices > much less pervasive through function calls "automatically", but it will > be safer. People who believe their code is safe for matrices, can use > asanyarray. I can see that there may well be sone code that could take any subclass of ndarray..perhaps if it's only accessing the data, for instance, so it's nice to have that option. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception From cjw at sympatico.ca Thu Oct 13 05:02:11 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu Oct 13 05:02:11 2005 Subject: [Numpy-discussion] Re: asarray behaviour In-Reply-To: <434DE709.20208@noaa.gov> References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> <434D73D9.5040802@ee.byu.edu> <434DB553.9040303@noaa.gov> <434DB93C.9010303@ee.byu.edu> <434DE709.20208@noaa.gov> Message-ID: <434E4C6C.3090705@sympatico.ca> Chris Barker wrote: > Travis Oliphant wrote: > >> Chris Barker wrote: >> >>> Travis Oliphant wrote: >>> >>>> 2) new behavior >>>> >>>> array (and asarray) always return an ndarray base-class object (or >>>> a big-nd array if the object is already one). >>> >>> >>> +1 ...I think. What is a big-nd array? >> >> >> A big-nd arrary is the parent-class of the ndarray object. It >> doesn't have the sequence protocols or the buffer protocols and >> should not suffer from the 32-bit limitations of those protocols. >> An ndarray inherits from the bigarray. Eventually, when Python cleans >> up its 32-bit limitations, the bigarray will disappear. >> >>>> asanyarray --- returns an nd-array or a sub-class if the object is >>>> already a sub-class. >>> >>> >>> What about having each subclass define it's own function-- asmatrix, >>> etc? Maybe that's not general enough, but I know I use asarray >>> because I know I want a NumPy Array, and nothing else. >> >> >> I guess the problem is that we are not used to coding to interfaces. > > > I'm not sure I get this. If we were coding to an interface, then I'd > write my functions to jsut expect the type I want, and be done iowth > it, I wouldn't need asarray(). > >> I'm going to make the change suggested by the second point, just >> because I think it's more explicit and will make porting scipy a lot >> easier. > > > um, I'm not sure I follow...what change are you making? > >> The fact that multiplication could be redefined by the matrix which >> still passes as an array, means that lots of code can choke on matrices. > > > Exactly. What I like about asarray is that I can write a function that > really does require a NumPy array, and let the user code pass in > anything that can be turned into one, without any performance penalty > if a NumPy array is passed in. However, it's very important that I > know EXACTLY what type asarray will return. The same thing applies to > matrixes, etc. If I write a function that manipulates a matrix, Iwant > to be darn sure that's what I've got...I'd need an asmatrix() function > to give it to me. > I'm puzzled by this. There is already a matrix class which should construct a matrix instance from a list, tuple or array or any subclass of array - I haven't tested all of these. >> Of course, this will have negative consequences. It will make >> matrices much less pervasive through function calls "automatically", >> but it will be safer. People who believe their code is safe for >> matrices, can use asanyarray. > > > I can see that there may well be sone code that could take any > subclass of ndarray..perhaps if it's only accessing the data, for > instance, so it's nice to have that option. > > -Chris > Colin W. From Chris.Barker at noaa.gov Thu Oct 13 09:58:00 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Thu Oct 13 09:58:00 2005 Subject: [Numpy-discussion] Re: asarray behaviour In-Reply-To: <434E4C6C.3090705@sympatico.ca> References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> <434D73D9.5040802@ee.byu.edu> <434DB553.9040303@noaa.gov> <434DB93C.9010303@ee.byu.edu> <434DE709.20208@noaa.gov> <434E4C6C.3090705@sympatico.ca> Message-ID: <434E91CA.5010208@noaa.gov> Colin J. Williams wrote: > I'm puzzled by this. There is already a matrix class which should > construct a matrix instance from a list, tuple or array or any subclass > of array - I haven't tested all of these. A) I was speaking generically. I haven't made use of any particular matrix class myself. B) I assume the standard matrix() constructor always creates a copy of the data, much like the NumPy array() constructor. In this case, the discussion was about the behavior of asarray(), which creates a new array if the argument is not an array, but returns the argument untouched if it already is an array. I was proposing that a matrix package (and any other subclass of ndarray) could benefit from a similar function. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From theller at python.net Thu Oct 13 10:44:46 2005 From: theller at python.net (Thomas Heller) Date: Thu Oct 13 10:44:46 2005 Subject: [Numpy-discussion] Re: Error involving import_libnumarray in packaged program References: <434BE46E.6090000@stsci.edu> Message-ID: <3bn51fzn.fsf@python.net> "Russell E. Owen" writes: > In article <434BE46E.6090000 at stsci.edu>, > Todd Miller wrote: > >> Russell E. Owen wrote: >> >> >If I convert my python code to an application (Windows via py2exe or Mac >> >via bundlebuilder) it fails with the following error: >> > >> >Fatal Python error: Call to API function without first calling >> >import_libnumarray() in Src/_convmodule.c >>... >> >I can force *all* of numarray into the application, which avoids the >>... >> Unfortunately, I think it's just necessary to (a) include all of core >> numarray and (b) help out automated tools (which choke on circular >> dependencies) by explicitly listing numarray's core extensions. > > Is there some practical way to include all of core numarray (without > getting the unused extensions)? > > Could "import numarray" itself do the importing of core numarray? Then > automatic packaging tools would "just work". The easiest way to build some 'hints' for those packaging tools that use modulefinder is to create a silly function that imports everything belonging to the package: def _give_py2exe_hints(): import this import that ... Note that it will NOT work to write this: if 0: import this import that ... because 'if 0' blocks are optimized away completely by the Python compiler. This should be in numarray's __init__.py file, probably. Thomas From bsouthey at gmail.com Fri Oct 14 12:14:53 2005 From: bsouthey at gmail.com (Bruce Southey) Date: Fri Oct 14 12:14:53 2005 Subject: [Numpy-discussion] Numeric precision In-Reply-To: <434AB6F6.1040500@csun.edu> References: <434AB6F6.1040500@csun.edu> Message-ID: Hi, The unaddressed problem is how the user will see the result. This type of thing leads type mismatches especially when doing array indexing. In that case it is a really bad thing to do. But in terms of mathematical computations it is usually a given that the highest possible precision is used when possible. Regards Bruce On 10/10/05, Stephen Walton wrote: > > Over on the scipy-dev mailing list, Travis Oliphant raised a question > which is of interest to the folks over here as well. To summarize very > briefly, Travis wondered whether the rule of "safest conversion" might > require that an integer to float computation conversion on a 64-bit > platform would require promotion of, for example, sqrt(2) to a long > double (128 bits). I suggested that it would make the most sense for an > M-bit integer to be converted to an N-bit real, where N>=M, on all > platforms. For example, sqrt(2) would become a 32-bit real if 2 was a > 32-bit integer on the plaform. > > I am not sure if this is a change from current Numeric and numarray > practice, but wanted to give a heads-up over here. You can read the > entire original thread beginning at > > > http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2005-October/003403.html > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From meng at are.berkeley.edu Mon Oct 17 14:12:23 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Mon Oct 17 14:12:23 2005 Subject: [Numpy-discussion] Question: why is this different? Message-ID: <5.1.1.5.2.20051017140354.03a06090@are.berkeley.edu> Hi there- I have a question on how this difference arises when I run the following simple code on windows and Debian: >>> from numarray import * >>> a=reshape(arange(81.0), (9,9))+identity(9) >>> ca = matrixmultiply(transpose(a),a) >>> import numarray.linear_algebra as la >>> ica = la.inverse(ca) >>> ica.min() -0.30951621414374736 >>> ica.max() 0.8888918586585135 The above is the result in Debian and below is that in Windows: >>> ica.min() -0.30951621414449404 >>> ica.max() 0.88889185865875686 So, what caused this difference (both machines have python2.4 and numarray 1.3.2)? The difference happens in the 10th/11th decimal point, which should be still in the floating point precision, right? This simple difference caused big differences in my later calculation. Best, Xiangyi From meng at are.berkeley.edu Tue Oct 18 13:28:40 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Tue Oct 18 13:28:40 2005 Subject: [Numpy-discussion] Question: why is this different? In-Reply-To: <4354215C.9080806@stsci.edu> References: <5.1.1.5.2.20051017140354.03a06090@are.berkeley.edu> <5.1.1.5.2.20051017140354.03a06090@are.berkeley.edu> Message-ID: <5.1.1.5.2.20051018132505.038c99f0@are.berkeley.edu> Todd and Paul- Thank you so much for your help! Any suggestions on my next step? This difference doesn't make my result qualitatively different but my final result differs in the 3rd decimal point across these two machines, which could be troublesome when you want something that's robust to these computing errors. Best, Xiangyi At 18:10 2005-10-17 -0400, Todd Miller wrote: >meng at are.berkeley.edu wrote: > >>Hi there- >> >>I have a question on how this difference arises when I run the following >>simple code on windows and Debian: >> >>> from numarray import * >> >>> a=reshape(arange(81.0), (9,9))+identity(9) >> >>> ca = matrixmultiply(transpose(a),a) >> >>> import numarray.linear_algebra as la >> >>> ica = la.inverse(ca) >> >>> ica.min() >>-0.30951621414374736 >> >>> ica.max() >>0.8888918586585135 >> >>The above is the result in Debian and below is that in Windows: >> >>> ica.min() >>-0.30951621414449404 >> >>> ica.max() >>0.88889185865875686 > >I'm able to reproduce your "Windows results" in Fedora Core 4 x86_64... >exactly. That says that a variant of gcc and VC.NET are producing >identical results to 18 digits... which is encouraging. Numeric agrees as >well. > >When I build numarray with LAPACK and ATLAS rather than the builit-in >lapack_lite on RHEL3 i386 I get different results from any of the above: > >ica.min: -0.30951621414375136 >ica.max: 0.88889185865854037 > >They are closest to Debian I think, differing in the 15th decimal >place. My suspicion is that your Debian version is being built against >different numerical libraries... but I'm not privy to exactly how. A key >point is that numarray on Windows is *not* built using LAPACK and ATLAS. > >>So, what caused this difference (both machines have python2.4 and >>numarray 1.3.2)? The difference happens in the 10th/11th decimal point, >>which should be still in the floating point precision, right? > >I would expect so. > Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng at are.berkeley.edu From timb at tbitc.com Wed Oct 19 22:43:11 2005 From: timb at tbitc.com (Tim Burgess) Date: Wed Oct 19 22:43:11 2005 Subject: [Numpy-discussion] numarray - was trying to freeze wxmpl matplotlib and wxpython Message-ID: <43572E26.4050902@tbitc.com> Hi all, First thanks to all those who have been helping out - it is really appreciated. I have cross posted to the numarray, py2exe and matplotlib lists I have attempted to use numarray from cvs but I can't build it - no VC7 clang - did a penny drop anywhere? I am working with python 2.4.1 and numarray 1.3.3 for 2.4 (obviously) Since I couldnt build numarray - I looked at the new import code in the cvs init module and ripped it off and stuffed it into my numarray installation and then when that didnt work - I jammed it right up front of my application. It looks like this def main(): import numarray.numarrayall from numarray.numinclude import version as __version__ # stolen from next numarray version in cvs TjB import numarray._conv import numarray._sort import numarray._bytes import numarray._ufunc import numarray._ufuncBool import numarray._ufuncInt8 import numarray._ufuncUInt8 import numarray._ufuncInt16 import numarray._ufuncUInt16 import numarray._ufuncInt32 import numarray._ufuncUInt32 import numarray._ufuncFloat32 import numarray._ufuncFloat64 import numarray._ufuncComplex32 import numarray._ufuncComplex64 import numarray._ndarray import numarray._numarray import numarray._chararray import numarray._objectarray import numarray.memory import numarray._converter import numarray._operator import numarray._numerictype import numarray.libnumarray import numarray.libnumeric import numarray._ufuncInt64 import numarray._ufuncUInt64 print numarray.__version__ application = BoaApp(0) Still no go - but a changed error message.... grrr I get Traceback (most recent call last): File "AEMdaApp.py", line 81, in ? File "AEMdaApp.py", line 41, in main File "numarray\__init__.pyc", line 42, in ? File "numarray\numarrayall.pyc", line 1, in ? File "numarray\numerictypes.pyc", line 35, in ? File "numarray\numinclude.pyc", line 4, in ? File "numarray\_ndarray.pyc", line 9, in ? File "numarray\_ndarray.pyc", line 7, in __load ImportError: init_ndarray: can't find memory.new_memory ok I confess my bottom lip trembled.... then - probably because I was reading the numarray manual this morning - I went to my numarray install and decided to run testall.py I got F:\Python24\Lib\site-packages\numarray>\python24\python testall.py Testing numarray 1.3.3 on normal Python (2, 4, 1, 'final', 0) on platform win32 ********************************************************************** File "F:\python24\lib\site-packages\numarray\numtest.py", line 2843, in cache p ss Failed example: cPickle.loads(cPickle.dumps(arange(5)+1j)) Exception raised: Traceback (most recent call last): File "F:\Python24\lib\doctest.py", line 1243, in __run compileflags, 1) in test.globs File "", line 1, in ? PicklingError: Can't pickle : it's no the same object as memory.memory_from_string ********************************************************************** File "F:\python24\lib\site-packages\numarray\numtest.py", line 2849, in cache p ss Failed example: p = p.dump(a) Exception raised: Traceback (most recent call last): File "F:\Python24\lib\doctest.py", line 1243, in __run compileflags, 1) in test.globs File "", line 1, in ? PicklingError: Can't pickle : it's no the same object as memory.memory_from_string ********************************************************************** File "F:\python24\lib\site-packages\numarray\numtest.py", line 2850, in cache p ss Failed example: p = p.dump(b) Exception raised: and so on for quite some time. Hmm the message about memory.memory_from_string and my applications ImportError: init_ndarray: can't find memory.new_memory are these a little related? Am I clutching at straws? Yet my app runs fine under any ide I wish to use. WAIT! - my install of numarray is broken.... I deleted it and reinstalled - still broken I installed on my laptop and tested - broken there too My legacy python 2.3 and numarray 1.3.3 installation passes all the tests just fine. Is there a problem with python2.4.1 and numarray 1.3.3? Could some kind soul build me a numarray windows installer for python 2.4 from cvs - please - so that I can see if that works better with py2exe. I need a stiff drink -- Tim Burgess IT Consultant RedHat Certified Engineer TBITC Pty Ltd Professional Computer Support for Business timb at tbitc.com Mobile 0422 942 972 Office 85 662 016 http://www.tbitc.com From Chris.Barker at noaa.gov Thu Oct 20 00:04:29 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Thu Oct 20 00:04:29 2005 Subject: [Numpy-discussion] Re: [Matplotlib-users] numarray - was trying to freeze wxmpl matplotlib and wxpython In-Reply-To: <43572E26.4050902@tbitc.com> References: <43572E26.4050902@tbitc.com> Message-ID: <43574114.30903@noaa.gov> Tim, Sorry I can't help much, but: > I have attempted to use numarray from cvs but I can't build it - no VC7 You can build Python extensions with MinGW. You have to do a little patching to Python, but it's not too hard, and it does work. Here's one link. Googling will get you others: http://www.mingw.org/MinGWiki/index.php/Python extensions -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From jmiller at stsci.edu Thu Oct 20 09:35:36 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu Oct 20 09:35:36 2005 Subject: [Numpy-discussion] Re: [Matplotlib-users] numarray - was trying to freeze wxmpl matplotlib and wxpython In-Reply-To: <43572E26.4050902@tbitc.com> References: <43572E26.4050902@tbitc.com> Message-ID: <4357C6F5.4080500@stsci.edu> Tim Burgess wrote: > Hi all, > > First thanks to all those who have been helping out - it is really > appreciated. > > Since I couldnt build numarray - I looked at the new import code in > the cvs init module and ripped it off and stuffed it into my numarray > installation and then when that didnt work - I jammed it right up > front of my application. > > It looks like this > > def main(): > import numarray.numarrayall > from numarray.numinclude import version as __version__ > > # stolen from next numarray version in cvs TjB > > import numarray._conv > import numarray._sort > import numarray._bytes > import numarray._ufunc > import numarray._ufuncBool > import numarray._ufuncInt8 > import numarray._ufuncUInt8 > import numarray._ufuncInt16 > import numarray._ufuncUInt16 > import numarray._ufuncInt32 > import numarray._ufuncUInt32 > import numarray._ufuncFloat32 > import numarray._ufuncFloat64 > import numarray._ufuncComplex32 > import numarray._ufuncComplex64 > import numarray._ndarray > import numarray._numarray > import numarray._chararray > import numarray._objectarray > import numarray.memory > import numarray._converter > import numarray._operator > import numarray._numerictype > import numarray.libnumarray > import numarray.libnumeric > import numarray._ufuncInt64 > import numarray._ufuncUInt64 > > print numarray.__version__ > > application = BoaApp(0) > > Still no go - but a changed error message.... grrr Well, that's progress anyway. > > I get > > Traceback (most recent call last): > File "AEMdaApp.py", line 81, in ? > File "AEMdaApp.py", line 41, in main > File "numarray\__init__.pyc", line 42, in ? > File "numarray\numarrayall.pyc", line 1, in ? > File "numarray\numerictypes.pyc", line 35, in ? > File "numarray\numinclude.pyc", line 4, in ? > File "numarray\_ndarray.pyc", line 9, in ? > File "numarray\_ndarray.pyc", line 7, in __load > ImportError: init_ndarray: can't find memory.new_memory Is numarray.memory actually present in your install directory? Can you CD to that directory and import memory by itself? > then - probably because I was reading the numarray manual this morning > - I went to my numarray install and decided to run testall.py > > I got There's two things going on here. (1) This manner of running the numarray self-tests isn't recommended since it doesn't work anywhere. Read Doc/INSTALL.txt for how to run them. (2) There's bit-rot in numarray CVS for windows. I'm working on the rot now. > F:\Python24\Lib\site-packages\numarray>\python24\python testall.py > Testing numarray 1.3.3 on normal Python (2, 4, 1, 'final', 0) on > platform win32 > ********************************************************************** > File "F:\python24\lib\site-packages\numarray\numtest.py", line 2843, > in cache p > ss > Failed example: > cPickle.loads(cPickle.dumps(arange(5)+1j)) > Exception raised: > Traceback (most recent call last): > File "F:\Python24\lib\doctest.py", line 1243, in __run > compileflags, 1) in test.globs > File "", line 1, in ? > PicklingError: Can't pickle memory_from_string>: it's no > the same object as memory.memory_from_string > ********************************************************************** > File "F:\python24\lib\site-packages\numarray\numtest.py", line 2849, > in cache p > ss > Failed example: > p = p.dump(a) > Exception raised: > Traceback (most recent call last): > File "F:\Python24\lib\doctest.py", line 1243, in __run > compileflags, 1) in test.globs > File "", line 1, in ? > PicklingError: Can't pickle memory_from_string>: it's no > the same object as memory.memory_from_string > ********************************************************************** > File "F:\python24\lib\site-packages\numarray\numtest.py", line 2850, > in cache p > ss > Failed example: > p = p.dump(b) > Exception raised: > > and so on for quite some time. > > Hmm the message about > memory.memory_from_string > and my applications > ImportError: init_ndarray: can't find memory.new_memory > > are these a little related? Am I clutching at straws? > > Yet my app runs fine under any ide I wish to use. > > WAIT! - my install of numarray is broken.... > I deleted it and reinstalled - still broken I'm a little confused here. How is numarray installed where your application actually works? How is numarray installed where numarray does not work? (I was under the impression that you're using a tool to build a self-contained installer... which isn't working. I figured your development systems have numarray installed independently... which is working. What are you doing?) > I installed on my laptop and tested - broken there too > > My legacy python 2.3 and numarray 1.3.3 installation passes all the > tests just fine. > > Is there a problem with python2.4.1 and numarray 1.3.3? Nope. > Could some kind soul build me a numarray windows installer for python > 2.4 from cvs - please - so that I can see if that works better with > py2exe. Let me know if you still want this and I'll make one for you. FYI, MinGW works great for matplotlib but you may have to row-your-own-boat when it comes to numarray. As far as I know, there are missing glibc dependencies for numarray, particularly IEEE exception handling. Regards, Todd From oliphant at ee.byu.edu Thu Oct 20 10:40:26 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu Oct 20 10:40:26 2005 Subject: [Numpy-discussion] Re: [Matplotlib-users] numarray - was trying to freeze wxmpl matplotlib and wxpython In-Reply-To: <4357C6F5.4080500@stsci.edu> References: <43572E26.4050902@tbitc.com> <4357C6F5.4080500@stsci.edu> Message-ID: <4357D658.8080506@ee.byu.edu> > Let me know if you still want this and I'll make one for you. > > FYI, MinGW works great for matplotlib but you may have to > row-your-own-boat when it comes to numarray. As far as I know, there > are missing glibc dependencies for numarray, particularly IEEE > exception handling. Actually, mingw handles IEEE exceptions fine (using the LINUX style). You just have to include the right directories (it took me a while to track that one down). Look in new scipy core ufuncobject.h for the test for MINGW. -Travis From jmiller at stsci.edu Thu Oct 20 11:38:18 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu Oct 20 11:38:18 2005 Subject: [Numpy-discussion] Re: [Matplotlib-users] numarray - was trying to freeze wxmpl matplotlib and wxpython In-Reply-To: <4357D658.8080506@ee.byu.edu> References: <43572E26.4050902@tbitc.com> <4357C6F5.4080500@stsci.edu> <4357D658.8080506@ee.byu.edu> Message-ID: <4357E38E.7000901@stsci.edu> Travis Oliphant wrote: > >> Let me know if you still want this and I'll make one for you. >> >> FYI, MinGW works great for matplotlib but you may have to >> row-your-own-boat when it comes to numarray. As far as I know, >> there are missing glibc dependencies for numarray, particularly IEEE >> exception handling. > > > Actually, mingw handles IEEE exceptions fine (using the LINUX style). > You just have to include the right directories (it took me a while to > track that one down). Look in new scipy core ufuncobject.h for the > test for MINGW. > > -Travis Thanks for the information Travis. I tried this and can get it to compile, link, and load OK, but the IEEE exception handling self-tests fail, multiplication overflow detection in particular. Todd From timb at tbitc.com Thu Oct 20 16:52:49 2005 From: timb at tbitc.com (Tim Burgess) Date: Thu Oct 20 16:52:49 2005 Subject: [Numpy-discussion] Numarray - doesnt pass installation test under Windblows (was freezing wxmpl etc) Message-ID: <43582D73.3050809@tbitc.com> Hi Todd, Yesterday I claimed that numarray wasnt passing the tests under python 2.4. Actually I was testing the 1.3.3 Windows exe installer version - not my cvs derrived version. *blush* The cvs version tests perfectly. I will take you up on your offer of a Windows exe from CVS please Todd. Perhaps it means it is time for a 1.3.3.1 version? :-} -- Tim Burgess IT Consultant RedHat Certified Engineer TBITC Pty Ltd Professional Computer Support for Business timb at tbitc.com Mobile 0422 942 972 Office 85 662 016 http://www.tbitc.com From jmiller at stsci.edu Fri Oct 21 16:43:16 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 21 16:43:16 2005 Subject: [Numpy-discussion] Numarray - doesnt pass installation test under Windblows (was freezing wxmpl etc) In-Reply-To: <43582D73.3050809@tbitc.com> References: <43582D73.3050809@tbitc.com> Message-ID: <43597CA8.4060505@stsci.edu> Tim Burgess wrote: > Hi Todd, > > Yesterday I claimed that numarray wasnt passing the tests under python > 2.4. > > Actually I was testing the 1.3.3 Windows exe installer version - not > my cvs derrived version. *blush* > > The cvs version tests perfectly. > > I will take you up on your offer of a Windows exe from CVS please Todd. > > Perhaps it means it is time for a 1.3.3.1 version? :-} > > Hi Tim, The good news is I tried to create this today. The bad news is that I failed to get a distribution that passes all the self-tests. It's late and Monday is booked so it's looking like COB Tuesday is the earliest a release will come from me. If my understanding of the way py2exe works is correct (sucks up installed code for inclusion in distro), you might consider rolling your own 1.3.3.1 by starting with a 1.3.3 installation, adding the __init__.py from the head of CVS, and updating generate.py with the new version numbers. Sorry this is dragging out but it *will* be resolved in the next week, possibly by 1.4.0. Todd From vanzwieten at science-and-technology.nl Mon Oct 24 00:52:27 2005 From: vanzwieten at science-and-technology.nl (Joris van Zwieten) Date: Mon Oct 24 00:52:27 2005 Subject: [Numpy-discussion] numarray.objects Message-ID: <49857.217.77.152.33.1130140184.squirrel@control.prolocation.net> Hi, Are arrays of objects (i.e. numarray.objects) still supported in Numeric3/Scipy? (I'm working on an application that should be able to work with arrays of numbers and arrays of objects. Without numarray.objects-like functionality, this would require using numarray arrays for the numbers, and nested python lists for the objects. ;) Thanks, Joris van Zwieten From oliphant at ee.byu.edu Tue Oct 25 11:45:26 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Oct 25 11:45:26 2005 Subject: [Numpy-discussion] Re: Numeric: Change of v24.0 In-Reply-To: <6B3FDE9C-E25C-402D-9C27-F17B7F412753@opendarwin.org> References: <6B3FDE9C-E25C-402D-9C27-F17B7F412753@opendarwin.org> Message-ID: <435E7CC7.1050103@ee.byu.edu> Weissmann Markus wrote: > Hi folks, > > Before beginning my rant: Thank you very much for Numeric - you're > doing a great job! > > I'm a bit pissed as you changed the - already released tarball of > version 24.0; > This is a real mess, as all build systems out there have problems > distinguishing versions, > the correct tarballs (hashes will fail) etc. I'm not sure what you are talking about. The old tarball was always a beta release. It took a long time to get the actual release out, but that is due to the lack of reports. > I don't think this is my personal, or my projects impression, as e. > g. Novell [1] & O'Reilly [2] > clearly were under the impression that the old tarball was v24.0. > Then they were mistaken. It was always clearly marked as version 24b2 (beta-release 2). > Whatever happened that lead to this confusion, it would be great if > you could at least > make a version bump to 24.1 right now so everyone can be sure that > 24.1 is 24.1 in contrast > to 24.0 not being 24.0 all the time. > Someone could make a 24.1, but I don't see the need, and I'm not really interested in doing that. I'm done with Numeric development as I've moved to scipy core. I'm sorry for your consternation. -Travis From jmiller at stsci.edu Wed Oct 26 08:23:40 2005 From: jmiller at stsci.edu (Todd Miller) Date: Wed Oct 26 08:23:40 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.0 Message-ID: <435F9F2F.2000005@stsci.edu> Numarray is an array processing package designed to efficiently manipulate large multi-dimensional arrays. Numarray is modeled after Numeric and features c-code generated from python template scripts, the capacity to operate directly on arrays in files, arrays of heterogeneous records, string arrays, and in-place operation on memory mapped files. ========================================================================= Release Notes for numarray-1.4.0 I. ENHANCEMENTS 1. Speed improvement for numarray operators. The Python level hook mapping numarray operators onto universal functions has been moved down to C. 2. Speed improvement for string-array comparisons, any(), all(). String correlation is ~10x faster. 3. Better operation with py2exe to help it automatically detect the core numarray extensions to include in an installer. 4. scipy newcore compatible lower case type names (e.g. int32 not Int32) 5. scipy newcore 'dtype' keyword and .dtypechar attribute. II. BUGS FIXED / CLOSED 1323355 Apps fail with import_libnumarray 1315212 Infinite loop converting some scalar strings into a list 1298916 rank-0 tostring() broken 1297948 records.array fails to create empty fields 1286291 import sys missing from array_persist.py 1286168 Generic sequences in ``strings.array()`` 1236392 Outdated web link in announcements 1235219 LinearAlgebraError not imported in linear_algebra See http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse for more details. III. CAUTIONS This release should be backward binary compatible with numarray-1.3.x WHERE ----------- Numarray-1.4.0 windows executable installers, source code, and manual is here: http://sourceforge.net/project/showfiles.php?group_id=1369 Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://www.stsci.edu/resources/software_hardware/numarray Trackers for Numarray Bugs, Feature Requests, Support, and Patches are at the Source Forge project for NumPy at: http://sourceforge.net/tracker/?group_id=1369 REQUIREMENTS ------------------------------ numarray-1.4.0 requires Python 2.2.2 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science Institute. We'd like to acknowledge the assitance of Francesc Alted, Paul Dubois, Sebastian Haase, Chuck Harris, Tim Hochberg, Nadav Horesh, Edward C. Jones, Eric Jones, Jochen Kuepper, Travis Oliphant, Pearu Peterson, Peter Verveer, Colin Williams, Rory Yorke, and everyone else who has contributed with comments and feedback. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. -- Todd Miller jmiller at stsci.edu From chaves at computer.org Wed Oct 26 18:57:32 2005 From: chaves at computer.org (John A Chaves) Date: Wed Oct 26 18:57:32 2005 Subject: [Numpy-discussion] numarray-1.4.0 string-array comparisons problem In-Reply-To: <435F9F2F.2000005@stsci.edu> References: <435F9F2F.2000005@stsci.edu> Message-ID: <200510262055.39204.chaves@computer.org> After installing numarray-1.4.0, I get a Segmentation fault when comparing two instances of the same string-array. For example: from numarray.strings import asarray for x in range(1000,1000000,1000): print x raw = ['abcde']*x arr = asarray(raw) arr == arr gives: 1000 2000 3000 4000 5000 6000 Segmentation fault I verified that this works correctly in numarray-1.3.3. Also, the problem can be avoided in 1.4.0 by replacing 'arr == arr' with 'arr = arr.copy()'. Can anyone else reproduce this, or is a problem with my environment? Thanks, John From jmiller at stsci.edu Thu Oct 27 10:51:40 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu Oct 27 10:51:40 2005 Subject: [Numpy-discussion] numarray-1.4.0 string-array comparisons problem In-Reply-To: <200510262055.39204.chaves@computer.org> References: <435F9F2F.2000005@stsci.edu> <200510262055.39204.chaves@computer.org> Message-ID: <436112CA.2050202@stsci.edu> John A Chaves wrote: >After installing numarray-1.4.0, I get a Segmentation fault when comparing >two instances of the same string-array. For example: > > from numarray.strings import asarray > > for x in range(1000,1000000,1000): > print x > raw = ['abcde']*x > arr = asarray(raw) > arr == arr > >gives: > > 1000 > 2000 > 3000 > 4000 > 5000 > 6000 > Segmentation fault > > >Can anyone else reproduce this, or is a problem with my environment? > > Yes, I was able to reproduce this too, thanks for the report and for logging on SF. I understand the problem and am working out a fix now. Regards, Todd From jmiller at stsci.edu Thu Oct 27 15:25:25 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu Oct 27 15:25:25 2005 Subject: [Numpy-discussion] numarray-1.4.0 string-array comparisons problem In-Reply-To: <436112CA.2050202@stsci.edu> References: <435F9F2F.2000005@stsci.edu> <200510262055.39204.chaves@computer.org> <436112CA.2050202@stsci.edu> Message-ID: <43615361.70404@stsci.edu> Todd Miller wrote: > John A Chaves wrote: > >> After installing numarray-1.4.0, I get a Segmentation fault when >> comparing >> two instances of the same string-array. For example: >> >> from numarray.strings import asarray >> >> for x in range(1000,1000000,1000): >> print x >> raw = ['abcde']*x >> arr = asarray(raw) >> arr == arr >> >> gives: >> >> 1000 >> 2000 >> 3000 >> 4000 >> 5000 >> 6000 >> Segmentation fault >> >> >> Can anyone else reproduce this, or is a problem with my environment? >> >> > Yes, I was able to reproduce this too, thanks for the report and for > logging on SF. I understand the problem and am working out a fix now. > > Regards, > Todd I committed the fix for this and put out numarray-1.4.1. Thanks again, Todd From a.h.jaffe at gmail.com Thu Oct 27 15:51:57 2005 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Thu Oct 27 15:51:57 2005 Subject: [Numpy-discussion] Re: ANN: numarray-1.4.0 In-Reply-To: <435F9F2F.2000005@stsci.edu> References: <435F9F2F.2000005@stsci.edu> Message-ID: <43615751.8080009@gmail.com> Hi- > 4. scipy newcore compatible lower case type names (e.g. int32 not Int32) Is this actually true? For example, numarray.Float64 exists as before, but numarray.float64 does not. > 5. scipy newcore 'dtype' keyword and .dtypechar attribute. And more generally, is there any more information on the relationship between numarray and scipy_core? Will numarray be phased out eventually (i.e., should we start moving over to scipy?)? But in any event thanks for the hard work! Andrew From jmiller at stsci.edu Fri Oct 28 02:20:26 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 28 02:20:26 2005 Subject: [Numpy-discussion] Re: ANN: numarray-1.4.0 In-Reply-To: <43615751.8080009@gmail.com> References: <435F9F2F.2000005@stsci.edu> <43615751.8080009@gmail.com> Message-ID: <4361ECFC.9000904@stsci.edu> Andrew Jaffe wrote: > Hi- > >> 4. scipy newcore compatible lower case type names (e.g. int32 not Int32) > > > Is this actually true? For example, numarray.Float64 exists as before, > but numarray.float64 does not. I added the "string aliases" but didn't create new Python object bindings. So you can currently say something like arange(10, dtype='float32') but cannot yet say: arange(10, dtype=float32) Good catch. I'll add those soon. >> 5. scipy newcore 'dtype' keyword and .dtypechar attribute. > > > And more generally, is there any more information on the relationship > between numarray and scipy_core? I think scipy_core is intended to add the best numarray-like features to Numeric. > Will numarray be phased out eventually (i.e., should we start moving > over to scipy?)? When scipy_core matures enough to replace numarray, numarray will be phased out. If you rely on numarray, it is definitely worth your time to keep an eye on scipy_core and switch when it meets your needs better. Regards, Todd From jmiller at stsci.edu Fri Oct 28 06:28:17 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 28 06:28:17 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 Message-ID: <43622697.9040302@stsci.edu> Numarray is an array processing package designed to efficiently manipulate large multi-dimensional arrays. Numarray is modelled after Numeric and features c-code generated from python template scripts, the capacity to operate directly on arrays in files, arrays of heterogeneous records, string arrays, and in-place operation on memory mapped files. ========================================================================= Release Notes for numarray-1.4.1 II. BUGS FIXED / CLOSED 1339713 segfault when comparing string-array to self ========================================================================= Release Notes for numarray-1.4.0 I. ENHANCEMENTS 1. Speed improvement for numarray operators. The Python level hook mapping numarray operators onto universal functions has been moved down to C. 2. Speed improvement for string-array comparisons, any(), all(). String correlation is ~10x faster. 3. Better operation with py2exe to help it automatically detect the core numarray extensions to include in an installer. 4. scipy newcore compatible lower case type names (e.g. int32 not Int32) 5. scipy newcore 'dtype' keyword and .dtypechar attribute. II. BUGS FIXED / CLOSED 1323355 Apps fail with import_libnumarray 1315212 Infinite loop converting some scalar strings into a list 1298916 rank-0 tostring() broken 1297948 records.array fails to create empty fields 1286291 import sys missing from array_persist.py 1286168 Generic sequences in ``strings.array()`` 1236392 Outdated web link in announcements 1235219 LinearAlgebraError not imported in linear_algebra See http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse for more details. III. CAUTIONS This release should be backward binary compatible with numarray-1.3.x WHERE ----------- Numarray-1.4.0 windows executable installers, source code, and manual is here: http://sourceforge.net/project/showfiles.php?group_id=1369 Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://www.stsci.edu/resources/software_hardware/numarray Trackers for Numarray Bugs, Feature Requests, Support, and Patches are at the Source Forge project for NumPy at: http://sourceforge.net/tracker/?group_id=1369 REQUIREMENTS ------------------------------ numarray-1.4.0 requires Python 2.2.2 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science Institute. We'd like to acknowledge the assitance of Francesc Alted, Paul Dubois, Sebastian Haase, Chuck Harris, Tim Hochberg, Nadav Horesh, Edward C. Jones, Eric Jones, Jochen Kuepper, Travis Oliphant, Pearu Peterson, Peter Verveer, Colin Williams, Rory Yorke, and everyone else who has contributed with comments and feedback. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. -- Todd Miller jmiller at stsci.edu From faltet at carabos.com Fri Oct 28 09:05:48 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 28 09:05:48 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <43622697.9040302@stsci.edu> References: <43622697.9040302@stsci.edu> Message-ID: <200510281803.46663.faltet@carabos.com> Hi Todd, I've just installed numarray 1.4.1 and pass my tests over it. Everything goes well, except some small detail: Python 2.4.2 (#2, Sep 29 2005, 00:23:59) [GCC 4.0.2 (Debian 4.0.1-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import Numeric >>> import numarray >>> Numeric.__version__ '24.0b2' >>> numarray.__version__ '1.4.1' >>> na=numarray.array([ 51.], type=numarray.Float64) >>> Numeric.array(na, typecode='d') Traceback (most recent call last): File "", line 1, in ? TypeError: expected a writeable buffer object >>> This worked in 1.3.x and before. I hope that this is not the intended behaviour. Thanks, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From jmiller at stsci.edu Fri Oct 28 11:00:32 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 28 11:00:32 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <200510281803.46663.faltet@carabos.com> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> Message-ID: <436266DB.1020308@stsci.edu> Francesc Altet wrote: >Hi Todd, > >I've just installed numarray 1.4.1 and pass my tests over it. >Everything goes well, except some small detail: > >Python 2.4.2 (#2, Sep 29 2005, 00:23:59) >[GCC 4.0.2 (Debian 4.0.1-9)] on linux2 >Type "help", "copyright", "credits" or "license" for more information. > > >>>>import Numeric >>>>import numarray >>>>Numeric.__version__ >>>> >>>> >'24.0b2' > > >>>>numarray.__version__ >>>> >>>> >'1.4.1' > > >>>>na=numarray.array([ 51.], type=numarray.Float64) >>>>Numeric.array(na, typecode='d') >>>> >>>> >Traceback (most recent call last): > File "", line 1, in ? >TypeError: expected a writeable buffer object > > > >This worked in 1.3.x and before. > >I hope that this is not the intended behaviour. > > This looks like a coordinated change in both numarray and Numeric. (Travis?) I updated to Numeric-24.0 and all was well. It appears the __array_data__ protocol improved to know about readonly-ness. In order to support the readonly behavior in numarray, I think the change must be made in lockstep. We could however back out support for readonly while Numeric-24.0 becomes more pervasive. Regards, Todd From faltet at carabos.com Fri Oct 28 11:23:26 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 28 11:23:26 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <436266DB.1020308@stsci.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> Message-ID: <200510282021.31288.faltet@carabos.com> A Divendres 28 Octubre 2005 19:58, Todd Miller va escriure: > This looks like a coordinated change in both numarray and Numeric. > (Travis?) I updated to Numeric-24.0 and all was well. Ops, I was not aware that Numeric-24.0 was out. Anyway, I've tried to compile it, but I did not succeed with my Debian system: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -IInclude -IPackages/FFT/Include -IPackages/RNG/Include -I/usr/include/python2.4 -c Src/lapack_litemodule.c -o build/temp.linux-i686-2.4/Src/lapack_litemodule.o gcc -pthread -shared build/temp.linux-i686-2.4/Src/lapack_litemodule.o -L/usr/lib/atlas -llapack -lcblas -lf77blas -latlas -lg2c -o build/lib.linux-i686-2.4/lapack_lite.so /usr/bin/ld: cannot find -llapack collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 The installation for Numeric usually worked on my system. Perhaps it is now assumed that lapack should be a pre-requisite? > It appears the __array_data__ protocol improved to know about > readonly-ness. In order to support the readonly behavior in numarray, > I think the change must be made in lockstep. We could however back out > support for readonly while Numeric-24.0 becomes more pervasive. Yes, please, backward compatibility with pre-Numeric 24.0 would be a nice thing to have. Regards, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From Chris.Barker at noaa.gov Fri Oct 28 11:39:50 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Fri Oct 28 11:39:50 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <200510282021.31288.faltet@carabos.com> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> <200510282021.31288.faltet@carabos.com> Message-ID: <43626FDA.8070201@noaa.gov> Francesc Altet wrote: > /usr/bin/ld: cannot find -llapack > collect2: ld returned 1 exit status > error: command 'gcc' failed with exit status 1 > > The installation for Numeric usually worked on my system. Perhaps it > is now assumed that lapack should be a pre-requisite? This has been a problem for a remarkably long time! I thought it had been fixed. It's not really a big deal, it's only a matter of what the defaults are in customize.py. The comments in there should be enough to figure it out. The default really should be to use the built-in lapack-lite. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From faltet at carabos.com Fri Oct 28 12:03:42 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 28 12:03:42 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <43626FDA.8070201@noaa.gov> References: <43622697.9040302@stsci.edu> <200510282021.31288.faltet@carabos.com> <43626FDA.8070201@noaa.gov> Message-ID: <200510282101.23736.faltet@carabos.com> A Divendres 28 Octubre 2005 20:37, Chris Barker va escriure: > This has been a problem for a remarkably long time! I thought it had > been fixed. It's not really a big deal, it's only a matter of what the > defaults are in customize.py. The comments in there should be enough to > figure it out. Yup, it worked. Thanks. Anyway, I think that the sensible default would be to not suppose that neither BLAS, LAPACK, ATALS are installed and update the README to give instructuctions on how to adapt Numeric to your system. Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From oliphant at ee.byu.edu Fri Oct 28 12:23:01 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri Oct 28 12:23:01 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <200510282101.23736.faltet@carabos.com> References: <43622697.9040302@stsci.edu> <200510282021.31288.faltet@carabos.com> <43626FDA.8070201@noaa.gov> <200510282101.23736.faltet@carabos.com> Message-ID: <43627A2F.4090307@ee.byu.edu> Francesc Altet wrote: >A Divendres 28 Octubre 2005 20:37, Chris Barker va escriure: > > >>This has been a problem for a remarkably long time! I thought it had >>been fixed. It's not really a big deal, it's only a matter of what the >>defaults are in customize.py. The comments in there should be enough to >>figure it out. >> >> > >Yup, it worked. Thanks. > >Anyway, I think that the sensible default would be to not suppose that >neither BLAS, LAPACK, ATALS are installed and update the README to >give instructuctions on how to adapt Numeric to your system. > > > Yes, that was always the plan. I just forgot to reset customize.py to the original before making a tar ball. I've fixed the problem now. The new tar ball does uses default lapack_lite. (I'm sure I'll upset somebody because the tar ball changed slightly with no version number increase --- but I don't want to make a new release --- the other files should be fine). -Travis From oliphant at ee.byu.edu Fri Oct 28 12:29:43 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri Oct 28 12:29:43 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <436266DB.1020308@stsci.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> Message-ID: <43627BBE.6070109@ee.byu.edu> Todd Miller wrote: > Francesc Altet wrote: > >> Hi Todd, >> >> I've just installed numarray 1.4.1 and pass my tests over it. >> Everything goes well, except some small detail: >> >> Python 2.4.2 (#2, Sep 29 2005, 00:23:59) >> [GCC 4.0.2 (Debian 4.0.1-9)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >> >>>>> import Numeric >>>>> import numarray >>>>> Numeric.__version__ >>>>> >>>> >> '24.0b2' >> >> >>>>> numarray.__version__ >>>>> >>>> >> '1.4.1' >> >> >>>>> na=numarray.array([ 51.], type=numarray.Float64) >>>>> Numeric.array(na, typecode='d') >>>>> >>>> >> Traceback (most recent call last): >> File "", line 1, in ? >> TypeError: expected a writeable buffer object >> >> > This looks like a coordinated change in both numarray and Numeric. > (Travis?) I updated to Numeric-24.0 and all was well. > It appears the __array_data__ protocol improved to know about > readonly-ness. In order to support the readonly behavior in > numarray, I think the change must be made in lockstep. We could > however back out support for readonly while Numeric-24.0 becomes more > pervasive. Yes, basically the __array_data__ protocol was essentially pointless before because it just returned a buffer object which 1) did nothing more than the object itself supporting the buffer protocol and 2) could not handle strided (discontiguous) arrays. The new array protocol handles the situation much better and allows discontiguous arrays to be shared. However, it does create some backward compatibility issues. But, because Numeric 24.0b2 was a *beta* release I don't see this as a real problem. Get the final release of Numeric, which will handle the array protocol correctly (note it will also handle the old __array_data__ format as well). Todd, I was not sure how to change numarray so it would consume the new protocol correctly. So, I'm not sure how numarray interprets the __array_data__ attribute. -Travis From edcjones at comcast.net Fri Oct 28 12:48:37 2005 From: edcjones at comcast.net (Edward C. Jones) Date: Fri Oct 28 12:48:37 2005 Subject: [Numpy-discussion] numarray 1.4.0: unitary operator "+" fails Message-ID: <43628028.8040705@comcast.net> The unitary "+" does not work for numarray 1.4.0: > python Python 2.4.2 (#1, Oct 1 2005, 18:47:35) [GCC 4.0.2 20050821 (prerelease) (Debian 4.0.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numarray >>> arr = numarray.zeros((2,2)) >>> +arr Traceback (most recent call last): File "", line 1, in ? AttributeError: copy From jmiller at stsci.edu Fri Oct 28 13:06:36 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 28 13:06:36 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <43627BBE.6070109@ee.byu.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> <43627BBE.6070109@ee.byu.edu> Message-ID: <4362844D.4080901@stsci.edu> Travis Oliphant wrote: > Todd Miller wrote: > >> Francesc Altet wrote: >> >>> Hi Todd, >>> >>> I've just installed numarray 1.4.1 and pass my tests over it. >>> Everything goes well, except some small detail: >>> >>> Python 2.4.2 (#2, Sep 29 2005, 00:23:59) >>> [GCC 4.0.2 (Debian 4.0.1-9)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> >>>>>> import Numeric >>>>>> import numarray >>>>>> Numeric.__version__ >>>>>> >>>>> >>>>> >>> '24.0b2' >>> >>> >>>>>> numarray.__version__ >>>>>> >>>>> >>>>> >>> '1.4.1' >>> >>> >>>>>> na=numarray.array([ 51.], type=numarray.Float64) >>>>>> Numeric.array(na, typecode='d') >>>>>> >>>>> >>>>> >>> Traceback (most recent call last): >>> File "", line 1, in ? >>> TypeError: expected a writeable buffer object >>> >>> > >> This looks like a coordinated change in both numarray and Numeric. >> (Travis?) I updated to Numeric-24.0 and all was well. > > > >> It appears the __array_data__ protocol improved to know about >> readonly-ness. In order to support the readonly behavior in >> numarray, I think the change must be made in lockstep. We could >> however back out support for readonly while Numeric-24.0 becomes more >> pervasive. > > > Yes, basically the __array_data__ protocol was essentially pointless > before because it just returned a buffer object which > > 1) did nothing more than the object itself supporting the buffer > protocol and > 2) could not handle strided (discontiguous) arrays. > > The new array protocol handles the situation much better and allows > discontiguous arrays to be shared. > > However, it does create some backward compatibility issues. But, > because Numeric 24.0b2 was a *beta* release I don't see this as a real > problem. Yeah, that sounds ok... I was thinking the array interface was a release or two older. > Get the final release of Numeric, which will handle the array > protocol correctly (note it will also handle the old __array_data__ > format as well). > > Todd, I was not sure how to change numarray so it would consume the > new protocol correctly. I think numarray's multiple factory functions in numarray, strings, and records all need to be beefed up to look for it. > So, I'm not sure how numarray interprets the __array_data__ attribute. I don't know if it does right now. Just to be clear, what Francesc was talking about is *supplying* the array buffer from numarray as apparently defined in the __get_array_data__ method. If 24b was the first time the array interface ever worked it sounds ok to leave your changes in and force an upgrade to 24 if a user uses that feature. Francesc? Todd From oliphant at ee.byu.edu Fri Oct 28 13:53:19 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri Oct 28 13:53:19 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <4362844D.4080901@stsci.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> <43627BBE.6070109@ee.byu.edu> <4362844D.4080901@stsci.edu> Message-ID: <43628F84.3040200@ee.byu.edu> > Todd Miller wrote: > I don't know if it does right now. Just to be clear, what Francesc > was talking about is *supplying* the array buffer from numarray as > apparently defined in the __get_array_data__ method. If 24b was the > first time the array interface ever worked it sounds ok to leave your > changes in and force an upgrade to 24 if a user uses that feature. > Francesc? Right, he was talking about just supplying the array data information from numarray. And, yes, Numeric 24b was the first time the array interface ever worked. -Travis From edcjones at comcast.net Fri Oct 28 18:26:53 2005 From: edcjones at comcast.net (Edward C. Jones) Date: Fri Oct 28 18:26:53 2005 Subject: [Numpy-discussion] numarray 1.4.1: Infinite loop during test Message-ID: <4362CF93.4050605@comcast.net> I have a PC with Debian unstable. I use Python 1.4.2. After installing numarray 1.4.1, I ran the tests: >>> import numarray.testall as testall >>> testall.test() Testing numarray 1.4.1 on normal Python (2, 4, 2, 'final', 0) on platform linux2 numarray.numtest: 2.89 ((0, 1205), (0, 1205)) numarray.ieeespecial: 0.07 (0, 86) numarray.records: 0.05 (0, 48) numarray.strings: 0.17 (0, 189) numarray.memmap: 0.09 (0, 82) numarray.objects: 0.14 (0, 105) numarray.memorytest: 0.01 (0, 16) numarray.examples.convolve: 0.09 ((0, 20), (0, 20), (0, 20), (0, 20)) numarray.convolve: 0.06 (0, 45) numarray.fft: 0.11 (0, 75) After the above, nothing happens. "ps ax" show a lot of time being used. Looks like an infinite loop. Process had to be killed. From jmiller at stsci.edu Sat Oct 29 05:11:07 2005 From: jmiller at stsci.edu (Todd Miller) Date: Sat Oct 29 05:11:07 2005 Subject: [Numpy-discussion] numarray 1.4.1: Infinite loop during test In-Reply-To: <4362CF93.4050605@comcast.net> References: <4362CF93.4050605@comcast.net> Message-ID: <4363667F.9070000@stsci.edu> Edward C. Jones wrote: > I have a PC with Debian unstable. I use Python 1.4.2. After installing > numarray 1.4.1, I ran the tests: > > >>> import numarray.testall as testall > >>> testall.test() > Testing numarray 1.4.1 on normal Python (2, 4, 2, 'final', 0) on > platform linux2 > numarray.numtest: 2.89 ((0, 1205), (0, 1205)) > numarray.ieeespecial: 0.07 (0, 86) > numarray.records: 0.05 (0, 48) > numarray.strings: 0.17 (0, 189) > numarray.memmap: 0.09 (0, 82) > numarray.objects: 0.14 (0, 105) > numarray.memorytest: 0.01 (0, 16) > numarray.examples.convolve: 0.09 ((0, 20), (0, 20), > (0, 20), (0, 20)) > numarray.convolve: 0.06 (0, 45) > numarray.fft: 0.11 (0, 75) > > After the above, nothing happens. "ps ax" show a lot of time being > used. Looks like an infinite loop. Process had to be killed. I don't have access to a Debian system but some things come to mind: a. The loop looks like it's in numarray.linear_algebra. b. I changed numarray's setup to make the "config" command automatic for 1.4.0. That changes how the builtin lapack_lite module is configured/built which affects linear_algebra. c. My impression is that debian builds numarray against external libraries so (b) shouldn't be the problem. Does the numarray tarball from source forge: http://prdownloads.sourceforge.net/numpy/numarray-1.4.1.tar.gz?download install and self-test OK using "python setup.py install"? Regards, Todd From faltet at carabos.com Sat Oct 29 09:07:22 2005 From: faltet at carabos.com (Francesc Altet) Date: Sat Oct 29 09:07:22 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <43628F84.3040200@ee.byu.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> <43627BBE.6070109@ee.byu.edu> <4362844D.4080901@stsci.edu> <43628F84.3040200@ee.byu.edu> Message-ID: <1130602931.2848.4.camel@localhost.localdomain> El dv 28 de 10 del 2005 a les 14:52 -0600, en/na Travis Oliphant va escriure: > > Todd Miller wrote: > > I don't know if it does right now. Just to be clear, what Francesc > > was talking about is *supplying* the array buffer from numarray as > > apparently defined in the __get_array_data__ method. If 24b was the > > first time the array interface ever worked it sounds ok to leave your > > changes in and force an upgrade to 24 if a user uses that feature. > > Francesc? > Right, he was talking about just supplying the array data information > from numarray. > > And, yes, Numeric 24b was the first time the array interface ever worked. Yes, I think that if Numeric 24b was the first implementing the array interface, then there is no need to touch nothing in numarray. As you said, I should make Numeric 24.0 a requisite for running our software. Thanks for clarificating the issue, -- >0,0< Francesc Altet http://www.carabos.com/ V V C?rabos Coop. V. Enjoy Data "-" From rowen at cesmail.net Mon Oct 31 16:56:21 2005 From: rowen at cesmail.net (Russell E. Owen) Date: Mon Oct 31 16:56:21 2005 Subject: [Numpy-discussion] byteswap questions Message-ID: I've got a module that can send array data to ds9. It screws up on byteswapped data and I'm trying to fix it. The code sends the data to ds9 via a pipe using arr.tofile(). To send byteswapped data I think I have two choices (given that I don't want to modify the input array): 1) Figure out the byte order and specify a suitable flag to ds9 (it accepts "bigendian" or "littleendian" flags). This avoids copying the data, so it sounds attractive, but can I do it without relying on "internals"? Numarrays have an "isbyteswapped" method, but all this says is "not native machine order"; it doesn't actually tell me the order. I need to know if the order is bigendian or littleendian, and I can't find a documented way to get that, just an undocumented attribute _byteorder. 2) If the array is not in native byte order, make a native byte ordered copy before sending it. The following seems to work: if arr.isbyteswapped(): arr = arr.copy() but is this the recommended way to get native-order copy? - Is "copy" guaranteed to return a copy in native byte order? The documentation doesn't say. - I was tempted to use num.array(arr) to make the copy, but the documentation doesn't say what "array" does if the input is a already a numarray array. As an aside, I tried to use the byteswapped method, but it has a totally different effect: >>> d array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.], ...type=Float32) >>> d.isbyteswapped() 1 >>> dcopy = d.copy() >>> d array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.], ...type=Float32) >>> dcopy.isbyteswapped() 0 >>> dswap = d.byteswapped() >>> dswap array([[ 1.91063654e-38, 1.91063654e-38, 1.91063654e-38, ..., ...type=Float32) which I guess means I'd have to byteswap the byteswapped data. Aargh. -- Russell From bgranger at scu.edu Sun Oct 2 11:03:07 2005 From: bgranger at scu.edu (Brian Granger) Date: Sun Oct 2 11:03:07 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Release of SciPy Core 0.4 (Beta) Message-ID: Which version of gccc are you using? As I understand it, g77 3.4 is not compatible with gcc 4.0 (the default on OS X). Switching to gcc 3.4 might help: gcc_select 3.3 I will soon try scipy_core on Tiger with Python 2.4.1. Brian Brian Granger, Ph.D. Assistant Professor of Physics Santa Clara University bgranger at scu.edu Phone: 408-551-1891 Fax: 408-554-6965 >>> managan at llnl.gov 09/30/05 11:21 AM >>> OK, I am giving this a go. I seem to be not setting a flag about some math functions availability. Here are the details (aorry about the length of it but I snipped a lot!): OSX 10.3.9, Python 2.4.1 Installed g77v3.4-bin.tar.gz from http://hpc.sourceforge.net/ as described into /usr/local/... Installed fftw-3.0.1-fma.tar.gz from http://www.fftw.org/download.html into ~/Documents/local/... where I have other libraries I have installed. Got scipy_core-0.4.1.tar.gz from http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=6222, wanted to try out the new replacement for Numeric tar -zxvf scipy_core-0.4.1.tar.gz cd scipy_core-0.4.1 python setup.py build Assuming default configuration (scipy/distutils/command/{setup_command,setup}.py was not found) Appending scipy.distutils.command configuration to scipy.distutils Assuming default configuration (scipy/distutils/fcompiler/{setup_fcompiler,setup}.py was not found) ... Appending scipy.base configuration to scipy blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] Appending scipy.lib configuration to scipy Assuming default configuration (scipy/fftpack/{setup_fftpack,setup}.py was not found) ... scipy_core version 0.4.1 running build running config_fc running build_src building extension "scipy.base.multiarray" sources creating build creating build/src creating build/src/scipy creating build/src/scipy/base Generating build/src/scipy/base/config.h customize NAGFCompiler customize AbsoftFCompiler customize IbmFCompiler Could not locate executable f77 customize GnuFCompiler customize GnuFCompiler ... compiling C sources gcc options: '-fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes' creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base/src compile options: '-Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c' gcc: build/src/scipy/base/src/umathmodule.c In file included from build/src/scipy/base/src/umathmodule.c:8: scipy/base/include/scipy/arrayobject.h:84: warning: redefinition of `ushort' /usr/include/sys/types.h:82: warning: `ushort' previously declared here scipy/base/include/scipy/arrayobject.h:85: warning: redefinition of `uint' /usr/include/sys/types.h:83: warning: `uint' previously declared here build/src/scipy/base/src/umathmodule.c: In function `nc_floor_quotl': build/src/scipy/base/src/umathmodule.c:1257: warning: implicit declaration of function `floorl' build/src/scipy/base/src/umathmodule.c: In function `nc_sqrtl': build/src/scipy/base/src/umathmodule.c:1269: warning: implicit declaration of function `sqrtl' ... build/src/scipy/base/__umath_generated.c:171: error: initializer element is not constant build/src/scipy/base/__umath_generated.c:171: error: (near initialization for `arccosh_data[2]') error: Command "gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dy namic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/s rc/scipy/base -Iscipy/base/src -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c b uild/src/scipy/base/src/umathmodule.c -o build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base /src/umathmodule.o" failed with exit status 1 At 12:59 AM -0600 9/30/05, Travis Oliphant wrote: >This is to announce the release of SciPy Core 0.4.X (beta) > >It is available at sourceforge which you can access by going to > >http://numeric.scipy.org > >Thanks to all who helped me with this release, especially > >Robert Kern >Pearu Peterson > >Now, let's start getting this stable.... > >-Travis Oliphant > > > >_______________________________________________ >SciPy-user mailing list >SciPy-user at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-user -- *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- Rob Managan email managan at llnl.gov LLNL phone: 925-423-0903 P.O. Box 808, L-095 FAX: 925-422-3389 Livermore, CA 94551-0808 _______________________________________________ SciPy-user mailing list SciPy-user at scipy.net http://www.scipy.net/mailman/listinfo/scipy-user This message scanned for viruses and SPAM by GWGuardian at SCU (MGW1) From gerard.vermeulen at grenoble.cnrs.fr Sun Oct 2 12:41:08 2005 From: gerard.vermeulen at grenoble.cnrs.fr (Gerard Vermeulen) Date: Sun Oct 2 12:41:08 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <433CE24A.6040509@ee.byu.edu> References: <433CE24A.6040509@ee.byu.edu> Message-ID: <20051002213837.70eec02c.gerard.vermeulen@grenoble.cnrs.fr> On Fri, 30 Sep 2005 00:59:22 -0600 Travis Oliphant wrote: > > This is to announce the release of SciPy Core 0.4.X (beta) > > It is available at sourceforge which you can access by going to > > http://numeric.scipy.org > > Thanks to all who helped me with this release, especially > > Robert Kern > Pearu Peterson > > Now, let's start getting this stable.... > > -Travis Oliphant > I have found two problems: (1) scipy_core-0.4.1 does not compile on this system without ATLAS, because there are missing files: ... creating build/temp.linux-i686-2.4/lapack_lite compile options: '-Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c' gcc: scipy/corelib/lapack_lite/lapack_litemodule.c gcc: lapack_lite/dlapack_lite.c gcc: lapack_lite/dlapack_lite.c: No such file or directory gcc: no input files gcc: lapack_lite/dlapack_lite.c: No such file or directory gcc: no input files error: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -fomit-frame-pointer -pipe -march=i586 -mtune=pentiumpro-g -fPIC -Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c lapack_lite/dlapack_lite.c -o build/temp.linux-i686-2.4/lapack_lite/dlapack_lite.o" failed with exit status 1 [packer at titan scipy_core-0.4.1]$ (2) when building scipy_core from SVN the header files do not get installed. The install section from my RPM SPEC file reads: %install rm -rf %{buildroot} python setup.py install --root=%{buildroot} # The API headers do not get installed mkdir -p %{buildroot}/%{_includedir}/python%{pyver}/scipy/ for header in scipy/base/include/scipy/*object.h ; do install -m 644 $header %{buildroot}/%{_includedir}/python%{pyver}/scipy/ done # The generated API headers and config.h do not get installed either for header in build/src/scipy/base/*.h ; do install -m 644 $header %{buildroot}/%{_includedir}/python%{pyver}/scipy/ done # Mandrake 10.2 needs this: %multiarch_includes %{buildroot}/%{_includedir}/python%{pyver}/scipy/config.h %clean #rm -rf %{buildroot} Regards -- Gerard From managan at llnl.gov Sun Oct 2 13:46:48 2005 From: managan at llnl.gov (Rob Managan) Date: Sun Oct 2 13:46:48 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Release of SciPy Core 0.4 (Beta) In-Reply-To: <433CE24A.6040509@ee.byu.edu> References: <433CE24A.6040509@ee.byu.edu> Message-ID: OK, I am giving this a go. I seem to be not setting a flag about some math functions availability. Here are the details (aorry about the length of it but I snipped a lot!): OSX 10.3.9, Python 2.4.1 Installed g77v3.4-bin.tar.gz from http://hpc.sourceforge.net/ as described into /usr/local/... Installed fftw-3.0.1-fma.tar.gz from http://www.fftw.org/download.html into ~/Documents/local/... where I have other libraries I have installed. Got scipy_core-0.4.1.tar.gz from http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=6222, wanted to try out the new replacement for Numeric tar -zxvf scipy_core-0.4.1.tar.gz cd scipy_core-0.4.1 python setup.py build Assuming default configuration (scipy/distutils/command/{setup_command,setup}.py was not found) Appending scipy.distutils.command configuration to scipy.distutils Assuming default configuration (scipy/distutils/fcompiler/{setup_fcompiler,setup}.py was not found) ... Appending scipy.base configuration to scipy blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] Appending scipy.lib configuration to scipy Assuming default configuration (scipy/fftpack/{setup_fftpack,setup}.py was not found) ... scipy_core version 0.4.1 running build running config_fc running build_src building extension "scipy.base.multiarray" sources creating build creating build/src creating build/src/scipy creating build/src/scipy/base Generating build/src/scipy/base/config.h customize NAGFCompiler customize AbsoftFCompiler customize IbmFCompiler Could not locate executable f77 customize GnuFCompiler customize GnuFCompiler ... compiling C sources gcc options: '-fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes' creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base/src compile options: '-Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c' gcc: build/src/scipy/base/src/umathmodule.c In file included from build/src/scipy/base/src/umathmodule.c:8: scipy/base/include/scipy/arrayobject.h:84: warning: redefinition of `ushort' /usr/include/sys/types.h:82: warning: `ushort' previously declared here scipy/base/include/scipy/arrayobject.h:85: warning: redefinition of `uint' /usr/include/sys/types.h:83: warning: `uint' previously declared here build/src/scipy/base/src/umathmodule.c: In function `nc_floor_quotl': build/src/scipy/base/src/umathmodule.c:1257: warning: implicit declaration of function `floorl' build/src/scipy/base/src/umathmodule.c: In function `nc_sqrtl': build/src/scipy/base/src/umathmodule.c:1269: warning: implicit declaration of function `sqrtl' ... build/src/scipy/base/__umath_generated.c:171: error: initializer element is not constant build/src/scipy/base/__umath_generated.c:171: error: (near initialization for `arccosh_data[2]') error: Command "gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dy namic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/s rc/scipy/base -Iscipy/base/src -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c b uild/src/scipy/base/src/umathmodule.c -o build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base /src/umathmodule.o" failed with exit status 1 At 12:59 AM -0600 9/30/05, Travis Oliphant wrote: >This is to announce the release of SciPy Core 0.4.X (beta) > >It is available at sourceforge which you can access by going to > >http://numeric.scipy.org > >Thanks to all who helped me with this release, especially > >Robert Kern >Pearu Peterson > >Now, let's start getting this stable.... > >-Travis Oliphant > > > >_______________________________________________ >SciPy-user mailing list >SciPy-user at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-user -- *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- Rob Managan email managan at llnl.gov LLNL phone: 925-423-0903 P.O. Box 808, L-095 FAX: 925-422-3389 Livermore, CA 94551-0808 From efiring at hawaii.edu Sun Oct 2 18:17:42 2005 From: efiring at hawaii.edu (Eric Firing) Date: Sun Oct 2 18:17:42 2005 Subject: [Numpy-discussion] Re: Release of SciPy Core 0.4 (Beta) In-Reply-To: <20051001031446.28F4210589F@sc8-sf-spam1.sourceforge.net> References: <20051001031446.28F4210589F@sc8-sf-spam1.sourceforge.net> Message-ID: <433F279D.8070109@hawaii.edu> Travis, > This is to announce the release of SciPy Core 0.4.X (beta) Wonderful progress--I can't thank you enough for the work you are doing on this. I initially tried to build from the tarball, but ran into missing pieces: compile options: '-Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c' gcc: lapack_lite/blas_lite.c gcc: lapack_lite/blas_lite.c: No such file or directory gcc: no input files I then installed from the rpm (on a Mandriva 2005 LE system) with no problems, and proceeded to poke around. It looks great! > Now, let's start getting this stable.... Bug: in function_base.py, there are four places where "nx" should be "_nx"; see attached diff. Also attached is a diff for arraymethods, to fix what look to me like erroneous docstrings for the reshape and resize methods. The present docstrings indicate that reshape makes the change in place, and resize returns a new array, but in both cases the opposite is true--reshape returns a reshaped copy, and resize returns None. Eric -------------- next part -------------- A non-text attachment was scrubbed... Name: function_base.diff Type: text/x-patch Size: 991 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: arraymethods.diff Type: text/x-patch Size: 1370 bytes Desc: not available URL: From tchur at optusnet.com.au Sun Oct 2 21:52:41 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Sun Oct 2 21:52:41 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <433CE24A.6040509@ee.byu.edu> References: <433CE24A.6040509@ee.byu.edu> Message-ID: <4340B872.8020102@optusnet.com.au> Travis Oliphant wrote: > > This is to announce the release of SciPy Core 0.4.X (beta) > > It is available at sourceforge which you can access by going to > > http://numeric.scipy.org > > Thanks to all who helped me with this release, especially > > Robert Kern > Pearu Peterson > > Now, let's start getting this stable.... Great work, and we are much relieved that NumPy has a firm future (albeit under a different name). However, can I ask if there are plans to add Masked Arrays to SciPy Core? If not, we'll have to stick with NumPy - missing values are ubiquitous in the biological sciences and any package which can't handle them isn't, to put it bluntly, of much use. We are happy to help test MA for SciPy Core, and our project (www.netepi.org) has over a thousand unit tests which give the basics of NumPy and MA a good work-out - so if/when we convert to SciPy Core the unit tests will be helpful. However we are not familiar enough with how SciPy Core differs from NumPy to help with the porting, I'm afraid. Tim C From tchur at optusnet.com.au Sun Oct 2 22:53:11 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Sun Oct 2 22:53:11 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340BE90.6080302@pfdubois.com> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> Message-ID: <4340C6FA.1050307@optusnet.com.au> Paul F. Dubois wrote: > I was going to see if I could port MA. It is pure Python so shouldn't be > too hard. That would eb a most welcome development. If you are in the mood for some minor extensions to MA (you probably aren't, but just on the off-chance...), then support for multiple masking values would be great. In other words, instead of the mask just taking the values zero or one, it can also take 2, 3, etc (within the constraints of the Int8 data type), where any non-zero value means masked. That would allow the reason why a value in an array is masked to be neatly encoded, not just the fact that it is masked (missing). For example, in (human) survey data, a scalar value (eg age at menopasue) may be missing because the respondent did not meet the precondition(s) for the question (eg sex not female), or because they have not yet undergone menopause, or because they refuse to answer, or because they don't know or don't recall. That's at least 4 different reasons why the value may be missing (masked). But don't ask me the the mask value should be when you multiply a masked-because-not-asked value by a masked-because-refused-to-answer value. Perhaps a reasonable rule would be max(mask0,mask1)? Tim C > > Tim Churches wrote: > >> Travis Oliphant wrote: >> >>> This is to announce the release of SciPy Core 0.4.X (beta) >>> >>> It is available at sourceforge which you can access by going to >>> >>> http://numeric.scipy.org >>> >>> Thanks to all who helped me with this release, especially >>> >>> Robert Kern >>> Pearu Peterson >>> >>> Now, let's start getting this stable.... >> >> >> >> Great work, and we are much relieved that NumPy has a firm future >> (albeit under a different name). >> >> However, can I ask if there are plans to add Masked Arrays to SciPy >> Core? If not, we'll have to stick with NumPy - missing values are >> ubiquitous in the biological sciences and any package which can't handle >> them isn't, to put it bluntly, of much use. We are happy to help test MA >> for SciPy Core, and our project (www.netepi.org) has over a thousand >> unit tests which give the basics of NumPy and MA a good work-out - so >> if/when we convert to SciPy Core the unit tests will be helpful. However >> we are not familiar enough with how SciPy Core differs from NumPy to >> help with the porting, I'm afraid. >> >> Tim C >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > From paul at pfdubois.com Sun Oct 2 23:23:34 2005 From: paul at pfdubois.com (Paul F. Dubois) Date: Sun Oct 2 23:23:34 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340B872.8020102@optusnet.com.au> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> Message-ID: <4340BE90.6080302@pfdubois.com> I was going to see if I could port MA. It is pure Python so shouldn't be too hard. Tim Churches wrote: > Travis Oliphant wrote: > >>This is to announce the release of SciPy Core 0.4.X (beta) >> >>It is available at sourceforge which you can access by going to >> >>http://numeric.scipy.org >> >>Thanks to all who helped me with this release, especially >> >>Robert Kern >>Pearu Peterson >> >>Now, let's start getting this stable.... > > > Great work, and we are much relieved that NumPy has a firm future > (albeit under a different name). > > However, can I ask if there are plans to add Masked Arrays to SciPy > Core? If not, we'll have to stick with NumPy - missing values are > ubiquitous in the biological sciences and any package which can't handle > them isn't, to put it bluntly, of much use. We are happy to help test MA > for SciPy Core, and our project (www.netepi.org) has over a thousand > unit tests which give the basics of NumPy and MA a good work-out - so > if/when we convert to SciPy Core the unit tests will be helpful. However > we are not familiar enough with how SciPy Core differs from NumPy to > help with the porting, I'm afraid. > > Tim C > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > From stephen.walton at csun.edu Mon Oct 3 08:25:09 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Mon Oct 3 08:25:09 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340C6FA.1050307@optusnet.com.au> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> <4340C6FA.1050307@optusnet.com.au> Message-ID: <43414CD7.8020507@csun.edu> Tim Churches wrote: >That would eb a most welcome development. If you are in the mood for >some minor extensions to MA (you probably aren't, but just on the >off-chance...), then support for multiple masking values would be great. > > This sounds like an overly complicated addition to MA. If I may be so bold, it seems to me that what you might want is your own object which maintains parallel arrays of values and "mask reasons," if you will, and generates a mask array accordingly From Chris.Barker at noaa.gov Mon Oct 3 09:55:40 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 3 09:55:40 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340BE90.6080302@pfdubois.com> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> Message-ID: <434161F0.80408@noaa.gov> > Tim Churches wrote: >> missing values are >> ubiquitous in the biological sciences and any package which can't handle >> them isn't, to put it bluntly, of much use. MA is great, but I wonder if many of the simple "missing value" use cases could be covered by robust handling of NaNs. Which brings up the question: How does scipy_core handle NaN and the other IEEE special values? This was a major weakness in Numeric for me. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From efiring at hawaii.edu Mon Oct 3 12:49:57 2005 From: efiring at hawaii.edu (Eric Firing) Date: Mon Oct 3 12:49:57 2005 Subject: [Numpy-discussion] ma is in scipy_core Message-ID: <43418AA8.2040809@hawaii.edu> Paul, Tim, Masked arrays *are* in scipy_core. import scipy.base.ma as ma In addition, a quick look indicates that NaN-handling is in good shape. Eric From stephen.walton at csun.edu Mon Oct 3 13:05:28 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Mon Oct 3 13:05:28 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <4340B872.8020102@optusnet.com.au> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> Message-ID: <43418E5A.5040105@csun.edu> Tim Churches wrote: >However, can I ask if there are plans to add Masked Arrays to SciPy >Core? > Eric Firing pointed out on the matplotlib mailing list that they're already there: import scipy.base.ma as ma From Chris.Barker at noaa.gov Mon Oct 3 13:49:50 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 3 13:49:50 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <43417A87.9000103@ee.byu.edu> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> <434161F0.80408@noaa.gov> <43417A87.9000103@ee.byu.edu> Message-ID: <43419904.7040509@noaa.gov> Travis Oliphant wrote: > Chris Barker wrote: >> Which brings up the question: How does scipy_core handle NaN and the >> other IEEE special values? This was a major weakness in Numeric for me. > > They can exist in your arrays and isnan, isinf, isposinf, isneginf > detects them. Wonderful! I'm really looking forward to this being stable. thanks, Travis! -CHB -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From oliphant at ee.byu.edu Mon Oct 3 14:05:01 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon Oct 3 14:05:01 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <43418E5A.5040105@csun.edu> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <43418E5A.5040105@csun.edu> Message-ID: <43419C54.6030003@ee.byu.edu> Stephen Walton wrote: > Tim Churches wrote: > >> However, can I ask if there are plans to add Masked Arrays to SciPy >> Core? >> > Eric Firing pointed out on the matplotlib mailing list that they're > already there: > > import scipy.base.ma as ma from scipy import ma also works (everything under scipy.base is also under scipy alone). -Travis From tchur at optusnet.com.au Mon Oct 3 14:09:14 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 14:09:14 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <43414CD7.8020507@csun.edu> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> <4340C6FA.1050307@optusnet.com.au> <43414CD7.8020507@csun.edu> Message-ID: <43419D88.3020006@optusnet.com.au> Stephen Walton wrote: > Tim Churches wrote: > >> That would eb a most welcome development. If you are in the mood for >> some minor extensions to MA (you probably aren't, but just on the >> off-chance...), then support for multiple masking values would be great. >> >> > This sounds like an overly complicated addition to MA. If I may be so > bold, it seems to me that what you might want is your own object which > maintains parallel arrays of values and "mask reasons," if you will, and > generates a mask array accordingly Yes, but the wish is that NumPy, or rather, SciPy Core, treats any mask value other than zero the same - that is, that the mask value doesn't have to be one. Teh machinery to attach meaning to mask values is indeed situation-specific and shouldn't be part of SciPy Core (what is the handy abbreviation of SciPy Core, BTW?). Tim C From tchur at optusnet.com.au Mon Oct 3 14:18:29 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 14:18:29 2005 Subject: [Numpy-discussion] Release of SciPy Core 0.4 (Beta) In-Reply-To: <434161F0.80408@noaa.gov> References: <433CE24A.6040509@ee.byu.edu> <4340B872.8020102@optusnet.com.au> <4340BE90.6080302@pfdubois.com> <434161F0.80408@noaa.gov> Message-ID: <43419FE3.2070109@optusnet.com.au> Chris Barker wrote: >> Tim Churches wrote: >> >>> missing values are >>> ubiquitous in the biological sciences and any package which can't handle >>> them isn't, to put it bluntly, of much use. > > > MA is great, but I wonder if many of the simple "missing value" use > cases could be covered by robust handling of NaNs. Most. > Which brings up the question: How does scipy_core handle NaN and the > other IEEE special values? This was a major weakness in Numeric for me. MA is indeed very flexible, well-designed and easy to use, but its weakness is that it is slow - necessarily so due to its "add-on" design - every operation is at least twice as slow as the equivalent oepration on a Numeric array. The mask arrays also eat some additional memory, which is sometimes an issue (untill everyone moves to 64 bit systems). Tim C From tchur at optusnet.com.au Mon Oct 3 14:36:36 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 14:36:36 2005 Subject: [Numpy-discussion] ma is in scipy_core In-Reply-To: <43418AA8.2040809@hawaii.edu> References: <43418AA8.2040809@hawaii.edu> Message-ID: <4341A410.5010504@optusnet.com.au> Eric Firing wrote: > Paul, Tim, > > Masked arrays *are* in scipy_core. > > import scipy.base.ma as ma OK, thanks. In the absence of documentation, I just looked for an MA subdirectory, couldn't find one and assumed that it wasn't (yet) supported. > In addition, a quick look indicates that NaN-handling is in good shape. How about other IEEE special values? Tim C From efiring at hawaii.edu Mon Oct 3 15:50:11 2005 From: efiring at hawaii.edu (Eric Firing) Date: Mon Oct 3 15:50:11 2005 Subject: [Numpy-discussion] ma is in scipy_core In-Reply-To: <4341A410.5010504@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> Message-ID: <4341B549.3000107@hawaii.edu> Tim Churches wrote: > Eric Firing wrote: > >>Paul, Tim, >> >>Masked arrays *are* in scipy_core. >> >>import scipy.base.ma as ma > > > OK, thanks. In the absence of documentation, I just looked for an MA > subdirectory, couldn't find one and assumed that it wasn't (yet) supported. > > >>In addition, a quick look indicates that NaN-handling is in good shape. > > > How about other IEEE special values? > > Tim C Tim, It has inf: >>> import scipy.base as nx >>> nx.inf inf >>> nx.isposinf(nx.inf) True >>> nx.isposinf(-nx.inf) False >>> nx.isneginf(-nx.inf) True >>> I don't know about signed floating point zero; I have never used such a beast, and am aware of it only because it is in numarray's ieeespecial module. See http://numeric.scipy.org/new_features.html; it includes: Errors are handled through the IEEE floating point status flags and there is flexibility on a per function / module / builtin level for handling these errors. I haven't looked into how this is done. Eric From efiring at hawaii.edu Mon Oct 3 15:56:19 2005 From: efiring at hawaii.edu (Eric Firing) Date: Mon Oct 3 15:56:19 2005 Subject: [Numpy-discussion] ma is in scipy_core In-Reply-To: <4341A410.5010504@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> Message-ID: <4341B659.7070503@hawaii.edu> > OK, thanks. In the absence of documentation, I just looked for an MA > subdirectory, couldn't find one and assumed that it wasn't (yet) supported. Tim, Documentation is coming along, but being made available in an unusual way: http://www.tramy.us/ Eric From tchur at optusnet.com.au Mon Oct 3 16:43:52 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 16:43:52 2005 Subject: [Numpy-discussion] ma is in scipy_core In-Reply-To: <4341B659.7070503@hawaii.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> Message-ID: <4341C1C8.3000506@optusnet.com.au> Eric Firing wrote: > >> OK, thanks. In the absence of documentation, I just looked for an MA >> subdirectory, couldn't find one and assumed that it wasn't (yet) >> supported. > > > Tim, > > Documentation is coming along, but being made available in an unusual > way: http://www.tramy.us/ Yes. I don't mind paying about AUD$50 for a copy of the SciPy Core documentation in order to expedite its production, but the Total Price and Total Time parameters as they currently stand are ridiculous: $300k represents about 7 or 8 thousand copies - and although there may be that many users of NumPy worldwide, realistically only a proportion of them will migrate to SciPy Core and of those, only a proportion will buy a copy of the documentation. Most likely, one copy of the documentation will be purchased to be shared between several (or many) users in a workgroup within an institution. Thus, the $300k mark is unlikely to be reached and it will be 7 years before the documentation is freely available. Given that the lifecycle of projects like NumPy or SciPy Core is of that same order of magnitude, it effectively means that the documentation for SciPy Core will not be free for most or all of its lifespan. That is a pretty severe limitation and one which end users should carefully consider before converting to SciPy Core. I would say that perhaps $30k or one year (after completion of the documentation) would be more reasonable criteria for making the documentation freely available (but then I am not writing it). Tim C From oliphant at ee.byu.edu Mon Oct 3 17:27:02 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon Oct 3 17:27:02 2005 Subject: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4341C1C8.3000506@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> Message-ID: <4341CB9E.4050306@ee.byu.edu> Tim Churches wrote: >Eric Firing wrote: > > >>>OK, thanks. In the absence of documentation, I just looked for an MA >>>subdirectory, couldn't find one and assumed that it wasn't (yet) >>>supported. >>> >>> >>Tim, >> >>Documentation is coming along, but being made available in an unusual >>way: http://www.tramy.us/ >> >> > >copy of the documentation. Most likely, one copy of the documentation >will be purchased to be shared between several (or many) users in a >workgroup within an institution. > Note that it is expressly against the agreement for one copy to be shared between multiple users at the same institution. I hope this is clear.... Of course you can let somebody else look at it a couple of times, but if they will use it regularly, they need to get their own copy. Prices are always a matter of supply and demand. The whole point of the system is to allow the price system to help coordinate what people think is valuable and what developers spend time doing. What you see currently is the maximum price (and time) I could possibly set as per the agreement with Trelgol. These things can always come down, however, as time proceeds, and the market responds. Now, obviously the cost of the documentation includes something of the cost of producing the actual code. Of course, you may disagree, but I did choose the numbers based on a little bit of market research. I don't think that 7000 copies of the documentation or 7 years is all that ridiculous given that there have been over 12000 downloads of the Numeric 24.0b2 source code since April and Numeric has been in stable use for nearly 10 years. If scipy does it's job correctly, then a user-base needing documentation of 7000 is rather low-balling it I would say. I want scipy to surpass the number of users of Numeric. I'm trying to make scipy core so that everybody can convert to it, eventually. The old Numeric manual still provides documentation, and the source is still available. I think you are still getting a great deal. Unless there is another majore re-write, the documentation will be updated as it goes (and you get the updates). >I would say that perhaps $30k or one year (after completion of the >documentation) would be more reasonable criteria for making the >documentation freely available (but then I am not writing it). > > Well, given the time I had to spend on this, that is quite a bit less than the market will bear for my services elsewhere. I suppose if I were rich, I could donate more of my time. But, I'm not.... I'm really not trying to make people upset. I'm really a huge fan of open information, and would love to see the documentation completely free. It's just that I cannot afford to create it for nothing. I have lots of demands on my time. Spending it doing scientific python has to be justified, somehow. I did not start the creation of a hybrid Numeric / numarray with the hope of making any money. I started it because there was a need. I thought I would get more help with its implementation. But, given the reality of people's scarce time (they need to make money...), nobody was able to help me. Out of this, the documentation idea was born to try and help fund development of scipy core. I hope people can understand that the reality of scarcity dictates that we coordinate efforts through some mechanism. The price mechanism has been the most succesful large-scale mechanism yet developed. I am interested in feedback. If you don't buy the book because you think I'm asking too much money, then let me know, as Tim has done. You can email me directly, as well. Best regards, -Travis From tchur at optusnet.com.au Mon Oct 3 17:59:25 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 17:59:25 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <4341CB9E.4050306@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> Message-ID: <4341D32A.5000000@optusnet.com.au> Travis Oliphant wrote: > Tim Churches wrote: > >> Eric Firing wrote: >> >> >>>> OK, thanks. In the absence of documentation, I just looked for an MA >>>> subdirectory, couldn't find one and assumed that it wasn't (yet) >>>> supported. >>>> >>> >>> Tim, >>> >>> Documentation is coming along, but being made available in an unusual >>> way: http://www.tramy.us/ >>> >> >> >> copy of the documentation. Most likely, one copy of the documentation >> will be purchased to be shared between several (or many) users in a >> workgroup within an institution. > > > Note that it is expressly against the agreement for one copy to be > shared between multiple users at the same institution. I hope this is > clear.... Of course you can let somebody else look at it a couple of > times, but if they will use it regularly, they need to get their own copy. > Prices are always a matter of supply and demand. The whole point of > the system is to allow the price system to help coordinate what people > think is valuable and what developers spend time doing. What you see > currently is the maximum price (and time) I could possibly set as per > the agreement with Trelgol. These things can always come down, > however, as time proceeds, and the market responds. The only agreement I can find is these words on the above site: "When you receive a copy of content under an MBTDR, you implicitly agree to make only back-up copies of the content and to not make a copy for others during the restriction period. If additional copies of the content are needed (for multiple users at a company, for example) you must purchase them. You may make exactly one "printed" or hard-copy for each copy of the content you purchase. For books, Trelgol may make available volume printing options at cost for those who have purchased a copy of the electronic content." Travis, are you saying that this agreement only allows a single person to read the single printed copy? If so, I think you need a formally worded legal license to make that stick - certainly Australian copyright law (nor copyright law in other countries, I suspect) does not provide any support for such severe restrictions in the use of a printed document. Under copyright law, you may not make unauthorised copies of a printed document, but you can certainly lend that copy to others, or sell or give it to them, or share it as a bench manual. > Now, obviously the cost of the documentation includes something of the > cost of producing the actual code. Of course, you may disagree, but I > did choose the numbers based on a little bit of market research. I > don't think that 7000 copies of the documentation or 7 years is all that > ridiculous given that there have been over 12000 downloads of the > Numeric 24.0b2 source code since April and Numeric has been in stable > use for nearly 10 years. Many of those downloads are for "t[y|i]re-kicking" - potential users determining whether it meets their needs. If those potential users have to pay for documentation up-front, then a large proportion will go elsewhere. The rest are probably existing NumPy users upgrading (or testing the upgrade waters). The latter group may well pay for documentation, but how large is that group? For example, how many people are subscribed to the numpy-discussion mailing list? > If scipy does it's job correctly, then a user-base needing > documentation of 7000 is rather low-balling it I would say. I want > scipy to surpass the number of users of Numeric. I'm trying to make > scipy core so that everybody can convert to it, eventually. The old > Numeric manual still provides documentation, and the source is still > available. I think you are still getting a great deal. Unless there > is another majore re-write, the documentation will be updated as it goes > (and you get the updates). Yes, maybe all that is needed is a (free) SciPy Core update of the existing (free) NumPy documentation. The (non-free) SciPy Core book which Travis is writing can complement that, just as the dozens of commercial Python books complement the free Python documentation. But imagine if everyone had to pay $50 to access the basic Python documentation - the number of Python users worldwide would be very much smaller than it is is now, I dare say. >> I would say that perhaps $30k or one year (after completion of the >> documentation) would be more reasonable criteria for making the >> documentation freely available (but then I am not writing it). >> >> > Well, given the time I had to spend on this, that is quite a bit less > than the market will bear for my services elsewhere. I suppose if I > were rich, I could donate more of my time. But, I'm not.... Travis, although many of us are grateful for your efforts on SciPy Core, no-one made you do it. If you wanted to earn (more) money by doing other things, you should have done them. > I'm really not trying to make people upset. I'm really a huge fan of > open information, and would love to see the documentation completely > free. It's just that I cannot afford to create it for nothing. I have > lots of demands on my time. Spending it doing scientific python has to > be justified, somehow. I did not start the creation of a hybrid > Numeric / numarray with the hope of making any money. I started it > because there was a need. I thought I would get more help with its > implementation. But, given the reality of people's scarce time (they > need to make money...), nobody was able to help me. Out of this, the > documentation idea was born to try and help fund development of scipy core. > I hope people can understand that the reality of scarcity dictates that > we coordinate efforts through some mechanism. The price mechanism has > been the most succesful large-scale mechanism yet developed. > > I am interested in feedback. If you don't buy the book because you > think I'm asking too much money, then let me know, as Tim has done. You > can email me directly, as well. I think there needs to be some community debate about this. Is there sufficient interest for people other than Travis to start with the Numeric documentation and update it as necessary to become a free SciPy Core documentation? The NumPy documentation is available in HTML format as the basis of this - perhaps the original source (LaTeX?) for the Numeric docs is also available? Tim C From oliphant at ee.byu.edu Mon Oct 3 20:55:20 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon Oct 3 20:55:20 2005 Subject: [Numpy-discussion] Contributions to development is one way to get documentation In-Reply-To: <4341F42F.2020802@pfdubois.com> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341F42F.2020802@pfdubois.com> Message-ID: <4341FCD0.5020405@ee.byu.edu> I have not made this explicitly clear, but if somebody cannot afford the documentation, I will give it to them in return for help with development (significant bug-fixes, optimizations, improved docstrings, new features etc.). Any copy I give away, counts against the total price as well... I've already given away several copies. If you feel like you should receive one, and I've forgotten about you, let me know. Best, -Travis From tchur at optusnet.com.au Mon Oct 3 21:29:57 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 21:29:57 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <4341F777.3030404@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> Message-ID: <434204B8.4050308@optusnet.com.au> Travis Oliphant wrote: > Tim Churches wrote: > >> Travis Oliphant wrote: >> >> >>> Tim Churches wrote: >>> >>> >> >> Travis, are you saying that this agreement only allows a single person >> to read the single printed copy? If so, I think you need a formally >> worded legal license to make that stick - certainly Australian copyright >> law (nor copyright law in other countries, I suspect) does not provide >> any support for such severe restrictions in the use of a printed >> document. Under copyright law, you may not make unauthorised copies of a >> printed document, but you can certainly lend that copy to others, or >> sell or give it to them, or share it as a bench manual. >> >> > I'm just stating what I think is fair. I'm not going to try and use the > power of the Australian state or any other to try and enforce it. My point was that you won't get any help from the State if what you think is fair extends to restricting who can read a single printed copy of your documentation. You will need to resort to tort law to enforce such restrictions, and thus you really need a formal documentation license, if that is your aim. >> Many of those downloads are for "t[y|i]re-kicking" - potential users >> determining whether it meets their needs. If those potential users have >> to pay for documentation up-front, then a large proportion will go >> elsewhere. The rest are probably existing NumPy users upgrading (or >> testing the upgrade waters). The latter group may well pay for >> documentation, but how large is that group? For example, how many people >> are subscribed to the numpy-discussion mailing list? >> >> > Don't know. But, there is enough free documentation to get started, I > think. Lack of documentation has hampered more people than cost of > existing documentation, I think. I have to disagree on this point. Granted the existing documentation for NumPy is not fantastic - some aspects of it are positively obscure - but overall it is better than good-enough, we have found, and was not a barrier to our decision to use NumPy. Having to cough up $50 to even peek at the documentation would be a far, far greater barrier to uptake than the occasional obscure wording of some of the function descriptions, IMHO. >> Travis, although many of us are grateful for your efforts on SciPy Core, >> no-one made you do it. If you wanted to earn (more) money by doing other >> things, you should have done them. >> > I'm really thinking about the future here. If I can succesfully raise > money to support work on this using a simple time-delay documentation > encouragement, then perhaps others will do the same, and we can all have > more. Waiting for people to "donate" their time to develop scientific > python is rather slow.... I've been freely donating time for a long > time, and there are few who help. So, we'll try a slightly different > route and see where that goes. Yes, fair enough. My point was that 7 years is far too long a time delay. Given that SciPy Core will probably take another year to become rock-solid, I would happily buy several copies of your documentation at $50 per copy if I knew that in a year the documentation could be freely distributed to all God's children. But in 7 years? Nope, too long to wait, not interested in supporting that. >> I think there needs to be some community debate about this. Is there >> sufficient interest for people other than Travis to start with the >> Numeric documentation and update it as necessary to become a free SciPy >> Core documentation? The NumPy documentation is available in HTML format >> as the basis of this - perhaps the original source (LaTeX?) for the >> Numeric docs is also available? > > I don't know why you would want to undermine my efforts in this way by > duplicating effort. Travis, I don't want to undermine your efforts, but you must understand that one of the attractions of NumPy and thus Scipy Core is that they are free, open source software, which implies that there exists adequate free, open source documentation for them, or if such free, open source documentation does not exist, then taht users are collectively at liberty to create it. That's a basic FOSS tenet. What you are are proposing is the creation of proprietary documentation for SciPy Core. No-one objects to this, but only if it is supplementary to free, open source "core" documentation, not instead of it. > Perhaps, instead you could have people donate $$ > instead of time to releasing the documentation. I give away copies of > the documentation to people who participate in the development process > all the time (and that comes off the total price --- though I haven't > advertised this as of yet). So, why don't you encourage people who > don't have the money to contribute to the project instead. I understand that open source software and its documentation don't just appear out of thin air, and I fully support the idea of funded or collectively-commissioned open source development. But only if the results of that funded or commissioned development is, in fact, free and open sourced. By preventing free distribution of the documentation you are creating for 7 years, the end product cannot even remotely be described as open source, thus personally I am disinterested in funding it. If the delay until open sourcing was just one year, then that's a different matter, but you do not seem to be proposing that. Hence I think the NumPy/SciPy Core user community should establish a SciPy Core documentation project, with the aim of updating and enhancing the existing NumPy documentation so that it covers the new and changed features of SciPy Core, and making that documentation available on teh same free, open source basis as NumPy/SciPy Core itself. To that end, are the original LaTeX or whatever sources for the currenet NumPy documenattion available somewhere (apart from the HTML and PDF versions, that is). Tim C From Chris.Barker at noaa.gov Mon Oct 3 22:12:48 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 3 22:12:48 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <434204B8.4050308@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> Message-ID: <43420ECD.5070806@noaa.gov> Tim Churches wrote: > I think there needs to be some community debate about this. Well, no, there doesn't. Travis is doing two things: 1) Spending a great deal of his time producing some wonderful open source software. 2) Embarking on a small business venture, trying to sell books. There is no need for community debate about an individual's business venture. It might benefit Travis to do a bit more market research, but as was pointed out, there's no reason he couldn't lower those limits and open source the book sooner if he chooses to. Travis Oliphant wrote: > I am interested in feedback. If you don't buy the book because you > think I'm asking too much money, then let me know. Since you ask, I think $39.99 is a bit steep for an electronic copy. As a quick comparison, I just bought the new wxWidgets book, documenting another open source project. You can get a printed copy from Amazon for $32.99. I also do think Tim has a point. Usually the open source model is that the basic reference docs are freely available, and there are nice user and newbie friendly books for sale. It will be much harder for scipy.base to catch on if there are no freely available docs. However, having looked at the little bit of the book that is now available, the quality and detail are far beyond what one usually finds in open-source references (with the possible exception of the python references). It certainly looks better than the old NumPy docs, which were still adequate. I have no doubt that the NumPy community will produce some free "getting started" docs anyway, so we can all be happy. Maybe, as Tim suggests, we could fill a Wiki with the existing Numpy docs, and all start editing away! -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From tchur at optusnet.com.au Mon Oct 3 22:28:18 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 22:28:18 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <43420ECD.5070806@noaa.gov> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420ECD.5070806@noaa.gov> Message-ID: <43421271.2010705@optusnet.com.au> Chris Barker wrote: > Tim Churches wrote: > >> I think there needs to be some community debate about this. > > Well, no, there doesn't. Travis is doing two things: > > 1) Spending a great deal of his time producing some wonderful open > source software. > > 2) Embarking on a small business venture, trying to sell books. > > There is no need for community debate about an individual's business > venture. It might benefit Travis to do a bit more market research, but > as was pointed out, there's no reason he couldn't lower those limits and > open source the book sooner if he chooses to. Chris, I think you misunderstand what I meant - whic is clarified in another message I just posted a minute ago. I meant that there needs to be community debate over whether there is a requirement for a project to create free, open source documentation for SciPy Core, not over whether Travis does or doesn't have the right to, as you put it "...embark on a small business venture, trying to sell books.". I completely agree that he has every legal and moral right in the world to do that. > Travis Oliphant wrote: > >> I am interested in feedback. If you don't buy the book because you >> think I'm asking too much money, then let me know. > > Since you ask, I think $39.99 is a bit steep for an electronic copy. As > a quick comparison, I just bought the new wxWidgets book, documenting > another open source project. You can get a printed copy from Amazon for > $32.99. > > I also do think Tim has a point. Usually the open source model is that > the basic reference docs are freely available, and there are nice user > and newbie friendly books for sale. It will be much harder for > scipy.base to catch on if there are no freely available docs. > > However, having looked at the little bit of the book that is now > available, the quality and detail are far beyond what one usually finds > in open-source references (with the possible exception of the python > references). It certainly looks better than the old NumPy docs, which > were still adequate. > > I have no doubt that the NumPy community will produce some free "getting > started" docs anyway, so we can all be happy. Maybe, as Tim suggests, we > could fill a Wiki with the existing Numpy docs, and all start editing away! Yup, that's all I had in mind. If, in fact we are allowed to edit the existing NumPy docs. If not, then a wiki to produce an addendum or set of annotations to them (like a Biblical concordance, perhaps) is the best that can be done. Tim C From tchur at optusnet.com.au Mon Oct 3 22:34:52 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Mon Oct 3 22:34:52 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <43420C28.5010701@pfdubois.com> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> Message-ID: <434213F0.30105@optusnet.com.au> Paul F. Dubois wrote: > The original sources were in Framemaker. I am not positive where they > are. In any case they are copyrighted by the Regents of the University > of California. I am not a lawyer and don't know what the consequences of > that are. LLNL granted free distribution of the printed document with > the Numeric source code but I don't know what their position would be on > using their copyrighted text in a new document or on giving away the > sources. OK, thanks Paul. That may have implications for you, Travis, if you are planning to base your SciPy Core book on the existing NumPy documentation. Given the circumstance which Paul describes, perhaps the only way forward is to create a freely available addendum to the freely available, existing NumPy documentation which describes how and where SciPy Core differs or extends NumPy? Or to write entirely new documentation from scratch. > I believe that under U.S. copyright law, a person may lend their copy of > a book to anyone but not reproduce it and give them a copy. Yes, likewise here in Oz, and in most countries. Of course, the publisher of a book can impose additional restrictions to which the purchaser of the book must agree, such as 'You may not show or lend your copy of this book to your colleagues, or to anyone else.' However, who in their right mind would buy a book which was sold with such restrictions attached to it? > Given the way open source capitalism works, I would not be surprised if > someone produces a 'quick reference guide' that they give away, or > writes a book on 'Using scipy.core in biology' that they sell for $88. > Let a thousand flowers bloom. Yes, I agree entirely. Travis, you are at perfect liberty to create commercial documentation for SciPy Core, but please don't object if others try to organise to create free open source documentation as well. Tim C > Tim Churches wrote: > >> Travis Oliphant wrote: >> >>> Tim Churches wrote: >>> >>> >>>> Travis Oliphant wrote: >>>> >>>> >>>> >>>>> Tim Churches wrote: >>>>> >>>>> >>>> >>>> >>>> Travis, are you saying that this agreement only allows a single person >>>> to read the single printed copy? If so, I think you need a formally >>>> worded legal license to make that stick - certainly Australian >>>> copyright >>>> law (nor copyright law in other countries, I suspect) does not provide >>>> any support for such severe restrictions in the use of a printed >>>> document. Under copyright law, you may not make unauthorised copies >>>> of a >>>> printed document, but you can certainly lend that copy to others, or >>>> sell or give it to them, or share it as a bench manual. >>>> >>>> >>> >>> I'm just stating what I think is fair. I'm not going to try and use the >>> power of the Australian state or any other to try and enforce it. >> >> >> >> My point was that you won't get any help from the State if what you >> think is fair extends to restricting who can read a single printed copy >> of your documentation. You will need to resort to tort law to enforce >> such restrictions, and thus you really need a formal documentation >> license, if that is your aim. >> >> >>>> Many of those downloads are for "t[y|i]re-kicking" - potential users >>>> determining whether it meets their needs. If those potential users have >>>> to pay for documentation up-front, then a large proportion will go >>>> elsewhere. The rest are probably existing NumPy users upgrading (or >>>> testing the upgrade waters). The latter group may well pay for >>>> documentation, but how large is that group? For example, how many >>>> people >>>> are subscribed to the numpy-discussion mailing list? >>>> >>>> >>> >>> Don't know. But, there is enough free documentation to get started, I >>> think. Lack of documentation has hampered more people than cost of >>> existing documentation, I think. >> >> >> >> I have to disagree on this point. Granted the existing documentation for >> NumPy is not fantastic - some aspects of it are positively obscure - but >> overall it is better than good-enough, we have found, and was not a >> barrier to our decision to use NumPy. Having to cough up $50 to even >> peek at the documentation would be a far, far greater barrier to uptake >> than the occasional obscure wording of some of the function >> descriptions, IMHO. >> >> >>>> Travis, although many of us are grateful for your efforts on SciPy >>>> Core, >>>> no-one made you do it. If you wanted to earn (more) money by doing >>>> other >>>> things, you should have done them. >>>> >>> >>> I'm really thinking about the future here. If I can succesfully raise >>> money to support work on this using a simple time-delay documentation >>> encouragement, then perhaps others will do the same, and we can all have >>> more. Waiting for people to "donate" their time to develop scientific >>> python is rather slow.... I've been freely donating time for a long >>> time, and there are few who help. So, we'll try a slightly different >>> route and see where that goes. >> >> >> >> Yes, fair enough. My point was that 7 years is far too long a time >> delay. Given that SciPy Core will probably take another year to become >> rock-solid, I would happily buy several copies of your documentation at >> $50 per copy if I knew that in a year the documentation could be freely >> distributed to all God's children. But in 7 years? Nope, too long to >> wait, not interested in supporting that. >> >> >>>> I think there needs to be some community debate about this. Is there >>>> sufficient interest for people other than Travis to start with the >>>> Numeric documentation and update it as necessary to become a free SciPy >>>> Core documentation? The NumPy documentation is available in HTML format >>>> as the basis of this - perhaps the original source (LaTeX?) for the >>>> Numeric docs is also available? >>> >>> >>> I don't know why you would want to undermine my efforts in this way by >>> duplicating effort. >> >> >> >> Travis, I don't want to undermine your efforts, but you must understand >> that one of the attractions of NumPy and thus Scipy Core is that they >> are free, open source software, which implies that there exists adequate >> free, open source documentation for them, or if such free, open source >> documentation does not exist, then taht users are collectively at >> liberty to create it. That's a basic FOSS tenet. What you are are >> proposing is the creation of proprietary documentation for SciPy Core. >> No-one objects to this, but only if it is supplementary to free, open >> source "core" documentation, not instead of it. >> >> >>> Perhaps, instead you could have people donate $$ >>> instead of time to releasing the documentation. I give away copies of >>> the documentation to people who participate in the development process >>> all the time (and that comes off the total price --- though I haven't >>> advertised this as of yet). So, why don't you encourage people who >>> don't have the money to contribute to the project instead. >> >> >> >> I understand that open source software and its documentation don't just >> appear out of thin air, and I fully support the idea of funded or >> collectively-commissioned open source development. But only if the >> results of that funded or commissioned development is, in fact, free and >> open sourced. By preventing free distribution of the documentation you >> are creating for 7 years, the end product cannot even remotely be >> described as open source, thus personally I am disinterested in funding >> it. If the delay until open sourcing was just one year, then that's a >> different matter, but you do not seem to be proposing that. >> >> Hence I think the NumPy/SciPy Core user community should establish a >> SciPy Core documentation project, with the aim of updating and enhancing >> the existing NumPy documentation so that it covers the new and changed >> features of SciPy Core, and making that documentation available on teh >> same free, open source basis as NumPy/SciPy Core itself. >> >> To that end, are the original LaTeX or whatever sources for the currenet >> NumPy documenattion available somewhere (apart from the HTML and PDF >> versions, that is). >> >> Tim C >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > From perry at stsci.edu Tue Oct 4 02:18:07 2005 From: perry at stsci.edu (Perry Greenfield) Date: Tue Oct 4 02:18:07 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <434213F0.30105@optusnet.com.au> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> <434213F0.30105@optusnet.com.au> Message-ID: First of all, I certainly don't want to prevent Travis from recovering something for all the effort he has put into scipy_core. Nor do I wish to discourage anyone that wishes to provide alternative free documentation if they so choose. But I will note some facts about numarray documentation below that may prove useful to anyone considering the latter. On Oct 4, 2005, at 1:32 AM, Tim Churches wrote: > Paul F. Dubois wrote: >> The original sources were in Framemaker. I am not positive where they >> are. In any case they are copyrighted by the Regents of the University >> of California. I am not a lawyer and don't know what the consequences >> of >> that are. LLNL granted free distribution of the printed document with >> the Numeric source code but I don't know what their position would be >> on >> using their copyrighted text in a new document or on giving away the >> sources. The numarray User's Manual was derived from the Numpy manual that LLNL originally sponsored David Ascher to write (if that is incorrect I'm sure Paul will correct me). We dutifully propagated the legal notice in the original to the numarray version. Although IANAL I'll note that the text seems to permit changes by others, but it also seems mis-worded as it refers to software, not documentation regarding the rights. What that really means in the end I'm not sure. In any case, Jochen Kupper (I hope I have that right) converted the format from Framemaker to the Python document latex style. The source for the numarray version is currently on sourceforge (under the numarray/doc directory). I'll also note much of the capabilities in scipy_core are very similar to those of numarray. There are differences, though I don't believe it would take a great deal of work to udpate the numarray version to reflect these (e.g. the changes in the types system, how rank-0 issues, the C-API, object array details and the names of the standard packages within scipy_core.) > OK, thanks Paul. That may have implications for you, Travis, if you are > planning to base your SciPy Core book on the existing NumPy > documentation. > From what I've seen of it, I don't believe it is based at all on the original manual or any derivative, but I'll leave that for Travis to comment further on. Perry From aisaac at american.edu Tue Oct 4 03:01:45 2005 From: aisaac at american.edu (Alan G Isaac) Date: Tue Oct 4 03:01:45 2005 Subject: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4341CB9E.4050306@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> Message-ID: On Mon, 03 Oct 2005, Travis Oliphant apparently wrote: > I hope people can understand that the reality of scarcity > dictates that we coordinate efforts through some > mechanism. The price mechanism has been the most > succesful large-scale mechanism yet developed. > I am interested in feedback. If you don't buy the book > because you think I'm asking too much money, then let me > know, as Tim has done. I found this an interesting approach to supporting the project. I plan to buy the book when it is released. Hmm, why wait? I should put my money where my mouth is. Just a moment ... ok, done. I view the book as a *complement* to other documentation that will appear and as a way to support the project. I agree with Tim that freely accessible online documentation will and must become available as well. As Chris notes, some of this can happen on the Wiki. I also plan to ask our library to purchase the book, but I am concerned that your statement that multiple users each need their own copy might mean a library purchase is forbidden. I assume it did not mean that, and that you just meant that making multiple copies is restricted. (Our library supports electronic book check out.) Ruling out library purchases would, I think, be a costly mistake for many reasons, which I can list if you are interested. Finally, I agree with Tim that seven years is too long and at the price I'd hope for a paperback copy. I think a better strategy would be two years copy protection, with an updated edition every two years. (But then I am not writing the code!) The basic concept is really nice, as long as it does not make it harder for you to - fully document your code, - smile on the free documentation that emerges, and - keep your sunny disposition. Cheers, Alan Isaac From jonas at cortical.mit.edu Tue Oct 4 04:50:54 2005 From: jonas at cortical.mit.edu (Eric Jonas) Date: Tue Oct 4 04:50:54 2005 Subject: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> Message-ID: <20051004114825.GX5015@convolution.mit.edu> I wanted to echo isaac's point: > I also plan to ask our library to purchase the book, but > I am concerned that your statement that multiple users each > need their own copy might mean a library purchase is > forbidden. I assume it did not mean that, and that you > just meant that making multiple copies is restricted. (Our > library supports electronic book check out.) Ruling out > library purchases would, I think, be a costly mistake for > many reasons, which I can list if you are interested. I couldn't agree more, although I'm not quite sure how a license could be worded such that it would allow a library copy and prevent a lab bench copy, which I think was Travis' intent. That said, have you considered selling "site licenses" of a sort? I know my lab would pay a few hundred to get a PDF that we could just stick in our fileserver and use in perpetuity. I know that right now there's nothing -preventing- us from buying just one copy and doing that (other than pesky copyright law), but we'd like to support the project. ...Eric From rays at blue-cove.com Tue Oct 4 08:08:51 2005 From: rays at blue-cove.com (RayS) Date: Tue Oct 4 08:08:51 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation Message-ID: <6.2.3.4.2.20051004070429.02bca890@pop-server.san.rr.com> As a lurker and neophyte user of Numeric/numarray, I inject my perspective... >the existing documentation for >NumPy is not fantastic - some aspects of it are positively >obscure - but overall it is better than good-enough I disagree here - I have found the docs to be a real stumbling block to learning and putting new (to me) methods to use. There is often a very high level of C and/or specific math familiarity assumed; I spend a large amount of time in those instances Googling for example code (usually not much Numpy code is just sitting, posted online) and Usenet examples and discussion. I then spent time at the prompt testing up my own examples. The printed Python books are usually very good resources, unfortunately, there is too much competition for general, intro Python and not enough depth into specific areas. This is the classic Publishers Dilemma. I am buying the new wx book, myself. A similar, in-depth book on Numpy does interest me at $29-$39, and Ch2 seems to be nicely done. Hopefully the new scipy_core will have the low overhead of Numeric for small arrays. Does it have easy access to the pointer to the array memory, as in numarray? (for use with ctypes.memmove()) That said, I still like more examples. People learn in different ways. I still maintain some PHP web sites, and the best (some say only ;-) ) thing is their style of online docs with user comments, ex.: array-rand: http://www.php.net/manual/en/function.array-rand.php I you need to do a particular Thing, it is very likely to already be _in_ the docs. I think this is a better approach than Wikis, and is easier and more likely for users to contribute, like this: http://www.php.net/manual/add-note.php?sect=index&redirect=http://www.php.net/manual/en/index.php (Notice also: "Please note that periodically, the developers may go through the notes and incorporate the information in them into the documentation. This means that any note submitted here becomes the property of the PHP Documentation Group.") Ray Schumacher hobby: http://rjs.org/astro/1004x/Python/ From meng at are.berkeley.edu Tue Oct 4 12:01:53 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Tue Oct 4 12:01:53 2005 Subject: [Numpy-discussion] how to use lu decomposition in lapack library? Message-ID: <5.1.1.5.2.20051004113223.0390ae40@are.berkeley.edu> Hi there- Can someone help me on this? Thanks! Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng at are.berkeley.edu From luszczek at cs.utk.edu Tue Oct 4 12:16:40 2005 From: luszczek at cs.utk.edu (Piotr Luszczek) Date: Tue Oct 4 12:16:40 2005 Subject: [Numpy-discussion] how to use lu decomposition in lapack library? In-Reply-To: <5.1.1.5.2.20051004113223.0390ae40@are.berkeley.edu> References: <5.1.1.5.2.20051004113223.0390ae40@are.berkeley.edu> Message-ID: <4342D48B.7070904@cs.utk.edu> You would have to be more specific. If you just want to solve a system of linear equations with matrix 'a' and right-hand-side 'b': import numarray.linear_algebra as LA x = LA.solve_linear_equations(a, b) Unlike LAPACK, the above will leave your 'a' untouched. So, if you have another right hand side 'b1' and the same matrix 'a' you'll have to pay the cost of factorization all over again. There is a way around it but I don't know what you really need. Piotr meng at are.berkeley.edu wrote: > Hi there- > > Can someone help me on this? Thanks! > > Best, > Xiangyi > > Xiangyi Meng > Department of Agricultural and Resource Economics > Room 303, Giannini Hall #3310 > University of California, Berkeley > Berkeley, CA 94720-3310 > Tel: (510) 643-4124 > Fax: (510) 643-8911 > Email: meng at are.berkeley.edu > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion From luszczek at cs.utk.edu Tue Oct 4 12:41:04 2005 From: luszczek at cs.utk.edu (Piotr Luszczek) Date: Tue Oct 4 12:41:04 2005 Subject: [Numpy-discussion] how to use lu decomposition in lapack library? Message-ID: <4342DAB4.40703@cs.utk.edu> Then you have to dig in deeper: import numarray import numarray.linear_algebra import numarray.linear_algebra.lapack_lite2 as lapack_lite piv = numarray.zeros((n,), "l") outcome = lapack_lite.dgesv(n, nrhs, a, lda, piv, b, ldb, 0) outcome['info'] is the 'info' parameter set by LAPACK. You have to make sure that 'a' and 'b' are in Fortran order because you're calling Fortran code. Here is a sample code: >>> a=numarray.array([[1,2,3],[4,5,6]], 'd') >>> a.is_fortran_contiguous() 0 >>> a.is_c_array() True >>> a.is_f_array() 0 >>> a.transpose() >>> a.is_fortran_contiguous() 1 >>> a.is_c_array() False >>> a.is_f_array() True Call to 'transpose()' made it a Fortran array but it transposed the array too. Piotr meng at are.berkeley.edu wrote: > Thanks, Piotr! > > What I actually want is the lower (L) and upper triangular matrix (U) > from matrix 'a'. How to get it? > > Xiangyi > > At 15:14 2005-10-4 -0400, you wrote: > >> You would have to be more specific. If you just want to >> solve a system of linear equations with matrix 'a' and >> right-hand-side 'b': >> >> import numarray.linear_algebra as LA >> >> x = LA.solve_linear_equations(a, b) >> >> Unlike LAPACK, the above will leave your 'a' untouched. >> So, if you have another right hand side 'b1' and the >> same matrix 'a' you'll have to pay the cost of factorization >> all over again. >> >> There is a way around it but I don't know what you really need. >> >> Piotr >> >> meng at are.berkeley.edu wrote: >> >>> Hi there- >>> Can someone help me on this? Thanks! >>> Best, >>> Xiangyi >>> Xiangyi Meng >>> Department of Agricultural and Resource Economics >>> Room 303, Giannini Hall #3310 >>> University of California, Berkeley >>> Berkeley, CA 94720-3310 >>> Tel: (510) 643-4124 >>> Fax: (510) 643-8911 >>> Email: meng at are.berkeley.edu >>> >>> >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: >>> Power Architecture Resource Center: Free content, downloads, >>> discussions, >>> and more. http://solutions.newsforge.com/ibmarch.tmpl >>> _______________________________________________ >>> Numpy-discussion mailing list >>> Numpy-discussion at lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > Best, > Xiangyi > > Xiangyi Meng > Department of Agricultural and Resource Economics > Room 303, Giannini Hall #3310 > University of California, Berkeley > Berkeley, CA 94720-3310 > Tel: (510) 643-4124 > Fax: (510) 643-8911 > Email: meng at are.berkeley.edu > From tchur at optusnet.com.au Tue Oct 4 14:56:32 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Tue Oct 4 14:56:32 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <87wtkt2x36.fsf@peds-pc311.bsd.uchicago.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> <434213F0.30105@optusnet.com.au> <87wtkt2x36.fsf@peds-pc311.bsd.uchicago.edu> Message-ID: <4342FA08.805@optusnet.com.au> John Hunter wrote: >>>>>>"Tim" == Tim Churches writes: > > > Tim> Yes, I agree entirely. Travis, you are at perfect liberty to > Tim> create commercial documentation for SciPy Core, but please > Tim> don't object if others try to organise to create free open > Tim> source documentation as well. > > Object? I'll bet dollars to doughnuts that Travis would be delighted > to see this, just as he would have been at any time over the last > several years. And be careful: Perry still owes me a doughnut. Travis did indeed object, which surprised me. Verbatim: Tim C: >> I think there needs to be some community debate about this. Is there >> sufficient interest for people other than Travis to start with the >> Numeric documentation and update it as necessary to become a free >> SciPy Core documentation? The NumPy documentation is available in >> HTML format as the basis of this - perhaps the original source >> (LaTeX?) for the Numeric docs is also available? Travis: > I don't know why you would want to undermine my efforts in this way > by duplicating effort. Perhaps, instead you could have people donate > $$ instead of time to releasing the documentation. I give away > copies of the documentation to people who participate in the > development process all the time (and that comes off the total price > --- though I haven't advertised this as of yet). So, why don't you > encourage people who don't have the money to contribute to the project > instead. So, Travis seems to be proposing that he write his documentation book, which will be available only on a commercial basis (or by barter for development work) and which cannot be shared as a lab bench manual, and that any effort to create freely available documentation for scipy.core undermines his efforts. Now, we are **extremely** grateful to Travis for rescuing NumPy from a slow death by decay, and for saving us the pain of having to convert all our code to numarray (whereas the conversion to scipy.core should be much less painful). But the foregoing sets off alarm bells ringing in my head... I really think it would be useful if Travis makes a clear and unambiguous statement about where he stands on the free, open source nature of both the scipy.core code AND on documentation for it, just so we all know where we respectively stand. I have stated my position (not that it really matters), but I will re-iterate: a) scipy.core deserves a basic set of documentation (similar to the current NumPy documentation) which is available to everyone on exactly the same basis as the scipy.core code itself - that is, freely available at no cost. b) It is not Travis's duty to create such freely-available documentation. c) Others at at liberty to create such freely-available documentation for scipy.core if they so chose. d) The documentation which Travis is proposing to write will not be made freely available until targets of $300k in sales or 7 years have been reached, and this effectively renders such documentation proprietary. e) I think that the creation of proprietary supplementary documentation for scipy.core, as Travis is doing, is a good idea and I will personally buy several copies provided that the work is licensed in a reasonable manner (eg sharing of single physical copies as a bench manual is permitted). Tim C From oliphant at ee.byu.edu Tue Oct 4 15:10:37 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Oct 4 15:10:37 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Release of SciPy Core 0.4 (Beta) In-Reply-To: References: <433CE24A.6040509@ee.byu.edu> <4341F044.8030900@shrogers.com> <4341FBD0.8090107@ee.byu.edu> Message-ID: <4342FD6B.20009@ee.byu.edu> Rob Managan wrote: > Does the new scipy_core support the Numeric function > PyArray_FromDimsAndData? > > I use that to create a front end to a stand alone program that > generates a lot of arrays that I want to be able to query. Not the > best approach maybe but it works! > Yes, All of the old Numeric C-API is available (I believe...). Direct access to descr->one and descr->zero does not work anymore, though (it's replaced with a function call). If you look at the source, you will see that these older calls are all special cases of the call to PyArray_New() -Travis From oliphant at ee.byu.edu Tue Oct 4 15:25:50 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Oct 4 15:25:50 2005 Subject: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> Message-ID: <434300FB.7070606@ee.byu.edu> Alan G Isaac wrote: >I also plan to ask our library to purchase the book, but >I am concerned that your statement that multiple users each >need their own copy might mean a library purchase is >forbidden. I assume it did not mean that, and that you >just meant that making multiple copies is restricted. (Our >library supports electronic book check out.) Ruling out >library purchases would, I think, be a costly mistake for >many reasons, which I can list if you are interested. > > A library purchase is fine. If that how a single copy is shared. I'll make that more clear. But, really, if multiple users need to use it at the same time, then the library should purchase several copies. >Finally, I agree with Tim that seven years is too long and >at the price I'd hope for a paperback copy. I think >a better strategy would be two years copy protection, with >an updated edition every two years. (But then I am not >writing the code!) T > Thanks for the feedback. I'm experimenting with the right combination of total price and total time so feedback is welcomed. I want to encourage people who can really afford the book to spend the money on it. What is the "right" time/$$ combination that will encourage this. I'm willing to shorten the time and come down on the total price. They are set so I cannot increase them. But, there is no problem with lowering them. I could also support the idea of a cheaper total price 1st edition with a need to spend for the 2nd edition again. Thanks for the feedback. >he basic concept is really nice, as >long as it does not make it harder for you to >- fully document your code, >- smile on the free documentation that emerges, and >- keep your sunny disposition. > > Don't worry, I'm not banking my future on this little experiment, so I won't worry about what other people do. In fact, as John Hunter inferred, I would be thrilled by more contributions however they come. I just want to see more people use Python for scientific computing, and am trying some things out to help it along. My only question about writing "free" documentation, is that it just seems rather wasteful to spend time writing free documentation when you can set free the documentation by spending a little money instead. If you think I'm charging too much (either in $$ or time-delay), then continue to give me feedback. I am interested in what works. -Travis From aisaac at american.edu Tue Oct 4 15:40:31 2005 From: aisaac at american.edu (Alan G Isaac) Date: Tue Oct 4 15:40:31 2005 Subject: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <434300FB.7070606@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> Message-ID: On Tue, 04 Oct 2005, Travis Oliphant apparently wrote: > A library purchase is fine. If that how a single copy is shared. I'll > make that more clear. But, really, if multiple users need to use it at > the same time, then the library should purchase several copies. Our library supports single copy checkout. I think this currently is a pretty standard library function these days. > My only question about writing "free" documentation, is that it just > seems rather wasteful to spend time writing free documentation when you > can set free the documentation by spending a little money instead. I've done my part. ;-) Cheers, Alan Isaac From Fernando.Perez at colorado.edu Tue Oct 4 15:54:25 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Tue Oct 4 15:54:25 2005 Subject: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <434300FB.7070606@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> Message-ID: <434307CC.7020303@colorado.edu> Travis Oliphant wrote: > Alan G Isaac wrote: > > >>I also plan to ask our library to purchase the book, but >>I am concerned that your statement that multiple users each >>need their own copy might mean a library purchase is >>forbidden. I assume it did not mean that, and that you >>just meant that making multiple copies is restricted. (Our >>library supports electronic book check out.) Ruling out >>library purchases would, I think, be a costly mistake for >>many reasons, which I can list if you are interested. >> >> > > A library purchase is fine. If that how a single copy is shared. I'll > make that more clear. But, really, if multiple users need to use it at > the same time, then the library should purchase several copies. Travis, I think that what confused some people (and which I believe was not your original intent) was the impression that you meant to have terms on the printed version of the book which were more restrictive than those of traditional paper books. With a single physical copy of a paper book, the rules are pretty simple and constrained by the laws of nature (one non-quantum object can't really be in more than one place at the same time). Lending, borrowing, library use, 'lab bench' use, etc, are all accepted practices because while one person is using the book, nobody else has access to it. Since your book is originally provided electronically, there is the technical possiblity to make multiple physical copies, which I believe is what you wish to prevent (and something I'm not arguing with). So perhaps a clarification along the lines of the following could help (this is my wording, of course, so you should say what _you_ want, not what I get from trying to read your mind :) 'a single printed copy can be made from the electronic version, which is subject to the same restrictions imposed on paper books (lending is OK but not wholesale photocopying for redistribution, for example)' This would put at ease a lot of people who normally buy a book in a lab or research group with the natural assumption that anyone in that lab can go to the shelf and read it. Obviously if it's a book with very frequent use, traditional book purchasers buy multiple copies. With your book, the exact same thing would be expected: just because they have the PDF doesn't mean they can print 10 copies of it for the whole lab. Or at least that's my understanding. Best regards, Fernando. From oliphant at ee.byu.edu Tue Oct 4 16:10:08 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Oct 4 16:10:08 2005 Subject: [SciPy-user] Re: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <434307CC.7020303@colorado.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au><4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> Message-ID: <43430B62.9090908@ee.byu.edu> Fernando Perez wrote: > Travis Oliphant wrote: > >> Alan G Isaac wrote: >> >> >>> I also plan to ask our library to purchase the book, but I am >>> concerned that your statement that multiple users each need their >>> own copy might mean a library purchase is forbidden. I assume it >>> did not mean that, and that you just meant that making multiple >>> copies is restricted. (Our library supports electronic book check >>> out.) Ruling out library purchases would, I think, be a costly >>> mistake for many reasons, which I can list if you are interested. >>> >> >> A library purchase is fine. If that how a single copy is shared. >> I'll make that more clear. But, really, if multiple users need to >> use it at the same time, then the library should purchase several >> copies. > > > 'a single printed copy can be made from the electronic version, which > is subject to the same restrictions imposed on paper books (lending is > OK but not wholesale photocopying for redistribution, for example)' > > This would put at ease a lot of people who normally buy a book in a > lab or research group with the natural assumption that anyone in that > lab can go to the shelf and read it. Obviously if it's a book with > very frequent use, traditional book purchasers buy multiple copies. > With your book, the exact same thing would be expected: just because > they have the PDF doesn't mean they can print 10 copies of it for the > whole lab. Or at least that's my understanding. > Thanks, I like this wording. It is exactly what I meant. I also think except for a library-checkout system (where only one digital copy is in circulation), somebody should not be able to buy an e-copy and then make electronic copies for everybody in their organization. That's really quite counter productive. If you want to share your e-copy with someone for a while (or give it away) fine... I'm really just asking that you treat the e-copy something like a physical book. I'll make some more details concerning the intent, available. Thanks for everybody's help. -Travis From cjw at sympatico.ca Wed Oct 5 16:57:17 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Wed Oct 5 16:57:17 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> <434213F0.30105@optusnet.com.au> Message-ID: <434467D9.3040707@sympatico.ca> Perry Greenfield wrote: > First of all, I certainly don't want to prevent Travis from recovering > something for all the effort he has put into scipy_core. Nor do I wish > to discourage anyone that wishes to provide alternative free > documentation > if they so choose. But I will note some facts about numarray > documentation > below that may prove useful to anyone considering the latter. > > On Oct 4, 2005, at 1:32 AM, Tim Churches wrote: > >> Paul F. Dubois wrote: >> >>> The original sources were in Framemaker. I am not positive where they >>> are. In any case they are copyrighted by the Regents of the University >>> of California. I am not a lawyer and don't know what the >>> consequences of >>> that are. LLNL granted free distribution of the printed document with >>> the Numeric source code but I don't know what their position would >>> be on >>> using their copyrighted text in a new document or on giving away the >>> sources. >> > > The numarray User's Manual was derived from the Numpy manual that LLNL > originally sponsored David Ascher to write (if that is incorrect I'm sure > Paul will correct me). We dutifully propagated the legal notice in the > original to the numarray version. Although IANAL I'll note that the > text seems to permit changes by others, but it also seems mis-worded as > it refers to software, not documentation regarding the rights. What that > really means in the end I'm not sure. > > In any case, Jochen Kupper (I hope I have that right) converted the > format > from Framemaker to the Python document latex style. The source for the > numarray version is currently on sourceforge (under the numarray/doc > directory). I'll also note much of the capabilities in scipy_core are > very similar to those of numarray. There are differences, though I don't > believe it would take a great deal of work to udpate the numarray version > to reflect these (e.g. the changes in the types system, how rank-0 > issues, the C-API, object array details and the names of the standard > packages within scipy_core.) > >> OK, thanks Paul. That may have implications for you, Travis, if you are >> planning to base your SciPy Core book on the existing NumPy >> documentation. >> > From what I've seen of it, I don't believe it is based at all on the > original manual or any derivative, but I'll leave that for Travis to > comment further on. > > Perry > > Perry, It would be helpful if you gave some indication of your organization's intention with respect to numarray. Will it be maintained, as in the past, fully in the public domain and without charge for either the code or the API information needed to make use of the code? New versions have been produced every few months, will this continue? Colin W. From faltet at carabos.com Thu Oct 6 02:53:59 2005 From: faltet at carabos.com (Francesc Altet) Date: Thu Oct 6 02:53:59 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <434467D9.3040707@sympatico.ca> References: <43418AA8.2040809@hawaii.edu> <434467D9.3040707@sympatico.ca> Message-ID: <200510061152.55708.faltet@carabos.com> Exactly, I'm wondering the same questions. Provided that numarray is an excellent and mature piece of code, I'd like to hear a declaration of principles on that subject. Regards, A Dijous 06 Octubre 2005 01:55, Colin J. Williams va escriure: > Perry, > > It would be helpful if you gave some indication of your organization's > intention with respect to numarray. > > Will it be maintained, as in the past, fully in the public domain and > without charge for either the code or the API information needed to make > use of the code? > > New versions have been produced every few months, will this continue? > > Colin W. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From cjw at sympatico.ca Thu Oct 6 04:53:56 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu Oct 6 04:53:56 2005 Subject: [Numpy-discussion] pysci base formerly Numeric3 Message-ID: <43451032.10702@sympatico.ca> I have just downloaded and installed the Windows version. The install appeared to go smoothly but test_basic.py gave some errors, is this what one should expect? --------------------------------------------------------------------------------------------------------------------- >pythonw -u "test_basic.py" !! No test file 'test_scimath.py' found for !! No test file 'test_umath.py' found for Found 23 tests for scipy.base.function_base !! No test file 'test_machar.py' found for !! No test file 'test_ma.py' found for !! No test file 'test_numerictypes.py' found for !! No test file 'test_getlimits.py' found for Found 9 tests for scipy.base.twodim_base !! No test file 'test__compiled_base.py' found for !! No test file 'test_info_scipy_base.py' found for !! No test file 'test_base.py' found for !! No test file 'test_convertcode.py' found for !! No test file 'test_multiarray.py' found for !! No test file 'test_matrix.py' found for !! No test file 'test_oldnumeric.py' found for Found 44 tests for scipy.base.shape_base !! No test file 'test_numeric.py' found for !! No test file 'test_polynomial.py' found for Found 42 tests for scipy.base.type_check !! No test file 'test_arrayprint.py' found for Found 4 tests for scipy.base.index_tricks Found 0 tests for __main__ ................................E.................................................E........E.E.E...........EEEEEE.E..E.... ====================================================================== ERROR: check_simple (scipy.base.shape_base.test_shape_base.test_apply_along_axis) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_shape_base.py", line 14, in check_simple assert_array_equal(apply_along_axis(len,0,a),len(a)*ones(shape(a)[1])) File "C:\Python24\Lib\site-packages\scipy\base\shape_base.py", line 31, in apply_along_axis res = func1d(arr[tuple(i)],*args) IndexError: each subindex must be either a slice, an integer, Ellipsis, or newaxis ====================================================================== ERROR: check_complex1 (scipy.base.type_check.test_type_check.test_isfinite) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 162, in check_complex1 assert_all(isfinite(array(1+1j)/0.) == 0) ZeroDivisionError: complex division ====================================================================== ERROR: check_neginf_scalar (scipy.base.type_check.test_type_check.test_isinf) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 176, in check_neginf_scalar assert_all(isinf(array(-1.)/0.) == 1) ZeroDivisionError: float division ====================================================================== ERROR: check_posinf_scalar (scipy.base.type_check.test_type_check.test_isinf) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 172, in check_posinf_scalar assert_all(isinf(array(1.,)/0.) == 1) ZeroDivisionError: float division ====================================================================== ERROR: check_complex1 (scipy.base.type_check.test_type_check.test_isnan) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 142, in check_complex1 assert_all(isnan(array(0+0j)/0.) == 1) ZeroDivisionError: complex division ====================================================================== ERROR: check_default_1 (scipy.base.type_check.test_type_check.test_mintypecode) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 18, in check_default_1 assert_equal(mintypecode(itype),'d') File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 30, in mintypecode typecodes = [(type(t) is type('') and t) or asarray(t).dtypechar\ UnboundLocalError: local variable 'typecodes' referenced before assignment ====================================================================== ERROR: check_default_2 (scipy.base.type_check.test_type_check.test_mintypecode) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 26, in check_default_2 assert_equal(mintypecode(itype+'f'),'f') File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 30, in mintypecode typecodes = [(type(t) is type('') and t) or asarray(t).dtypechar\ UnboundLocalError: local variable 'typecodes' referenced before assignment ====================================================================== ERROR: check_default_3 (scipy.base.type_check.test_type_check.test_mintypecode) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 50, in check_default_3 assert_equal(mintypecode('fdF'),'D') File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 30, in mintypecode typecodes = [(type(t) is type('') and t) or asarray(t).dtypechar\ UnboundLocalError: local variable 'typecodes' referenced before assignment ====================================================================== ERROR: check_complex_bad (scipy.base.type_check.test_type_check.test_nan_to_num) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 211, in check_complex_bad v += array(0+1.j)/0. ZeroDivisionError: complex division ====================================================================== ERROR: check_complex_bad2 (scipy.base.type_check.test_type_check.test_nan_to_num) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 217, in check_complex_bad2 v += array(-1+1.j)/0. ZeroDivisionError: complex division ====================================================================== ERROR: check_complex_good (scipy.base.type_check.test_type_check.test_nan_to_num) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 207, in check_complex_good vals = nan_to_num(1+1j) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 99, in nan_to_num y = nan_to_num(x.real) + 1j * nan_to_num(x.imag) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 106, in nan_to_num y[are_nan] = 0 TypeError: object does not support item assignment ====================================================================== ERROR: check_integer (scipy.base.type_check.test_type_check.test_nan_to_num) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 204, in check_integer vals = nan_to_num(1) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 105, in nan_to_num maxf, minf = _getmaxmin(y.dtype) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 86, in _getmaxmin f = getlimits.finfo(t) File "C:\Python24\Lib\site-packages\scipy\base\getlimits.py", line 29, in __init__ raise ValueError, "data type not inexact" ValueError: data type not inexact ====================================================================== ERROR: check_basic (scipy.base.type_check.test_type_check.test_real_if_close) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python24\Lib\site-packages\scipy\base\tests\test_type_check.py", line 230, in check_basic b = real_if_close(a+1e-15j) File "C:\Python24\Lib\site-packages\scipy\base\type_check.py", line 120, in real_if_close tol = f.epsilon * tol AttributeError: 'finfo' object has no attribute 'epsilon' ---------------------------------------------------------------------- Ran 122 tests in 0.521s FAILED (errors=13) >Exit code: -1073741819 ------------------------------------------------------------------------------------------------------------------- Colin W. From matthew.brett at gmail.com Thu Oct 6 05:31:25 2005 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu Oct 6 05:31:25 2005 Subject: [SciPy-user] Re: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <43430B62.9090908@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> Message-ID: <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> Hi Travis, So, can I summarize for my own benefit? I think we all agree that software without documentation is not usable, and therefore that free software requires that the basic documentation also be free. Obviously if the _basic_ documentation is not free, this will dramatically reduce software uptake. And I think we all also agree that there is no problem of any sort with not-free documentation that expands in a useful way on the basic documentation. So, just to clarify - does Fenando win his bet (was it dollars or doughnuts)? Are you happy for the community to write and release free basic documentation for scipy.base? Thanks a lot, Matthew From cjw at sympatico.ca Thu Oct 6 06:22:25 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu Oct 6 06:22:25 2005 Subject: [Numpy-discussion] A possible problem with PythonWin and Numeric3 Message-ID: <434524A0.9020400@sympatico.ca> PythonWin 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32. Portions Copyright 1994-2004 Mark Hammond (mhammond at skippinet.com.au) - see 'Help/About PythonWin' for further copyright information. >>> import scipy <== Executing this causes a crash. >>> import scipy.base <== This has the same effect Colin W. From faltet at carabos.com Thu Oct 6 07:59:19 2005 From: faltet at carabos.com (Francesc Altet) Date: Thu Oct 6 07:59:19 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <200510061152.55708.faltet@carabos.com> References: <43418AA8.2040809@hawaii.edu> <434467D9.3040707@sympatico.ca> <200510061152.55708.faltet@carabos.com> Message-ID: <200510061657.09349.faltet@carabos.com> A Dijous 06 Octubre 2005 11:52, Francesc Altet va escriure: > Exactly, I'm wondering the same questions. Provided that numarray is > an excellent and mature piece of code, I'd like to hear a declaration > of principles on that subject. Er, I think I'd better said "declaration of intentions", sorry. Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From oliphant at ee.byu.edu Thu Oct 6 08:24:10 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu Oct 6 08:24:10 2005 Subject: [SciPy-user] Re: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> Message-ID: <43454135.503@ee.byu.edu> Matthew Brett wrote: >Hi Travis, > >So, can I summarize for my own benefit? > >I think we all agree that software without documentation is not >usable, and therefore that free software requires that the basic >documentation also be free. > >Obviously if the _basic_ documentation is not free, this will >dramatically reduce software uptake. > >And I think we all also agree that there is no problem of any sort >with not-free documentation that expands in a useful way on the basic >documentation. > >So, just to clarify - does Fenando win his bet (was it dollars or >doughnuts)? Are you happy for the community to write and release free >basic documentation for scipy.base? > > Yes, yes. Fernando knows me pretty well in that regard. I'll probably contribute to it with useful selections from the book (once I have the book finished). It really motivates me to produce more (of everything) when I see people actually purchasing the documentation. And I certainly don't want "lack of free documentation" to hamper adoption of scipy core. So, it benefits everybody to have free version available. It would be great if somebody could at least post the pydoc output for the core. That is a good start. I'll add basic C-API calls to free documentation and we'll be ready to go. -Travis From oliphant at ee.byu.edu Thu Oct 6 08:31:38 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu Oct 6 08:31:38 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Fwd: Scipy_core and numarray In-Reply-To: <36BF0393-F2E6-46FA-8A2A-978B1D51535C@gmail.com> References: <17221.11905.227711.3197@imperial.ac.uk> <36BF0393-F2E6-46FA-8A2A-978B1D51535C@gmail.com> Message-ID: <434542BE.3010005@ee.byu.edu> Andrew Jaffe wrote: >Hi All, > >I've tried to post via the newsgroup interface, but my messages have >never appeared, so here's another try!... > >I'm currently a numarray user from the astrophysics community and I >suppose I want to understand a bit more about the relationship between >the two projects. Do the two projects exist side-by-side? Will numarray >be superseded by scipy_core? I like the speed advantage of scipy_core, >but I there may be features of numarray I don't want to give up (or >legacy code I need to interfact with). > > I'm interested in the "features" you don't want to give up. SciPy core should have all features of numarray. Also, scipy core, numarray, and Numeric can all live together. If you are using sufficient versions of each, you can exchange data via the array interface. The ONLY reason I started scipy core was to unify Numeric and numarray users. My selling documentation, should not be construed as anything else but trying to help people who can't find time to contribute in other ways to contribute to that goal. If I was "just trying to make money," I would be doing something far different than this... > >Along the latter lines, I have a more practical question: one >numarray capability that I used a lot was the ability to create >uninitialized arrays, such as > > arr = numarray.array(shape=(3,5), type=Float64) > > It's actually there in Numeric (since 23.7 I think). Numeric.empty((3,5),'d') and also in (new) scipy scipy.empty((3,5),scipy.float64) In (new) scipy you can also create new arrays from buffers very easily. -Travis From jswhit at fastmail.fm Thu Oct 6 08:43:52 2005 From: jswhit at fastmail.fm (Jeff Whitaker) Date: Thu Oct 6 08:43:52 2005 Subject: [Numpy-discussion] Re: [SciPy-user] Fwd: Scipy_core and numarray In-Reply-To: <434542BE.3010005@ee.byu.edu> References: <17221.11905.227711.3197@imperial.ac.uk> <36BF0393-F2E6-46FA-8A2A-978B1D51535C@gmail.com> <434542BE.3010005@ee.byu.edu> Message-ID: <434545C5.80709@fastmail.fm> Travis Oliphant wrote: > Andrew Jaffe wrote: > >> Hi All, >> >> I've tried to post via the newsgroup interface, but my messages have >> never appeared, so here's another try!... >> >> I'm currently a numarray user from the astrophysics community and I >> suppose I want to understand a bit more about the relationship between >> the two projects. Do the two projects exist side-by-side? Will numarray >> be superseded by scipy_core? I like the speed advantage of scipy_core, >> but I there may be features of numarray I don't want to give up (or >> legacy code I need to interfact with). >> >> > I'm interested in the "features" you don't want to give up. SciPy > core should have all features of numarray. > Also, scipy core, numarray, and Numeric can all live together. If you > are using sufficient versions of each, you can exchange data via the > array interface. Will scipy_core have nd_image and memory mapping? -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/CDC R/CDC1 Email : Jeffrey.S.Whitaker at noaa.gov 325 Broadway Office : Skaggs Research Cntr 1D-124 Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg From matthew.brett at gmail.com Thu Oct 6 11:16:15 2005 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu Oct 6 11:16:15 2005 Subject: [SciPy-user] Re: [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <43454135.503@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> Message-ID: <1e2af89e0510061115l57351184id6ebba394b4b5b26@mail.gmail.com> Hi, > >So, just to clarify - does Fenando win his bet (was it dollars or > >doughnuts)? Are you happy for the community to write and release free > >basic documentation for scipy.base? > > Yes, yes. Fernando knows me pretty well in that regard. I'll probably > contribute to it with useful selections from the book (once I have the > book finished). It really motivates me to produce more (of > everything) when I see people actually purchasing the documentation. Excellent - thanks very much for that. When my credit cards have recovered from their address changes I will be early in line! Best, Matthew From cjw at sympatico.ca Thu Oct 6 11:57:37 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu Oct 6 11:57:37 2005 Subject: [Numpy-discussion] A comparative test Message-ID: <43457338.2010503@sympatico.ca> # NumericNumarrayTest.py ''' A test, provided by Francesc Altet http://sourceforge.net/mailarchive/forum.php?thread_id=6396059&forum_id=4890 ''' from timeit import Timer setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" print 'Timer-Numeric23.8:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) # setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" print 'Timer-numarray1.3.3:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) # scipy/Numeric3 added setup = "import scipy.base; a = scipy.base.arange(2000);a.shape=(1000,2)" print 'Timer-Numeric3:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) RESULTS: >pythonw -u "NumericNumarrayTest.py" Timer-Numeric23.8: 0.179712784725 Timer-numarray1.3.3: 0.21674933546 Timer-Numeric3: 0.253077136899 >Exit code: 0 Colin W. From faltet at carabos.com Thu Oct 6 12:43:46 2005 From: faltet at carabos.com (Francesc Altet) Date: Thu Oct 6 12:43:46 2005 Subject: [Numpy-discussion] A comparative test In-Reply-To: <43457338.2010503@sympatico.ca> References: <43457338.2010503@sympatico.ca> Message-ID: <200510062142.56692.faltet@carabos.com> A Dijous 06 Octubre 2005 20:55, Colin J. Williams va escriure: > RESULTS: > >pythonw -u "NumericNumarrayTest.py" > > Timer-Numeric23.8: 0.179712784725 > Timer-numarray1.3.3: 0.21674933546 > Timer-Numeric3: 0.253077136899 Is that really true? From my original post: >>> from timeit import Timer >>> setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) 0.12782907485961914 >>> setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) 1.2013700008392334 My post was from January, so, in the interim numarray has improved *a lot* it's object creation time. In that case, why the numarray team hasn't publisized that more? Well, I think I should be missing something. Time for nap. Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From rlw at stsci.edu Thu Oct 6 13:16:34 2005 From: rlw at stsci.edu (Rick White) Date: Thu Oct 6 13:16:34 2005 Subject: [Numpy-discussion] A comparative test In-Reply-To: <200510062142.56692.faltet@carabos.com> References: <43457338.2010503@sympatico.ca> <200510062142.56692.faltet@carabos.com> Message-ID: On Thu, 6 Oct 2005, Francesc Altet wrote: > A Dijous 06 Octubre 2005 20:55, Colin J. Williams va escriure: > > RESULTS: > > >pythonw -u "NumericNumarrayTest.py" > > > > Timer-Numeric23.8: 0.179712784725 > > Timer-numarray1.3.3: 0.21674933546 > > Timer-Numeric3: 0.253077136899 > > Is that really true? From my original post: > > >>> from timeit import Timer > >>> setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" > >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) > 0.12782907485961914 > >>> setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" > >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) > 1.2013700008392334 > > My post was from January, so, in the interim numarray has improved *a > lot* it's object creation time. In that case, why the numarray team > hasn't publisized that more? Well, I think I should be missing > something. Time for nap. Well, this seems pretty clear: On Thu, 14 Apr 2005, Todd Miller wrote: > Subject: [Numpy-discussion] ANN: numarray-1.3.0 > > I. ENHANCEMENTS > > 2. Removal of dictionary update from array view creation improves > performance of view/slice/subarray creation. This should e.g. improve > the performance of wxPython sequence protocol access to Nx2 arrays. You might wonder why everyone didn't notice when their numarray programs started to run blazingly fast after 1.3.0. My (jaded) view is that it is very rare to have an application where the overall execution time is dominated by the time to create small arrays. When you're in that limit, there are a million other overheads that are also slowing you down. In the vast majority of applications you could reduce the array creation time to zero and probably wouldn't notice the difference without running timing tests. IMHO the numarray developers put their emphasis in the right place by focusing on large array performance and improved functionality, and all the noise around small array indexing performance was just a red herring that convinced some folks not to try numarray because they heard it was slow. I hope Travis doesn't devote a lot of effort to this optimization right now. I'd be much more interested in seeing a large array benchmark. Rick From Fernando.Perez at colorado.edu Thu Oct 6 13:28:36 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Thu Oct 6 13:28:36 2005 Subject: [Numpy-discussion] A comparative test In-Reply-To: References: <43457338.2010503@sympatico.ca> <200510062142.56692.faltet@carabos.com> Message-ID: <4345889B.6050909@colorado.edu> Rick White wrote: > IMHO the numarray developers put their emphasis in the right place > by focusing on large array performance and improved functionality, > and all the noise around small array indexing performance was just > a red herring that convinced some folks not to try numarray because > they heard it was slow. I hope Travis doesn't devote a lot of > effort to this optimization right now. I'd be much more interested > in seeing a large array benchmark. Except for codes like our PDE solvers, which need to create 10s of millions of small arrays very, very fast. So no, it's not a red herring [1]. Just because you don't need something doesn't mean there's no valid need for it. I am not disputing in any way the value of the many, many improvements made by numarray, nor the importance of fixing large array performance for many applications (such as I imagine is the typical workflow of astronomical data analysis). The fact that Travis tried to incorporate all of numarrays' improvements into the new library is a recognition of the value of all this work. But calling the small array performance problem a 'red herring' is inaccurate and unfair, at best. Regards, f [1] Yes, I could preallocate pools of memory for this, manage them manually, etc. But then I wouldn't be writing my code in python, would I? The whole point of using python is to have my cake and eat it too: I want high-level code that gives me near-hardware max speeds. With carefully tuned (python) code, judiciously sprinkled minimal amounts of C/C++/Fortran, and a LOT of thinking about algorithm design, I get that. From oliphant at ee.byu.edu Thu Oct 6 15:15:40 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu Oct 6 15:15:40 2005 Subject: [Numpy-discussion] Simple optimization In-Reply-To: <43457338.2010503@sympatico.ca> References: <43457338.2010503@sympatico.ca> Message-ID: <4345A1C4.9080409@ee.byu.edu> Colin J. Williams wrote: > # NumericNumarrayTest.py > ''' A test, provided by Francesc Altet > > http://sourceforge.net/mailarchive/forum.php?thread_id=6396059&forum_id=4890 > > ''' > from timeit import Timer > setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" > print 'Timer-Numeric23.8:', Timer("for i in range(len(a)): row=a[i]", > setup).timeit(number=100) > > # > setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" > print 'Timer-numarray1.3.3:', Timer("for i in range(len(a)): > row=a[i]", setup).timeit(number=100) > > # scipy/Numeric3 added > setup = "import scipy.base; a = scipy.base.arange(2000);a.shape=(1000,2)" > print 'Timer-Numeric3:', Timer("for i in range(len(a)): row=a[i]", > setup).timeit(number=100) > > RESULTS: > >pythonw -u "NumericNumarrayTest.py" > Timer-Numeric23.8: 0.179712784725 > Timer-numarray1.3.3: 0.21674933546 > Timer-Numeric3: 0.253077136899 After a simple optimization in the array_subscript function (to check for scalar selection up front): new results are (for 1000 runs on my system): Numeric: 0.56431102752685547 numarray: 1.0897960662841797 scipy: 0.57508182525634766 (it's now executing nearly the identical code as Numeric) Any more optimization ideas? -Travis From Fernando.Perez at colorado.edu Thu Oct 6 15:26:03 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Thu Oct 6 15:26:03 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <43454135.503@ee.byu.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> Message-ID: <4345A449.8040303@colorado.edu> Travis Oliphant wrote: > Yes, yes. Fernando knows me pretty well in that regard. I'll probably > contribute to it with useful selections from the book (once I have the > book finished). It really motivates me to produce more (of > everything) when I see people actually purchasing the documentation. To reiterate before my reputation is tarnished forever: the bet was with John, who is the doughnuts-eating one. Being a man of taste, I eat bagels :) [and I wasn't present when the bet was made] With that critical point out of the way, on to minor details. > It would be great if somebody could at least post the pydoc output for > the core. That is a good start. I'll add basic C-API calls to free > documentation and we'll be ready to go. While I fully respect the opinion of those concerned about Travis' decision, I have to say that I am personally not so worried. At scipy'05, when Travis announced the book idea, I asked a very simple question: 'are the docstrings complete?'. His answer was 'yes'. There was a good reason for my asking this question: I personally find that when coding, the most productive and efficient to learn a library is by typing import foo foo. and then foo.bar? or foo.bar?? if I want to see the source (for python-implemented things). I understand that some people prefer to read manuals/books, and there are topics beyond the single-function API that are best discussed in a book format. But given the instantaneous response of the above, I would rather use this than wade through an index in most cases. In fact, I'd like docstrings to provide (simple) examples in them, and one thing on my radar now that ipython is growing GUI/XML legs is to add support for latex in docstrings. I think that a library where all individual methods/functions have full, accurate and example-loaded docstrings, and where class declarations and top-level module docstrings provide the higher-level overview, is extremely usable as is. Especially when you can also keep pydoc running in html server mode (pydoc -p) to browse at your leisure, and trivially dump that HTML info to disk for static reference if needed. If this is done, I think that the space for a 'real book' is better defined, providing less of a detail reference and more of a teaching coverage. I sense that this is Travis' approach, and I personally have no beef with it whatsoever. If he had deliberately stripped or crippled all docstrings from new_scipy to artificially create a need for his book, that would be a different story. But he certainly didn't do such a thing. I do think that there's a lot of room for higher-level books on python for scientific computing. Langtangen's is already out there, and Perry's excellent tutorial is already free for all to use (while slightly focused on astronomy, I find it a very useful document). I guess what I'm trying to say is that the _library_ documentation can be considered to be a (good) set of docstrings. As long as scipy ships with such a thing (which can be improved with the easiest of all patches by users, since no code is touched), I am not so worried that it will be perceived as lacking real docs. Please note that I am not discussing the price/time constraints of his licensing, which is strictly his choice to make. Again, this is just my _personal_ opinion. And while I have great appreciation for Travis, he's not paying me to say any of this :) Regards, f From arnd.baecker at web.de Fri Oct 7 00:12:54 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Fri Oct 7 00:12:54 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4345A449.8040303@colorado.edu> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> Message-ID: On Thu, 6 Oct 2005, Fernando Perez wrote: [...] > To reiterate before my reputation is tarnished forever: the bet was with John, > who is the doughnuts-eating one. Being a man of taste, I eat bagels :) [and I > wasn't present when the bet was made] > > With that critical point out of the way, on to minor details. Thanks for the clarification - this definitively made my day ;-) [...] > In fact, I'd like docstrings to provide (simple) examples in them, +(some large positive number ;-) > and one thing on my radar now that > ipython is growing GUI/XML legs is to add support for latex in docstrings. Are you thinking of restructured text http://docutils.sourceforge.net/ together with the mathhack http://docutils.sourceforge.net/sandbox/cben/rolehack/ or the tools used in the LaTeXwiki, http://mcelrath.org/Notes/LatexWiki ? Independent of this, I see two main points: - how should the examples be added - who should add the examples to the doc-strings? Getting "end-users" to help with this task sounds quite attractive to me. Would it make sense to revive the pynotes idea http://www.scipy.org/documentation/mailman?fn=scipy-dev/2004-November/002536.html which would make it possible to add remarks to any python command? However, I don't know if there is a good solution on how to submit these notes and how they finally should get merged into the doc-string (One possibility would be to combine this in some way with Travis' live-docs). It might be unavoidable that at the end some knowledgable person has to go over suggested additions and incorporate them on a case-by-case basis? Best, Arnd From Chris.Barker at noaa.gov Fri Oct 7 00:14:45 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Fri Oct 7 00:14:45 2005 Subject: [Numpy-discussion] A comparative test In-Reply-To: References: <43457338.2010503@sympatico.ca> <200510062142.56692.faltet@carabos.com> Message-ID: <43461FF1.4010300@noaa.gov> Rick White wrote: > You might wonder why everyone didn't notice when their numarray > programs started to run blazingly fast after 1.3.0. Maybe because those of us for whom small array performance is important don't use numarray? I know I haven't tested my wxPython FloatCanvas with numarray since I discovered painfully slow performance a while back. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From perry at stsci.edu Fri Oct 7 05:50:21 2005 From: perry at stsci.edu (Perry Greenfield) Date: Fri Oct 7 05:50:21 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <200510061152.55708.faltet@carabos.com> Message-ID: Sorry for the tardy response. Transatlantic flights don't yet have convenient internet service. I've been meaning to put out a message of this sort soon anyway. This is few days earlier than I was planning, but close enough. Our basic position has been and continues to be that if scipy_core provides all the capabilities that we need, we will switch to it. We have been involved in installing it on the platforms we need it for (namely Solaris, which has proven difficult to build on) and testing it (which is ongoing). So far things look pretty promising and barring any unforseen problems, we will likely begin porting recarray to it next week. As Travis has already mentioned, the whole point of scipy_core was to unify the two existing projects, not introduce a 3rd. I have been involved all along in design and interface discussions with Travis on this project. We will maintain numarray until the point at which we switch all of our code to scipy_core (i.e., all libraries and applications that depend on it). This may take several months for some (likely the most work is for C extensions that use the numarray C-API). When this is complete (including testing), we may support numarray for a while in the sense of answering questions, but it would not likely see any more new releases after that point. We do not have the resources to support two different numeric packages indefinitely. We will report on our progress in this switchover (most likely our libraries and applications will be set up to work with both in the interim). While I appreciate the pain that this may cause those who have written C-extensions for numarray, it's hard to deny that unification will bring great benefits. It's as much pain for us as anyone. It should not be as painful to port python code since scipy_core should support all capabilities. But some changes will be needed since things are not always done the same way (e.g., dtype for type). Perry Francesc Altet wrote: > > Exactly, I'm wondering the same questions. Provided that numarray is > an excellent and mature piece of code, I'd like to hear a declaration > of principles on that subject. > > Regards, > > A Dijous 06 Octubre 2005 01:55, Colin J. Williams va escriure: > > Perry, > > > > It would be helpful if you gave some indication of your organization's > > intention with respect to numarray. > > > > Will it be maintained, as in the past, fully in the public domain and > > without charge for either the code or the API information needed to make > > use of the code? > > > > New versions have been produced every few months, will this continue? > > > > Colin W. > > > From usr01873 at prolocation.net Fri Oct 7 05:55:22 2005 From: usr01873 at prolocation.net (usr01873 at prolocation.net) Date: Fri Oct 7 05:55:22 2005 Subject: [Numpy-discussion] wrapping question Message-ID: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> Hi all, I'm using the numarray package. I would like to use the numarray C API to create a new array object and fill it with data without copying. Also, I would like numarray to take care of freeing the data once the numarray object is destructed. I've managed to find some related posts in the mailing list archive, but it is really unclear which of these posts are still accurate and what the current 'recommended' approach is. See: http://sourceforge.net/mailarchive/message.php?msg_id=11788304 http://sourceforge.net/mailarchive/message.php?msg_id=2494535 Currently, I first create a new PyArrayObject that is large enough to hold the data: array = NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims); then I pass array->data to a C function that reads the data from disk. Any comments on this approach? Also, I was wondering if numarray frees the data using this approach (because it also allocates the memory in the first place within the NA_vNewArray function.) Thanks, Joris van Zwieten From jmiller at stsci.edu Fri Oct 7 06:46:29 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 7 06:46:29 2005 Subject: [Numpy-discussion] wrapping question In-Reply-To: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> References: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> Message-ID: <43467B61.7080101@stsci.edu> usr01873 at prolocation.net wrote: >Hi all, > >I'm using the numarray package. I would like to use the numarray C API to >create a new array object and fill it with data without copying. Also, I > > The numarray C-API has a compatibility layer which supports most Numeric C-API functionality. So, using numarray, you can write most Numeric/Numeric3/scipy_core-like code in C. For historical reasons numarray also has a wider set of native APIs (i.e. NA_vNewArray()), but since numarray is likely to be replaced by scipy_core when it matures, it's wisest to use the compatibility API. >would like numarray to take care of freeing the data once the numarray >object is destructed. I've managed to find some related posts in the >mailing list archive, but it is really unclear which of these posts are >still accurate and what the current 'recommended' approach is. See: > >http://sourceforge.net/mailarchive/message.php?msg_id=11788304 >http://sourceforge.net/mailarchive/message.php?msg_id=2494535 > >Currently, I first create a new PyArrayObject that is large enough to hold >the data: >array = NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims); > >then I pass array->data to a C function that reads the data from disk. Any >comments on this approach? > That should work fine; 'array' owns the data and will free it when 'array' is destructed. A better approach, however, is to use numarray's PyArray_FromDims() which is the Numeric/Numeric3/scipy_core API. >Also, I was wondering if numarray frees the >data using this approach (because it also allocates the memory in the >first place within the NA_vNewArray function.) > > Yes, numarray will free the data allocated internally by NA_vNewArray(). numarray will also free the data allocated by PyArray_FromDims(). Regards, Todd From usr01873 at prolocation.net Fri Oct 7 07:27:28 2005 From: usr01873 at prolocation.net (usr01873 at prolocation.net) Date: Fri Oct 7 07:27:28 2005 Subject: [Numpy-discussion] wrapping question In-Reply-To: <43467B61.7080101@stsci.edu> References: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> <43467B61.7080101@stsci.edu> Message-ID: <52411.217.77.152.33.1128695128.squirrel@control.prolocation.net> > usr01873 at prolocation.net wrote: > >> Hi all, >> >> >> I'm using the numarray package. I would like to use the numarray C API >> to create a new array object and fill it with data without copying. >> Also, I >> >> >> > The numarray C-API has a compatibility layer which supports most Numeric > C-API functionality. So, using numarray, you can write most > Numeric/Numeric3/scipy_core-like code in C. For historical reasons > numarray also has a wider set of native APIs (i.e. NA_vNewArray()), but > since numarray is likely to be replaced by scipy_core when it matures, > it's wisest to use the compatibility API. > >> would like numarray to take care of freeing the data once the numarray >> object is destructed. I've managed to find some related posts in the >> mailing list archive, but it is really unclear which of these posts are >> still accurate and what the current 'recommended' approach is. See: >> >> http://sourceforge.net/mailarchive/message.php?msg_id=11788304 >> http://sourceforge.net/mailarchive/message.php?msg_id=2494535 >> >> >> Currently, I first create a new PyArrayObject that is large enough to >> hold the data: array = >> NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims); >> >> >> then I pass array->data to a C function that reads the data from disk. >> Any >> comments on this approach? >> > That should work fine; 'array' owns the data and will free it when > 'array' is destructed. > > > A better approach, however, is to use numarray's PyArray_FromDims() > which is the Numeric/Numeric3/scipy_core API. > >> Also, I was wondering if numarray frees the >> data using this approach (because it also allocates the memory in the >> first place within the NA_vNewArray function.) >> >> > Yes, numarray will free the data allocated internally by > NA_vNewArray(). numarray will also free the data allocated by > PyArray_FromDims(). Thanx for the reply! Just to be sure I understand this correctly: to conform to the Numeric/Numeric3/scipy api I could/should do the following: 1. allocate a new array using PyArray_FromDims() 2. pass the ->data pointer to a C routine that reads the data 3. use PyArray_Return to return the array to Python 4. No DECREF's are necessary? Also, PyArray_FromDims() will not copy the data, right? Thanks, Joris /* (From http://stsdas.stsci.edu/numarray/numarray-1.3.html/node58.html) PyObject* PyArray_FromDims(int nd, int *dims, int type) This function will allocate a new numarray. An array created with PyArray_FromDims can be used as a temporary or returned using PyArray_Return. Used as a temporary, calling Py_DECREF deallocates it. */ > > Regards, > Todd > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > From jmiller at stsci.edu Fri Oct 7 07:51:50 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 7 07:51:50 2005 Subject: [Numpy-discussion] wrapping question In-Reply-To: <52411.217.77.152.33.1128695128.squirrel@control.prolocation.net> References: <35664.217.77.152.33.1128689530.squirrel@control.prolocation.net> <43467B61.7080101@stsci.edu> <52411.217.77.152.33.1128695128.squirrel@control.prolocation.net> Message-ID: <43468AED.1030403@stsci.edu> usr01873 at prolocation.net wrote: >>usr01873 at prolocation.net wrote: >> >> >> >>>Hi all, >>> >>> >>>I'm using the numarray package. I would like to use the numarray C API >>>to create a new array object and fill it with data without copying. >>>Also, I >>> >>> >>> >>> >>> >>The numarray C-API has a compatibility layer which supports most Numeric >>C-API functionality. So, using numarray, you can write most >>Numeric/Numeric3/scipy_core-like code in C. For historical reasons >>numarray also has a wider set of native APIs (i.e. NA_vNewArray()), but >>since numarray is likely to be replaced by scipy_core when it matures, >>it's wisest to use the compatibility API. >> >> >> >>>would like numarray to take care of freeing the data once the numarray >>>object is destructed. I've managed to find some related posts in the >>>mailing list archive, but it is really unclear which of these posts are >>> still accurate and what the current 'recommended' approach is. See: >>> >>>http://sourceforge.net/mailarchive/message.php?msg_id=11788304 >>>http://sourceforge.net/mailarchive/message.php?msg_id=2494535 >>> >>> >>>Currently, I first create a new PyArrayObject that is large enough to >>>hold the data: array = >>>NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims); >>> >>> >>>then I pass array->data to a C function that reads the data from disk. >>>Any >>>comments on this approach? >>> >>> >>> >>That should work fine; 'array' owns the data and will free it when >>'array' is destructed. >> >> >>A better approach, however, is to use numarray's PyArray_FromDims() >>which is the Numeric/Numeric3/scipy_core API. >> >> >> >>>Also, I was wondering if numarray frees the >>>data using this approach (because it also allocates the memory in the >>>first place within the NA_vNewArray function.) >>> >>> >>> >>> >>Yes, numarray will free the data allocated internally by >>NA_vNewArray(). numarray will also free the data allocated by >>PyArray_FromDims(). >> >> > >Thanx for the reply! Just to be sure I understand this correctly: to >conform to the Numeric/Numeric3/scipy api I could/should do the following: > >1. allocate a new array using PyArray_FromDims() >2. pass the ->data pointer to a C routine that reads the data >3. use PyArray_Return to return the array to Python >4. No DECREF's are necessary? > >Also, PyArray_FromDims() will not copy the data, right? >Thanks, > > 1. yes 2. yes 3. PyArray_Return is mostly important if you want rank-0 arrays converted into equivalent Python scalars. Otherwise I think it's not really needed and is commonly omitted. 4. When you call PyArray_FromDims(), the returned array has one reference. If the array is a temporary you want to deallocate at extension function exit, then DECREF; if you want to return the array, don't DECREF. Todd >Joris > >/* >(From http://stsdas.stsci.edu/numarray/numarray-1.3.html/node58.html) >PyObject* PyArray_FromDims(int nd, int *dims, int type) > This function will allocate a new numarray. > An array created with PyArray_FromDims can be used as a temporary or > returned using PyArray_Return. > > Used as a temporary, calling Py_DECREF deallocates it. >*/ > > > >>Regards, >>Todd >> >> >> >>------------------------------------------------------- >>This SF.Net email is sponsored by: >>Power Architecture Resource Center: Free content, downloads, discussions, >>and more. http://solutions.newsforge.com/ibmarch.tmpl >>_______________________________________________ >>Numpy-discussion mailing list >>Numpy-discussion at lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> >> >> > > > > >------------------------------------------------------- >This SF.Net email is sponsored by: >Power Architecture Resource Center: Free content, downloads, discussions, >and more. http://solutions.newsforge.com/ibmarch.tmpl >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion at lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > From faltet at carabos.com Fri Oct 7 08:07:40 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 7 08:07:40 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: References: Message-ID: <200510071704.55100.faltet@carabos.com> A Divendres 07 Octubre 2005 14:51, Perry Greenfield va escriure: > Our basic position has been and continues to be that if scipy_core provides > all the capabilities that we need, we will switch to it. We have been > involved in installing it on the platforms we need it for (namely Solaris, > which has proven difficult to build on) and testing it (which is ongoing). > So far things look pretty promising and barring any unforseen problems, we > will likely begin porting recarray to it next week. As Travis has already > mentioned, the whole point of scipy_core was to unify the two existing > projects, not introduce a 3rd. I have been involved all along in design and > interface discussions with Travis on this > project. Ok. That's clear enough and satifies my curiosity, thank you. I must confess, though, that I'm a bit disappointed becuase I was hoping that numarray, now that has arrived to maturity, was going to stay for long time (as any other good piece of software deserves). Anyway, as a long-time user of numarray I can't express enough my gratitude towards you and your team for suporting and enhancing a beast like numarray, that introduced a lot of new features that missed Numeric and that without it all of our current projects (and I guess many others around the world) simply would not have been possible. > We will maintain numarray until the point at which we switch all of our > code to scipy_core (i.e., all libraries and applications that depend on > it). This may take several months for some (likely the most work is for C > extensions that use the numarray C-API). Yeah, I do think so. However, what it worries me is, in my (limited) experience, that substituting large packages like Numeric or numarray, is not a trivial task at all (after all, how much time took until numarray was able to be a good substitute of Numeric?). So, I forsee a transitional time no shorter than a year before arriving to the high standards of quality that numarray has nowadays. This is why I'm not precisely eager to quickly switch to scipy_core until it has proven to be, at least, as solid as it is numarray. Having said that, we are in the mood of offering our implementation of a NestedRecArray class that allows nested fields in RecArray objects. This class has been included in PyTables for some months and has proven to be stable, comes with a nice set of unit tests, and is highly efficient (at least, as much as RecArray for most operations). Also, until ready-for-production scipy_core arrives, I'd ask you (and the maintainers of scipy_core) to provide an array protocol to share data as efficiently as possible between numarray and scipy_core, just to easy the transition between the packages. If you want, we will be happy to collaborate on that area as well. > While I appreciate the pain that this may cause those who have written > C-extensions for numarray, it's hard to deny that unification will bring > great benefits. It's as much pain for us as anyone. Sure, but migration work is not the biggest problem (hopefully) for us. As I said before, the key point to migrate to scipy_core is to make sure that is stable enough, featured enough, coner-cases-proof enough and well-documented enough as it is numarray now. Thanks again for all your work in numarray, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From oliphant at ee.byu.edu Fri Oct 7 10:00:51 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri Oct 7 10:00:51 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <200510071704.55100.faltet@carabos.com> References: <200510071704.55100.faltet@carabos.com> Message-ID: <4346A954.5000400@ee.byu.edu> Francesc Altet wrote: >Also, until ready-for-production scipy_core arrives, I'd ask you (and >the maintainers of scipy_core) to provide an array protocol to share >data as efficiently as possible between numarray and scipy_core, just >to easy the transition between the packages. If you want, we will be >happy to collaborate on that area as well. > > There is an array protocol in place already (see the Array Interface section on numeric.scipy.org). I think it is fully supported in numarray. Also, I have a PEP for placing a simple array object in Python that is nothing more than a shell for passing data around. If somebody would like to push that forward they are welcome. Right now what I have is available in an SVN server. http://svn.scipy.org/svn/PEP get it with svn co http://svn.scipy.org/svn/PEP PEP -Travis From faltet at carabos.com Fri Oct 7 10:07:06 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 7 10:07:06 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <4346A4ED.80404@sympatico.ca> References: <200510071704.55100.faltet@carabos.com> <4346A4ED.80404@sympatico.ca> Message-ID: <200510071905.25174.faltet@carabos.com> A Divendres 07 Octubre 2005 18:40, Colin J. Williams va escriure: > I share these feelings and would like to express my thanks to Todd > Miller and to Perry Greenfield for their help over the years. Of course, I did not explicitely mention Todd in my previous message, but I would like to note that he has always been extremely responsive to suggestions and very efficient in solving many, many issues that appeared in numarray through the years. A big thanks to you as well, Todd :-) -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From jmiller at stsci.edu Fri Oct 7 11:39:42 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 7 11:39:42 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <200510071905.25174.faltet@carabos.com> References: <200510071704.55100.faltet@carabos.com> <4346A4ED.80404@sympatico.ca> <200510071905.25174.faltet@carabos.com> Message-ID: <4346C097.3020703@stsci.edu> Francesc Altet wrote: >A Divendres 07 Octubre 2005 18:40, Colin J. Williams va escriure: > > >>I share these feelings and would like to express my thanks to Todd >>Miller and to Perry Greenfield for their help over the years. >> >> > >Of course, I did not explicitely mention Todd in my previous message, >but I would like to note that he has always been extremely responsive >to suggestions and very efficient in solving many, many issues that >appeared in numarray through the years. > >A big thanks to you as well, Todd :-) > > Ah... you're welcome! Working on numarray has been a great privilege. Regards, Todd From cjw at sympatico.ca Fri Oct 7 11:49:28 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri Oct 7 11:49:28 2005 Subject: [Numpy-discussion] Numeric3.0 - unable to retrieve data typecode Message-ID: <4346C2BC.8020109@sympatico.ca> # temp.py import scipy.base as num3 a= num3.array([[1, 2], [9, 8]]) print a.typecode C:\>python temp.py Traceback (most recent call last): File "temp.py", line 4, in ? print a.typecode AttributeError: 'scipy.ndarray' object has no attribute 'typecode' Is there any simple way to retrieve the dataType, as with numarray's numerictypes? Colin W. From cjw at sympatico.ca Fri Oct 7 13:20:24 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Fri Oct 7 13:20:24 2005 Subject: [Numpy-discussion] Problem with name numeric Message-ID: <4346D825.70309@sympatico.ca> With the following sequence I had expected "scipy.base.numeric" to be a module, instead it is reported as a type. Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy.base.numeric >>> dir() ['__builtins__', '__doc__', '__name__', 'scipy'] >>> scipy >>> scipy.base >>> scipy.base.numeric >>> I need to be able to access ndarray to create a subclass. Colin W. From mantqygk at donahue.com Fri Oct 7 18:34:22 2005 From: mantqygk at donahue.com (Fa Mantz) Date: Fri Oct 7 18:34:22 2005 Subject: [Numpy-discussion] Re: Meaghan exhibitionism Message-ID: <003001c5cba8$5ae79d80$9b1ea8c0@pouter> Hi Do you want t VE UP n your Med o SA TO 70% o dications? It's easy with us - The WebSite VCXAVL iagialanabiealiitr ra $is $xnum $a 134 (30 p.)169 (30 p.) 218 (180 p.) And much more , Good luck -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjw at sympatico.ca Sat Oct 8 07:31:49 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Sat Oct 8 07:31:49 2005 Subject: [Numpy-discussion] Problem with Windows version of Numeric3.0 Message-ID: <4347D83C.10002@sympatico.ca> Either of these import lines crashes in Pythonw, with Boa-constructor, SPE or PythonWin. # subclass.py import scipy.base as num3 import scipy.base.multiarray as M Colin W. From cjw at sympatico.ca Sat Oct 8 09:44:34 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Sat Oct 8 09:44:34 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray Message-ID: <4347F755.2080304@sympatico.ca> With a view to exploring Numeric3, below is a script to check the availability of doc strings. Some are not yet available. There is a method __class__ module but the purpose is not clear. Colin W. # checkDocs.py ''' To check the availabilty of doc strings for the ndarray class. Among the attributes of ndarray is: __class__ module(name[, doc]) Create a module object. The name must be a string; the optional doc argument can have any type. ndarray is referenced as a class and reported as "" but the common Python usage is see the last output line. ''' import scipy.base.multiarray as M print type(M.ndarray) print '>', type( M) print 'module doc:', M.__doc__ for i in dir(M.ndarray): print i, try: print eval('M.'+i+'.__doc__') except: print 'No doc' print '\n', M.ndarray.__doc__ print type(M.ndarray) # Check normal usage import timeit print timeit.Timer, type(timeit.Timer) From oliphant at ee.byu.edu Sat Oct 8 12:48:07 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat Oct 8 12:48:07 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <4347F755.2080304@sympatico.ca> References: <4347F755.2080304@sympatico.ca> Message-ID: <4348224A.3030006@ee.byu.edu> Colin J. Williams wrote: > With a view to exploring Numeric3, below is a script to check the > availability of doc strings. > > Some are not yet available. > I don't think your script will pick up the docs for the ndarray methods. You need to replace eval('M.'+i+'.__doc__') with eval('M.ndarray.'+i+'.__doc__') -Travis From oliphant at ee.byu.edu Sat Oct 8 12:55:20 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat Oct 8 12:55:20 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <4347F755.2080304@sympatico.ca> References: <4347F755.2080304@sympatico.ca> Message-ID: <4348240D.6010308@ee.byu.edu> Colin J. Williams wrote: > With a view to exploring Numeric3, below is a script to check the > availability of doc strings. > > Some are not yet available. > > There is a method __class__ module but the purpose is not clear. > > Colin W. > > # checkDocs.py > ''' To check the availabilty of doc strings for the ndarray class. > > Among the attributes of ndarray is: > __class__ module(name[, doc]) I'm not sure what you are talking about here? Where are you getting this? I don't know of any such attribute. There is a __class__ attribute. But I don't see a __class__ module(name[, doc]) attribute -Travis From oliphant at ee.byu.edu Sat Oct 8 21:50:10 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat Oct 8 21:50:10 2005 Subject: [Numpy-discussion] Another PEP needed for the Numeric community. Message-ID: <4348A153.5050405@ee.byu.edu> Full, 64-bit support in Python is problematic because of the buffer and sequence protocols which require int's in their interfaces. Right now, numarray support memory mapped arrays. SciPy can easily support memory mapped arrays by modifying the same module that numarray has forged (using the frombuffer function). However, the module will be limited on 64-bit systems because memory-map support in Python is limited to 32-bit files even on 64-bit systems. This was an intentional limitation of the memory map module in Python so that the sequence and buffer protocols could be supported. However, the choice to not even allow a 64-bit size memory map to be created seems wrong. There is no need to go through the sequence and buffer interface at all times. If the memory-map module in Python were re-written to inherit from a big-map object that did not export the buffer and sequence protocols (or did so in a limited fashion), then all that is needed is for the object to export it's data pointer and size. Then a function could be written to create an array in scipy that used the mmap's exported buffer as its data region. This would allow very big memory mapped arrays without waiting for Python to fix it's sequence and buffer protocols (which may not be fixed until Python 3.0). Thus, a PEP stating how the mmap module should change to support 64-bit systems is needed. Another possible project idea for any lurking people desiring to help. The other possibility is to just copy the nice muliplatform code in the Python mmap module and use an ndarray as the memory-mapped object. -Travis From cjw at sympatico.ca Sun Oct 9 10:54:43 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Sun Oct 9 10:54:43 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <4348240D.6010308@ee.byu.edu> References: <4347F755.2080304@sympatico.ca> <4348240D.6010308@ee.byu.edu> Message-ID: <43495948.9020402@sympatico.ca> Travis Oliphant wrote: > Colin J. Williams wrote: > >> With a view to exploring Numeric3, below is a script to check the >> availability of doc strings. >> >> Some are not yet available. >> >> There is a method __class__ module but the purpose is not clear. >> >> Colin W. >> >> # checkDocs.py >> ''' To check the availabilty of doc strings for the ndarray class. >> >> Among the attributes of ndarray is: >> __class__ module(name[, doc]) > > > > I'm not sure what you are talking about here? Where are you getting > this? > > > I don't know of any such attribute. There is a __class__ attribute. > But I don't see a > > __class__ module(name[, doc]) attribute > > -Travis > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > It might have been clearer if I had referred to the name "__class__" in the multiarray module. Please see the sequence below: >>> import scipy.base.multiarray as M >>> M.__class__ >>> I am surprised that this name points to a module. It usually is an attribute of a class instance which points to the class object of that instance. Colin W. From matthew.brett at gmail.com Sun Oct 9 13:07:46 2005 From: matthew.brett at gmail.com (Matthew Brett) Date: Sun Oct 9 13:07:46 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4345A449.8040303@colorado.edu> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> Message-ID: <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> Hi, > While I fully respect the opinion of those concerned about Travis' decision, I > have to say that I am personally not so worried. At scipy'05, when Travis > announced the book idea, I asked a very simple question: 'are the docstrings > complete?'. His answer was 'yes'. > > There was a good reason for my asking this question: I personally find that > when coding, the most productive and efficient to learn a library is by typing > > import foo > foo. > > and then > > foo.bar? Just a tiny addition. I know what you mean, but my experience is of someone recently trying to learn numarray / Numeric, and I suspect I am not alone in printing out the documentation and reading a moderate amount of it before I get started. Call me old-fashioned (it wouldn't be the first time!), See you, Matthew From Fernando.Perez at colorado.edu Sun Oct 9 16:19:30 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Sun Oct 9 16:19:30 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> Message-ID: <4349A524.40208@colorado.edu> Matthew Brett wrote: >>While I fully respect the opinion of those concerned about Travis' decision, I >>have to say that I am personally not so worried. At scipy'05, when Travis >>announced the book idea, I asked a very simple question: 'are the docstrings >>complete?'. His answer was 'yes'. >> >>There was a good reason for my asking this question: I personally find that >>when coding, the most productive and efficient to learn a library is by typing >> >>import foo >>foo. >> >>and then >> >>foo.bar? > > > Just a tiny addition. I know what you mean, but my experience is of > someone recently trying to learn numarray / Numeric, and I suspect I > am not alone in printing out the documentation and reading a moderate > amount of it before I get started. Call me old-fashioned (it > wouldn't be the first time!), Certainly. As I said, I do think there is room for 'book-style' (as opposed to API reference) books for scipy. Langtangen's ($$$) and Perry's (free) are two such existing offers, and now Travis' comes in as well (and I still believe there is room for more). My idea was of top-level (module) docstrings which would provide a reasonable overview, along with single-function ones providing not only API reference but also a few examples. Since pydoc -w can generate permanent HTML for browsing/printing out of any module (and docutils has even more sophisticated facilities), I think this provides acceptable coverage of the basic library. And it does give anyone who wants 'material to read on the bus' (which I often need myself) a reasonable solution, I think. I just wanted to clarify that a docstring-based set of docs is not limited either to interactive usage via ipython, nor to raw API information. It can both be printed for offline use, and can cover enough overview and examples to be genuinely useful standalone. Not a substitute for a full book, but not a crippled tool either, I think. Cheers, f From tchur at optusnet.com.au Sun Oct 9 16:56:38 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Sun Oct 9 16:56:38 2005 Subject: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation In-Reply-To: <4349A524.40208@colorado.edu> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> <4349A524.40208@colorado.edu> Message-ID: <4349ADD4.7060701@optusnet.com.au> Fernando Perez wrote: > Certainly. As I said, I do think there is room for 'book-style' (as > opposed to API reference) books for scipy. Langtangen's ($$$) and > Perry's (free) are two such existing offers, and now Travis' comes in as > well (and I still believe there is room for more). > > My idea was of top-level (module) docstrings which would provide a > reasonable overview, along with single-function ones providing not only > API reference but also a few examples. Since pydoc -w can generate > permanent HTML for browsing/printing out of any module (and docutils has > even more sophisticated facilities), I think this provides acceptable > coverage of the basic library. And it does give anyone who wants > 'material to read on the bus' (which I often need myself) a reasonable > solution, I think. > > I just wanted to clarify that a docstring-based set of docs is not > limited either to interactive usage via ipython, nor to raw API > information. It can both be printed for offline use, and can cover > enough overview and examples to be genuinely useful standalone. Not a > substitute for a full book, but not a crippled tool either, I think. Sounds like the ideal solution to me. Now, I wonder if it is possible to resolve the legal situation over the existing NumPy documentation? By which I mean, I wonder if it is possible to just copy suitable parts of the NumPy docs into the scipy.core docstrings (improving on them if possible) - with attribution of course. Or is it necessary or safer to start from scratch, or at least to extensively paraphrase the NumPy docs? Finally, to whom should we send docstrings? To Travis? Tim C From oliphant at ee.byu.edu Sun Oct 9 17:07:21 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sun Oct 9 17:07:21 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <43495948.9020402@sympatico.ca> References: <4347F755.2080304@sympatico.ca> <4348240D.6010308@ee.byu.edu> <43495948.9020402@sympatico.ca> Message-ID: <4349B055.8060006@ee.byu.edu> >> I'm not sure what you are talking about here? Where are you getting >> this? >> >> >> I don't know of any such attribute. There is a __class__ attribute. >> But I don't see a >> >> __class__ module(name[, doc]) attribute >> >> -Travis >> > > Please see the sequence below: > > >>> import scipy.base.multiarray as M > >>> M.__class__ > > >>> > > I am surprised that this name points to a module. It usually is an > attribute of a class instance which > points to the class object of that instance. Why is this surprising? scipy.base.multiarray is a module, therefore it's "class" is type 'module'. This seems fine to me. At any rate, Python is assigning the __class__ attribute to the extension module multiarray, so it is what it is. -Travis From greg.ewing at canterbury.ac.nz Sun Oct 9 19:39:50 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun Oct 9 19:39:50 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <43495948.9020402@sympatico.ca> References: <4347F755.2080304@sympatico.ca> <4348240D.6010308@ee.byu.edu> <43495948.9020402@sympatico.ca> Message-ID: <4349D442.6050507@canterbury.ac.nz> Colin J. Williams wrote: > >>> import scipy.base.multiarray as M > >>> M.__class__ > > >>> > > I am surprised that this name points to a module. is NOT a module, it's the type (or class) to which all modules belong. This is to be expected if scipy.base.multiarray is itself a module. The reason it says and not is because it's a built-in type/class rather than one defined in Python. But ever since the type/class unification, there's very little difference in meaning between the terms 'type' and 'class'. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From oliphant at ee.byu.edu Sun Oct 9 21:28:33 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sun Oct 9 21:28:33 2005 Subject: [Numpy-discussion] Question about 64-bit integers being cast to double precision Message-ID: <4349EDA2.2090802@ee.byu.edu> What is the opinion of people here regarding the casting of 64-bit integers to double precision. In scipy (as in Numeric), there is the concept of "Casting safely" to a type. This concept is used when choosing a ufunc, for example. My understanding is that a 64-bit integer cannot be cast safely to a double-precision floating point number, because precision is lost in the conversion. However, at least a signed 64-bit integer can usually be cast safely to a long double precision floating point number. This is not too big a deal on 32-bit systems where people rarely request 64-bit integers. However, on some 64-bit systems (where the C long is 64-bit), Python's default integer is 64-bit. Therefore, simple expressions like sqrt(2) which require conversion to floating point will look for the first floating point number that it can convert a 64-bit integer to safely. This can only be a long double. The result is that on 64-bit systems, the long double type gets used a lot more. Is this acceptable? expected? What do those of you on 64-bit systems think? -Travis From Chris.Barker at noaa.gov Sun Oct 9 21:38:25 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Sun Oct 9 21:38:25 2005 Subject: [Numpy-discussion] Can you enumerarte an array? Message-ID: <4349F00C.7080605@noaa.gov> Hi all, A freind of mine that I jsut introduced to NumPy asked me about enumerating an array. What he'd like to see is something like: A = N.ones((3,4)) for indexes, item in enumeate(A): print indexes result in: (0,0) (0,1) (0,2) (1,0) . . . You get the idea. Can you do this with any of the three NumPys? If not consider this a feature request for scipy_base. -Chris From arnd.baecker at web.de Sun Oct 9 23:39:01 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Sun Oct 9 23:39:01 2005 Subject: [Numpy-discussion] Can you enumerarte an array? In-Reply-To: <4349F00C.7080605@noaa.gov> References: <4349F00C.7080605@noaa.gov> Message-ID: Hi Chris, On Sun, 9 Oct 2005, Chris Barker wrote: > Hi all, > > A freind of mine that I jsut introduced to NumPy asked me about > enumerating an array. What he'd like to see is something like: > > A = N.ones((3,4)) > > for indexes, item in enumeate(A): > print indexes > > result in: > > (0,0) > (0,1) > (0,2) > (1,0) > . > . > . > You get the idea. Can you do this with any of the three NumPys? Not that I know: import Numeric as N A = N.ones((3,4)) for indizes, item in enumerate(A): print indizes,item gives 0 [1 1 1 1] 1 [1 1 1 1] 2 [1 1 1 1] (the same applies for numarrary and the new scipy_core) It might be the best to define your own iterator (e.g. `enumerate_ind`) because the present behaviour can be useful as well. > If not > consider this a feature request for scipy_base. Best, Arnd From cjw at sympatico.ca Mon Oct 10 04:23:56 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Mon Oct 10 04:23:56 2005 Subject: [Numpy-discussion] Access to online documeation - was Purchasing Documentation In-Reply-To: <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> Message-ID: <434A4EF1.90004@sympatico.ca> Matthew Brett wrote: >Hi, > > > >>While I fully respect the opinion of those concerned about Travis' decision, I >>have to say that I am personally not so worried. At scipy'05, when Travis >>announced the book idea, I asked a very simple question: 'are the docstrings >>complete?'. His answer was 'yes'. >> >>There was a good reason for my asking this question: I personally find that >>when coding, the most productive and efficient to learn a library is by typing >> >>import foo >>foo. >> >>and then >> >>foo.bar? >> >> > >Just a tiny addition. I know what you mean, but my experience is of >someone recently trying to learn numarray / Numeric, and I suspect I >am not alone in printing out the documentation and reading a moderate >amount of it before I get started. Call me old-fashioned (it >wouldn't be the first time!), > >See you, > >Matthew > > > > Could someone elaborate on the remark below please?: There was a good reason for my asking this question: I personally find that when coding, the most productive and efficient to learn a library is by typing import foo foo. and then foo.bar? Is this with some particular IDE? Using Windows, I don't get this. This sort of help is available with PythonWin but Numeric3 and pythonw seem to be mutually incompatible at this stage. Colin W. From cjw at sympatico.ca Mon Oct 10 04:53:30 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Mon Oct 10 04:53:30 2005 Subject: [Numpy-discussion] Dcoument strings for ndarray In-Reply-To: <4349B055.8060006@ee.byu.edu> References: <4347F755.2080304@sympatico.ca> <4348240D.6010308@ee.byu.edu> <43495948.9020402@sympatico.ca> <4349B055.8060006@ee.byu.edu> Message-ID: <434A55B7.5010900@sympatico.ca> Travis Oliphant wrote: > >>> I'm not sure what you are talking about here? Where are you getting >>> this? >>> >>> >>> I don't know of any such attribute. There is a __class__ >>> attribute. But I don't see a >>> >>> __class__ module(name[, doc]) attribute >>> >>> -Travis >>> >> >> Please see the sequence below: >> >> >>> import scipy.base.multiarray as M >> >>> M.__class__ >> >> >>> >> >> I am surprised that this name points to a module. It usually is an >> attribute of a class instance which >> points to the class object of that instance. > > > Why is this surprising? scipy.base.multiarray is a module, therefore > it's "class" is type 'module'. This seems fine to me. At any rate, > Python is assigning the __class__ attribute to the extension module > multiarray, so it is what it is. > > -Travis > Yes, this is the Python practice: >>> import timeit >>> timeit.__class__ >>> import anydbm >>> anydbm.__class__ >>> Yes, timeit.__class__ points to the internal module creator function: >>> import new >>> new.module >>> new.module.__doc__ 'module(name[, doc])\n\nCreate a module object.\nThe name must be a string; the optional doc argumen t can have any type.' >>> new.module is timeit.__class__ True >>> With a little digging, this is logical, consistent and no longer surprising. Colin W. From kosh at skynet.be Mon Oct 10 10:26:00 2005 From: kosh at skynet.be (Kosta Gaitanis) Date: Mon Oct 10 10:26:00 2005 Subject: [Numpy-discussion] Problems compiling ufuncs example In-Reply-To: <20051010170845.6C9A7FFCB@sc8-sf-spam2.sourceforge.net> References: <20051010170845.6C9A7FFCB@sc8-sf-spam2.sourceforge.net> Message-ID: <434AA3D2.80105@skynet.be> Hi all, I'm trying to run the UFunc example in the Numarray manual (http://stsdas.stsci.edu/numarray/numarray-1.1.html/node32.html) but when I run the command python setup.py buil_ext I get the following output : running build_ext error: Python was built with version 7.1 of Visual Studio, and extensions need t o be built with the same version of the compiler, but it isn't installed so I installed Visual C++ Toolkit 2003 and then run the same command again and got the following output : running build_ext error: the .NET Framework SDK needs to be installed before building extensions f or Python so I installed .NET Framework SDK 1.1 and then tried again. I got back the first error... does anyone have an idea ? What am I doing wrong ? thanks in advance Kosta I'm using Python24 and NumArray 1.3.3 -- setup.py -- """ This setup demonstrates how to use the numarray code generation facilities to define additional ufuncs at the user level. Universal functions apply operators or C-functions elementwise to all of the elements of an array or pair of arrays. This setup generates code defining three universal functions which are installed as the numarray.foo package: Cos, bessel, and bar. The ufuncs are used like this: >>> import numarray as na >>> import numarray.foo as foo >>> a = na.array([1,2,3,4], type='Int32') >>> foo.Cos(a) array([ 0.54030228, -0.41614684, -0.9899925 , -0.65364361 ], type=Float32) Note that since a is an Int32 array, Cos(a) first does blockwise conversion of a to Float32 before calling the cosine library function. """ import distutils, os from distutils.core import setup from numarray.numarrayext import NumarrayExtension from numarray.codegenerator import UfuncModule, make_stub # ====================================================================== # # Generate the C-code for the numarray.foo._foo extension: # m = UfuncModule("_foo") # Inline the code for a couple C functions to be turned into ufuncs. # This method lets you define your function here in the setup.py. An # alternate approach would be to link with a libarary or additional C # module. m.add_code(""" static double c_bessel(double x, double y) { double x2=x*x, y2=y*y, diff=x2-y2; return diff*diff/(x2+y2); } static UInt8 c_bar(UInt32 x ) { return (x >> 24) & 0xFF; } """) # Method add_ufunc() defines the name of a ufunc, it's corresponding C # function which is applied element-wise to all elements of an array, # The arity of the ufunc (unary or binary only), and the I/O type # signatures which are directly supported by the ufunc. Binary Ufunc # "bessel" is implemented by the inline function "c_bessel" from the # add_code() call above. m.add_ufunc("bessel", "c_bessel", 2, [('Float32', 'Float32'), ('Float64', 'Float64')]) # Unary Ufunc "bar" only supports one builtin type signature: # UInt32-->UInt8. Other types are blockwise converted by the ufunc # machinery to UInt32 before "c_bar" is called. m.add_ufunc("bar", "c_bar", 1, [('UInt32','UInt8')]) # Unary Ufunc "cos" is implemented by the C standard library function # "cos". Given a Float32 array, it returns a Float32 array. Given a # Float64 array, it returns a Float64 array. For other arrays, # transparent conversion to one of the supported input types is performed # by the ufunc machinery. m.add_ufunc("cos", "cos", 1, [('Float32', 'Float32'), ('Float64', 'Float64')]) # The generate() method writes out the complete extension module to the # specified C file. m.generate("foo/Src/_foo.c") # ====================================================================== # Create foo's __init__.py for defining UFuncs corresponding to CFuncs # in _foo. make_stub() emits boiler-plate which makes your extension # a package. The __init__ file imports all the public symbols from # C extension _foo making them visible as numarray.foo. make_stub("foo/Lib/__init__", "_foo") # ====================================================================== # # Standard distutils setup for the generated code. # setup(name = "foo", version = "0.1", maintainer = "Todd Miller", maintainer_email = "jmiller at stsci.edu", description = "foo universal functions for numarray", url = "http://www.stsci.edu/projects/foo/", packages = ["numarray.foo"], package_dir = { "numarray.foo":"foo/Lib" }, ext_modules = [ NumarrayExtension( 'numarray.foo._foo', ['foo/Src/_foo.c'], # libraries = ['ttf'], # include_dirs = ['include'], # library_dirs = ['lib'] ) ] ) From efiring at hawaii.edu Mon Oct 10 10:34:09 2005 From: efiring at hawaii.edu (Eric Firing) Date: Mon Oct 10 10:34:09 2005 Subject: [Numpy-discussion] Access to online documeation - was Purchasing Documentation In-Reply-To: <434A4EF1.90004@sympatico.ca> References: <43418AA8.2040809@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <434300FB.7070606@ee.byu.edu> <434307CC.7020303@colorado.edu> <43430B62.9090908@ee.byu.edu> <1e2af89e0510060529s5319192ayea270ebe2949b6b8@mail.gmail.com> <43454135.503@ee.byu.edu> <4345A449.8040303@colorado.edu> <1e2af89e0510091306w5b435c32j3e59cc453a35be30@mail.gmail.com> <434A4EF1.90004@sympatico.ca> Message-ID: <434AA585.4090900@hawaii.edu> Colin, The tab completion works on Linux from the standard python prompt and from ipython; the "?" syntax is added by ipython (http://ipython.scipy.org/). Eric > Could someone elaborate on the remark below please?: > > There was a good reason for my asking this question: I personally find > that > when coding, the most productive and efficient to learn a library is by > typing > > import foo > foo. > > and then > > foo.bar? > > > Is this with some particular IDE? Using Windows, I don't get this. From Chris.Barker at noaa.gov Mon Oct 10 11:37:55 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 10 11:37:55 2005 Subject: [Numpy-discussion] Can you enumerarte an array? In-Reply-To: <434A952F.7030909@ee.byu.edu> References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> Message-ID: <434AB4E0.9040607@noaa.gov> Travis Oliphant wrote: > All flavors of Numeric would return > > 0 a[0] > 1 a[1] > . > . > . > n-1 a[n-1] > > where n=a.shape[0] > > in response to index, item in enumerate(A): Right, as expected, and as Anrd pointed out, this is useful behaviour. > In scipy you could also get > > 0 a.flat[0] > 1 a.flat[1] I do like the new flat! > To do something like he wants, I think we would have to construct a > different iterator then enumerate. Exactly,. I didn't mean to imply that that's what the built-in enumerate should do. > It wouldn't be that hard using the > a.flat iterator (it's just a remapping of the 1-d index). Cool. then again, consider this a feature request, for a nd_enumerate, or whatever you want to call it. I'm sorry that implimenting something like this myself is way out my depth! -Chris From jochen at fhi-berlin.mpg.de Mon Oct 10 11:58:41 2005 From: jochen at fhi-berlin.mpg.de (=?iso-8859-1?Q?Jochen_K=FCpper?=) Date: Mon Oct 10 11:58:41 2005 Subject: NumPies future (was: [Numpy-discussion] Re: Purchasing Documentation) In-Reply-To: (Perry Greenfield's message of "Fri, 7 Oct 2005 08:51:41 -0400") References: Message-ID: All, following all this up and down I decide to write a short email for two reasons: First of all I want to thank all developers of Numeric, numarray, and the yet to fly scipy_core, esp. Perry/Todd and Travis. I hope that this apparent joining of forces will really get Numeric Python a big step ahead. On the other hand I have personally stepped down from using Python for larger numerical calculation projects, because it was too much of an hassle to get sysops convinced to install it, and because overall the hassles to follow releases, adopting code and explaining colleagues was to big to make up for the ease of coding compared to C++. Yep, there we are. I am REALLY looking forward to the day when I can start a larger project in Python again. I wish and hope that you guys are paving the road for me;) [1] Using scipy_core I would also be happy to work on public documentation, as that is where everybody using scipy_core can easily contribute his share. Maybe it would be a nice team effort to get the numarray Python-style documentation cut down and updated to a good and current reference-manual, pretty much a nicer and cross-linked version of the doc-strings, maybe with more of the examples that were discussed here before. I am sure that Travis writes a good user-manual, and, voila, both would be in the right place! To all the guys actually doing the work: Thank you very much, nevertheless! Greetings, Jochen -- Einigkeit und Recht und Freiheit http://www.Jochen-Kuepper.de Libert?, ?galit?, Fraternit? GnuPG key: CC1B0B4D (Part 3 you find in my messages before fall 2003.) From oliphant at ee.byu.edu Mon Oct 10 14:40:50 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Mon Oct 10 14:40:50 2005 Subject: [Numpy-discussion] Re: NumPies future In-Reply-To: References: Message-ID: <434ADF7A.6070300@ee.byu.edu> Jochen K?pper wrote: >All, > >following all this up and down I decide to write a short email for two >reasons: > >First of all I want to thank all developers of Numeric, numarray, and >the yet to fly scipy_core, esp. Perry/Todd and Travis. I hope that >this apparent joining of forces will really get Numeric Python a big >step ahead. > >On the other hand I have personally stepped down from using Python for >larger numerical calculation projects, because it was too much of an >hassle to get sysops convinced to install it, and because overall the >hassles to follow releases, adopting code and explaining colleagues >was to big to make up for the ease of coding compared to C++. Yep, >there we are. I am REALLY looking forward to the day when I can start >a larger project in Python again. I wish and hope that you guys are >paving the road for me;) [1] > >Using scipy_core I would also be happy to work on public >documentation, as that is where everybody using scipy_core can easily >contribute his share. Maybe it would be a nice team effort to get the >numarray Python-style documentation cut down and updated to a good and >current reference-manual, pretty much a nicer and cross-linked version >of the doc-strings, maybe with more of the examples that were >discussed here before. I am sure that Travis writes a good >user-manual, and, voila, both would be in the right place! > > I actually do think that converting the numarray manual will be easier. Because it is closer in function to what current scipy is. Advanced indexing has all been explained (the PEP for SciPy explains what is in current SciPy a bit better --- especially when it comes to mixing slices and integer indexing arrays. So, I would start from there. Hopefully in response to Jochen's other question (regarding hassles of install) we can get to the point where scipy core is installed by default on everybody's system.... :-) No need to give up C++ coding either with weave and pyrex (and f2py which can wrap C-code as well). -Travis From rowen at cesmail.net Mon Oct 10 16:18:20 2005 From: rowen at cesmail.net (Russell E. Owen) Date: Mon Oct 10 16:18:20 2005 Subject: [Numpy-discussion] Error involving import_libnumarray in packaged program Message-ID: If I convert my python code to an application (Windows via py2exe or Mac via bundlebuilder) it fails with the following error: Fatal Python error: Call to API function without first calling import_libnumarray() in Src/_convmodule.c I can force *all* of numarray into the application, which avoids the issue. But that is overkill. Is there some simpler way to solve this, e.g. some an import or command in my code that will prevent this from happening? -- Russell From Chris.Barker at noaa.gov Mon Oct 10 22:21:46 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Mon Oct 10 22:21:46 2005 Subject: [Numpy-discussion] Re: NumPies future In-Reply-To: <434ADF7A.6070300@ee.byu.edu> References: <434ADF7A.6070300@ee.byu.edu> Message-ID: <434B4BF2.7010505@noaa.gov> Travis Oliphant wrote: > No need to give up C++ coding either with weave and pyrex And SWIG and Boost::Python, which appears to have NumPy support...Has anyone used that, and can tell me about how well it works? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception From jochen at fhi-berlin.mpg.de Tue Oct 11 00:18:38 2005 From: jochen at fhi-berlin.mpg.de (=?iso-8859-1?Q?Jochen_K=FCpper?=) Date: Tue Oct 11 00:18:38 2005 Subject: [Numpy-discussion] Re: NumPies future In-Reply-To: <434ADF7A.6070300@ee.byu.edu> (Travis Oliphant's message of "Mon, 10 Oct 2005 15:39:06 -0600") References: <434ADF7A.6070300@ee.byu.edu> Message-ID: <9eoe5wbjyb.fsf@gowron.rz-berlin.mpg.de> Travis Oliphant writes: > I actually do think that converting the numarray manual will be easier. easier than? > Because it is closer in function to what current scipy is. closer than what? > No need to give up C++ coding either with weave and pyrex (and f2py > which can wrap C-code as well). I don't really want to program C++ -- I want a simple environment that's available on the typical number cruncher... Therefore, having scipy on every system would be a solution;) Greetings, Jochen -- Einigkeit und Recht und Freiheit http://www.Jochen-Kuepper.de Libert?, ?galit?, Fraternit? GnuPG key: CC1B0B4D (Part 3 you find in my messages before fall 2003.) From faltet at carabos.com Tue Oct 11 00:32:19 2005 From: faltet at carabos.com (Francesc Altet) Date: Tue Oct 11 00:32:19 2005 Subject: [Numpy-discussion] Re: NumPies future In-Reply-To: <434ADF7A.6070300@ee.byu.edu> References: <434ADF7A.6070300@ee.byu.edu> Message-ID: <200510110931.09415.faltet@carabos.com> A Dilluns 10 Octubre 2005 23:39, Travis Oliphant va escriure: > No need to give up C++ coding either with weave and pyrex (and f2py > which can wrap C-code as well). No, pyrex is not ready for C++ yet (at least natively, i.e. without C wrappers), but it would eventually be, with Greg permission. -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From kosh at skynet.be Tue Oct 11 05:08:38 2005 From: kosh at skynet.be (Kosta Gaitanis) Date: Tue Oct 11 05:08:38 2005 Subject: [Numpy-discussion] Problems compiling ufuncs example (2) In-Reply-To: <434AA3D2.80105@skynet.be> References: <20051010170845.6C9A7FFCB@sc8-sf-spam2.sourceforge.net> <434AA3D2.80105@skynet.be> Message-ID: <434BAACC.4010302@skynet.be> Kosta Gaitanis wrote: Hi again, I downloaded mingw32 and this seemed to fix the compilator problem. However I still can't build the ufunc example ! This time I tried : python setup.py build_ext --compiler=mingw32 and get the following output: running build_ext building 'numarray.foo._foo' extension c:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -Iinclude -Iinclude/numarray -Ic :\python24\Include umarray -IC:\Python24\include -IC:\Python24\PC -c foo/Src/_foo.c -o build\temp.w in32-2.4\Release\foo\src\_foo.o foo/Src/_foo.c:8: warning: ignoring #pragma warning foo/Src/_foo.c: In function `init_funcDict': foo/Src/_foo.c:26: warning: unused variable `keytuple' foo/Src/_foo.c:26: warning: unused variable `cfunc' writing build\temp.win32-2.4\Release\foo\src\_foo.def c:\mingw\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.4\Release\foo\src \_foo.o build\temp.win32-2.4\Release\foo\src\_foo.def -LLib -LC:\Python24\libs - LC:\Python24\PCBuild -lpython24 -lmsvcr71 -o build\lib.win32-2.4\numarray\foo\_f oo.pyd build\temp.win32-2.4\Release\foo\src\_foo.o(.text+0x87):_foo.c: undefined refere nce to `_imp__PyCObject_Type' build\temp.win32-2.4\Release\foo\src\_foo.o(.text+0xa6):_foo.c: undefined refere nce to `_imp__PyExc_ImportError' collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 If I remove the comments from the line libraries = ['ttf'] (at the end of setup.py) and try again I get this error : running build_ext building 'numarray.foo._foo' extension c:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -Iinclude -Iinclude/numarray -Ic :\python24\Include umarray -IC:\Python24\include -IC:\Python24\PC -c foo/Src/_foo.c -o build\temp.w in32-2.4\Release\foo\src\_foo.o foo/Src/_foo.c:8: warning: ignoring #pragma warning foo/Src/_foo.c: In function `init_funcDict': foo/Src/_foo.c:26: warning: unused variable `keytuple' foo/Src/_foo.c:26: warning: unused variable `cfunc' writing build\temp.win32-2.4\Release\foo\src\_foo.def c:\mingw\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.4\Release\foo\src \_foo.o build\temp.win32-2.4\Release\foo\src\_foo.def -LLib -LC:\Python24\libs - LC:\Python24\PCBuild -lttf -lpython24 -lmsvcr71 -o build\lib.win32-2.4\numarray\ foo\_foo.pyd c:\mingw\bin\..\lib\gcc\mingw32\3.4.2\..\..\..\..\mingw32\bin\ld.exe: cannot fin d -lttf collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 Can someone tell me what I'm doing wrong please. Thanks Kosta > Hi all, > > I'm trying to run the UFunc example in the Numarray manual > (http://stsdas.stsci.edu/numarray/numarray-1.1.html/node32.html) > but when I run the command > > python setup.py buil_ext > > > I get the following output : > > running build_ext > error: Python was built with version 7.1 of Visual Studio, and > extensions need t > o be built with the same version of the compiler, but it isn't > installed > > > so I installed Visual C++ Toolkit 2003 and then run the same command > again and got the following output : > > running build_ext > error: the .NET Framework SDK needs to be installed before building > extensions f > or Python > > > so I installed .NET Framework SDK 1.1 and then tried again. > I got back the first error... > > does anyone have an idea ? What am I doing wrong ? > > thanks in advance > Kosta > > I'm using Python24 and NumArray 1.3.3 > > > -- setup.py -- > """ This setup demonstrates how to use the numarray code generation > facilities to define additional ufuncs at the user level. Universal > functions apply operators or C-functions elementwise to all of the > elements of an array or pair of arrays. This setup generates code > defining three universal functions which are installed as the > numarray.foo package: Cos, bessel, and bar. > > The ufuncs are used like this: > > >>> import numarray as na > >>> import numarray.foo as foo > >>> a = na.array([1,2,3,4], type='Int32') > >>> foo.Cos(a) > array([ 0.54030228, -0.41614684, -0.9899925 , -0.65364361 ], > type=Float32) > > Note that since a is an Int32 array, Cos(a) first does blockwise > conversion of a to Float32 before calling the cosine library function. > """ > > import distutils, os > from distutils.core import setup > from numarray.numarrayext import NumarrayExtension > from numarray.codegenerator import UfuncModule, make_stub > > # ====================================================================== > # > # Generate the C-code for the numarray.foo._foo extension: > # > > m = UfuncModule("_foo") > > # Inline the code for a couple C functions to be turned into ufuncs. > # This method lets you define your function here in the setup.py. An > # alternate approach would be to link with a libarary or additional C > # module. > m.add_code(""" > > static double c_bessel(double x, double y) > { > double x2=x*x, y2=y*y, diff=x2-y2; > return diff*diff/(x2+y2); > } > > static UInt8 c_bar(UInt32 x ) > { > return (x >> 24) & 0xFF; > } > > """) > > # Method add_ufunc() defines the name of a ufunc, it's corresponding C > # function which is applied element-wise to all elements of an array, > # The arity of the ufunc (unary or binary only), and the I/O type > # signatures which are directly supported by the ufunc. Binary Ufunc > # "bessel" is implemented by the inline function "c_bessel" from the > # add_code() call above. > > m.add_ufunc("bessel", "c_bessel", 2, [('Float32', 'Float32'), > ('Float64', 'Float64')]) > > # Unary Ufunc "bar" only supports one builtin type signature: > # UInt32-->UInt8. Other types are blockwise converted by the ufunc > # machinery to UInt32 before "c_bar" is called. > > m.add_ufunc("bar", "c_bar", 1, [('UInt32','UInt8')]) > > # Unary Ufunc "cos" is implemented by the C standard library function > # "cos". Given a Float32 array, it returns a Float32 array. Given a > # Float64 array, it returns a Float64 array. For other arrays, > # transparent conversion to one of the supported input types is performed > # by the ufunc machinery. > > m.add_ufunc("cos", "cos", 1, [('Float32', 'Float32'), > ('Float64', 'Float64')]) > > # The generate() method writes out the complete extension module to the > # specified C file. > m.generate("foo/Src/_foo.c") > > # ====================================================================== > # Create foo's __init__.py for defining UFuncs corresponding to CFuncs > # in _foo. make_stub() emits boiler-plate which makes your extension > # a package. The __init__ file imports all the public symbols from > # C extension _foo making them visible as numarray.foo. > > make_stub("foo/Lib/__init__", "_foo") > # ====================================================================== > # > # Standard distutils setup for the generated code. > # > setup(name = "foo", > version = "0.1", > maintainer = "Todd Miller", > maintainer_email = "jmiller at stsci.edu", > description = "foo universal functions for numarray", > url = "http://www.stsci.edu/projects/foo/", > packages = ["numarray.foo"], > package_dir = { "numarray.foo":"foo/Lib" }, > ext_modules = [ NumarrayExtension( 'numarray.foo._foo', > ['foo/Src/_foo.c'], > # libraries = ['ttf'], > # include_dirs = ['include'], > # library_dirs = ['lib'] > ) > ] > ) > > From jh at oobleck.astro.cornell.edu Tue Oct 11 07:34:46 2005 From: jh at oobleck.astro.cornell.edu (Joe Harrington) Date: Tue Oct 11 07:34:46 2005 Subject: [Numpy-discussion] ASP status, docs, and supporting development Message-ID: <200510111433.j9BEXo0p013993@oobleck.astro.cornell.edu> After SciPy '04, Perry, Janet, and a few others and I put together a roadmap and framework, called ASP, for people to contribute to all the parts of SciPy other than the software: docs, packaging, website, etc. There was much interest expressed, and a few people even signed on the electronic dotted line to be involved: http://www.scipy.org/wikis/accessible_scipy/AccessibleSciPy The roadmap is laid out in this thread, which is linked in the first paragraph of the page above: http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2004-October/002412.html The first thing we did was to gather all the external projects using SciPy that we could find, and make an index page. The community found them and Fernando Perez did the hard work (more than he bargained for) of collating everything into the index page: http://www.scipy.org/wikis/topical_software/TopicalSoftware That is now a live wiki page, so if your project isn't on there, please add it! We were gearing up for effort #2, a web-site overhaul, early this year, when Travis announced his intention to heal the rift between the small- and large-array communities. We held up our push on web development, which was a few days from kickoff, so that it wouldn't take volunteers from his more-crucial effort. We all know the story of last year: Travis worked like crazy, called for volunteers, got only a few, and finished the job anyway. Now he's publishing a book in hopes of supporting some of his work from the revenues. He's made it clear that he will not be offended by or opposed to free docs, and may even contribute to them. He's also still Head Nummie and is leading the hard work of testing and bug swatting. Of course, we all want him to continue, and most of us freely admit our getting more than we are giving, and our gratitude to Travis, Todd, Robert, and the other core developers. Meanwhile, we have the problem of needing some basic docs that are free, and there seems to be quite a bit of interest in the community for doing one or more free docs. This seems to have more community energy behind it now than a web overhaul, so let's do it. The wiki and procedures are all set up to do some docs, and have been for about a year now. If you're interested in doing some docs, either as the lead author of a doc, as a contributor, or as a reviewer, please sign up on http://www.scipy.org/wikis/accessible_scipy/AccessibleSciPy The goal of the signups page is to make it easy for people to find each other: for lead authors to find people who will help them, for the community to identify who is taking part in what efforts, for low-level-of-effort volunteers to become hooked up with bigger projects, etc. There should be dozens of names there, not just three! So: If you're interested in LEADING A DOC, please add your name to the page, make a page for your doc on the wiki, and hang it off the main page, as "Scipy Cookbook" has done (there's a help link at the top of the page with instructions). A project can be anything from writing a collaborative book from scratch, to writing a monograph, to editing and revising existing docs. Announce your project on scipy-dev. If you would like to do a little work but not take the lead on something, you can contribute to the Cookbook or sign up to be a reviewer or contributor, either on an existing doc or at large. Or, contact a doc lead directly and sign up under that project. Please read the roadmap for ideas of docs we thought the community needs. The roadmap document is meant to be amended. For example, is the idea of using the docstrings to make a full-blown reference manual a good idea? I think so, since it's a rather self-updating format, but it will require some substantial work to get them all fleshed out and up to par. Discuss plans, changes, and ongoing efforts for docs on the scipy-dev mailing. It would be nice to have each project have a home on scipy.org, and Plone has excellent workflow-management tools. But, it's ok to home a project on your own site and just put a link on scipy.org. Finally, PLEASE everyone buy Travis's book if you can! Wait for the price to go up. Buy copies for everyone in your lab, if you can afford that. Buy one for your grandmother. It looks like it will be a really nice book, but it's more than that. This is a way to support development, which everyone desperately needs but few have the time (and fewer the skill and experience) to do. If you're at a company that benefits from SciPy and that hasn't already contributed resources to the effort, please consider parting with a larger chunk of change, either by buying more copies or by hiring Travis or others directly to do ongoing maintenance and package development. --jh-- From jmiller at stsci.edu Tue Oct 11 09:13:15 2005 From: jmiller at stsci.edu (Todd Miller) Date: Tue Oct 11 09:13:15 2005 Subject: [Numpy-discussion] Error involving import_libnumarray in packaged program In-Reply-To: References: Message-ID: <434BE46E.6090000@stsci.edu> Russell E. Owen wrote: >If I convert my python code to an application (Windows via py2exe or Mac >via bundlebuilder) it fails with the following error: > >Fatal Python error: Call to API function without first calling >import_libnumarray() in Src/_convmodule.c > > This currently (1.3.3) happens when a numarray API function is called before the API is successfully initialized. Although the message was intended as an aid to extension writers, in this case numarray is failing to import altogether. At one point numarray had a fatal error for import failures but I removed it at someone's request. I've restored it because I think it's most commonly fatal anyway and removing the message just obfuscated the problem. The non-fatal behavior is now in the _import_libnumarray() macro. >I can force *all* of numarray into the application, which avoids the > > This is what you need to do. It should be possible to factor out (or not explicitly list) numarray's Packages, but core numarray is not meant to be distributed in pieces. The many type-specific extensions were only added to work around a compiler problem, not to lighten binary distributions. >issue. But that is overkill. Is there some simpler way to solve this, > > Unfortunately, I think it's just necessary to (a) include all of core numarray and (b) help out automated tools (which choke on circular dependencies) by explicitly listing numarray's core extensions. Regards, Todd From rowen at cesmail.net Tue Oct 11 13:33:26 2005 From: rowen at cesmail.net (Russell E. Owen) Date: Tue Oct 11 13:33:26 2005 Subject: [Numpy-discussion] Re: Error involving import_libnumarray in packaged program References: <434BE46E.6090000@stsci.edu> Message-ID: In article <434BE46E.6090000 at stsci.edu>, Todd Miller wrote: > Russell E. Owen wrote: > > >If I convert my python code to an application (Windows via py2exe or Mac > >via bundlebuilder) it fails with the following error: > > > >Fatal Python error: Call to API function without first calling > >import_libnumarray() in Src/_convmodule.c >... > >I can force *all* of numarray into the application, which avoids the >... > Unfortunately, I think it's just necessary to (a) include all of core > numarray and (b) help out automated tools (which choke on circular > dependencies) by explicitly listing numarray's core extensions. Is there some practical way to include all of core numarray (without getting the unused extensions)? Could "import numarray" itself do the importing of core numarray? Then automatic packaging tools would "just work". -- Russell From meng at are.berkeley.edu Tue Oct 11 14:14:22 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Tue Oct 11 14:14:22 2005 Subject: [Numpy-discussion] question on differential results in windows and debian Message-ID: <5.1.1.5.2.20051011140513.0393b080@are.berkeley.edu> Hi there- I have a matrix X, calculate cX=matrixmultiply(transpose(X), X) and finally invert it by icX = linear_algebra.inverse(cX) cX is kind of ill-conditioned, which caused the icX differs a lot between windows and debian, after a tiny difference on cX across platforms. Here are the results: In windows: >>> x.min() -0.51187150837988826 >>> x.max() 0.81517690875232773 >>> cX = matrixmultiply(transpose(x), x) >>> cX.min() -666.96857541904126 >>> cX.max() 1073.3945530725441 >>> import numarray.linear_algebra as la >>> icX = la.inverse(cX) >>> icX.min() -3287030277.3580685 >>> icX.max() 0.0 In debian: >>> x.min() -0.51187150837988826 >>> x.max() 0.81517690875232773 >>> cX = matrixmultiply(transpose(x), x) >>> cX.min() -666.96857541898294 >>> cX.max() 1073.3945530726264 >>> import numarray.linear_algebra as la >>> icX = la.inverse(cX) >>> icX.min() 0.0 >>> icX.max() 84778028554.337051 I don't understand why a multiplication of a matrix and its transpose will create a tiny difference across windows and debian, which could be disastrous when inverting it. Please help me. Thanks a lot for your time! BTW, I am using Python2.4 and numarray 1.3.2 in both machines. Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng at are.berkeley.edu From arnd.baecker at web.de Wed Oct 12 06:06:45 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Wed Oct 12 06:06:45 2005 Subject: [Numpy-discussion] Can you enumerarte an array? In-Reply-To: <434AB4E0.9040607@noaa.gov> References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> Message-ID: Hi Chris, On Mon, 10 Oct 2005, Chris Barker wrote: > Travis Oliphant wrote: > > All flavors of Numeric would return > > > > 0 a[0] > > 1 a[1] > > . > > . > > . > > n-1 a[n-1] > > > > where n=a.shape[0] > > > > in response to index, item in enumerate(A): > > Right, as expected, and as Anrd pointed out, this is useful behaviour. > > > In scipy you could also get > > > > 0 a.flat[0] > > 1 a.flat[1] > > I do like the new flat! > > > To do something like he wants, I think we would have to construct a > > different iterator then enumerate. > > Exactly,. I didn't mean to imply that that's what the built-in enumerate > should do. > > > It wouldn't be that hard using the > > a.flat iterator (it's just a remapping of the 1-d index). > > Cool. then again, consider this a feature request, for a nd_enumerate, > or whatever you want to call it. > > I'm sorry that implimenting something like this myself is way out my depth! Below is one way to do it. BUT: - it is the first iterator I ever wrote... - it is not using yield, which might be better style - Oh and indeed it does give nicer code (see example 1). - it does not use a.flat. (BTW: I found http://heather.cs.ucdavis.edu/~matloff/Python/PyIterGen.pdf helpful) Best, Arnd 1) Short variant ---------------- from Numeric import * def arr_enumerate(arr): for indy in xrange(arr.shape[1]): for indx in xrange(arr.shape[0]): yield (indy,indx),arr[indy,indx] if __name__=="__main__": x=reshape(arange(25),(5,5)) print x for ind,value in arr_enumerate(x): print ind,value 2) Longer variant (just left in for historical reasons ...;-) ------------------------------------------------------------- from Numeric import * class arr_enumerate: def __init__(self,arr): self.Nx,self.Ny=arr.shape # just assume 2D here self.arr=arr self.indx=0 self.indy=0 def next(self): indx=self.indx indy=self.indy self.indx+=1 if self.indx==self.Nx: self.indx=0 self.indy+=1 if self.indy==self.Ny and self.indx==1: raise StopIteration return (indy,indx),self.arr[indy,indx] def __iter__(self): return self if __name__=="__main__": x=reshape(arange(25),(5,5)) print x for ind,value in arr_enumerate(x): print ind,value ###################################### From Chris.Barker at noaa.gov Wed Oct 12 08:52:24 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Wed Oct 12 08:52:24 2005 Subject: [Numpy-discussion] Can you enumerate an array? In-Reply-To: References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> Message-ID: <434D3106.8090305@noaa.gov> Arnd, You've written some nice compact code, but it only works for rank-2 arrays. Did you see Travis' solution? I think he's put it (or a C version, I'm not sure about that) in scipy_base. However, not using flat is nice, as in Numeric and numarray is sometimes makes a copy, and sometimes doesn't. -Chris Travis Oliphant wrote: > class ndenumerate(enumerate): > def __init__(self, arr): > self.iter = enumerate(arr.flat) > self.ashape = arr.shape > self.nd = arr.ndim > self.tot = arr.size > self.factors = [None]*(self.nd-1) > val = self.ashape[-1] > for i in range(self.nd-1,0,-1): > self.factors[i-1] = val > val *= self.ashape[i-1] > > def __len__(self): > return self.tot > > def next(self): > res = self.iter.next() > indxs = [None]*self.nd > val = res[0] > for i in range(self.nd-1): > indxs[i] = val / self.factors[i] > val = val % self.factors[i] > indxs[self.nd-1] = val > return tuple(indxs), res[1] -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From arnd.baecker at web.de Wed Oct 12 10:29:29 2005 From: arnd.baecker at web.de (Arnd Baecker) Date: Wed Oct 12 10:29:29 2005 Subject: [Numpy-discussion] Can you enumerate an array? In-Reply-To: <434D3106.8090305@noaa.gov> References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> <434D3106.8090305@noaa.gov> Message-ID: Hi Chris, On Wed, 12 Oct 2005, Chris Barker wrote: > Arnd, > > You've written some nice compact code, but it only works for rank-2 > arrays. Yup - I was aware of this limitation ;-) > Did you see Travis' solution? I think he's put it (or a C > version, I'm not sure about that) in scipy_base. Somehow I missed that - (his speed of coding is obviously way too fast for me to follow even in terms of the routines being added ;-). > However, not using flat > is nice, as in Numeric and numarray is sometimes makes a copy, and > sometimes doesn't. Whatever is most suitable for the task, at least I learnt about about iteraters ;-). Best, Arnd From oliphant at ee.byu.edu Wed Oct 12 12:10:44 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 12:10:44 2005 Subject: ***[Possible UCE]*** Re: [Numpy-discussion] Can you enumerate an array? In-Reply-To: <434D3106.8090305@noaa.gov> References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> <434D3106.8090305@noaa.gov> Message-ID: <434D5F66.1070901@ee.byu.edu> Chris Barker wrote: > Arnd, > > You've written some nice compact code, but it only works for rank-2 > arrays. Did you see Travis' solution? I think he's put it (or a C > version, I'm not sure about that) in scipy_base. However, not using > flat is nice, as in Numeric and numarray is sometimes makes a copy, > and sometimes doesn't. Yes, for Numeric and numarray a.flat does that. But, for scipy core, a.flat never makes a copy. It just returns an iterator object (that can be indexed and has an .__array__() method). So using a.flat is not a problem. Right now the solution I have is in Python. -Travis From oliphant at ee.byu.edu Wed Oct 12 12:12:21 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 12:12:21 2005 Subject: [Numpy-discussion] Can you enumerate an array? In-Reply-To: References: <4349F00C.7080605@noaa.gov> <434A952F.7030909@ee.byu.edu> <434AB4E0.9040607@noaa.gov> <434D3106.8090305@noaa.gov> Message-ID: <434D5FCE.9080202@ee.byu.edu> >Whatever is most suitable for the task, at least I learnt >about about iteraters ;-). > > A very worth pursuit. Iterators are a very nice abstraction. I used them extensively in writing the new scipy core. It made N-dimensional coding, broadcasting, and indexing much, much easier. -Travis From oliphant at ee.byu.edu Wed Oct 12 13:37:49 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 13:37:49 2005 Subject: [Numpy-discussion] Re: [SciPy-dev] matrix and ravel() In-Reply-To: References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> Message-ID: <434D73D9.5040802@ee.byu.edu> Alan G Isaac wrote: >>Travis Oliphant wrote: >> >> >>>Not really a side-effect. An intended feature. Matrices are supposed >>>to be matrices (i.e. 2-d arrays). They cannot be 1-d arrays no matter >>>how hard you try.... If you want to ravel a matrix and get a 1-d array >>>out, you are not using matrices right.... Use a 1-d array (i.e. the .A >>>attribute). >>> >>> >1. Better support for matrices is very welcome; I use them > a lot. >2. I expect ravel to return a 1-d array, not a matrix. My > background is a matrix programming language (GAUSS from > before its N-dimensional array support), and yet the > > > behavior of ravel never surprised me. > > I'm less concerned about this issue because it is so easy to get an array out of a matrix. >3. I expect a row and column vectorization functions (named > e.g., vecr and vecc) to do row and column vectorization. > I do not have a strong opinion on whether what is really > wanted is a vec function that takes an axis argument, > although this is probably right. In particular, both row > and column vectorization of matrices is common, and this > is just a choice of axis. > > These are m.T.ravel() and m.ravel() right now. To get a 1-d array it is m.A.ravel() >The same goes double for asarray: it will be a really >unwanted surprise if it does not return an array. > > I'm much more concerned about this issue. I would like to hear more people way in on what should be the behavior here. In particular, here are two proposals. 1) current behavior array (and asarray) --- returns the object itself if it is a sub-class of the array (or an array scalar). asndarray --- returns an actual nd array object 2) new behavior array (and asarray) always return an ndarray base-class object (or a big-nd array if the object is already one). asanyarray --- returns an nd-array or a sub-class if the object is already a sub-class. Please weigh in... --Travis >fwiw, >Alan Isaac > > >_______________________________________________ >Scipy-dev mailing list >Scipy-dev at scipy.net >http://www.scipy.net/mailman/listinfo/scipy-dev > > From tchur at optusnet.com.au Wed Oct 12 13:45:50 2005 From: tchur at optusnet.com.au (Tim Churches) Date: Wed Oct 12 13:45:50 2005 Subject: [Numpy-discussion] Re: Purchasing Documentation In-Reply-To: <43420C28.5010701@pfdubois.com> References: <43418AA8.2040809@hawaii.edu> <4341A410.5010504@optusnet.com.au> <4341B659.7070503@hawaii.edu> <4341C1C8.3000506@optusnet.com.au> <4341CB9E.4050306@ee.byu.edu> <4341D32A.5000000@optusnet.com.au> <4341F777.3030404@ee.byu.edu> <434204B8.4050308@optusnet.com.au> <43420C28.5010701@pfdubois.com> Message-ID: <4342107C.8070404@optusnet.com.au> Paul F. Dubois wrote: > The original sources were in Framemaker. I am not positive where they > are. In any case they are copyrighted by the Regents of the University > of California. I am not a lawyer and don't know what the consequences of > that are. LLNL granted free distribution of the printed document with > the Numeric source code but I don't know what their position would be on > using their copyrighted text in a new document or on giving away the > sources. OK, thanks Paul. That may have implications for you, Travis, if you are planning to base your SciPy Core book on the existing NumPy documentation. Given the circumstance which Paul describes, perhaps the only way forward is to create a freely available addendum to the freely available, existing NumPy documentation which describes how and where SciPy Core differs or extends NumPy? Or to write entirely new documentation from scratch. > I believe that under U.S. copyright law, a person may lend their copy of > a book to anyone but not reproduce it and give them a copy. Yes, likewise here in Oz, and in most countries. Of course, the publisher of a book can impose additional restrictions to which the purchaser of the book must agree, such as 'You may not show or lend your copy of this book to your colleagues, or to anyone else.' However, who in their right mind would buy a book which was sold with such restrictions attached to it? > Given the way open source capitalism works, I would not be surprised if > someone produces a 'quick reference guide' that they give away, or > writes a book on 'Using scipy.core in biology' that they sell for $88. > Let a thousand flowers bloom. Yes, I agree entirely. Travis, you are at perfect liberty to create commercial documentation for SciPy Core, but please don't object if others try to organise to create free open source documentation as well. Tim C > Tim Churches wrote: > >> Travis Oliphant wrote: >> >>> Tim Churches wrote: >>> >>> >>>> Travis Oliphant wrote: >>>> >>>> >>>> >>>>> Tim Churches wrote: >>>>> >>>>> >>>> >>>> >>>> Travis, are you saying that this agreement only allows a single person >>>> to read the single printed copy? If so, I think you need a formally >>>> worded legal license to make that stick - certainly Australian >>>> copyright >>>> law (nor copyright law in other countries, I suspect) does not provide >>>> any support for such severe restrictions in the use of a printed >>>> document. Under copyright law, you may not make unauthorised copies >>>> of a >>>> printed document, but you can certainly lend that copy to others, or >>>> sell or give it to them, or share it as a bench manual. >>>> >>>> >>> >>> I'm just stating what I think is fair. I'm not going to try and use the >>> power of the Australian state or any other to try and enforce it. >> >> >> >> My point was that you won't get any help from the State if what you >> think is fair extends to restricting who can read a single printed copy >> of your documentation. You will need to resort to tort law to enforce >> such restrictions, and thus you really need a formal documentation >> license, if that is your aim. >> >> >>>> Many of those downloads are for "t[y|i]re-kicking" - potential users >>>> determining whether it meets their needs. If those potential users have >>>> to pay for documentation up-front, then a large proportion will go >>>> elsewhere. The rest are probably existing NumPy users upgrading (or >>>> testing the upgrade waters). The latter group may well pay for >>>> documentation, but how large is that group? For example, how many >>>> people >>>> are subscribed to the numpy-discussion mailing list? >>>> >>>> >>> >>> Don't know. But, there is enough free documentation to get started, I >>> think. Lack of documentation has hampered more people than cost of >>> existing documentation, I think. >> >> >> >> I have to disagree on this point. Granted the existing documentation for >> NumPy is not fantastic - some aspects of it are positively obscure - but >> overall it is better than good-enough, we have found, and was not a >> barrier to our decision to use NumPy. Having to cough up $50 to even >> peek at the documentation would be a far, far greater barrier to uptake >> than the occasional obscure wording of some of the function >> descriptions, IMHO. >> >> >>>> Travis, although many of us are grateful for your efforts on SciPy >>>> Core, >>>> no-one made you do it. If you wanted to earn (more) money by doing >>>> other >>>> things, you should have done them. >>>> >>> >>> I'm really thinking about the future here. If I can succesfully raise >>> money to support work on this using a simple time-delay documentation >>> encouragement, then perhaps others will do the same, and we can all have >>> more. Waiting for people to "donate" their time to develop scientific >>> python is rather slow.... I've been freely donating time for a long >>> time, and there are few who help. So, we'll try a slightly different >>> route and see where that goes. >> >> >> >> Yes, fair enough. My point was that 7 years is far too long a time >> delay. Given that SciPy Core will probably take another year to become >> rock-solid, I would happily buy several copies of your documentation at >> $50 per copy if I knew that in a year the documentation could be freely >> distributed to all God's children. But in 7 years? Nope, too long to >> wait, not interested in supporting that. >> >> >>>> I think there needs to be some community debate about this. Is there >>>> sufficient interest for people other than Travis to start with the >>>> Numeric documentation and update it as necessary to become a free SciPy >>>> Core documentation? The NumPy documentation is available in HTML format >>>> as the basis of this - perhaps the original source (LaTeX?) for the >>>> Numeric docs is also available? >>> >>> >>> I don't know why you would want to undermine my efforts in this way by >>> duplicating effort. >> >> >> >> Travis, I don't want to undermine your efforts, but you must understand >> that one of the attractions of NumPy and thus Scipy Core is that they >> are free, open source software, which implies that there exists adequate >> free, open source documentation for them, or if such free, open source >> documentation does not exist, then taht users are collectively at >> liberty to create it. That's a basic FOSS tenet. What you are are >> proposing is the creation of proprietary documentation for SciPy Core. >> No-one objects to this, but only if it is supplementary to free, open >> source "core" documentation, not instead of it. >> >> >>> Perhaps, instead you could have people donate $$ >>> instead of time to releasing the documentation. I give away copies of >>> the documentation to people who participate in the development process >>> all the time (and that comes off the total price --- though I haven't >>> advertised this as of yet). So, why don't you encourage people who >>> don't have the money to contribute to the project instead. >> >> >> >> I understand that open source software and its documentation don't just >> appear out of thin air, and I fully support the idea of funded or >> collectively-commissioned open source development. But only if the >> results of that funded or commissioned development is, in fact, free and >> open sourced. By preventing free distribution of the documentation you >> are creating for 7 years, the end product cannot even remotely be >> described as open source, thus personally I am disinterested in funding >> it. If the delay until open sourcing was just one year, then that's a >> different matter, but you do not seem to be proposing that. >> >> Hence I think the NumPy/SciPy Core user community should establish a >> SciPy Core documentation project, with the aim of updating and enhancing >> the existing NumPy documentation so that it covers the new and changed >> features of SciPy Core, and making that documentation available on teh >> same free, open source basis as NumPy/SciPy Core itself. >> >> To that end, are the original LaTeX or whatever sources for the currenet >> NumPy documenattion available somewhere (apart from the HTML and PDF >> versions, that is). >> >> Tim C >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> > From meng at are.berkeley.edu Wed Oct 12 13:58:26 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Wed Oct 12 13:58:26 2005 Subject: [Numpy-discussion] question on differential results in windows and debian Message-ID: <5.1.1.5.2.20051012135240.03920290@are.berkeley.edu> >Hi there- > >I have a matrix X, calculate cX=matrixmultiply(transpose(X), X) and >finally invert it by icX = linear_algebra.inverse(cX) >cX is kind of ill-conditioned, which caused the icX differs a lot between >windows and debian, after a tiny difference on cX across platforms. I >don't understand why a multiplication of a matrix and its transpose will >create a tiny difference across windows and debian, which could be >disastrous when inverting it. Here are the results: >In windows: > >>> cX = matrixmultiply(transpose(x), x) > >>> cX.min() >-666.96857541904126 > >>> cX.max() >1073.3945530725441 > >>> import numarray.linear_algebra as la > >>> icX = la.inverse(cX) > >>> icX.min() >-3287030277.3580685 > >>> icX.max() >0.0 > >In debian: > >>> cX = matrixmultiply(transpose(x), x) > >>> cX.min() >-666.96857541898294 > >>> cX.max() >1073.3945530726264 > >>> import numarray.linear_algebra as la > >>> icX = la.inverse(cX) > >>> icX.min() >0.0 > >>> icX.max() >84778028554.337051 > >Thanks a lot for your time! BTW, I am using Python2.4 and numarray 1.3.2 >in both machines. > >Best, >Xiangyi > >Xiangyi Meng >Department of Agricultural and Resource Economics >Room 303, Giannini Hall #3310 >University of California, Berkeley >Berkeley, CA 94720-3310 >Tel: (510) 643-4124 >Fax: (510) 643-8911 >Email: meng at are.berkeley.edu > Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng at are.berkeley.edu From oliphant at ee.byu.edu Wed Oct 12 14:00:15 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 14:00:15 2005 Subject: [Numpy-discussion] New 0.4.2 beta release of scipy core Message-ID: <434D791B.9060809@ee.byu.edu> I made another beta release of scipy core last night. There are windows binaries for Python 2.4 and Python 2.3. If you are already a user of scipy, the new __init__ file installed for newcore will break your current installation of scipy (but the problem with linalg, fftpack, and stats is no longer there). There have been many improvements: - bug fixes (including 64-bit fixes) - threading support fixes - optimizations - more random numbers (thanks Robert Kern). - more distutils fixes (thanks Pearu Peterson). More tests are welcome. We are moving towards a release (but still need to get Masked Arrays working and all of scipy to build on top of the new scipy core before a full release). -Travis From stephen.walton at csun.edu Wed Oct 12 15:24:30 2005 From: stephen.walton at csun.edu (Stephen Walton) Date: Wed Oct 12 15:24:30 2005 Subject: [Numpy-discussion] Numeric precision Message-ID: <434AB6F6.1040500@csun.edu> Over on the scipy-dev mailing list, Travis Oliphant raised a question which is of interest to the folks over here as well. To summarize very briefly, Travis wondered whether the rule of "safest conversion" might require that an integer to float computation conversion on a 64-bit platform would require promotion of, for example, sqrt(2) to a long double (128 bits). I suggested that it would make the most sense for an M-bit integer to be converted to an N-bit real, where N>=M, on all platforms. For example, sqrt(2) would become a 32-bit real if 2 was a 32-bit integer on the plaform. I am not sure if this is a change from current Numeric and numarray practice, but wanted to give a heads-up over here. You can read the entire original thread beginning at http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2005-October/003403.html From Fernando.Perez at colorado.edu Wed Oct 12 15:37:26 2005 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Wed Oct 12 15:37:26 2005 Subject: [Numpy-discussion] Re: [SciPy-dev] New 0.4.2 beta release of scipy core In-Reply-To: <434D791B.9060809@ee.byu.edu> References: <434D791B.9060809@ee.byu.edu> Message-ID: <434D8FDC.2050306@colorado.edu> Travis Oliphant wrote: > I made another beta release of scipy core last night. There are > windows binaries for Python 2.4 and Python 2.3. If you are already a Sorry if I missed the blindingly obvious, but a URL for the tarballs (do they exist?) would be most appreciated at this point (I have a friend looking for it, and I'm embarrassed to say that I can't seem to find it). As a hook: this friend may test the build on Itanium boxes, so I'd like to respond to him soonish :) If it's just an SVN pull that's fine, just let me know and I'll pass the info along. Cheers, f From Chris.Barker at noaa.gov Wed Oct 12 18:15:58 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Wed Oct 12 18:15:58 2005 Subject: [Numpy-discussion] Re: [SciPy-dev] matrix and ravel() In-Reply-To: <434D73D9.5040802@ee.byu.edu> References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> <434D73D9.5040802@ee.byu.edu> Message-ID: <434DB553.9040303@noaa.gov> Travis Oliphant wrote: > In particular, here are two proposals. > 1) current behavior > array (and asarray) --- returns the object itself if it is a sub-class > of the array > (or an array scalar). > asndarray --- returns an actual nd array object -1. I expect as array to return an array. Period. That's why I use it. > 2) new behavior > > array (and asarray) always return an ndarray base-class object (or a > big-nd array if the object is already one). +1 ...I think. What is a big-nd array? > asanyarray --- returns an nd-array or a sub-class if the object is > already a sub-class. What about having each subclass define it's own function-- asmatrix, etc? Maybe that's not general enough, but I know I use asarray because I know I want a NumPy Array, and nothing else. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception From oliphant at ee.byu.edu Wed Oct 12 18:49:38 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Wed Oct 12 18:49:38 2005 Subject: [Numpy-discussion] Problem with name numeric In-Reply-To: <4346D825.70309@sympatico.ca> References: <4346D825.70309@sympatico.ca> Message-ID: <4346DADA.6050200@ee.byu.edu> Colin J. Williams wrote: > With the following sequence I had expected "scipy.base.numeric" to be > a module, instead it is reported as a type. Yes numeric is a type under scipy.base you just need scipy.ndarray The idea is not not import all the little submodules that make up scipy (that way we can change them around in the future should we need to). The only modules that should import numeric directly are those that are part of the scipy.base package itself. All you need to do is import scipy alone import scipy scipy.ndarray is what you want. -Travis From Chris.Barker at noaa.gov Wed Oct 12 21:47:11 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Wed Oct 12 21:47:11 2005 Subject: [Numpy-discussion] Re: asarray behaviour In-Reply-To: <434DB93C.9010303@ee.byu.edu> References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> <434D73D9.5040802@ee.byu.edu> <434DB553.9040303@noaa.gov> <434DB93C.9010303@ee.byu.edu> Message-ID: <434DE709.20208@noaa.gov> Travis Oliphant wrote: > Chris Barker wrote: >> Travis Oliphant wrote: >>> 2) new behavior >>> >>> array (and asarray) always return an ndarray base-class object (or a >>> big-nd array if the object is already one). >> >> +1 ...I think. What is a big-nd array? > > A big-nd arrary is the parent-class of the ndarray object. It doesn't > have the sequence protocols or the buffer protocols and should not > suffer from the 32-bit limitations of those protocols. An ndarray > inherits from the bigarray. Eventually, when Python cleans up its > 32-bit limitations, the bigarray will disappear. > >>> asanyarray --- returns an nd-array or a sub-class if the object is >>> already a sub-class. >> >> What about having each subclass define it's own function-- asmatrix, >> etc? Maybe that's not general enough, but I know I use asarray because >> I know I want a NumPy Array, and nothing else. > > I guess the problem is that we are not used to coding to interfaces. I'm not sure I get this. If we were coding to an interface, then I'd write my functions to jsut expect the type I want, and be done iowth it, I wouldn't need asarray(). > I'm going to make the change suggested by the second point, just because > I think it's more explicit and will make porting scipy a lot easier. um, I'm not sure I follow...what change are you making? > The fact that multiplication could be redefined by the matrix which > still passes as an array, means that lots of code can choke on matrices. Exactly. What I like about asarray is that I can write a function that really does require a NumPy array, and let the user code pass in anything that can be turned into one, without any performance penalty if a NumPy array is passed in. However, it's very important that I know EXACTLY what type asarray will return. The same thing applies to matrixes, etc. If I write a function that manipulates a matrix, Iwant to be darn sure that's what I've got...I'd need an asmatrix() function to give it to me. > Of course, this will have negative consequences. It will make matrices > much less pervasive through function calls "automatically", but it will > be safer. People who believe their code is safe for matrices, can use > asanyarray. I can see that there may well be sone code that could take any subclass of ndarray..perhaps if it's only accessing the data, for instance, so it's nice to have that option. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception From cjw at sympatico.ca Thu Oct 13 05:02:11 2005 From: cjw at sympatico.ca (Colin J. Williams) Date: Thu Oct 13 05:02:11 2005 Subject: [Numpy-discussion] Re: asarray behaviour In-Reply-To: <434DE709.20208@noaa.gov> References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> <434D73D9.5040802@ee.byu.edu> <434DB553.9040303@noaa.gov> <434DB93C.9010303@ee.byu.edu> <434DE709.20208@noaa.gov> Message-ID: <434E4C6C.3090705@sympatico.ca> Chris Barker wrote: > Travis Oliphant wrote: > >> Chris Barker wrote: >> >>> Travis Oliphant wrote: >>> >>>> 2) new behavior >>>> >>>> array (and asarray) always return an ndarray base-class object (or >>>> a big-nd array if the object is already one). >>> >>> >>> +1 ...I think. What is a big-nd array? >> >> >> A big-nd arrary is the parent-class of the ndarray object. It >> doesn't have the sequence protocols or the buffer protocols and >> should not suffer from the 32-bit limitations of those protocols. >> An ndarray inherits from the bigarray. Eventually, when Python cleans >> up its 32-bit limitations, the bigarray will disappear. >> >>>> asanyarray --- returns an nd-array or a sub-class if the object is >>>> already a sub-class. >>> >>> >>> What about having each subclass define it's own function-- asmatrix, >>> etc? Maybe that's not general enough, but I know I use asarray >>> because I know I want a NumPy Array, and nothing else. >> >> >> I guess the problem is that we are not used to coding to interfaces. > > > I'm not sure I get this. If we were coding to an interface, then I'd > write my functions to jsut expect the type I want, and be done iowth > it, I wouldn't need asarray(). > >> I'm going to make the change suggested by the second point, just >> because I think it's more explicit and will make porting scipy a lot >> easier. > > > um, I'm not sure I follow...what change are you making? > >> The fact that multiplication could be redefined by the matrix which >> still passes as an array, means that lots of code can choke on matrices. > > > Exactly. What I like about asarray is that I can write a function that > really does require a NumPy array, and let the user code pass in > anything that can be turned into one, without any performance penalty > if a NumPy array is passed in. However, it's very important that I > know EXACTLY what type asarray will return. The same thing applies to > matrixes, etc. If I write a function that manipulates a matrix, Iwant > to be darn sure that's what I've got...I'd need an asmatrix() function > to give it to me. > I'm puzzled by this. There is already a matrix class which should construct a matrix instance from a list, tuple or array or any subclass of array - I haven't tested all of these. >> Of course, this will have negative consequences. It will make >> matrices much less pervasive through function calls "automatically", >> but it will be safer. People who believe their code is safe for >> matrices, can use asanyarray. > > > I can see that there may well be sone code that could take any > subclass of ndarray..perhaps if it's only accessing the data, for > instance, so it's nice to have that option. > > -Chris > Colin W. From Chris.Barker at noaa.gov Thu Oct 13 09:58:00 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Thu Oct 13 09:58:00 2005 Subject: [Numpy-discussion] Re: asarray behaviour In-Reply-To: <434E4C6C.3090705@sympatico.ca> References: <434C594B.40501@ucsd.edu> <434C79D6.4000002@ee.byu.edu> <434C7EFE.3060308@ucsd.edu> <434D73D9.5040802@ee.byu.edu> <434DB553.9040303@noaa.gov> <434DB93C.9010303@ee.byu.edu> <434DE709.20208@noaa.gov> <434E4C6C.3090705@sympatico.ca> Message-ID: <434E91CA.5010208@noaa.gov> Colin J. Williams wrote: > I'm puzzled by this. There is already a matrix class which should > construct a matrix instance from a list, tuple or array or any subclass > of array - I haven't tested all of these. A) I was speaking generically. I haven't made use of any particular matrix class myself. B) I assume the standard matrix() constructor always creates a copy of the data, much like the NumPy array() constructor. In this case, the discussion was about the behavior of asarray(), which creates a new array if the argument is not an array, but returns the argument untouched if it already is an array. I was proposing that a matrix package (and any other subclass of ndarray) could benefit from a similar function. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From theller at python.net Thu Oct 13 10:44:46 2005 From: theller at python.net (Thomas Heller) Date: Thu Oct 13 10:44:46 2005 Subject: [Numpy-discussion] Re: Error involving import_libnumarray in packaged program References: <434BE46E.6090000@stsci.edu> Message-ID: <3bn51fzn.fsf@python.net> "Russell E. Owen" writes: > In article <434BE46E.6090000 at stsci.edu>, > Todd Miller wrote: > >> Russell E. Owen wrote: >> >> >If I convert my python code to an application (Windows via py2exe or Mac >> >via bundlebuilder) it fails with the following error: >> > >> >Fatal Python error: Call to API function without first calling >> >import_libnumarray() in Src/_convmodule.c >>... >> >I can force *all* of numarray into the application, which avoids the >>... >> Unfortunately, I think it's just necessary to (a) include all of core >> numarray and (b) help out automated tools (which choke on circular >> dependencies) by explicitly listing numarray's core extensions. > > Is there some practical way to include all of core numarray (without > getting the unused extensions)? > > Could "import numarray" itself do the importing of core numarray? Then > automatic packaging tools would "just work". The easiest way to build some 'hints' for those packaging tools that use modulefinder is to create a silly function that imports everything belonging to the package: def _give_py2exe_hints(): import this import that ... Note that it will NOT work to write this: if 0: import this import that ... because 'if 0' blocks are optimized away completely by the Python compiler. This should be in numarray's __init__.py file, probably. Thomas From bsouthey at gmail.com Fri Oct 14 12:14:53 2005 From: bsouthey at gmail.com (Bruce Southey) Date: Fri Oct 14 12:14:53 2005 Subject: [Numpy-discussion] Numeric precision In-Reply-To: <434AB6F6.1040500@csun.edu> References: <434AB6F6.1040500@csun.edu> Message-ID: Hi, The unaddressed problem is how the user will see the result. This type of thing leads type mismatches especially when doing array indexing. In that case it is a really bad thing to do. But in terms of mathematical computations it is usually a given that the highest possible precision is used when possible. Regards Bruce On 10/10/05, Stephen Walton wrote: > > Over on the scipy-dev mailing list, Travis Oliphant raised a question > which is of interest to the folks over here as well. To summarize very > briefly, Travis wondered whether the rule of "safest conversion" might > require that an integer to float computation conversion on a 64-bit > platform would require promotion of, for example, sqrt(2) to a long > double (128 bits). I suggested that it would make the most sense for an > M-bit integer to be converted to an N-bit real, where N>=M, on all > platforms. For example, sqrt(2) would become a 32-bit real if 2 was a > 32-bit integer on the plaform. > > I am not sure if this is a change from current Numeric and numarray > practice, but wanted to give a heads-up over here. You can read the > entire original thread beginning at > > > http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2005-October/003403.html > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From meng at are.berkeley.edu Mon Oct 17 14:12:23 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Mon Oct 17 14:12:23 2005 Subject: [Numpy-discussion] Question: why is this different? Message-ID: <5.1.1.5.2.20051017140354.03a06090@are.berkeley.edu> Hi there- I have a question on how this difference arises when I run the following simple code on windows and Debian: >>> from numarray import * >>> a=reshape(arange(81.0), (9,9))+identity(9) >>> ca = matrixmultiply(transpose(a),a) >>> import numarray.linear_algebra as la >>> ica = la.inverse(ca) >>> ica.min() -0.30951621414374736 >>> ica.max() 0.8888918586585135 The above is the result in Debian and below is that in Windows: >>> ica.min() -0.30951621414449404 >>> ica.max() 0.88889185865875686 So, what caused this difference (both machines have python2.4 and numarray 1.3.2)? The difference happens in the 10th/11th decimal point, which should be still in the floating point precision, right? This simple difference caused big differences in my later calculation. Best, Xiangyi From meng at are.berkeley.edu Tue Oct 18 13:28:40 2005 From: meng at are.berkeley.edu (meng at are.berkeley.edu) Date: Tue Oct 18 13:28:40 2005 Subject: [Numpy-discussion] Question: why is this different? In-Reply-To: <4354215C.9080806@stsci.edu> References: <5.1.1.5.2.20051017140354.03a06090@are.berkeley.edu> <5.1.1.5.2.20051017140354.03a06090@are.berkeley.edu> Message-ID: <5.1.1.5.2.20051018132505.038c99f0@are.berkeley.edu> Todd and Paul- Thank you so much for your help! Any suggestions on my next step? This difference doesn't make my result qualitatively different but my final result differs in the 3rd decimal point across these two machines, which could be troublesome when you want something that's robust to these computing errors. Best, Xiangyi At 18:10 2005-10-17 -0400, Todd Miller wrote: >meng at are.berkeley.edu wrote: > >>Hi there- >> >>I have a question on how this difference arises when I run the following >>simple code on windows and Debian: >> >>> from numarray import * >> >>> a=reshape(arange(81.0), (9,9))+identity(9) >> >>> ca = matrixmultiply(transpose(a),a) >> >>> import numarray.linear_algebra as la >> >>> ica = la.inverse(ca) >> >>> ica.min() >>-0.30951621414374736 >> >>> ica.max() >>0.8888918586585135 >> >>The above is the result in Debian and below is that in Windows: >> >>> ica.min() >>-0.30951621414449404 >> >>> ica.max() >>0.88889185865875686 > >I'm able to reproduce your "Windows results" in Fedora Core 4 x86_64... >exactly. That says that a variant of gcc and VC.NET are producing >identical results to 18 digits... which is encouraging. Numeric agrees as >well. > >When I build numarray with LAPACK and ATLAS rather than the builit-in >lapack_lite on RHEL3 i386 I get different results from any of the above: > >ica.min: -0.30951621414375136 >ica.max: 0.88889185865854037 > >They are closest to Debian I think, differing in the 15th decimal >place. My suspicion is that your Debian version is being built against >different numerical libraries... but I'm not privy to exactly how. A key >point is that numarray on Windows is *not* built using LAPACK and ATLAS. > >>So, what caused this difference (both machines have python2.4 and >>numarray 1.3.2)? The difference happens in the 10th/11th decimal point, >>which should be still in the floating point precision, right? > >I would expect so. > Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng at are.berkeley.edu From timb at tbitc.com Wed Oct 19 22:43:11 2005 From: timb at tbitc.com (Tim Burgess) Date: Wed Oct 19 22:43:11 2005 Subject: [Numpy-discussion] numarray - was trying to freeze wxmpl matplotlib and wxpython Message-ID: <43572E26.4050902@tbitc.com> Hi all, First thanks to all those who have been helping out - it is really appreciated. I have cross posted to the numarray, py2exe and matplotlib lists I have attempted to use numarray from cvs but I can't build it - no VC7 clang - did a penny drop anywhere? I am working with python 2.4.1 and numarray 1.3.3 for 2.4 (obviously) Since I couldnt build numarray - I looked at the new import code in the cvs init module and ripped it off and stuffed it into my numarray installation and then when that didnt work - I jammed it right up front of my application. It looks like this def main(): import numarray.numarrayall from numarray.numinclude import version as __version__ # stolen from next numarray version in cvs TjB import numarray._conv import numarray._sort import numarray._bytes import numarray._ufunc import numarray._ufuncBool import numarray._ufuncInt8 import numarray._ufuncUInt8 import numarray._ufuncInt16 import numarray._ufuncUInt16 import numarray._ufuncInt32 import numarray._ufuncUInt32 import numarray._ufuncFloat32 import numarray._ufuncFloat64 import numarray._ufuncComplex32 import numarray._ufuncComplex64 import numarray._ndarray import numarray._numarray import numarray._chararray import numarray._objectarray import numarray.memory import numarray._converter import numarray._operator import numarray._numerictype import numarray.libnumarray import numarray.libnumeric import numarray._ufuncInt64 import numarray._ufuncUInt64 print numarray.__version__ application = BoaApp(0) Still no go - but a changed error message.... grrr I get Traceback (most recent call last): File "AEMdaApp.py", line 81, in ? File "AEMdaApp.py", line 41, in main File "numarray\__init__.pyc", line 42, in ? File "numarray\numarrayall.pyc", line 1, in ? File "numarray\numerictypes.pyc", line 35, in ? File "numarray\numinclude.pyc", line 4, in ? File "numarray\_ndarray.pyc", line 9, in ? File "numarray\_ndarray.pyc", line 7, in __load ImportError: init_ndarray: can't find memory.new_memory ok I confess my bottom lip trembled.... then - probably because I was reading the numarray manual this morning - I went to my numarray install and decided to run testall.py I got F:\Python24\Lib\site-packages\numarray>\python24\python testall.py Testing numarray 1.3.3 on normal Python (2, 4, 1, 'final', 0) on platform win32 ********************************************************************** File "F:\python24\lib\site-packages\numarray\numtest.py", line 2843, in cache p ss Failed example: cPickle.loads(cPickle.dumps(arange(5)+1j)) Exception raised: Traceback (most recent call last): File "F:\Python24\lib\doctest.py", line 1243, in __run compileflags, 1) in test.globs File "", line 1, in ? PicklingError: Can't pickle : it's no the same object as memory.memory_from_string ********************************************************************** File "F:\python24\lib\site-packages\numarray\numtest.py", line 2849, in cache p ss Failed example: p = p.dump(a) Exception raised: Traceback (most recent call last): File "F:\Python24\lib\doctest.py", line 1243, in __run compileflags, 1) in test.globs File "", line 1, in ? PicklingError: Can't pickle : it's no the same object as memory.memory_from_string ********************************************************************** File "F:\python24\lib\site-packages\numarray\numtest.py", line 2850, in cache p ss Failed example: p = p.dump(b) Exception raised: and so on for quite some time. Hmm the message about memory.memory_from_string and my applications ImportError: init_ndarray: can't find memory.new_memory are these a little related? Am I clutching at straws? Yet my app runs fine under any ide I wish to use. WAIT! - my install of numarray is broken.... I deleted it and reinstalled - still broken I installed on my laptop and tested - broken there too My legacy python 2.3 and numarray 1.3.3 installation passes all the tests just fine. Is there a problem with python2.4.1 and numarray 1.3.3? Could some kind soul build me a numarray windows installer for python 2.4 from cvs - please - so that I can see if that works better with py2exe. I need a stiff drink -- Tim Burgess IT Consultant RedHat Certified Engineer TBITC Pty Ltd Professional Computer Support for Business timb at tbitc.com Mobile 0422 942 972 Office 85 662 016 http://www.tbitc.com From Chris.Barker at noaa.gov Thu Oct 20 00:04:29 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Thu Oct 20 00:04:29 2005 Subject: [Numpy-discussion] Re: [Matplotlib-users] numarray - was trying to freeze wxmpl matplotlib and wxpython In-Reply-To: <43572E26.4050902@tbitc.com> References: <43572E26.4050902@tbitc.com> Message-ID: <43574114.30903@noaa.gov> Tim, Sorry I can't help much, but: > I have attempted to use numarray from cvs but I can't build it - no VC7 You can build Python extensions with MinGW. You have to do a little patching to Python, but it's not too hard, and it does work. Here's one link. Googling will get you others: http://www.mingw.org/MinGWiki/index.php/Python extensions -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From jmiller at stsci.edu Thu Oct 20 09:35:36 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu Oct 20 09:35:36 2005 Subject: [Numpy-discussion] Re: [Matplotlib-users] numarray - was trying to freeze wxmpl matplotlib and wxpython In-Reply-To: <43572E26.4050902@tbitc.com> References: <43572E26.4050902@tbitc.com> Message-ID: <4357C6F5.4080500@stsci.edu> Tim Burgess wrote: > Hi all, > > First thanks to all those who have been helping out - it is really > appreciated. > > Since I couldnt build numarray - I looked at the new import code in > the cvs init module and ripped it off and stuffed it into my numarray > installation and then when that didnt work - I jammed it right up > front of my application. > > It looks like this > > def main(): > import numarray.numarrayall > from numarray.numinclude import version as __version__ > > # stolen from next numarray version in cvs TjB > > import numarray._conv > import numarray._sort > import numarray._bytes > import numarray._ufunc > import numarray._ufuncBool > import numarray._ufuncInt8 > import numarray._ufuncUInt8 > import numarray._ufuncInt16 > import numarray._ufuncUInt16 > import numarray._ufuncInt32 > import numarray._ufuncUInt32 > import numarray._ufuncFloat32 > import numarray._ufuncFloat64 > import numarray._ufuncComplex32 > import numarray._ufuncComplex64 > import numarray._ndarray > import numarray._numarray > import numarray._chararray > import numarray._objectarray > import numarray.memory > import numarray._converter > import numarray._operator > import numarray._numerictype > import numarray.libnumarray > import numarray.libnumeric > import numarray._ufuncInt64 > import numarray._ufuncUInt64 > > print numarray.__version__ > > application = BoaApp(0) > > Still no go - but a changed error message.... grrr Well, that's progress anyway. > > I get > > Traceback (most recent call last): > File "AEMdaApp.py", line 81, in ? > File "AEMdaApp.py", line 41, in main > File "numarray\__init__.pyc", line 42, in ? > File "numarray\numarrayall.pyc", line 1, in ? > File "numarray\numerictypes.pyc", line 35, in ? > File "numarray\numinclude.pyc", line 4, in ? > File "numarray\_ndarray.pyc", line 9, in ? > File "numarray\_ndarray.pyc", line 7, in __load > ImportError: init_ndarray: can't find memory.new_memory Is numarray.memory actually present in your install directory? Can you CD to that directory and import memory by itself? > then - probably because I was reading the numarray manual this morning > - I went to my numarray install and decided to run testall.py > > I got There's two things going on here. (1) This manner of running the numarray self-tests isn't recommended since it doesn't work anywhere. Read Doc/INSTALL.txt for how to run them. (2) There's bit-rot in numarray CVS for windows. I'm working on the rot now. > F:\Python24\Lib\site-packages\numarray>\python24\python testall.py > Testing numarray 1.3.3 on normal Python (2, 4, 1, 'final', 0) on > platform win32 > ********************************************************************** > File "F:\python24\lib\site-packages\numarray\numtest.py", line 2843, > in cache p > ss > Failed example: > cPickle.loads(cPickle.dumps(arange(5)+1j)) > Exception raised: > Traceback (most recent call last): > File "F:\Python24\lib\doctest.py", line 1243, in __run > compileflags, 1) in test.globs > File "", line 1, in ? > PicklingError: Can't pickle memory_from_string>: it's no > the same object as memory.memory_from_string > ********************************************************************** > File "F:\python24\lib\site-packages\numarray\numtest.py", line 2849, > in cache p > ss > Failed example: > p = p.dump(a) > Exception raised: > Traceback (most recent call last): > File "F:\Python24\lib\doctest.py", line 1243, in __run > compileflags, 1) in test.globs > File "", line 1, in ? > PicklingError: Can't pickle memory_from_string>: it's no > the same object as memory.memory_from_string > ********************************************************************** > File "F:\python24\lib\site-packages\numarray\numtest.py", line 2850, > in cache p > ss > Failed example: > p = p.dump(b) > Exception raised: > > and so on for quite some time. > > Hmm the message about > memory.memory_from_string > and my applications > ImportError: init_ndarray: can't find memory.new_memory > > are these a little related? Am I clutching at straws? > > Yet my app runs fine under any ide I wish to use. > > WAIT! - my install of numarray is broken.... > I deleted it and reinstalled - still broken I'm a little confused here. How is numarray installed where your application actually works? How is numarray installed where numarray does not work? (I was under the impression that you're using a tool to build a self-contained installer... which isn't working. I figured your development systems have numarray installed independently... which is working. What are you doing?) > I installed on my laptop and tested - broken there too > > My legacy python 2.3 and numarray 1.3.3 installation passes all the > tests just fine. > > Is there a problem with python2.4.1 and numarray 1.3.3? Nope. > Could some kind soul build me a numarray windows installer for python > 2.4 from cvs - please - so that I can see if that works better with > py2exe. Let me know if you still want this and I'll make one for you. FYI, MinGW works great for matplotlib but you may have to row-your-own-boat when it comes to numarray. As far as I know, there are missing glibc dependencies for numarray, particularly IEEE exception handling. Regards, Todd From oliphant at ee.byu.edu Thu Oct 20 10:40:26 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Thu Oct 20 10:40:26 2005 Subject: [Numpy-discussion] Re: [Matplotlib-users] numarray - was trying to freeze wxmpl matplotlib and wxpython In-Reply-To: <4357C6F5.4080500@stsci.edu> References: <43572E26.4050902@tbitc.com> <4357C6F5.4080500@stsci.edu> Message-ID: <4357D658.8080506@ee.byu.edu> > Let me know if you still want this and I'll make one for you. > > FYI, MinGW works great for matplotlib but you may have to > row-your-own-boat when it comes to numarray. As far as I know, there > are missing glibc dependencies for numarray, particularly IEEE > exception handling. Actually, mingw handles IEEE exceptions fine (using the LINUX style). You just have to include the right directories (it took me a while to track that one down). Look in new scipy core ufuncobject.h for the test for MINGW. -Travis From jmiller at stsci.edu Thu Oct 20 11:38:18 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu Oct 20 11:38:18 2005 Subject: [Numpy-discussion] Re: [Matplotlib-users] numarray - was trying to freeze wxmpl matplotlib and wxpython In-Reply-To: <4357D658.8080506@ee.byu.edu> References: <43572E26.4050902@tbitc.com> <4357C6F5.4080500@stsci.edu> <4357D658.8080506@ee.byu.edu> Message-ID: <4357E38E.7000901@stsci.edu> Travis Oliphant wrote: > >> Let me know if you still want this and I'll make one for you. >> >> FYI, MinGW works great for matplotlib but you may have to >> row-your-own-boat when it comes to numarray. As far as I know, >> there are missing glibc dependencies for numarray, particularly IEEE >> exception handling. > > > Actually, mingw handles IEEE exceptions fine (using the LINUX style). > You just have to include the right directories (it took me a while to > track that one down). Look in new scipy core ufuncobject.h for the > test for MINGW. > > -Travis Thanks for the information Travis. I tried this and can get it to compile, link, and load OK, but the IEEE exception handling self-tests fail, multiplication overflow detection in particular. Todd From timb at tbitc.com Thu Oct 20 16:52:49 2005 From: timb at tbitc.com (Tim Burgess) Date: Thu Oct 20 16:52:49 2005 Subject: [Numpy-discussion] Numarray - doesnt pass installation test under Windblows (was freezing wxmpl etc) Message-ID: <43582D73.3050809@tbitc.com> Hi Todd, Yesterday I claimed that numarray wasnt passing the tests under python 2.4. Actually I was testing the 1.3.3 Windows exe installer version - not my cvs derrived version. *blush* The cvs version tests perfectly. I will take you up on your offer of a Windows exe from CVS please Todd. Perhaps it means it is time for a 1.3.3.1 version? :-} -- Tim Burgess IT Consultant RedHat Certified Engineer TBITC Pty Ltd Professional Computer Support for Business timb at tbitc.com Mobile 0422 942 972 Office 85 662 016 http://www.tbitc.com From jmiller at stsci.edu Fri Oct 21 16:43:16 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 21 16:43:16 2005 Subject: [Numpy-discussion] Numarray - doesnt pass installation test under Windblows (was freezing wxmpl etc) In-Reply-To: <43582D73.3050809@tbitc.com> References: <43582D73.3050809@tbitc.com> Message-ID: <43597CA8.4060505@stsci.edu> Tim Burgess wrote: > Hi Todd, > > Yesterday I claimed that numarray wasnt passing the tests under python > 2.4. > > Actually I was testing the 1.3.3 Windows exe installer version - not > my cvs derrived version. *blush* > > The cvs version tests perfectly. > > I will take you up on your offer of a Windows exe from CVS please Todd. > > Perhaps it means it is time for a 1.3.3.1 version? :-} > > Hi Tim, The good news is I tried to create this today. The bad news is that I failed to get a distribution that passes all the self-tests. It's late and Monday is booked so it's looking like COB Tuesday is the earliest a release will come from me. If my understanding of the way py2exe works is correct (sucks up installed code for inclusion in distro), you might consider rolling your own 1.3.3.1 by starting with a 1.3.3 installation, adding the __init__.py from the head of CVS, and updating generate.py with the new version numbers. Sorry this is dragging out but it *will* be resolved in the next week, possibly by 1.4.0. Todd From vanzwieten at science-and-technology.nl Mon Oct 24 00:52:27 2005 From: vanzwieten at science-and-technology.nl (Joris van Zwieten) Date: Mon Oct 24 00:52:27 2005 Subject: [Numpy-discussion] numarray.objects Message-ID: <49857.217.77.152.33.1130140184.squirrel@control.prolocation.net> Hi, Are arrays of objects (i.e. numarray.objects) still supported in Numeric3/Scipy? (I'm working on an application that should be able to work with arrays of numbers and arrays of objects. Without numarray.objects-like functionality, this would require using numarray arrays for the numbers, and nested python lists for the objects. ;) Thanks, Joris van Zwieten From oliphant at ee.byu.edu Tue Oct 25 11:45:26 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Oct 25 11:45:26 2005 Subject: [Numpy-discussion] Re: Numeric: Change of v24.0 In-Reply-To: <6B3FDE9C-E25C-402D-9C27-F17B7F412753@opendarwin.org> References: <6B3FDE9C-E25C-402D-9C27-F17B7F412753@opendarwin.org> Message-ID: <435E7CC7.1050103@ee.byu.edu> Weissmann Markus wrote: > Hi folks, > > Before beginning my rant: Thank you very much for Numeric - you're > doing a great job! > > I'm a bit pissed as you changed the - already released tarball of > version 24.0; > This is a real mess, as all build systems out there have problems > distinguishing versions, > the correct tarballs (hashes will fail) etc. I'm not sure what you are talking about. The old tarball was always a beta release. It took a long time to get the actual release out, but that is due to the lack of reports. > I don't think this is my personal, or my projects impression, as e. > g. Novell [1] & O'Reilly [2] > clearly were under the impression that the old tarball was v24.0. > Then they were mistaken. It was always clearly marked as version 24b2 (beta-release 2). > Whatever happened that lead to this confusion, it would be great if > you could at least > make a version bump to 24.1 right now so everyone can be sure that > 24.1 is 24.1 in contrast > to 24.0 not being 24.0 all the time. > Someone could make a 24.1, but I don't see the need, and I'm not really interested in doing that. I'm done with Numeric development as I've moved to scipy core. I'm sorry for your consternation. -Travis From jmiller at stsci.edu Wed Oct 26 08:23:40 2005 From: jmiller at stsci.edu (Todd Miller) Date: Wed Oct 26 08:23:40 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.0 Message-ID: <435F9F2F.2000005@stsci.edu> Numarray is an array processing package designed to efficiently manipulate large multi-dimensional arrays. Numarray is modeled after Numeric and features c-code generated from python template scripts, the capacity to operate directly on arrays in files, arrays of heterogeneous records, string arrays, and in-place operation on memory mapped files. ========================================================================= Release Notes for numarray-1.4.0 I. ENHANCEMENTS 1. Speed improvement for numarray operators. The Python level hook mapping numarray operators onto universal functions has been moved down to C. 2. Speed improvement for string-array comparisons, any(), all(). String correlation is ~10x faster. 3. Better operation with py2exe to help it automatically detect the core numarray extensions to include in an installer. 4. scipy newcore compatible lower case type names (e.g. int32 not Int32) 5. scipy newcore 'dtype' keyword and .dtypechar attribute. II. BUGS FIXED / CLOSED 1323355 Apps fail with import_libnumarray 1315212 Infinite loop converting some scalar strings into a list 1298916 rank-0 tostring() broken 1297948 records.array fails to create empty fields 1286291 import sys missing from array_persist.py 1286168 Generic sequences in ``strings.array()`` 1236392 Outdated web link in announcements 1235219 LinearAlgebraError not imported in linear_algebra See http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse for more details. III. CAUTIONS This release should be backward binary compatible with numarray-1.3.x WHERE ----------- Numarray-1.4.0 windows executable installers, source code, and manual is here: http://sourceforge.net/project/showfiles.php?group_id=1369 Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://www.stsci.edu/resources/software_hardware/numarray Trackers for Numarray Bugs, Feature Requests, Support, and Patches are at the Source Forge project for NumPy at: http://sourceforge.net/tracker/?group_id=1369 REQUIREMENTS ------------------------------ numarray-1.4.0 requires Python 2.2.2 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science Institute. We'd like to acknowledge the assitance of Francesc Alted, Paul Dubois, Sebastian Haase, Chuck Harris, Tim Hochberg, Nadav Horesh, Edward C. Jones, Eric Jones, Jochen Kuepper, Travis Oliphant, Pearu Peterson, Peter Verveer, Colin Williams, Rory Yorke, and everyone else who has contributed with comments and feedback. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. -- Todd Miller jmiller at stsci.edu From chaves at computer.org Wed Oct 26 18:57:32 2005 From: chaves at computer.org (John A Chaves) Date: Wed Oct 26 18:57:32 2005 Subject: [Numpy-discussion] numarray-1.4.0 string-array comparisons problem In-Reply-To: <435F9F2F.2000005@stsci.edu> References: <435F9F2F.2000005@stsci.edu> Message-ID: <200510262055.39204.chaves@computer.org> After installing numarray-1.4.0, I get a Segmentation fault when comparing two instances of the same string-array. For example: from numarray.strings import asarray for x in range(1000,1000000,1000): print x raw = ['abcde']*x arr = asarray(raw) arr == arr gives: 1000 2000 3000 4000 5000 6000 Segmentation fault I verified that this works correctly in numarray-1.3.3. Also, the problem can be avoided in 1.4.0 by replacing 'arr == arr' with 'arr = arr.copy()'. Can anyone else reproduce this, or is a problem with my environment? Thanks, John From jmiller at stsci.edu Thu Oct 27 10:51:40 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu Oct 27 10:51:40 2005 Subject: [Numpy-discussion] numarray-1.4.0 string-array comparisons problem In-Reply-To: <200510262055.39204.chaves@computer.org> References: <435F9F2F.2000005@stsci.edu> <200510262055.39204.chaves@computer.org> Message-ID: <436112CA.2050202@stsci.edu> John A Chaves wrote: >After installing numarray-1.4.0, I get a Segmentation fault when comparing >two instances of the same string-array. For example: > > from numarray.strings import asarray > > for x in range(1000,1000000,1000): > print x > raw = ['abcde']*x > arr = asarray(raw) > arr == arr > >gives: > > 1000 > 2000 > 3000 > 4000 > 5000 > 6000 > Segmentation fault > > >Can anyone else reproduce this, or is a problem with my environment? > > Yes, I was able to reproduce this too, thanks for the report and for logging on SF. I understand the problem and am working out a fix now. Regards, Todd From jmiller at stsci.edu Thu Oct 27 15:25:25 2005 From: jmiller at stsci.edu (Todd Miller) Date: Thu Oct 27 15:25:25 2005 Subject: [Numpy-discussion] numarray-1.4.0 string-array comparisons problem In-Reply-To: <436112CA.2050202@stsci.edu> References: <435F9F2F.2000005@stsci.edu> <200510262055.39204.chaves@computer.org> <436112CA.2050202@stsci.edu> Message-ID: <43615361.70404@stsci.edu> Todd Miller wrote: > John A Chaves wrote: > >> After installing numarray-1.4.0, I get a Segmentation fault when >> comparing >> two instances of the same string-array. For example: >> >> from numarray.strings import asarray >> >> for x in range(1000,1000000,1000): >> print x >> raw = ['abcde']*x >> arr = asarray(raw) >> arr == arr >> >> gives: >> >> 1000 >> 2000 >> 3000 >> 4000 >> 5000 >> 6000 >> Segmentation fault >> >> >> Can anyone else reproduce this, or is a problem with my environment? >> >> > Yes, I was able to reproduce this too, thanks for the report and for > logging on SF. I understand the problem and am working out a fix now. > > Regards, > Todd I committed the fix for this and put out numarray-1.4.1. Thanks again, Todd From a.h.jaffe at gmail.com Thu Oct 27 15:51:57 2005 From: a.h.jaffe at gmail.com (Andrew Jaffe) Date: Thu Oct 27 15:51:57 2005 Subject: [Numpy-discussion] Re: ANN: numarray-1.4.0 In-Reply-To: <435F9F2F.2000005@stsci.edu> References: <435F9F2F.2000005@stsci.edu> Message-ID: <43615751.8080009@gmail.com> Hi- > 4. scipy newcore compatible lower case type names (e.g. int32 not Int32) Is this actually true? For example, numarray.Float64 exists as before, but numarray.float64 does not. > 5. scipy newcore 'dtype' keyword and .dtypechar attribute. And more generally, is there any more information on the relationship between numarray and scipy_core? Will numarray be phased out eventually (i.e., should we start moving over to scipy?)? But in any event thanks for the hard work! Andrew From jmiller at stsci.edu Fri Oct 28 02:20:26 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 28 02:20:26 2005 Subject: [Numpy-discussion] Re: ANN: numarray-1.4.0 In-Reply-To: <43615751.8080009@gmail.com> References: <435F9F2F.2000005@stsci.edu> <43615751.8080009@gmail.com> Message-ID: <4361ECFC.9000904@stsci.edu> Andrew Jaffe wrote: > Hi- > >> 4. scipy newcore compatible lower case type names (e.g. int32 not Int32) > > > Is this actually true? For example, numarray.Float64 exists as before, > but numarray.float64 does not. I added the "string aliases" but didn't create new Python object bindings. So you can currently say something like arange(10, dtype='float32') but cannot yet say: arange(10, dtype=float32) Good catch. I'll add those soon. >> 5. scipy newcore 'dtype' keyword and .dtypechar attribute. > > > And more generally, is there any more information on the relationship > between numarray and scipy_core? I think scipy_core is intended to add the best numarray-like features to Numeric. > Will numarray be phased out eventually (i.e., should we start moving > over to scipy?)? When scipy_core matures enough to replace numarray, numarray will be phased out. If you rely on numarray, it is definitely worth your time to keep an eye on scipy_core and switch when it meets your needs better. Regards, Todd From jmiller at stsci.edu Fri Oct 28 06:28:17 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 28 06:28:17 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 Message-ID: <43622697.9040302@stsci.edu> Numarray is an array processing package designed to efficiently manipulate large multi-dimensional arrays. Numarray is modelled after Numeric and features c-code generated from python template scripts, the capacity to operate directly on arrays in files, arrays of heterogeneous records, string arrays, and in-place operation on memory mapped files. ========================================================================= Release Notes for numarray-1.4.1 II. BUGS FIXED / CLOSED 1339713 segfault when comparing string-array to self ========================================================================= Release Notes for numarray-1.4.0 I. ENHANCEMENTS 1. Speed improvement for numarray operators. The Python level hook mapping numarray operators onto universal functions has been moved down to C. 2. Speed improvement for string-array comparisons, any(), all(). String correlation is ~10x faster. 3. Better operation with py2exe to help it automatically detect the core numarray extensions to include in an installer. 4. scipy newcore compatible lower case type names (e.g. int32 not Int32) 5. scipy newcore 'dtype' keyword and .dtypechar attribute. II. BUGS FIXED / CLOSED 1323355 Apps fail with import_libnumarray 1315212 Infinite loop converting some scalar strings into a list 1298916 rank-0 tostring() broken 1297948 records.array fails to create empty fields 1286291 import sys missing from array_persist.py 1286168 Generic sequences in ``strings.array()`` 1236392 Outdated web link in announcements 1235219 LinearAlgebraError not imported in linear_algebra See http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse for more details. III. CAUTIONS This release should be backward binary compatible with numarray-1.3.x WHERE ----------- Numarray-1.4.0 windows executable installers, source code, and manual is here: http://sourceforge.net/project/showfiles.php?group_id=1369 Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://www.stsci.edu/resources/software_hardware/numarray Trackers for Numarray Bugs, Feature Requests, Support, and Patches are at the Source Forge project for NumPy at: http://sourceforge.net/tracker/?group_id=1369 REQUIREMENTS ------------------------------ numarray-1.4.0 requires Python 2.2.2 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science Institute. We'd like to acknowledge the assitance of Francesc Alted, Paul Dubois, Sebastian Haase, Chuck Harris, Tim Hochberg, Nadav Horesh, Edward C. Jones, Eric Jones, Jochen Kuepper, Travis Oliphant, Pearu Peterson, Peter Verveer, Colin Williams, Rory Yorke, and everyone else who has contributed with comments and feedback. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. -- Todd Miller jmiller at stsci.edu From faltet at carabos.com Fri Oct 28 09:05:48 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 28 09:05:48 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <43622697.9040302@stsci.edu> References: <43622697.9040302@stsci.edu> Message-ID: <200510281803.46663.faltet@carabos.com> Hi Todd, I've just installed numarray 1.4.1 and pass my tests over it. Everything goes well, except some small detail: Python 2.4.2 (#2, Sep 29 2005, 00:23:59) [GCC 4.0.2 (Debian 4.0.1-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import Numeric >>> import numarray >>> Numeric.__version__ '24.0b2' >>> numarray.__version__ '1.4.1' >>> na=numarray.array([ 51.], type=numarray.Float64) >>> Numeric.array(na, typecode='d') Traceback (most recent call last): File "", line 1, in ? TypeError: expected a writeable buffer object >>> This worked in 1.3.x and before. I hope that this is not the intended behaviour. Thanks, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From jmiller at stsci.edu Fri Oct 28 11:00:32 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 28 11:00:32 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <200510281803.46663.faltet@carabos.com> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> Message-ID: <436266DB.1020308@stsci.edu> Francesc Altet wrote: >Hi Todd, > >I've just installed numarray 1.4.1 and pass my tests over it. >Everything goes well, except some small detail: > >Python 2.4.2 (#2, Sep 29 2005, 00:23:59) >[GCC 4.0.2 (Debian 4.0.1-9)] on linux2 >Type "help", "copyright", "credits" or "license" for more information. > > >>>>import Numeric >>>>import numarray >>>>Numeric.__version__ >>>> >>>> >'24.0b2' > > >>>>numarray.__version__ >>>> >>>> >'1.4.1' > > >>>>na=numarray.array([ 51.], type=numarray.Float64) >>>>Numeric.array(na, typecode='d') >>>> >>>> >Traceback (most recent call last): > File "", line 1, in ? >TypeError: expected a writeable buffer object > > > >This worked in 1.3.x and before. > >I hope that this is not the intended behaviour. > > This looks like a coordinated change in both numarray and Numeric. (Travis?) I updated to Numeric-24.0 and all was well. It appears the __array_data__ protocol improved to know about readonly-ness. In order to support the readonly behavior in numarray, I think the change must be made in lockstep. We could however back out support for readonly while Numeric-24.0 becomes more pervasive. Regards, Todd From faltet at carabos.com Fri Oct 28 11:23:26 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 28 11:23:26 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <436266DB.1020308@stsci.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> Message-ID: <200510282021.31288.faltet@carabos.com> A Divendres 28 Octubre 2005 19:58, Todd Miller va escriure: > This looks like a coordinated change in both numarray and Numeric. > (Travis?) I updated to Numeric-24.0 and all was well. Ops, I was not aware that Numeric-24.0 was out. Anyway, I've tried to compile it, but I did not succeed with my Debian system: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -IInclude -IPackages/FFT/Include -IPackages/RNG/Include -I/usr/include/python2.4 -c Src/lapack_litemodule.c -o build/temp.linux-i686-2.4/Src/lapack_litemodule.o gcc -pthread -shared build/temp.linux-i686-2.4/Src/lapack_litemodule.o -L/usr/lib/atlas -llapack -lcblas -lf77blas -latlas -lg2c -o build/lib.linux-i686-2.4/lapack_lite.so /usr/bin/ld: cannot find -llapack collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 The installation for Numeric usually worked on my system. Perhaps it is now assumed that lapack should be a pre-requisite? > It appears the __array_data__ protocol improved to know about > readonly-ness. In order to support the readonly behavior in numarray, > I think the change must be made in lockstep. We could however back out > support for readonly while Numeric-24.0 becomes more pervasive. Yes, please, backward compatibility with pre-Numeric 24.0 would be a nice thing to have. Regards, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From Chris.Barker at noaa.gov Fri Oct 28 11:39:50 2005 From: Chris.Barker at noaa.gov (Chris Barker) Date: Fri Oct 28 11:39:50 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <200510282021.31288.faltet@carabos.com> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> <200510282021.31288.faltet@carabos.com> Message-ID: <43626FDA.8070201@noaa.gov> Francesc Altet wrote: > /usr/bin/ld: cannot find -llapack > collect2: ld returned 1 exit status > error: command 'gcc' failed with exit status 1 > > The installation for Numeric usually worked on my system. Perhaps it > is now assumed that lapack should be a pre-requisite? This has been a problem for a remarkably long time! I thought it had been fixed. It's not really a big deal, it's only a matter of what the defaults are in customize.py. The comments in there should be enough to figure it out. The default really should be to use the built-in lapack-lite. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From faltet at carabos.com Fri Oct 28 12:03:42 2005 From: faltet at carabos.com (Francesc Altet) Date: Fri Oct 28 12:03:42 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <43626FDA.8070201@noaa.gov> References: <43622697.9040302@stsci.edu> <200510282021.31288.faltet@carabos.com> <43626FDA.8070201@noaa.gov> Message-ID: <200510282101.23736.faltet@carabos.com> A Divendres 28 Octubre 2005 20:37, Chris Barker va escriure: > This has been a problem for a remarkably long time! I thought it had > been fixed. It's not really a big deal, it's only a matter of what the > defaults are in customize.py. The comments in there should be enough to > figure it out. Yup, it worked. Thanks. Anyway, I think that the sensible default would be to not suppose that neither BLAS, LAPACK, ATALS are installed and update the README to give instructuctions on how to adapt Numeric to your system. Cheers, -- >0,0< Francesc Altet ? ? http://www.carabos.com/ V V C?rabos Coop. V. ??Enjoy Data "-" From oliphant at ee.byu.edu Fri Oct 28 12:23:01 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri Oct 28 12:23:01 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <200510282101.23736.faltet@carabos.com> References: <43622697.9040302@stsci.edu> <200510282021.31288.faltet@carabos.com> <43626FDA.8070201@noaa.gov> <200510282101.23736.faltet@carabos.com> Message-ID: <43627A2F.4090307@ee.byu.edu> Francesc Altet wrote: >A Divendres 28 Octubre 2005 20:37, Chris Barker va escriure: > > >>This has been a problem for a remarkably long time! I thought it had >>been fixed. It's not really a big deal, it's only a matter of what the >>defaults are in customize.py. The comments in there should be enough to >>figure it out. >> >> > >Yup, it worked. Thanks. > >Anyway, I think that the sensible default would be to not suppose that >neither BLAS, LAPACK, ATALS are installed and update the README to >give instructuctions on how to adapt Numeric to your system. > > > Yes, that was always the plan. I just forgot to reset customize.py to the original before making a tar ball. I've fixed the problem now. The new tar ball does uses default lapack_lite. (I'm sure I'll upset somebody because the tar ball changed slightly with no version number increase --- but I don't want to make a new release --- the other files should be fine). -Travis From oliphant at ee.byu.edu Fri Oct 28 12:29:43 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri Oct 28 12:29:43 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <436266DB.1020308@stsci.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> Message-ID: <43627BBE.6070109@ee.byu.edu> Todd Miller wrote: > Francesc Altet wrote: > >> Hi Todd, >> >> I've just installed numarray 1.4.1 and pass my tests over it. >> Everything goes well, except some small detail: >> >> Python 2.4.2 (#2, Sep 29 2005, 00:23:59) >> [GCC 4.0.2 (Debian 4.0.1-9)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >> >>>>> import Numeric >>>>> import numarray >>>>> Numeric.__version__ >>>>> >>>> >> '24.0b2' >> >> >>>>> numarray.__version__ >>>>> >>>> >> '1.4.1' >> >> >>>>> na=numarray.array([ 51.], type=numarray.Float64) >>>>> Numeric.array(na, typecode='d') >>>>> >>>> >> Traceback (most recent call last): >> File "", line 1, in ? >> TypeError: expected a writeable buffer object >> >> > This looks like a coordinated change in both numarray and Numeric. > (Travis?) I updated to Numeric-24.0 and all was well. > It appears the __array_data__ protocol improved to know about > readonly-ness. In order to support the readonly behavior in > numarray, I think the change must be made in lockstep. We could > however back out support for readonly while Numeric-24.0 becomes more > pervasive. Yes, basically the __array_data__ protocol was essentially pointless before because it just returned a buffer object which 1) did nothing more than the object itself supporting the buffer protocol and 2) could not handle strided (discontiguous) arrays. The new array protocol handles the situation much better and allows discontiguous arrays to be shared. However, it does create some backward compatibility issues. But, because Numeric 24.0b2 was a *beta* release I don't see this as a real problem. Get the final release of Numeric, which will handle the array protocol correctly (note it will also handle the old __array_data__ format as well). Todd, I was not sure how to change numarray so it would consume the new protocol correctly. So, I'm not sure how numarray interprets the __array_data__ attribute. -Travis From edcjones at comcast.net Fri Oct 28 12:48:37 2005 From: edcjones at comcast.net (Edward C. Jones) Date: Fri Oct 28 12:48:37 2005 Subject: [Numpy-discussion] numarray 1.4.0: unitary operator "+" fails Message-ID: <43628028.8040705@comcast.net> The unitary "+" does not work for numarray 1.4.0: > python Python 2.4.2 (#1, Oct 1 2005, 18:47:35) [GCC 4.0.2 20050821 (prerelease) (Debian 4.0.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numarray >>> arr = numarray.zeros((2,2)) >>> +arr Traceback (most recent call last): File "", line 1, in ? AttributeError: copy From jmiller at stsci.edu Fri Oct 28 13:06:36 2005 From: jmiller at stsci.edu (Todd Miller) Date: Fri Oct 28 13:06:36 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <43627BBE.6070109@ee.byu.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> <43627BBE.6070109@ee.byu.edu> Message-ID: <4362844D.4080901@stsci.edu> Travis Oliphant wrote: > Todd Miller wrote: > >> Francesc Altet wrote: >> >>> Hi Todd, >>> >>> I've just installed numarray 1.4.1 and pass my tests over it. >>> Everything goes well, except some small detail: >>> >>> Python 2.4.2 (#2, Sep 29 2005, 00:23:59) >>> [GCC 4.0.2 (Debian 4.0.1-9)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> >>>>>> import Numeric >>>>>> import numarray >>>>>> Numeric.__version__ >>>>>> >>>>> >>>>> >>> '24.0b2' >>> >>> >>>>>> numarray.__version__ >>>>>> >>>>> >>>>> >>> '1.4.1' >>> >>> >>>>>> na=numarray.array([ 51.], type=numarray.Float64) >>>>>> Numeric.array(na, typecode='d') >>>>>> >>>>> >>>>> >>> Traceback (most recent call last): >>> File "", line 1, in ? >>> TypeError: expected a writeable buffer object >>> >>> > >> This looks like a coordinated change in both numarray and Numeric. >> (Travis?) I updated to Numeric-24.0 and all was well. > > > >> It appears the __array_data__ protocol improved to know about >> readonly-ness. In order to support the readonly behavior in >> numarray, I think the change must be made in lockstep. We could >> however back out support for readonly while Numeric-24.0 becomes more >> pervasive. > > > Yes, basically the __array_data__ protocol was essentially pointless > before because it just returned a buffer object which > > 1) did nothing more than the object itself supporting the buffer > protocol and > 2) could not handle strided (discontiguous) arrays. > > The new array protocol handles the situation much better and allows > discontiguous arrays to be shared. > > However, it does create some backward compatibility issues. But, > because Numeric 24.0b2 was a *beta* release I don't see this as a real > problem. Yeah, that sounds ok... I was thinking the array interface was a release or two older. > Get the final release of Numeric, which will handle the array > protocol correctly (note it will also handle the old __array_data__ > format as well). > > Todd, I was not sure how to change numarray so it would consume the > new protocol correctly. I think numarray's multiple factory functions in numarray, strings, and records all need to be beefed up to look for it. > So, I'm not sure how numarray interprets the __array_data__ attribute. I don't know if it does right now. Just to be clear, what Francesc was talking about is *supplying* the array buffer from numarray as apparently defined in the __get_array_data__ method. If 24b was the first time the array interface ever worked it sounds ok to leave your changes in and force an upgrade to 24 if a user uses that feature. Francesc? Todd From oliphant at ee.byu.edu Fri Oct 28 13:53:19 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Fri Oct 28 13:53:19 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <4362844D.4080901@stsci.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> <43627BBE.6070109@ee.byu.edu> <4362844D.4080901@stsci.edu> Message-ID: <43628F84.3040200@ee.byu.edu> > Todd Miller wrote: > I don't know if it does right now. Just to be clear, what Francesc > was talking about is *supplying* the array buffer from numarray as > apparently defined in the __get_array_data__ method. If 24b was the > first time the array interface ever worked it sounds ok to leave your > changes in and force an upgrade to 24 if a user uses that feature. > Francesc? Right, he was talking about just supplying the array data information from numarray. And, yes, Numeric 24b was the first time the array interface ever worked. -Travis From edcjones at comcast.net Fri Oct 28 18:26:53 2005 From: edcjones at comcast.net (Edward C. Jones) Date: Fri Oct 28 18:26:53 2005 Subject: [Numpy-discussion] numarray 1.4.1: Infinite loop during test Message-ID: <4362CF93.4050605@comcast.net> I have a PC with Debian unstable. I use Python 1.4.2. After installing numarray 1.4.1, I ran the tests: >>> import numarray.testall as testall >>> testall.test() Testing numarray 1.4.1 on normal Python (2, 4, 2, 'final', 0) on platform linux2 numarray.numtest: 2.89 ((0, 1205), (0, 1205)) numarray.ieeespecial: 0.07 (0, 86) numarray.records: 0.05 (0, 48) numarray.strings: 0.17 (0, 189) numarray.memmap: 0.09 (0, 82) numarray.objects: 0.14 (0, 105) numarray.memorytest: 0.01 (0, 16) numarray.examples.convolve: 0.09 ((0, 20), (0, 20), (0, 20), (0, 20)) numarray.convolve: 0.06 (0, 45) numarray.fft: 0.11 (0, 75) After the above, nothing happens. "ps ax" show a lot of time being used. Looks like an infinite loop. Process had to be killed. From jmiller at stsci.edu Sat Oct 29 05:11:07 2005 From: jmiller at stsci.edu (Todd Miller) Date: Sat Oct 29 05:11:07 2005 Subject: [Numpy-discussion] numarray 1.4.1: Infinite loop during test In-Reply-To: <4362CF93.4050605@comcast.net> References: <4362CF93.4050605@comcast.net> Message-ID: <4363667F.9070000@stsci.edu> Edward C. Jones wrote: > I have a PC with Debian unstable. I use Python 1.4.2. After installing > numarray 1.4.1, I ran the tests: > > >>> import numarray.testall as testall > >>> testall.test() > Testing numarray 1.4.1 on normal Python (2, 4, 2, 'final', 0) on > platform linux2 > numarray.numtest: 2.89 ((0, 1205), (0, 1205)) > numarray.ieeespecial: 0.07 (0, 86) > numarray.records: 0.05 (0, 48) > numarray.strings: 0.17 (0, 189) > numarray.memmap: 0.09 (0, 82) > numarray.objects: 0.14 (0, 105) > numarray.memorytest: 0.01 (0, 16) > numarray.examples.convolve: 0.09 ((0, 20), (0, 20), > (0, 20), (0, 20)) > numarray.convolve: 0.06 (0, 45) > numarray.fft: 0.11 (0, 75) > > After the above, nothing happens. "ps ax" show a lot of time being > used. Looks like an infinite loop. Process had to be killed. I don't have access to a Debian system but some things come to mind: a. The loop looks like it's in numarray.linear_algebra. b. I changed numarray's setup to make the "config" command automatic for 1.4.0. That changes how the builtin lapack_lite module is configured/built which affects linear_algebra. c. My impression is that debian builds numarray against external libraries so (b) shouldn't be the problem. Does the numarray tarball from source forge: http://prdownloads.sourceforge.net/numpy/numarray-1.4.1.tar.gz?download install and self-test OK using "python setup.py install"? Regards, Todd From faltet at carabos.com Sat Oct 29 09:07:22 2005 From: faltet at carabos.com (Francesc Altet) Date: Sat Oct 29 09:07:22 2005 Subject: [Numpy-discussion] ANN: numarray-1.4.1 In-Reply-To: <43628F84.3040200@ee.byu.edu> References: <43622697.9040302@stsci.edu> <200510281803.46663.faltet@carabos.com> <436266DB.1020308@stsci.edu> <43627BBE.6070109@ee.byu.edu> <4362844D.4080901@stsci.edu> <43628F84.3040200@ee.byu.edu> Message-ID: <1130602931.2848.4.camel@localhost.localdomain> El dv 28 de 10 del 2005 a les 14:52 -0600, en/na Travis Oliphant va escriure: > > Todd Miller wrote: > > I don't know if it does right now. Just to be clear, what Francesc > > was talking about is *supplying* the array buffer from numarray as > > apparently defined in the __get_array_data__ method. If 24b was the > > first time the array interface ever worked it sounds ok to leave your > > changes in and force an upgrade to 24 if a user uses that feature. > > Francesc? > Right, he was talking about just supplying the array data information > from numarray. > > And, yes, Numeric 24b was the first time the array interface ever worked. Yes, I think that if Numeric 24b was the first implementing the array interface, then there is no need to touch nothing in numarray. As you said, I should make Numeric 24.0 a requisite for running our software. Thanks for clarificating the issue, -- >0,0< Francesc Altet http://www.carabos.com/ V V C?rabos Coop. V. Enjoy Data "-" From rowen at cesmail.net Mon Oct 31 16:56:21 2005 From: rowen at cesmail.net (Russell E. Owen) Date: Mon Oct 31 16:56:21 2005 Subject: [Numpy-discussion] byteswap questions Message-ID: I've got a module that can send array data to ds9. It screws up on byteswapped data and I'm trying to fix it. The code sends the data to ds9 via a pipe using arr.tofile(). To send byteswapped data I think I have two choices (given that I don't want to modify the input array): 1) Figure out the byte order and specify a suitable flag to ds9 (it accepts "bigendian" or "littleendian" flags). This avoids copying the data, so it sounds attractive, but can I do it without relying on "internals"? Numarrays have an "isbyteswapped" method, but all this says is "not native machine order"; it doesn't actually tell me the order. I need to know if the order is bigendian or littleendian, and I can't find a documented way to get that, just an undocumented attribute _byteorder. 2) If the array is not in native byte order, make a native byte ordered copy before sending it. The following seems to work: if arr.isbyteswapped(): arr = arr.copy() but is this the recommended way to get native-order copy? - Is "copy" guaranteed to return a copy in native byte order? The documentation doesn't say. - I was tempted to use num.array(arr) to make the copy, but the documentation doesn't say what "array" does if the input is a already a numarray array. As an aside, I tried to use the byteswapped method, but it has a totally different effect: >>> d array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.], ...type=Float32) >>> d.isbyteswapped() 1 >>> dcopy = d.copy() >>> d array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.], ...type=Float32) >>> dcopy.isbyteswapped() 0 >>> dswap = d.byteswapped() >>> dswap array([[ 1.91063654e-38, 1.91063654e-38, 1.91063654e-38, ..., ...type=Float32) which I guess means I'd have to byteswap the byteswapped data. Aargh. -- Russell