From Oliphant.Travis@mayo.edu Tue Aug 3 00:26:21 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Mon, 2 Aug 1999 18:26:21 -0500 (CDT) Subject: [Matrix-SIG] Dislin is a nice plotting package Message-ID: A users experience with plotting under Python: I know plotting has been discussed on this list before. Many people indicate that the lack of quality plotting is preventing them from using Numerical Python as a complete replacement for IDL or MATLAB. I have been using gist for the past several months and found it to be excellent for investigative plotting (except I miss axes on surface plots). The plot quality and flexibility with gist is excellent (it is particularly easy to use for me with the matlab-like plotting commands I implemented) However, recently I needed to produce plots with legends on them so I turned to DISLIN. I had seen DISLIN before and even had it installed but it looked pretty complicated to use so I hadn't tried it since I wasn't desperate for good plots. Well, I've now tried it and I'm impressed. It is a COMPLETE plotting system. After a bit of reading to understand the plotting paradigm (it is a very powerful but non-obvious system) I can now make any plot I need with the system, and configure it to my hearts content. There are even quick plotting functions defined so that I can make surface plots (with axes) and color plots with an automatic color bar, with a single plot command. I can even use dislin and gist together which works very well. An object-oriented wrapper around the quick-plots so that all the properties could be set easily would make dislin quite phenomenal for use with Python. I don't mean this post to discourage work on the fine plotting efforts underway with PIDDLE, GRAPHITE, and Snow. These packages will only improve the plotting choices under Python. I just wanted to express my pleasure at using DISLIN under Python and encourage others who may be frustrated about their ability to produce quality output now or who may be holding back from adopting Python as their interactive analysis environment to check out DISLIN and see for yourself if it will serve you. -Travis P.S. The dislin homepage is http://www.linmpi.mpg.de/dislin From godzilla@netmeg.net Tue Aug 3 02:59:41 1999 From: godzilla@netmeg.net (Les Schaffer) Date: Mon, 02 Aug 1999 21:59:41 -0400 Subject: [Matrix-SIG] Dislin is a nice plotting package In-Reply-To: Message-ID: <3.0.1.32.19990802215941.00901d90@pop.netmeg.net> At 06:26 PM 8/2/99 -0500, Travis Oliphant opined: ->An object-oriented wrapper around the quick-plots so that all the ->properties could be set easily would make dislin quite phenomenal for use ->with Python. a hearty 'i second this'..... i have been using dislin and python for about a year and a half, and am very happy with it. it should also be pointed out that its author, Helmut Michels, has always responded quickly to fix the few tiny glitches that occur from time to time in releases. he also has a java interface to his package as well. les schaffer From scolsen@uci.edu Tue Aug 3 19:58:07 1999 From: scolsen@uci.edu (scolsen@uci.edu) Date: Tue, 03 Aug 1999 11:58:07 -0700 Subject: [Matrix-SIG] Re: Dislin is a nice plotting package In-Reply-To: Message-ID: <7o7e3v$cq7h@eGroups.com> wrote: original article:http://www.egroups.com/group/matrix-sig/?start=1742 > A users experience with plotting under Python: > > I know plotting has been discussed on this list before. Many people > indicate that the lack of quality plotting is preventing them from using > Numerical Python as a complete replacement for IDL or MATLAB. > > I have been using gist for the past several months and found it to be > excellent for investigative plotting (except I miss axes on surface > plots). The plot quality and flexibility with gist is excellent (it is > particularly easy to use for me with the matlab-like plotting commands I > implemented) > > However, recently I needed to produce plots with legends on them so I > turned to DISLIN. I had seen DISLIN before and even had it installed but > it looked pretty complicated to use so I hadn't tried it since I wasn't > desperate for good plots. Well, I've now tried it and I'm impressed. > It is a COMPLETE plotting system. After a bit of reading to understand > the plotting paradigm (it is a very powerful but non-obvious system) I can > now make any plot I need with the system, and configure it to my hearts > content. > > There are even quick plotting functions defined so that I can make > surface plots (with axes) and color plots with an automatic color bar, > with a single plot command. I can even use dislin and gist together which > works very well. > > An object-oriented wrapper around the quick-plots so that all the > properties could be set easily would make dislin quite phenomenal for use > with Python. > > I don't mean this post to discourage work on the fine plotting efforts > underway with PIDDLE, GRAPHITE, and Snow. These packages will only > improve the plotting choices under Python. I just wanted to express my > pleasure at using DISLIN under Python and encourage others who may be > frustrated about their ability to produce quality output now or who > may be holding back from adopting Python as their interactive > analysis environment to check out DISLIN and see for yourself if it will > serve you. > > -Travis > > P.S. The dislin homepage is > > http://www.linmpi.mpg.de/dislin I've had the same good experience with DISLIN and I've hacked together a little bit of a class structure based on DISLIN. It's far from perfect but it does what I need it to do. It doesn't do any 3-d stuff but that would be easy to add. It could probably be cleaned up into something useable. It's here along with a few other Python things. http://essgrad.ps.uci.edu/~olsen/Pages/LinuxPython/Tools.html Seth scolsen@uci.edu From Oliphant.Travis@mayo.edu Thu Aug 5 05:46:45 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Wed, 4 Aug 1999 23:46:45 -0500 (CDT) Subject: [Matrix-SIG] NumPy Array as buffer. Message-ID: A couple of posts on the python news group made me aware of a buffer object that was apparently added in Python 1.5.2. From my first look at it this looks like a good way to "connect" packages which use different objects. I'm always wanting to get the data in NumPy arrays into some other object (such as the PIL). My understanding is that the buffer object should (could) allow this without a data copy. Are appropriate methods going to be added to NumPy to allow arrays to be read and written to as a buffer? Has this been looked into before? I'm hesitant to do anything right now because of the "long-promised" update to NumPy. Perhaps the coming Extension Class model will solve all my problems :-) Fishing for info, Travis From Oliphant.Travis@mayo.edu Thu Aug 5 07:29:16 1999 From: Oliphant.Travis@mayo.edu (Travis Oliphant) Date: Thu, 5 Aug 1999 01:29:16 -0500 (CDT) Subject: [Matrix-SIG] Question for the NumPy gurus Message-ID: I have two arrays with in1.shape = (N,M,P) and in2.shape = (N,P) Is there a fast way (no Python for loop) to perform an innerproduct element-wise along the first dimension to return an array of shape (N,M)? I was hoping for the equivalent of out = zeros((N,M)) for k in xrange(N): out[k] = innerproduct(in1[k],in2[k]) but with the for loop in C since N is very big. Thanks, Travis From janne@avocado.pc.helsinki.fi Thu Aug 5 11:44:14 1999 From: janne@avocado.pc.helsinki.fi (Janne Sinkkonen) Date: 05 Aug 1999 13:44:14 +0300 Subject: [Matrix-SIG] Question for the NumPy gurus In-Reply-To: Travis Oliphant's message of "Thu, 5 Aug 1999 01:29:16 -0500 (CDT)" References: Message-ID: Travis Oliphant writes: > I have two arrays with in1.shape = (N,M,P) and in2.shape = (N,P) > > Is there a fast way (no Python for loop) to perform > an innerproduct element-wise along the first dimension to > return an array of shape (N,M)? I was hoping for the equivalent of In general, you could compute the outer product and then sum and take a diagonal. No loops, just some extra computing and astronomical memory requirements. :) How about something like: reshape(dot(reshape(in1,(-1,P)),in2),(N,M)) dot() should really be extended for general tensor products, i.e. to handle N-dimensional arrays with user-specified dimension pairs over which inner products are computed: dot(A,B,((dimA1,dimB1),(dimA2,dimB2),...)) Then you could write dot(in1,in2,((-1,-1))) -- Janne From r.hooft@euromail.net Thu Aug 5 12:09:28 1999 From: r.hooft@euromail.net (Rob Hooft) Date: Thu, 5 Aug 1999 13:09:28 +0200 (MZT) Subject: [Matrix-SIG] Question for the NumPy gurus In-Reply-To: References: Message-ID: <14249.28904.621732.24473@octopus.chem.uu.nl> >>>>> "JS" == Janne Sinkkonen writes: JS> dot() should really be extended for general tensor products, JS> i.e. to handle N-dimensional arrays with user-specified dimension JS> pairs over which inner products are computed: JS> dot(A,B,((dimA1,dimB1),(dimA2,dimB2),...)) JS> Then you could write JS> dot(in1,in2,((-1,-1))) I guess: dot(in1,in2,((-1,-1),)) This would definitely be a nice feature! I now have a routine: def matvec(m,v): """Multiply a matrix with a vector, or any array of vectors""" v=Numeric.array(v,copy=0) ax=range(len(v.shape))[1:]+[0] return transpose(Numeric.innerproduct(m,v),axes=ax) Which I use often with a 3x3-matrix and a 1340x1300 array of 3-vectors. If/When the resulting transposed matrix needs to be made contiguous, that copy actually takes longer than the matvec() operation itself. OTOH, it doesn't make sense to optimize this, since the progam that uses this spends even more time in Numeric.arrayrange() :-( Rob -- ===== R.Hooft@EuroMail.net http://www.xs4all.nl/~hooft/rob/ ===== ===== R&D, Nonius BV, Delft http://www.nonius.nl/ ===== ===== PGPid 0xFA19277D ========================== Use Linux! ========= From da@ski.org Thu Aug 5 18:16:17 1999 From: da@ski.org (David Ascher) Date: Thu, 5 Aug 1999 10:16:17 -0700 (Pacific Daylight Time) Subject: [Matrix-SIG] NumPy Array as buffer. In-Reply-To: Message-ID: On Wed, 4 Aug 1999, Travis Oliphant wrote: > Are appropriate methods going to be added to NumPy to allow arrays to be > read and written to as a buffer? Has this been looked into before? > > I'm hesitant to do anything right now because of the "long-promised" > update to NumPy. Perhaps the coming Extension Class model will solve all > my problems :-) 1) I've started looking into the buffer interface, but haven't made any concrete changes. I agree that it is a very promising notion. Note also that PIL currently doesn't support the buffer interface, but Fredrik is aware of its potential benefits. 2) I think you'll see a beta release of the EC work within a day or two. --david From mhagger@blizzard.harvard.edu Mon Aug 9 04:33:14 1999 From: mhagger@blizzard.harvard.edu (Michael Haggerty) Date: 9 Aug 1999 03:33:14 -0000 Subject: [Matrix-SIG] ANNOUNCE: Gnuplot.py 1.2 Message-ID: <19990809033314.4154.qmail@cyclone.harvard.edu> This is to announce the release of version 1.2 of Gnuplot.py. Gnuplot.py is a Python [1] module that allows you to create graphs from within Python using the gnuplot [2] plotting package. This version mainly adds support for MS Windows and for sending data to gnuplot as inline data and in binary format. Gnuplot.py can be obtained from http://monsoon.harvard.edu/~mhagger/Gnuplot/Gnuplot.html Prerequisites (see footnotes): the Python interpreter [1] the Python Numeric module [3] the gnuplot program [2] Some ways this package can be used: 1. Interactive data processing: Use Python's excellent Numeric package to create and manipulate arrays of numbers, and use Gnuplot.py to visualize the results. 2. Web graphics: write CGI scripts in Python that use gnuplot to output plots in GIF format and return them to the client. 3. Glue for numerical applications (this is my favorite): wrap your C++/C/Fortran subroutines so that they are callable from Python, then you can perform numerical computations interactively from scripts or from the command line and use Gnuplot.py to plot the output on the fly. 4. Compute a series of datasets in Python and plot them one after the other using Gnuplot.py to produce a crude animation. Features added in version 1.2: + Support for MS Windows, using the `pgnuplot.exe' program. Thanks go especially to Craig Schardt for help with this. + Support for sending data to gnuplot as `inline data' (i.e., "plot '-'"). This method should be faster than the older method, temporary files (which are also still supported). + Support for using binary files to send grid data to gnuplot. This saves a lot of time and usually saves space compared with text files. However, gnuplot only supports binary data for grid data used by the splot command. + Allows PlotItem options to be modified after the PlotItem is constructed. + Simplified the PlotItem inheritance hierarchy. + Added several configuration options to aid porting (see top of Gnuplot.py). + Separated function-based interface into a separate file (Gnuplot_plot.py). + Added a test module, Gnuplot_test.py, which tests most of the features of Gnuplot.py. + A README file, lots of documentation changes, etc. Features already present in older versions: + Two and three-dimensional plots. + Plot data from memory, from a file, or from an expression. + Support for multiple simultaneous gnuplot sessions. + Can pass arbitrary commands to the gnuplot program. + Object oriented, extensible design with several built-in types of plot items. + Portable and easy to install (nothing to compile except on Windows). Footnotes: ---------- [1] Python is an excellent object-oriented scripting/rapid development language that is also especially good at gluing programs together. [2] gnuplot is a free, popular, very portable plotting program with a command-line interface. It can make 2-d and 3-d plots and can output to myriad printers. [3] The Numeric Python extension is a Python module that adds fast and convenient array manipulations to the Python language. Yours, Michael -- Michael Haggerty mhagger@blizzard.harvard.edu From dubois1@llnl.gov Mon Aug 9 19:44:15 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Mon, 9 Aug 1999 11:44:15 -0700 Subject: [Matrix-SIG] LLNLDistribution12beta Message-ID: <000d01bee297$2e5df080$f4160218@plstn1.sfba.home.com> This is a multi-part message in MIME format. ------=_NextPart_000_0009_01BEE25C.81A38B00 Content-Type: multipart/alternative; boundary="----=_NextPart_001_000A_01BEE25C.81A38B00" ------=_NextPart_001_000A_01BEE25C.81A38B00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable ftp://ftp-icf.llnl.gov/pub/python/LLNLDistribution12beta.tgz is = available. It contains substantial bug fixes to Gist, the = EGCS-compatible CXX, and a beta version of Numeric. (Only Numeric is = beta; the other packages contain no experiments that we are aware of).=20 Please, please, please read the release notes before trying this = package. Numeric arrays can now be inherited from IN PYTHON: but only if = you follow the directions. ------=_NextPart_001_000A_01BEE25C.81A38B00 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
ftp= ://ftp-icf.llnl.gov/pub/python/LLNLDistribution12beta.tgz=20 is available. It contains substantial bug fixes to Gist, the = EGCS-compatible=20 CXX, and a beta version of Numeric. (Only Numeric is beta; the other = packages=20 contain no experiments that we are aware of).
 
Please, please, please read the release notes before = trying=20 this package. Numeric arrays can now be inherited from IN PYTHON: but = only if=20 you follow the directions.
 
------=_NextPart_001_000A_01BEE25C.81A38B00-- ------=_NextPart_000_0009_01BEE25C.81A38B00 Content-Type: text/html; name="Release_Notes.htm" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="Release_Notes.htm"

Release notes for LLNLPython Distribution

Send bug reports to support@icf.llnl.gov.

See the file README for information on sources, documentation, etc.

For package-specific information see the README files for each = package:=20

Release 12 (beta)

CXX -- Building Python extensions with C++

(Paul Dubois)

1. The long awaited day has arrived -- CXX compiles with EGCS! I have = tested it with snapshot 19990616 on a RedHat Linux 5.2 on a PC. EGCS requires a = standard library class random_access_iterator that is not yet available in some other compilers = (such as Windows VC6). Therefore a new switch:

STANDARD_LIBRARY_HAS_ITERATOR_TRAITS

has been added to CXX_Config.h that you may need to toggle if you get = an error on the two lines that mention random_access_iterator. The current definition is = correct for VC6 and EGCS-19990616.

2. A new constructor was added to Module to allow construction from a = string containing the module name. A test was added for this to the demo.

3. In the demo CXX_array.h the include file was changed to = Numeric/arrayobject.h.

Numerical (BETA)

Authors: John Barnard, David Ascher, Paul Dubois.

This is a beta release of the Numeric Extension.

In addition to some bug fixes, the major feature of this release is that = array objects are now ExtensionClass objects. This is a major change and is likely to have = some bugs. Do not use this release for production work, but please exercise it and let us = know of any bugs (email bug reports to support@icf.llnl.gov).

What are ExtensionClasses?

The ExtensionClass (EC for short) mechanism was designed by Jim = Fulton, of Digital Creations. ExtensionClasses (EC) are documented at http://www.digicool.com/releases/ExtensionClass.

EC provides many features, but the one of most interest to us is the = ability to subclass in Python a Python type written in C.

What features does this give the NumPy user right now?

Not much so far. This is mostly groundwork for future features. The = only significant feature is that one can now use arrays produced by logical operations as = masks in setting elements in arrays. For example:

>>> from Numeric import *
>>> x =3D arange(-4, 5)
>>> print x
[-4 -3 -2 -1 0 1 2 3 4]
>>> x[less(x,0)] =3D 0 # new feature
>>> print x
[0 0 0 0 0 1 2 3 4]

This mechanism is currently limited to setting such 'masked' areas to = scalars or arrays that have the same shape as the target array, but contributions which = extend this behavior are wecome.

What features can the NumPy user look forward to?

There are plans or prototypes for various features, such as:

* arrays with 'missing values'

* data frame arrays as in S+

* arrays with different casting or coercion rules

If you have new kinds of arrays which you wish to have added, please = contact support@icf.llnl.gov.

How does one use this?

Look in Numeric.py at the code which defines the Array class and the = MaskArray subclass. All user-defined subclasses should subclass Array, which will = be considered the 'default' class for NumPy code. (note that all Python-defined functions = now return Array instances by default).

Things to watch out for:

ExtensionClass subclasses are hybrids of Python code and C code. = Mostly, one can use them just as Python classes. However, it is very important that a = subclass of Array with an __init__ method call the __init__ method for Array (or = _numpy.multiarray, the 'true' base class), as that method will do the memory allocation required in = C.

We know of at least one bug in the current release, and are working = to track it down. The purpose of the beta release is to elicit comments as well as bug = report.

The C API should be 100% backwards compatible.

Notes on the ExtensionClass code:

The version of the software available at the digicool.com URL is not = the latest version, nor is it the one we used. The latest "official" EC = code is part of the Zope CVS tree, which is available either via CVS (http://www.zope.org/Community/CVS_public_access) or through a snapshot(http://starship.python.net/crew/da/zopedists/). The = ExtensionClass code that we use is derived from Jim Fulton's, but we will work with Jim Fulton to = end this code fork as soon as possible. Jim is aware of our changes and understands the = need for the changes (although he may choose a different solution, which we'd adjust to).

Plans:

The code in Numeric.py will be simplified.

 

Other changes:

UserArray.py: changed indentation to all spaces for consistency. = Fixed many bugs in UserArray.py and Matrix.py (Charles Wadman)

Tools/installtool.py was modified to allow an extra argument that = specifies the directory into which the install will be made. Normally this is not = necessary because the python you use to run the script is the python into which the = installation is made. Usage:
python installthis.py Numeric prefix-dir
(Holger Duerer)

MLab.py: made several constants explicitly decimal instead of integer = to improve performance, as suggested by Katsunori Waragai.

In the RNG package the include file was changed to = Numeric/arrayobject.h.

Tools

The compile.py tool allows multiple "configurations" on = Windows, works with VC5 and VC6, and automatically builds the .pyd's unattended. (Mark = Hammond, David Ascher).

Release 11 (April 1, 1999)

New Installation Procedure, Unix and Windows

The install script (Tools/installtool.py) has been modified so that = it uses the new .pth facility in Python 1.5. This tool should work on Windows also. = Instead of distributing a Windows installer we will distribute a zip file and have = the user run this script. While not as elegant as an installer, it avoids all the problems = we have had with the installer.

Each package is installed in its own directory and a .pth file is = created so that this directory is included in the python path. The include files are = installed inside the Python include directory in their own file. Thus, after installation = extension packages wishing to use arrayobject.h in their build can use a statement = like:

#include "numeric/arrayobject.h"

rather than further modify their compiler include path. A change to = RNG has been made to illustrate this capability.

Gist

Fixed error introduced in version 10 that caused segv.

Numpy

Attribute  information facility added

The array object now has an attribute named "attributes". = This field can be set or examined by using it as an attribute in a Python statement = (x.attributes). The purpose of this field is to make it easy for an application to attach = information to an array that might be used by another part of the software, such as a = label for a curve, an indication of units, etc. Its use and meaning is completely up to the = user. It is copied by copy.copy, but not much else. If otherwise not set it is None.

Sample usage

from Numeric import *
def show(x):
    "Display an array with a label"
    if x.attributes:
        print x.attributes['label'], = ' (', x.attributes['units'], ')'
    print x

x =3D arange(5) * 0.5
x.attributes=3D{"label": "x", "units", = "cm"}
show(x)

Note to extension writers

This field is initialized to Py_None. This field is copied by = PyArray_CopyArray. On object deallocation, its reference count is decremented. Assignment to = the attributes field in Python is done reference-count correctly; if using this = facility from C, you are responsible for this. The recommended sequence to set attributes to foo = is:

Py_INCREF(foo);
Py_DECREF(object->attributes);
object->attributes =3D foo;

Never set attributes to NULL, but rather back to Py_None.

Users wishing to create a version of NumPy that does not have this = facility may do so by defining NUMPY_NOATTRIBUTES before compiling.

Bug fixes

Added change to array_zeros to fix problem in indices introduced by = change in release 10. This was causing fromfunction to fail.
Several warning errors in arrayobject.h and multiarrayobject.h were = eliminated.
Fixed bug in searchsorted with patch from Travis Oliphant. Can now = determine the type of result based on both inputs.

Release 10 (March 18, 1999)

Gist

Fixed mesh3d.py to handle tetrahedra, and to allow data of either = type 'd' or type 'b'.
Fixed slice3.py so that data type will not be changed.
Numerous changes in slice2 and _slice2_part in gistCmodule to support = data values of either type 'b' or type 'd'.
Bug fixes in gistCmodule: make sure result arrays are returned as = Py_None in case a slice is empty; clear the OWN_DATA pointer in a PyArrayObject when the pointer = has been given to a newly created object.
The color card specification for a 3D object will now overrule the = specification (if any) for the Graph containing the object.
The default style sheet for a Graph3d will be "nobox.gs". A = Graph3D style specification will be overruled by "z_nobox.gs" if a color bar = is requested, otherwise uncritically accepted.

Numpy

Fixed a bug in the NumPy exponentiation routine which was causing a = SEGV when an array of Python longs was an argument.
Fixed memory leak in array to list conversion. (Jonah Lee)
Fixed segv if you attempt to delete an array element. (Warren Focke)
Fixed bug in convolve in multiarray.c (Travis Oliphant)
Added missing #!... to makethis.py in Numerical (Pearu Peterson, who = also reported Gist probs).
Fixed documentation of clip. (Andrew Sterian)
Changed array so that it will accept a typecode of None. This allows = simplifications to indices, asarray, and UserArray. (Keith Junius)
Performance improvement for FFT.py (Konrad Hinsen)

PDB

Added import_array to PyPDB module init routine.

 

Release 8 (December 28, 1998)

Fixes to Matlab.py and Numerical.py in Numerical
Changed CXX_Extensions.py to match 1.5.2's definition of the type table. = Assume this would no longer compile with previous Pythons.

Release 7 (December 21, 1998)

  • Numerical -- described in Numerical/README.htm
  • There have been bugs reported when loading on a platform where = there is a native version of the BLAS or LAPACK. Somehow there is a confusion between = different versions of the routines. Please be sure to alter the file Setup to remove the files = indicated from the build of module lapack_lite and to set the location of your = libraries appropriately if working on such a platform.

Release 6 (October 7, 1998)

  • Numerical -- described in Numerical/README.htm
  • Graphics -- extensive bug fixes in Gist package.
  • PyPDB -- removed dependency missing file EB.py
  • PyHistory -- See xfiles.llnl.gov documentation. columnar_history = folded into history module.
  • CXX -- Beta release, still in development.

Release 5 (August 20, 1998)

  • Changes to Numerical, Gist (see their README files)
  • Removed multiarray comparisons, and changed "bitwise_" = operations to "boolean_" operations, per latest changes in NumPy (Zane = Motteler).
  • Added automatic installation script (first version) (David = Ascher)
  • Changed all references to LEGAL.LLNL in .py and .c files to = Legal.htm (Zane Motteler).
  • Changed Konrad Hinsen's URL in various documents and in = RNGtest2.py to the starship one (Zane Motteler).
  • PyHistory upgraded to version 2. Documentation now available in = this distribution as well as on web site. Function event changed to event_tag for = consistency. Columnar text tags now in history.py rather than a separate module.

Release 4 (June 19, 1998)

  • Added Complex class to CXX.
  • Release 1.3 of Numerical fixes bugs in LAPack, Matrix.py. See = README in Numerical.

Release 3 (June 12, 1998)

  • Fixes to Gist files required by the changes to NumPy.
  • Bug fix to Numerical/Lib/UserArray.py.
  • Renamed CXX/cxxmodule.cxx to cxxsupport.cxx since it isn't a = Python module.

Release 2 (June 5, 1998)

General

  • Several "Setup" files had a space after the *shared* = which mixed up FreeBSD. (Thomas Gellekum).

NumPy

Gist

  • Bug fixes in Gist3D/Demo/gistdemohigh.py, Gist3D/Lib/quadmesh.py = (Thomas Gellekum).

Release 1 (May, 1998)

This release contains all the Python extension packages currently = maintained by LLNL. It supercedes the previous system of distributing our work as part of a = complete Python source tree. 

------=_NextPart_000_0009_01BEE25C.81A38B00-- From minxu@sci.ccny.cuny.edu Wed Aug 11 01:53:17 1999 From: minxu@sci.ccny.cuny.edu (minxu@sci.ccny.cuny.edu) Date: Tue, 10 Aug 1999 20:53:17 -0400 Subject: [Matrix-SIG] Bug of Gist Message-ID: <199908110053.UAA08531@star.future.com> Hi, I try to use OOG is LLNL Graphics to export a postscript file. It does work, but the exported ps file or cgm file is different from what O see in the X-window. (1) when dump=1 is used, the color of exported image changed and some different colors appears same in output postscript files (2) the color of color_bar becomes uniform in output postscript files. It happens for 12beta and 11. Is this a known bug? -- Min Xu City College of NY, CUNY Email: mxu1@email.gc.cuny.edu minxu@sci.ccny.cuny.edu Tel: (O) (212) 650-6865 (O) (212) 650-5046 (H) (212) 690-2119 From zcm@llnl.gov Wed Aug 11 15:32:53 1999 From: zcm@llnl.gov (Zane Motteler) Date: Wed, 11 Aug 1999 07:32:53 -0700 Subject: [Matrix-SIG] Bug of Gist In-Reply-To: <199908110053.UAA08531@star.future.com> Message-ID: Min Xu, You wrote: >I try to use OOG is LLNL Graphics to export a postscript file. It does work, >but the exported ps file or cgm file is different from what O see in the >X-window. > >(1) when dump=1 is used, the color of exported image changed and some >different colors appears same in output postscript files > >(2) the color of color_bar becomes uniform in output postscript files. > > >It happens for 12beta and 11. > >Is this a known bug? If it's a bug, it is not a known one. ;-} I will look into these problems ASAP; thanks for calling them to my attention. Zane -------------------- Zane C. Motteler, Ph. D. Professor Emeritus of Computer Science and Engineering California Polytechnic State University, San Luis Obispo zmottel@calpoly.edu Currently: Computer Scientist Lawrence Livermore National Laboratory P O Box 808, L-038 (Street address 7000 East Avenue, L-038) Livermore, CA 94551-9900 zcm@llnl.gov 925/423-2143, FAX 925/423-9969 From heather@v1.wustl.edu Wed Aug 11 18:31:19 1999 From: heather@v1.wustl.edu (Heather Drury) Date: Wed, 11 Aug 1999 12:31:19 -0500 (CDT) Subject: [Matrix-SIG] bitwise combos of arrays? Message-ID: <199908111731.MAA01466@v1.wustl.edu> Hi, I'm trying to combine 16 binary volumes (10 meg each) into one 16-bit volume. For each of the 16 volumes, the output volume would have the appropriate bit set. In other words, if for voxel "idx" only the 1st, 4th, and 9th volumes had an entry of 255 for voxel "idx" the output voxel at idx would be 0000000100001001. The pseudocode for this would look approximately like: create output volume for j = 0; j < 16; j++ read jth volume for all voxels in volume if voxel [idx] == 255 currentvoxel = output [idx] adjust output [idx] so jth bit is set endif end end Normally, I would do this in "c" (my brain is just coded that way), but I'm trying to use python for more and more programming. So, I have a couple of questions: - Where is the most recent NumPy documentation? - Can I do this kind of thing in python/NumPy? - Can someone point me in the right direction? TIA, Heather Heather Drury heather@v1.wustl.edu Washington University School of Medicine http://v1.wustl.edu Department of Anatomy & Neurobiology Phone: 314-362-4325 660 S. Euclid, MS 8108 FAX: 314-747-4370 St. Louis, MO 63110-1093 From dubois1@llnl.gov Wed Aug 11 18:52:25 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Wed, 11 Aug 1999 10:52:25 -0700 Subject: [Matrix-SIG] bitwise combos of arrays? References: <199908111731.MAA01466@v1.wustl.edu> Message-ID: <99081110555500.06485@almanac> On Wed, 11 Aug 1999, Heather Drury wrote: > > - Where is the most recent NumPy documentation? > - Can I do this kind of thing in python/NumPy? > - Can someone point me in the right direction? > > TIA, > http://xfiles.llnl.gov will get you to the documentation. Numeric has functions such as logical_and, logical_or. But it does not (yet) have a bit vector type. From dubois1@llnl.gov Wed Aug 11 19:02:42 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Wed, 11 Aug 1999 11:02:42 -0700 Subject: [Matrix-SIG] bitwise combos of arrays? addendum References: <99081110555500.06485@almanac> Message-ID: <99081111052600.06750@almanac> On Wed, 11 Aug 1999, Paul F. Dubois wrote: > On Wed, 11 Aug 1999, Heather Drury wrote: > > > > > - Where is the most recent NumPy documentation? > > - Can I do this kind of thing in python/NumPy? > > - Can someone point me in the right direction? > > > > TIA, > > > > http://xfiles.llnl.gov will get you to the documentation. > > Numeric has functions such as logical_and, logical_or. But it does not (yet) > have a bit vector type. > Well, that was true but not helpful. I meant, bitwise_... >>> from Numeric import * >>> a=array([1,3,4]) >>> b=array([1,1,1]) >>> bitwise_and(a,b) array([1, 1, 0]) >>> From da@ski.org Wed Aug 11 19:03:39 1999 From: da@ski.org (David Ascher) Date: Wed, 11 Aug 1999 11:03:39 -0700 (Pacific Daylight Time) Subject: [Matrix-SIG] bitwise combos of arrays? In-Reply-To: <199908111731.MAA01466@v1.wustl.edu> Message-ID: > I'm trying to combine 16 binary volumes (10 meg each) into one 16-bit > volume. For each of the 16 volumes, the output volume would have the > appropriate bit set. In other words, if for voxel "idx" only the 1st, > 4th, and 9th volumes had an entry of 255 for voxel "idx" the output > voxel at idx would be 0000000100001001. As paul mentioned, there are bitwise operations. You could do something like the following, assuming that volumes is a list of 16 arrays of shape 'volumeshape' and containing zeros and ones (they will not be one-bit, arrays, alas, as Paul also mentioned). newvolume = zeros(volumeshape, Int16) shift = 0 for volume in volumes: newvolume = bitwise_or(newvolume, volume << shift) shift = shift + 1 (where I most probably got the direction of the shift wrong, cf. Murphy's law) The key concept for the C programmer is to use 'en-masse' opertions like bitwise_or which operate on arrays, as opposed to thinking about things in terms of loops. --david From hinsen@cnrs-orleans.fr Wed Aug 11 21:12:08 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 11 Aug 1999 22:12:08 +0200 Subject: [Matrix-SIG] Question for the NumPy gurus In-Reply-To: (message from Travis Oliphant on Thu, 5 Aug 1999 01:29:16 -0500 (CDT)) References: Message-ID: <199908112012.WAA26374@chinon.cnrs-orleans.fr> I have two arrays with in1.shape = (N,M,P) and in2.shape = (N,P) Is there a fast way (no Python for loop) to perform an innerproduct element-wise along the first dimension to return an array of shape (N,M)? I was hoping for the equivalent of Not with dot(), but it's easy enough to do it in two steps: out = add.reduce(in1*in2[:,Numeric.NewAxis,:], 1) (not tested, of course, after all I am on vacation!) -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From heather@v1.wustl.edu Thu Aug 12 22:45:51 1999 From: heather@v1.wustl.edu (Heather Drury) Date: Thu, 12 Aug 1999 16:45:51 -0500 (CDT) Subject: [Matrix-SIG] NumPy installation Message-ID: <199908122145.QAA02996@v1.wustl.edu> Hi, I just downloaded the most recent NumPy installation (LLNLDistribution11) and from the Numerical subdirectory did a "python makethis.py" and then a "python installthis.py" and now I get: v1 19# python Python 1.5.2b2 (#44, Mar 24 1999, 14:52:32) [C] on irix646 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>import Numeric Traceback (innermost last): File "", line 1, in ? File "/usr/local/lib/python1.5/site-packages/numeric/Numeric.py", line 45, in ? cross_correlate = multiarray.cross_correlate AttributeError: cross_correlate I suppose this should be in multiarraymodule.c, but I don't find it. Hints? Thanks, heather -- Heather Drury heather@v1.wustl.edu Washington University School of Medicine http://v1.wustl.edu Department of Anatomy & Neurobiology Phone: 314-362-4325 660 S. Euclid, MS 8108 FAX: 314-747-4370 St. Louis, MO 63110-1093 From heather@v1.wustl.edu Thu Aug 12 23:14:15 1999 From: heather@v1.wustl.edu (Heather Drury) Date: Thu, 12 Aug 1999 17:14:15 -0500 (CDT) Subject: [Matrix-SIG] NumPy installation: follow up In-Reply-To: <199908122145.QAA02996@v1.wustl.edu> from "Heather Drury" at Aug 12, 99 04:45:51 pm Message-ID: <199908122214.RAA03324@v1.wustl.edu> Hi, I'm sure I'm not supposed to respond to my own post, but as a follow up: if I comment out the "cross_correlate" lines in Numeric.py: cross_correlate = multiarray.cross_correlate def convolve(a,v,mode=0): if (len(v) > len(a)): temp = a a = v v = temp del temp return cross_correlate(a,asarray(v)[::-1],mode) then I can get it to run but I'm still not able to use any of the "bitwise" ufuncs (the original reason for the upgrade): Python 1.5.2b2 (#44, Mar 24 1999, 14:52:32) [C] on irix646 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import Numeric >>> print Numeric.__version__ 11 >>> print Numeric.less >>> print Numeric.logical_not >>> print Numeric.bitwise_and Traceback (innermost last): File "", line 1, in ? AttributeError: bitwise_and >>> print Numeric.bitwise_xor Traceback (innermost last): File "", line 1, in ? AttributeError: bitwise_xor So, what's going on here? Heather > I just downloaded the most recent NumPy installation > (LLNLDistribution11) and from the Numerical > subdirectory did a "python makethis.py" and then a > "python installthis.py" and now I get: > > v1 19# python > Python 1.5.2b2 (#44, Mar 24 1999, 14:52:32) [C] on irix646 > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam > >>import Numeric > Traceback (innermost last): > File "", line 1, in ? > File "/usr/local/lib/python1.5/site-packages/numeric/Numeric.py", line 45, in ? > cross_correlate = multiarray.cross_correlate > AttributeError: cross_correlate > > I suppose this should be in multiarraymodule.c, but I don't find it. > > Hints? > > Thanks, > > heather > > > -- > > Heather Drury heather@v1.wustl.edu > Washington University School of Medicine http://v1.wustl.edu > Department of Anatomy & Neurobiology Phone: 314-362-4325 > 660 S. Euclid, MS 8108 FAX: 314-747-4370 > St. Louis, MO 63110-1093 > > _______________________________________________ > Matrix-SIG maillist - Matrix-SIG@python.org > http://www.python.org/mailman/listinfo/matrix-sig > From taylor@rhino.llnl.gov Fri Aug 13 23:29:53 1999 From: taylor@rhino.llnl.gov (Lee Taylor) Date: Fri, 13 Aug 1999 15:29:53 -0700 (PDT) Subject: [Matrix-SIG] LLNLDistribution12beta problem Message-ID: I was compiling the LLNLDistribution12beta Numerical extension into python (i.e. not as a shared library) on an alpha running OSF1 4.0 (OSF1 tc02 V4.0 878 alpha), I ran into the following problem: ranlib libpython1.5.a ar: Warning: ignoring second definition of PyExtensionClassCAPI defined in archive true cd Modules; gmake OPT="-g" VERSION="1.5" \ prefix="/g/g17/taylor/python/beta12" exec_prefix="/g/g17/taylor/python/beta12" \ LIBRARY=../libpython1.5.a link gmake[1]: Entering directory `/g/g17/taylor/python/beta12/Python-1.5.2/Modules' cc python.o \ ../libpython1.5.a -lnet -lm -o python ld: ../libpython1.5.a(_numpymodule.o): PyExtensionClassCAPI: multiply defined After looking around, it seems that DEFINE_EXTENSIONCLASS should not be defined in _numpymodule.c or arrayobject.h. It is defined in ExtensionClass.c and then used in ExtensionClass.h as: #ifdef DEFINE_EXTENSIONCLASS struct ExtensionClassCAPIstruct *PyExtensionClassCAPI = NULL; #else extern struct ExtensionClassCAPIstruct *PyExtensionClassCAPI; #endif By defining DEFINE_EXTENSIONCLASS too many times more definitions are created instead of declarations. I also tried this on an IBM running AIX4.3, and both versions (original and patched) seem to work. *** old/_numpymodule.c Fri Aug 13 12:05:07 1999 --- _numpymodule.c Fri Aug 13 13:21:28 1999 *************** *** 1,7 **** #include "Python.h" #define _ARRAY_MODULE - #define DEFINE_EXTENSIONCLASS #include "arrayobject.h" #include "ExtensionClassMacros.h" #define _UFUNC_MODULE --- 1,6 ---- *** old/arrayobject.h Fri Aug 13 12:04:57 1999 --- arrayobject.h Fri Aug 13 13:21:51 1999 *************** *** 5,11 **** #endif #ifndef _ARRAY_MODULE - #define DEFINE_EXTENSIONCLASS #ifdef COMPILING_NUMPY #include "ExtensionClassMacros.h" #else --- 5,10 ---- Lee Taylor ltaylor@llnl.gov From kernr@ncifcrf.gov Sat Aug 14 19:15:45 1999 From: kernr@ncifcrf.gov (Robert Kern) Date: Sat, 14 Aug 1999 14:15:45 -0400 Subject: [Matrix-SIG] sparsemodule 0.2, MMTK 2.0b9 Win32 binaries Message-ID: <37B5B251.25060A8D@mail.ncifcrf.gov> ... are now available. http://starship.python.net/crew/kernr/binaries/Binaries.html -- Robert Kern | ----------------------|"In the fields of Hell where the grass grows high This space | Are the graves of dreams allowed to die." intentionally | - Richard Harter left blank. | From patd@wam.umd.edu Tue Aug 17 14:41:30 1999 From: patd@wam.umd.edu (Patrick Dahiroc) Date: Tue, 17 Aug 1999 09:41:30 -0400 (EDT) Subject: [Matrix-SIG] Modules: Shared Lib In-Reply-To: <37B5B251.25060A8D@mail.ncifcrf.gov> Message-ID: hi all i'm having problems making a shared object for a python extension module that i wrote. below is the steps i took to make the module. (i just followed the steps in the GCC-HOWTO) gcc -fPIC -Imyinc -Lmylib -c maptrans.c gcc -shared -Wl,-soname,maptrans.so.1 -o maptrans.so.1.0 maptrans.o ln -s maptrans.so.1.0 maptrans.so.1 ln -s maptrans.so.1 maptrans.so maptrans.c wraps a map transformation routine in one of my library. when i run python and 'import maptrans', i get an error that says something like ' undefined init function (initmaptrans) '. but in maptrans.c i do have the function static PyObject *wrap_mapconvert(self, args){ char *blah, *BLAH; PyArg_ParseTuple(args, "ss", &blah, &BLAH) MAPCONV(BLAH,blah); /* MAPCONV - function from mylib/libMAP.a */ return PyBuildValue("s",blah); } static PyMethodDef maptrans_method[] = { {"mapconvert", wrap_mapconvert,METH_VARARGS}, {NULL,NUL}}; void initmaptrans(){ Py_InitModule("maptrans",maptrans_method)} i have never made a shared library nor extended python before. any help you could suggest is much appreciated. TYA pd From dubois1@llnl.gov Tue Aug 17 17:13:59 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Tue, 17 Aug 1999 09:13:59 -0700 Subject: [Matrix-SIG] Re: NumPy changes References: Message-ID: <001401bee8cb$b48d0b20$f4160218@plstn1.sfba.home.com> Lee Taylor sent me some changes we discussed so that PyArray_From... routines are factored so that one has the ability to create the array descriptor oneself. This proved useful to Lee in a database handler. I updated the repo with Lee's changes after testing on Windows. However, I did not include his change about DEFINE_EXTENSIONCLASS because they don't work on Windows. I think the difference is that Lee loads his code with Numerical statically, while David and I have been testing dynamically. The static load gets multiple declarations of the same external. Clearly something is wrong here. Maybe we need help. Konrad, this CAPI trick was your idea, wasn't it? Do you see what to do? Maybe this is related to the bug report about reloads causing trouble. (For the purposes of understanding the problem, the beta release is fine; Lee's changes are unrelated.) ----- Original Message ----- From: Lee Taylor To: Paul Dubois Sent: Friday, August 13, 1999 4:15 PM Subject: NumPy changes > > Paul, > Attached are the changes I talked about to allow arrays > of structures to be index via NumPy. > > I don't think that this will address the bit array problem. > All of the indexing is done with byte addresses because that's > the smallest item a pointer can point at. The bit array would > require some shifting and masking. > > Also, the IBM compiler comments about line 1607 in arrayobject.c > having a /* within a comment. > > Finally, these files have the DEFINE_EXTENSIONCLASS removed > from arrayobject.h and _numpymodule.c as describe in my mail > to the MATRIX-sig. > > Lee > > ---------------------------------------------------------------------------- ---- > #include "Python.h" > > #define _ARRAY_MODULE > #include "arrayobject.h" > #include "ExtensionClassMacros.h" > #define _UFUNC_MODULE > #include "ufuncobject.h" > > /* Table of functions defined in the module */ > > static PyMethodDef numpy_methods[] = { > {NULL, NULL} /* sentinel */ > }; > > /* Module initialization */ > > void > init_numpy() > { > PyObject *m, *d; > static void *PyArray_API[PyArray_API_pointers]; > static void *Py_UFunc_API[PyUFunc_API_pointers]; > > /* Create the module and add the functions */ > PyArray_Type.ob_type = &PyType_Type; > PyUFunc_Type.ob_type = &PyType_Type; > > m = Py_InitModule("_numpy", numpy_methods); > d = PyModule_GetDict(m); > > /* Initialize C API pointer arrays and store them in module */ > PyArray_API[PyArray_Type_NUM] = (void *)&PyArray_Type; > PyArray_API[PyArray_SetNumericOps_NUM] = (void *)&PyArray_SetNumericOps; > PyArray_API[PyArray_INCREF_NUM] = (void *)&PyArray_INCREF; > PyArray_API[PyArray_XDECREF_NUM] = (void *)&PyArray_XDECREF; > #if 0 > PyArray_API[PyArrayError_NUM] = (void *)&PyArrayError; > #endif > PyArray_API[PyArray_SetStringFunction_NUM] = > (void *)&PyArray_SetStringFunction; > PyArray_API[PyArray_DescrFromType_NUM] = (void *)&PyArray_DescrFromType; > PyArray_API[PyArray_Cast_NUM] = (void *)&PyArray_Cast; > PyArray_API[PyArray_CanCastSafely_NUM] = (void *)&PyArray_CanCastSafely; > PyArray_API[PyArray_ObjectType_NUM] = (void *)&PyArray_ObjectType; > PyArray_API[_PyArray_multiply_list_NUM] = (void *)&_PyArray_multiply_list; > PyArray_API[PyArray_Size_NUM] = (void *)&PyArray_Size; > PyArray_API[PyArray_FromDims_NUM] = (void *)&PyArray_FromDims; > PyArray_API[PyArray_FromDimsAndData_NUM] = (void *)&PyArray_FromDimsAndData; > PyArray_API[PyArray_FromDimsAndDataAndDescr_NUM] = (void *)&PyArray_FromDimsAndDataAndDescr; > PyArray_API[PyArray_ContiguousFromObject_NUM] = > (void *)&PyArray_ContiguousFromObject; > PyArray_API[PyArray_CopyFromObject_NUM] = (void *)&PyArray_CopyFromObject; > PyArray_API[PyArray_FromObject_NUM] = (void *)&PyArray_FromObject; > PyArray_API[PyArray_Return_NUM] = (void *)&PyArray_Return; > PyArray_API[PyArray_Reshape_NUM] = (void *)&PyArray_Reshape; > PyArray_API[PyArray_Copy_NUM] = (void *)&PyArray_Copy; > PyArray_API[PyArray_Take_NUM] = (void *)&PyArray_Take; > PyArray_API[PyArray_As1D_NUM] = (void *)&PyArray_As1D; > PyArray_API[PyArray_As2D_NUM] = (void *)&PyArray_As2D; > PyArray_API[PyArray_Free_NUM] = (void *)&PyArray_Free; > PyDict_SetItemString(d, "_ARRAY_API", > PyCObject_FromVoidPtr((void *)PyArray_API, NULL)); > > Py_UFunc_API[PyUFunc_Type_NUM] = (void *)&PyUFunc_Type; > Py_UFunc_API[PyUFunc_FromFuncAndData_NUM] = (void *)&PyUFunc_FromFuncAndData; > Py_UFunc_API[PyUFunc_GenericFunction_NUM] = (void *)&PyUFunc_GenericFunction; > Py_UFunc_API[PyUFunc_f_f_As_d_d_NUM] = (void *)&PyUFunc_f_f_As_d_d; > Py_UFunc_API[PyUFunc_d_d_NUM] = (void *)&PyUFunc_d_d; > Py_UFunc_API[PyUFunc_F_F_As_D_D_NUM] = (void *)&PyUFunc_F_F_As_D_D; > Py_UFunc_API[PyUFunc_D_D_NUM] = (void *)&PyUFunc_D_D; > Py_UFunc_API[PyUFunc_O_O_NUM] = (void *)&PyUFunc_O_O; > Py_UFunc_API[PyUFunc_ff_f_As_dd_d_NUM] = (void *)&PyUFunc_ff_f_As_dd_d; > Py_UFunc_API[PyUFunc_dd_d_NUM] = (void *)&PyUFunc_dd_d; > Py_UFunc_API[PyUFunc_FF_F_As_DD_D_NUM] = (void *)&PyUFunc_FF_F_As_DD_D; > Py_UFunc_API[PyUFunc_DD_D_NUM] = (void *)&PyUFunc_DD_D; > Py_UFunc_API[PyUFunc_OO_O_NUM] = (void *)&PyUFunc_OO_O; > Py_UFunc_API[PyUFunc_O_O_method_NUM] = (void *)&PyUFunc_O_O_method; > #if 0 > Py_UFunc_API[PyArray_Map_NUM] = (void *)&PyArray_Map; > #endif > PyDict_SetItemString(d, "_UFUNC_API", > PyCObject_FromVoidPtr((void *)Py_UFunc_API, NULL)); > > PyExtensionClass_Export(d, "multiarray", PyArray_Type); > > PyDict_SetItemString(d, "_UseExtensionClass", PyInt_FromLong(1)); > /* Check for errors */ > if (PyErr_Occurred()) > Py_FatalError("can't initialize module _numpy"); > } > ---------------------------------------------------------------------------- ---- > /* > Python Array Object -- Provide multidimensional arrays as a basic > object type in python. > > Copyright (c) 1995, 1996, 1997 Jim Hugunin, hugunin@mit.edu > See file COPYING for details. > > > These arrays are primarily designed for supporting multidimensional, > homogeneous arrays of basic C numeric types. They also can support > arrays of arbitrary Python Objects, if you are willing to sacrifice > performance for heterogeneity. > */ > > /* $Id: arrayobject.c,v 0.16 1999/07/11 21:42:29 barnard Exp $ */ > > #include "Python.h" > /*Silly trick to make dll library work right under Win32*/ > #ifdef MS_WIN32 > #undef DL_IMPORT > #define DL_IMPORT(RTYPE) __declspec(dllexport) RTYPE > #endif > #define _ARRAY_MODULE > #define _UFUNC_MODULE > #include "arrayobject.h" > #include "ExtensionClassMacros.h" > #include "ufuncobject.h" > > #define OBJECT(O) ((PyObject*)(O)) > > /* There are several places in the code where an array of dimensions is */ > /* allocated statically. This is the size of that static allocation. I */ > /* can't come up with any reasonable excuse for a larger array than this. */ > > #define MAX_DIMS 40 > > /* Helper Functions */ > extern int _PyArray_multiply_list(int *l1, int n) { > int s=1, i=0; > while (i < n) s *= l1[i++]; > return s; > } > extern int _PyArray_compare_lists(int *l1, int *l2, int n) { > int i; > for(i=0;i if (l1[i] != l2[i]) return 0; > } > return 1; > } > > /* These can be cleaned up */ > #define SIZE(mp) (_PyArray_multiply_list((mp)->dimensions, (mp)->nd)) > #define NBYTES(mp) ((mp)->descr->elsize * SIZE(mp)) > /* Obviously this needs some work. */ > #define ISCONTIGUOUS(m) ((m)->flags & CONTIGUOUS) > > #define PyArray_CONTIGUOUS(m) (ISCONTIGUOUS(m) ? Py_INCREF(m), m : \ > (PyArrayObject *)(PyArray_ContiguousFromObject((PyObject *)(m), (m)->descr->type_num, 0,0))) > > int do_sliced_copy(char *dest, int *dest_strides, int *dest_dimensions, int dest_nd, > char *src, int *src_strides, int *src_dimensions, int src_nd, > int elsize, int copies) { > int i, j; > > if (src_nd == 0 && dest_nd == 0) { > for(j=0; j memcpy(dest, src, elsize); > dest += elsize; > } > return 0; > } > > if (dest_nd > src_nd) { > for(i=0; i<*dest_dimensions; i++, dest += *dest_strides) { > if (do_sliced_copy(dest, dest_strides+1, dest_dimensions+1, dest_nd-1, > src, src_strides, src_dimensions, src_nd, elsize, copies) == -1) return -1; > } > return 0; > } > > if (dest_nd == 1) { > if (*dest_dimensions != *src_dimensions) { > PyErr_SetString(PyExc_ValueError, "matrices are not aligned for copy"); > return -1; > } > for(i=0; i<*dest_dimensions; i++, src += *src_strides) { > for(j=0; j memcpy(dest, src, elsize); > dest += *dest_strides; > } > } > } else { > for(i=0; i<*dest_dimensions; i++, dest += *dest_strides, src += *src_strides) { > if (do_sliced_copy(dest, dest_strides+1, dest_dimensions+1, dest_nd-1, > src, src_strides+1, src_dimensions+1, src_nd-1, elsize, copies) == -1) return -1; > } > } > return 0; > } > > int optimize_slices(int **dest_strides, int **dest_dimensions, int *dest_nd, > int **src_strides, int **src_dimensions, int *src_nd, > int *elsize, int *copies) { > while (*src_nd > 0) { > if (((*dest_strides)[*dest_nd-1] == *elsize) && ((*src_strides)[*src_nd-1] == *elsize)) { > if ((*dest_dimensions)[*dest_nd-1] != (*src_dimensions)[*src_nd-1]) { > PyErr_SetString(PyExc_ValueError, "matrices are not aligned for copy"); > return -1; > } > *elsize *= (*dest_dimensions)[*dest_nd-1]; > *dest_nd-=1; *src_nd-=1; > } else { > break; > } > } > if (*src_nd == 0) { > while (*dest_nd > 0) { > if (((*dest_strides)[*dest_nd-1] == *elsize)) { > *copies *= (*dest_dimensions)[*dest_nd-1]; > *dest_nd-=1; > } else { > break; > } > } > } > return 0; > } > > static char *contiguous_data(PyArrayObject *src) { > int dest_strides[MAX_DIMS], *dest_strides_ptr; > int *dest_dimensions=src->dimensions; > int dest_nd=src->nd; > int *src_strides = src->strides; > int *src_dimensions=src->dimensions; > int src_nd=src->nd; > int elsize=src->descr->elsize; > int copies=1; > int ret, i; > int stride=elsize; > char *new_data; > > for(i=dest_nd-1; i>=0; i--) { > dest_strides[i] = stride; > stride *= dest_dimensions[i]; > } > > dest_strides_ptr = dest_strides; > > if (optimize_slices(&dest_strides_ptr, &dest_dimensions, &dest_nd, &src_strides, &src_dimensions, &src_nd, > &elsize, &copies) == -1) return NULL; > > new_data = (char *)malloc(stride); > > ret = do_sliced_copy(new_data, dest_strides_ptr, dest_dimensions, dest_nd, > src->data, src_strides, src_dimensions, src_nd, > elsize, copies); > > if (ret != -1) { return new_data; } > else { free(new_data); return NULL; } > } > > > /* Used for arrays of python objects to increment the reference count of */ > /* every python object in the array. */ > extern int PyArray_INCREF(PyArrayObject *mp) { > int i, n; > PyObject **data; > > if (mp->descr->type_num != PyArray_OBJECT) return 0; > > if (ISCONTIGUOUS(mp)) { > data = (PyObject **)mp->data; > } else { > if ((data = (PyObject **)contiguous_data(mp)) == NULL) return -1; > } > > n = SIZE(mp); > for(i=0; i > if (!ISCONTIGUOUS(mp)) free(data); > > return 0; > } > > extern int PyArray_XDECREF(PyArrayObject *mp) { > int i, n; > PyObject **data; > > if (mp->descr->type_num != PyArray_OBJECT) return 0; > > if (ISCONTIGUOUS(mp)) { > data = (PyObject **)mp->data; > } else { > if ((data = (PyObject **)contiguous_data(mp)) == NULL) return -1; > } > > n = SIZE(mp); > for(i=0; i > if (!ISCONTIGUOUS(mp)) free(data); > > return 0; > } > > /* Including a C file is admittedly ugly. Look at the C file if you want to */ > /* see something even uglier. This is computer generated code. */ > #include "arraytypes.c" > > char *index2ptr(PyArrayObject *mp, int i) { > if (i==0 && (mp->nd == 0 || mp->dimensions[0] > 0)) return mp->data; > > if (mp->nd>0 && i>0 && i < mp->dimensions[0]) { > return mp->data+i*mp->strides[0]; > } > PyErr_SetString(PyExc_IndexError,"index out of bounds"); > return NULL; > } > > extern int PyArray_Size(PyObject *op) { > if (PyArray_Check(op)) { > return SIZE((PyArrayObject *)op); > } else { > return 0; > } > } > > int PyArray_CopyArray(PyArrayObject *dest, PyArrayObject *src) { > int *dest_strides=dest->strides; > int *dest_dimensions=dest->dimensions; > int dest_nd=dest->nd; > int *src_strides = src->strides; > int *src_dimensions=src->dimensions; > int src_nd=src->nd; > int elsize=src->descr->elsize; > int copies=1; > int ret; > > if (src->nd > dest->nd) { > PyErr_SetString(PyExc_ValueError, "array too large for destination"); > return -1; > } > if (dest->descr->type_num != src->descr->type_num) { > PyErr_SetString(PyExc_ValueError, "can only copy from a array of the same type."); > return -1; > } > > if (optimize_slices(&dest_strides, &dest_dimensions, &dest_nd, &src_strides, &src_dimensions, &src_nd, > &elsize, &copies) == -1) return -1; > > ret = do_sliced_copy(dest->data, dest_strides, dest_dimensions, dest_nd, > src->data, src_strides, src_dimensions, src_nd, > elsize, copies); > > if (ret != -1) { ret = PyArray_INCREF(dest); } > return ret; > } > > int PyArray_CopyObject(PyArrayObject *dest, PyObject *src_object) { > PyArrayObject *src; > PyObject *tmp; > int ret, n_new, n_old; > char *new_string; > > /* Special function added here to try and make arrays of strings work out. */ > if ((dest->descr->type_num == PyArray_CHAR) && dest->nd > 0 && PyString_Check(src_object)) { > n_new = dest->dimensions[dest->nd-1]; > n_old = PyString_Size(src_object); > if (n_new > n_old) { > new_string = (char *)malloc(n_new*sizeof(char)); > memcpy(new_string, PyString_AS_STRING((PyStringObject *)src_object), n_old); > memset(new_string+n_old, ' ', n_new-n_old); > tmp = PyString_FromStringAndSize(new_string, n_new); > free(new_string); > src_object = tmp; > } > } > src = (PyArrayObject *)PyArray_FromObject(src_object, > dest->descr->type_num, 0, > dest->nd); > if (src == NULL) return -1; > > ret = PyArray_CopyArray(dest, src); > Py_DECREF(src); > return ret; > } > > /* This is the basic array allocation function. */ > PyObject *PyArray_FromDimsAndDataAndDescr(int nd, int *d, PyArray_Descr *descr, > char *data) { > PyArrayObject *self; > int i,sd; > int *dimensions, *strides; > int flags=CONTIGUOUS | OWN_DIMENSIONS | OWN_STRIDES; > > dimensions = strides = NULL; > > if (nd < 0) { > PyErr_SetString(PyExc_ValueError, "number of dimensions must be >= 0"); > return NULL; > } > > if (nd > 0) { > if ((dimensions = (int *)malloc(nd*sizeof(int))) == NULL) { > PyErr_SetString(PyExc_MemoryError, "can't allocate memory for array"); > goto fail; > } > if ((strides = (int *)malloc(nd*sizeof(int))) == NULL) { > PyErr_SetString(PyExc_MemoryError, "can't allocate memory for array"); > goto fail; > } > memcpy(dimensions, d, sizeof(int)*nd); > } > > sd = descr->elsize; > for(i=nd-1;i>=0;i--) { > if (flags & OWN_STRIDES) strides[i] = sd; > if (dimensions[i] < 0) { > PyErr_SetString(PyExc_ValueError, "negative dimensions are not allowed"); > goto fail; > } > /* > This may waste some space, but it seems to be > (unsuprisingly) unhealthy to allow strides that are > longer than sd. > */ > sd *= dimensions[i] ? dimensions[i] : 1; > /* sd *= dimensions[i]; */ > } > > /* Make sure we're alligned on ints. */ > sd += sizeof(int) - sd%sizeof(int); > > if (data == NULL) { > if ((data = (char *)malloc(sd)) == NULL) { > PyErr_SetString(PyExc_MemoryError, "can't allocate memory for array"); > goto fail; > } > flags |= OWN_DATA; > } > > /* > if((self = PyObject_NEW(PyArrayObject, &PyArray_Type)) == NULL) goto fail; > */ > > if ((self = (PyArrayObject *)PyObject_CallObject(OBJECT(&PyArray_Type), > Py_BuildValue("()"))) > == NULL) goto fail; > if (flags & OWN_DATA) memset(data, 0, sd); > > self->data=data; > self->dimensions = dimensions; > self->strides = strides; > self->nd=nd; > self->descr=descr; > self->base = (PyObject *)NULL; > self->flags = flags; > > return (PyObject*)self; > > fail: > > if (flags & OWN_DATA) free(data); > if (dimensions != NULL) free(dimensions); > if (strides != NULL) free(strides); > return NULL; > } > > PyObject *PyArray_FromDimsAndData(int nd, int *d, int type, char *data) { > PyArray_Descr *descr; > if ((descr = PyArray_DescrFromType(type)) == NULL) return NULL; > return PyArray_FromDimsAndDataAndDescr(nd, d, descr, data); > } > > PyObject *PyArray_FromDims(int nd, int *d, int type) { > PyArray_Descr *descr; > if ((descr = PyArray_DescrFromType(type)) == NULL) return NULL; > return PyArray_FromDimsAndDataAndDescr(nd, d, descr, NULL); > } > > > extern PyObject *PyArray_Copy(PyArrayObject *m1) { > PyArrayObject *ret = > (PyArrayObject *)PyArray_FromDims(m1->nd, m1->dimensions, m1->descr->type_num); > > if (PyArray_CopyArray(ret, m1) == -1) return NULL; > > return (PyObject *)ret; > } > > static void array_dealloc(PyArrayObject *self) { > if(self->base) Py_DECREF(self->base); > > if (self->flags & OWN_DATA) { > PyArray_XDECREF(self); > free(self->data); > } > > if (self->flags & OWN_DIMENSIONS && self->dimensions != NULL) free(self->dimensions); > if (self->flags & OWN_STRIDES && self->strides != NULL) free(self->strides); > > PyMem_DEL(self); > } > > /* Code to handle accessing Array objects as sequence objects */ > static int array_length(PyArrayObject *self) { > if (self->nd != 0) { > return self->dimensions[0]; > } else { > return 1; /* Because a[0] works on 0d arrays. */ > } > } > > static PyObject *array_item(PyArrayObject *self, int i) { > char *item; > > if ((item = index2ptr(self, i)) == NULL) return NULL; > > if(self->nd > 1) { > PyArrayObject *r; > if ((r = (PyArrayObject *)PyObject_CallObject(OBJECT(&PyArray_Type), NULL)) == NULL) return NULL; > > r->nd = self->nd-1; > r->dimensions = self->dimensions+1; > r->strides = self->strides+1; > r->descr = self->descr; > r->data = item; > r->base = (PyObject *)self; > r->flags = self->flags & CONTIGUOUS; > Py_INCREF(self); > return (PyObject*)r; > } else { > return self->descr->getitem(item); > } > } > > extern PyObject * PyArray_Item(PyObject *op, int i) { > if (PyArray_Check(op)) { > return array_item((PyArrayObject *)op, i); > } else { > PyErr_SetString(PyExc_ValueError, "Not an array object"); > return NULL; > } > } > > extern PyObject *PyArray_Return(PyArrayObject *mp) { > PyObject *op; > > if (PyErr_Occurred()) { > if (mp != NULL) Py_DECREF(mp); > return NULL; > } > if (mp->nd == 0) { > op = array_item(mp, 0); > Py_DECREF(mp); > return op; > } else { > return (PyObject *)mp; > } > } > > static PyObject * array_slice(PyArrayObject *self, int ilow, int ihigh) { > PyArrayObject *r; > int l; > char *data; > > if (self->nd == 0) { > PyErr_SetString(PyExc_ValueError, "can't slice a scalar"); > return NULL; > } > > l=self->dimensions[0]; > if (ihigh < 0) ihigh += l; > if (ilow < 0) ilow += l; > if (ilow < 0) ilow = 0; > else if (ilow > l) ilow = l; > if (ihigh < 0) ihigh = 0; > else if (ihigh > l) ihigh = l; > if (ihigh < ilow) ihigh = ilow; > > if (ihigh != ilow) { > data = index2ptr(self, ilow); > if (data == NULL) return NULL; > } else { > data = self->data; > } > > self->dimensions[0] = ihigh-ilow; > r = (PyArrayObject *)PyArray_FromDimsAndDataAndDescr(self->nd, self->dimensions, self->descr, data); > self->dimensions[0] = l; > > if (!ISCONTIGUOUS(self)) r->flags &= ~CONTIGUOUS; > memcpy(r->strides, self->strides, sizeof(int)*self->nd); > r->base = (PyObject *)self; > Py_INCREF(self); > > return (PyObject *)r; > } > > /* These will need some serious work when I feel like it. */ > static int array_ass_item(PyArrayObject *self, int i, PyObject *v) { > PyObject *c=NULL; > PyArrayObject *tmp; > char *item; > int ret; > > if (v == NULL) { > PyErr_SetString(PyExc_ValueError, "Can't delete array elements."); > return -1; > } > > if (i < 0) i = i+self->dimensions[0]; > > if (self->nd > 1) { > if((tmp = (PyArrayObject *)array_item(self, i)) == NULL) return -1; > ret = PyArray_CopyObject(tmp, v); > Py_DECREF(tmp); > return ret; > } > > if ((item = index2ptr(self, i)) == NULL) return -1; > > if(self->descr->type_num != PyArray_OBJECT && PyString_Check(v) && PyObject_Length(v) == 1) { > char *s; > if ((s=PyString_AsString(v)) == NULL) return -1; > if(self->descr->type == 'c') { > ((char*)self->data)[i]=*s; > return 0; > } > if(s) c=PyInt_FromLong((long)*s); > if(c) v=c; > } > > self->descr->setitem(v, item); > > if(c) Py_DECREF(c); > if(PyErr_Occurred()) return -1; > return 0; > } > > > static int array_ass_slice(PyArrayObject *self, int ilow, int ihigh, PyObject *v) { > int ret; > PyArrayObject *tmp; > > if (v == NULL) { > PyErr_SetString(PyExc_ValueError, "Can't delete array elements."); > return -1; > } > if ((tmp = (PyArrayObject *)array_slice(self, ilow, ihigh)) == NULL) return -1; > ret = PyArray_CopyObject(tmp, v); > Py_DECREF(tmp); > > return ret; > } > > /* -------------------------------------------------------------- */ > static int slice_GetIndices(PySliceObject *r, int length, > int *start, int *stop, int *step) > { > if (r->step == Py_None) { > *step = 1; > } else { > if (!PyInt_Check(r->step)) return -1; > *step = PyInt_AsLong(r->step); > } > if (r->start == Py_None) { > *start = *step < 0 ? length-1 : 0; > } else { > if (!PyInt_Check(r->start)) return -1; > *start = PyInt_AsLong(r->start); > if (*start < 0) *start += length; > } > if (r->stop == Py_None) { > *stop = *step < 0 ? -1 : length; > } else { > if (!PyInt_Check(r->stop)) return -1; > *stop = PyInt_AsLong(r->stop); > if (*stop < 0) *stop += length; > } > > if (*start > (length-1)) *start = length; > if (*start < 0) *start = 0; > if (*stop < -1) *stop = -1; > else if (*stop > length) *stop = length; > return 0; > } > > > static int get_slice(PyObject *op, int max, int *np, int *sp) { > int start, stop, step; > > if (PySlice_Check(op)) { > if (slice_GetIndices((PySliceObject *)op, max, > &start, &stop, &step) == -1) return -1; > > if (step != 0) { > if (step < 0) *np = (stop-start+1+step)/step; > else *np = (stop-start-1+step)/step; > } else { > if (stop == start) { > *np = 0; step = 1; > } > else return -1; > } > if (*np < 0) *np = 0; > *sp = step; > return start; > } > return -1; > } > > #define PseudoIndex -1 > #define RubberIndex -2 > #define SingleIndex -3 > > static int parse_subindex(PyObject *op, int *step_size, int *n_steps, int max) { > int i, tmp; > > if (op == Py_None) { > *n_steps = PseudoIndex; > return 0; > } > > if (op == Py_Ellipsis) { > *n_steps = RubberIndex; > return 0; > } > > if (PySlice_Check(op)) { > if ((i = get_slice(op, max, n_steps, step_size)) >= 0) { > return i; > } else { > PyErr_SetString(PyExc_IndexError, "invalid slice"); > return -1; > } > } > > if (PyInt_Check(op)) { > *n_steps=SingleIndex; > *step_size=0; > tmp = PyInt_AsLong(op); > if (tmp < 0) tmp += max; > if (tmp >= max || tmp < 0) { > PyErr_SetString(PyExc_IndexError, "invalid index"); > return -1; > } > return tmp; > } > PyErr_SetString(PyExc_IndexError, > "each subindex must be either a slice, an integer, Ellipsis, or NewAxis"); > return -1; > } > > static int parse_index(PyArrayObject *self, PyObject *op, > int *dimensions, int *strides, int *offset_ptr) { > int i, j, n; > int nd_old, nd_new, start, offset, n_add, n_pseudo; > int step_size, n_steps; > PyObject *op1; > int is_slice; > > if (PySlice_Check(op) || op == Py_Ellipsis) { > n = 1; > op1 = op; > Py_INCREF(op); /* this relies on the fact that n==1 for loop below */ > is_slice = 1; > } else { > if (!PySequence_Check(op)) { > PyErr_SetString(PyExc_IndexError, "index must be either an int or a sequence"); > return -1; > } > n = PySequence_Length(op); > is_slice = 0; > } > > nd_old = nd_new = 0; > > offset = 0; > for(i=0; i if (!is_slice) { > if (!(op1=PySequence_GetItem(op, i))) { > PyErr_SetString(PyExc_IndexError, "invalid index"); > return -1; > } > } > > start = parse_subindex(op1, &step_size, &n_steps, > nd_old < self->nd ? self->dimensions[nd_old] : 0); > Py_DECREF(op1); > if (start == -1) break; > > if (n_steps == PseudoIndex) { > dimensions[nd_new] = 1; strides[nd_new] = 0; nd_new++; > } else { > if (n_steps == RubberIndex) { > for(j=i+1, n_pseudo=0; j op1 = PySequence_GetItem(op, j); > if (op1 == Py_None) n_pseudo++; > Py_DECREF(op1); > } > n_add = self->nd-(n-i-n_pseudo-1+nd_old); > if (n_add < 0) { > PyErr_SetString(PyExc_IndexError, "too many indices"); > return -1; > } > for(j=0; j dimensions[nd_new] = self->dimensions[nd_old]; > strides[nd_new] = self->strides[nd_old]; > nd_new++; nd_old++; > } > } else { > if (nd_old >= self->nd) { > PyErr_SetString(PyExc_IndexError, "too many indices"); > return -1; > } > offset += self->strides[nd_old]*start; > nd_old++; > if (n_steps != SingleIndex) { > dimensions[nd_new] = n_steps; > strides[nd_new] = step_size*self->strides[nd_old-1]; > nd_new++; > } > } > } > } > if (i < n) return -1; > n_add = self->nd-nd_old; > for(j=0; j dimensions[nd_new] = self->dimensions[nd_old]; > strides[nd_new] = self->strides[nd_old]; > nd_new++; nd_old++; > } > *offset_ptr = offset; > return nd_new; > } > > static PyObject *array_subscript(PyArrayObject *self, PyObject *op) { > int dimensions[MAX_DIMS], strides[MAX_DIMS]; > int nd, offset, i, elsize; > PyArrayObject *other; > > if (PyInt_Check(op)) { > i = PyInt_AsLong(op); > if (i < 0 && self->nd > 0) i = i+self->dimensions[0]; > return array_item(self, i); > } > > if ((nd = parse_index(self, op, dimensions, strides, &offset)) == -1) { > return NULL; > } > > if ((other = (PyArrayObject *)PyArray_FromDimsAndDataAndDescr(nd, > dimensions, > self->descr, > self->data+offset)) == NULL) { > return NULL; > } > memcpy(other->strides, strides, sizeof(int)*other->nd); > other->base = (PyObject *)self; > Py_INCREF(self); > > elsize=other->descr->elsize; > for (i=other->nd-1; i>=0; i--) { > if (other->strides[i] == elsize) { > elsize *= other->dimensions[i]; > } else { > break; > } > } > if (i >= 0) other->flags &= ~CONTIGUOUS; > > return (PyObject *)other; > } > > static PyObject *array_subscript_nice(PyArrayObject *self, PyObject *op) { > PyObject *ret; > > if ((ret = array_subscript(self, op)) == NULL) return NULL; > if (PyArray_Check(ret)) return PyArray_Return((PyArrayObject *)ret); > else return ret; > } > > /* Another assignment hacked by using CopyObject. */ > > static int array_ass_sub(PyArrayObject *self, PyObject *index, PyObject *op) { > int ret; > PyArrayObject *tmp; > > if (op == NULL) { > PyErr_SetString(PyExc_ValueError, "Can't delete array elements."); > return -1; > } > > if (PyInt_Check(index)) return array_ass_item(self, PyInt_AsLong(index), op); > > if ((tmp = (PyArrayObject *)array_subscript(self, index)) == NULL) return -1; > ret = PyArray_CopyObject(tmp, op); > Py_DECREF(tmp); > > return ret; > } > > static PyMappingMethods array_as_mapping = { > (inquiry)array_length, /*mp_length*/ > (binaryfunc)array_subscript_nice, /*mp_subscript*/ > (objobjargproc)array_ass_sub, /*mp_ass_subscript*/ > }; > > /* -------------------------------------------------------------- */ > > typedef struct { > PyUFuncObject *add, *subtract, *multiply, *divide, *remainder, *power, *negative, *absolute; > PyUFuncObject *invert, *left_shift, *right_shift, *bitwise_and, *bitwise_xor, > *bitwise_or; > } NumericOps; > > static NumericOps n_ops; > > #define GET(op) n_ops.op = (PyUFuncObject *)PyDict_GetItemString(dict, #op) > > int PyArray_SetNumericOps(PyObject *dict) { > GET(add); > GET(subtract); > GET(multiply); > GET(divide); > GET(remainder); > GET(power); > GET(negative); > GET(absolute); > GET(invert); > GET(left_shift); > GET(right_shift); > GET(bitwise_and); > GET(bitwise_or); > GET(bitwise_xor); > return 0; > } > > static int array_coerce(PyArrayObject **pm, PyObject **pw) { > PyObject *new_op; > if ((new_op = PyArray_FromObject(*pw, PyArray_NOTYPE, 0, 0)) == NULL) return -1; > Py_INCREF(*pm); > *pw = new_op; > return 0; > > /**Eliminate coercion at this stage. Handled by umath now > PyObject *new_op; > char t1, t2; > > t1 = (*pm)->descr->type_num; > t2 = PyArray_ObjectType(*pw, 0); > > if (PyArray_CanCastSafely(t2, t1)) > if ((new_op = PyArray_FromObject(*pw, t1, 0, 0)) == NULL) return -1; > Py_INCREF(*pm); > *pw = new_op; > } else { > if (PyArray_CanCastSafely(t1, t2)) { > *pm = (PyArrayObject *)PyArray_Cast(*pm, t2); > if ((new_op = PyArray_FromObject(*pw, t2, 0, 0)) == NULL) return -1; > *pw = new_op; > } else { > PyErr_SetString(PyExc_TypeError, "cannot perform this operation on these types"); > return -1; > } > } > return 0; > ***/ > } > > static PyObject *PyUFunc_BinaryFunction(PyUFuncObject *s, PyArrayObject *mp1, PyObject *mp2) { > PyObject *arglist; > PyArrayObject *mps[3]; > > arglist = Py_BuildValue("(OO)", mp1, mp2); > > mps[0] = mps[1] = mps[2] = NULL; > if (PyUFunc_GenericFunction(s, arglist, mps) == -1) { > Py_XDECREF(mps[0]); Py_XDECREF(mps[1]); Py_XDECREF(mps[2]); > return NULL; > } > > Py_DECREF(mps[0]); Py_DECREF(mps[1]); > Py_DECREF(arglist); > return PyArray_Return(mps[2]); > } > > static PyObject *PyUFunc_UnaryFunction(PyUFuncObject *s, PyArrayObject *mp1) { > PyObject *arglist; > PyArrayObject *mps[3]; > > arglist = Py_BuildValue("(O)", mp1); > > mps[0] = mps[1] = NULL; > if (PyUFunc_GenericFunction(s, arglist, mps) == -1) { > Py_XDECREF(mps[0]); Py_XDECREF(mps[1]); > return NULL; > } > > Py_DECREF(mps[0]); > Py_DECREF(arglist); > return PyArray_Return(mps[1]); > } > > static PyObject *array_add(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.add, m1, m2); > } > static PyObject *array_subtract(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.subtract, m1, m2); > } > static PyObject *array_multiply(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.multiply, m1, m2); > } > static PyObject *array_divide(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.divide, m1, m2); > } > static PyObject *array_remainder(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.remainder, m1, m2); > } > static PyObject *array_power(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.power, m1, m2); > } > static PyObject *array_negative(PyArrayObject *m1) > return PyUFunc_UnaryFunction(n_ops.negative, m1); > } > static PyObject *array_absolute(PyArrayObject *m1) > return PyUFunc_UnaryFunction(n_ops.absolute, m1); > } > static PyObject *array_invert(PyArrayObject *m1) > return PyUFunc_UnaryFunction(n_ops.invert, m1); > } > static PyObject *array_left_shift(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.left_shift, m1, m2); > } > static PyObject *array_right_shift(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.right_shift, m1, m2); > } > static PyObject *array_bitwise_and(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.bitwise_and, m1, m2); > } > static PyObject *array_bitwise_or(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.bitwise_or, m1, m2); > } > static PyObject *array_bitwise_xor(PyArrayObject *m1, PyObject *m2) { > return PyUFunc_BinaryFunction(n_ops.bitwise_xor, m1, m2); > } > > > static int array_nonzero(PyArrayObject *mp) { > char *zero; > PyArrayObject *self; > char *data; > int i, s, elsize; > > self = PyArray_CONTIGUOUS(mp); > zero = self->descr->zero; > > s = SIZE(self); > elsize = self->descr->elsize; > data = self->data; > for(i=0; i if (memcmp(zero, data, elsize) != 0) break; > } > > Py_DECREF(self); > > return i!=s; > } > > static PyObject *array_divmod(PyArrayObject *op1, PyObject *op2) { > PyObject *divp, *modp; > > divp = array_divide(op1, op2); > if (divp == NULL) return NULL; > modp = array_remainder(op1, op2); > if (modp == NULL) { > Py_DECREF(divp); > return NULL; > } > return Py_BuildValue("OO", divp, modp); > } > > > static PyNumberMethods array_as_number = { > (binaryfunc)array_add, /*nb_add*/ > (binaryfunc)array_subtract, /*nb_subtract*/ > (binaryfunc)array_multiply, /*nb_multiply*/ > (binaryfunc)array_divide, /*nb_divide*/ > (binaryfunc)array_remainder, /*nb_remainder*/ > (binaryfunc)array_divmod, /*nb_divmod*/ > (ternaryfunc)array_power, /*nb_power*/ > (unaryfunc)array_negative, > (unaryfunc)PyArray_Copy, /*nb_pos*/ > (unaryfunc)array_absolute, /* (unaryfunc)array_abs, */ > (inquiry)array_nonzero, /*nb_nonzero*/ > (unaryfunc)array_invert, /*nb_invert*/ > (binaryfunc)array_left_shift, /*nb_lshift*/ > (binaryfunc)array_right_shift, /*nb_rshift*/ > (binaryfunc)array_bitwise_and, /*nb_and*/ > (binaryfunc)array_bitwise_xor, /*nb_xor*/ > (binaryfunc)array_bitwise_or, /*nb_or*/ > (coercion)array_coerce, /*nb_coerce*/ > 0, /*nb_int*/ > 0, /*nb_long*/ > 0, /*nb_float*/ > 0, /*nb_oct*/ > 0, /*nb_hex*/ > }; > > static PySequenceMethods array_as_sequence = { > (inquiry)array_length, /*sq_length*/ > (binaryfunc)array_add, /*nb_add, concat is numeric add*/ > (intargfunc)NULL, /*nb_multiply, repeat is numeric multiply*/ > (intargfunc)array_item, /*sq_item*/ > (intintargfunc)array_slice, /*sq_slice*/ > (intobjargproc)array_ass_item, /*sq_ass_item*/ > (intintobjargproc)array_ass_slice, /*sq_ass_slice*/ > }; > > /* -------------------------------------------------------------- */ > > /*This ugly function quickly prints out an array. It's likely to disappear */ > /*in the near future. */ > static int dump_data(char **string, int *n, int *max_n, > char *data, int nd, int *dimensions, > int *strides, PyArray_Descr *descr) { > PyObject *op, *sp; > char *ostring; > int i, N; > > #define CHECK_MEMORY if (*n >= *max_n-16) { *max_n *= 2; *string = (char *)realloc(*string, *max_n); } > > if (nd == 0) { > > if ((op = descr->getitem(data)) == NULL) return -1; > sp = PyObject_Repr(op); > if (sp == NULL) {Py_DECREF(op); return -1;} > ostring = PyString_AsString(sp); > N = PyString_Size(sp)*sizeof(char); > *n += N; > CHECK_MEMORY > memcpy(*string+(*n-N), ostring, N); > Py_DECREF(sp); > Py_DECREF(op); > return 0; > } else { > if (nd == 1 && descr->type_num == PyArray_CHAR) { > N = (dimensions[0]+2)*sizeof(char); > *n += N; > CHECK_MEMORY > (*string)[*n-N] = '"'; > memcpy(*string+(*n-N+1), data, N-2); > (*string)[*n-1] = '"'; > return 0; > } else { > CHECK_MEMORY > (*string)[*n] = '['; > *n += 1; > for(i=0; i if (dump_data(string, n, max_n, data+(*strides)*i, nd-1, dimensions+1, strides+1, descr) < 0) return -1; > CHECK_MEMORY > if (i (*string)[*n] = ','; > (*string)[*n+1] = ' '; > *n += 2; > } > } > CHECK_MEMORY > (*string)[*n] = ']'; *n += 1; > return 0; > } > } > > #undef CHECK_MEMORY > } > > static PyObject * array_repr_builtin(PyArrayObject *self) { > PyObject *ret; > char *string; > int n, max_n; > > max_n = NBYTES(self)*4*sizeof(char) + 7; > > if ((string = (char *)malloc(max_n)) == NULL) { > PyErr_SetString(PyExc_MemoryError, "out of memory"); > return NULL; > } > > n = 6; > sprintf(string, "array("); > > if (dump_data(&string, &n, &max_n, self->data, self->nd, self->dimensions, > self->strides, self->descr) < 0) { free(string); return NULL; } > > sprintf(string+n, ", '%c')", self->descr->type); > > ret = PyString_FromStringAndSize(string, n+6); > free(string); > return ret; > } > > static PyObject *PyArray_StrFunction=NULL; > static PyObject *PyArray_ReprFunction=NULL; > > extern void PyArray_SetStringFunction(PyObject *op, int repr) { > if (repr) { > Py_XDECREF(PyArray_ReprFunction); /* Dispose of previous callback */ > Py_XINCREF(op); /* Add a reference to new callback */ > PyArray_ReprFunction = op; /* Remember new callback */ > } else { > Py_XDECREF(PyArray_StrFunction); /* Dispose of previous callback */ > Py_XINCREF(op); /* Add a reference to new callback */ > PyArray_StrFunction = op; /* Remember new callback */ > } > } > > static PyObject *array_repr(PyArrayObject *self) { > PyObject *s, *arglist; > > if (PyArray_ReprFunction == NULL) { > s = array_repr_builtin(self); > } else { > arglist = Py_BuildValue("(O)", self); > s = PyEval_CallObject(PyArray_ReprFunction, arglist); > Py_DECREF(arglist); > } > return s; > } > > static PyObject *array_str(PyArrayObject *self) { > PyObject *s, *arglist; > > if (PyArray_StrFunction == NULL) { > s = array_repr(self); > } else { > arglist = Py_BuildValue("(O)", self); > s = PyEval_CallObject(PyArray_StrFunction, arglist); > Py_DECREF(arglist); > } > return s; > } > > /* No longer implemented here. Use the default in object.c > static int array_print(PyArrayObject *self, FILE *fp, int flags) { > PyObject *s; > int r=-1, n; > > s = array_str(self); > > if (s == NULL || !PyString_Check(s)) { > Py_XDECREF(s); > return -1; > } > n = PyString_Size(s); > r = fwrite(PyString_AsString(s), sizeof(char), n, fp); > Py_DECREF(s); > return r==n ? 0 : -1; > } > */ > > static void byte_swap_vector(void *p, int n, int size) { > char *a, *b, c; > > switch(size) { > case 2: > for (a = (char*)p ; n > 0; n--, a += 1) { > b = a + 1; > c = *a; *a++ = *b; *b = c; > } > break; > case 4: > for (a = (char*)p ; n > 0; n--, a += 2) { > b = a + 3; > c = *a; *a++ = *b; *b-- = c; > c = *a; *a++ = *b; *b = c; > } > break; > case 8: > for (a = (char*)p ; n > 0; n--, a += 4) { > b = a + 7; > c = *a; *a++ = *b; *b-- = c; > c = *a; *a++ = *b; *b-- = c; > c = *a; *a++ = *b; *b-- = c; > c = *a; *a++ = *b; *b = c; > } > break; > default: > break; > } > } > > static char doc_byteswapped[] = "m.byteswapped(). Swaps the bytes in the contents of array m. Useful for reading data written by a machine with a different byte order. Returns a new array with the byteswapped data."; > > static PyObject * array_byteswap(PyArrayObject *self, PyObject *args) { > PyArrayObject *ret; > > if (!PyArg_ParseTuple(args, "")) return NULL; > > if ((ret = (PyArrayObject *)PyArray_Copy(self)) == NULL) return NULL; > > if (self->descr->type_num < PyArray_CFLOAT) { > byte_swap_vector(ret->data, SIZE(self), self->descr->elsize); > } else { > byte_swap_vector(ret->data, SIZE(self)*2, self->descr->elsize/2); > } > > return (PyObject *)ret; > } > > static char doc_tostring[] = "m.tostring(). Copy the data portion of the array to a python string and return that string."; > > static PyObject *array_tostring(PyArrayObject *self, PyObject *args) { > PyObject *so; > PyArrayObject *mo; > > if (!PyArg_ParseTuple(args, "")) return NULL; > > if ((mo = PyArray_CONTIGUOUS(self)) ==NULL) return NULL; > > so = PyString_FromStringAndSize(mo->data, NBYTES(mo)); > > Py_DECREF(mo); > > return so; > } > > > static PyObject *PyArray_ToList(PyObject *self) { > PyObject *lp; > PyObject *v; > int sz, i; > > if (!PyArray_Check(self)) return self; > > if (((PyArrayObject *)self)->nd == 0) return array_item((PyArrayObject *)self, 0); > > sz = ((PyArrayObject *)self)->dimensions[0]; > lp = PyList_New(sz); > > for (i=0; i v=array_item((PyArrayObject *)self, i); > PyList_SetItem(lp, i, PyArray_ToList(v)); > if (((PyArrayObject *)self)->nd>1){ > Py_DECREF(v); > } > } > > return lp; > } > > > static char doc_tolist[] = "m.tolist(). Copy the data portion of the array to a hierarchical python list and return that list."; > > static PyObject *array_tolist(PyArrayObject *self, PyObject *args) { > if (!PyArg_ParseTuple(args, "")) return NULL; > if (self->nd <= 0) { > PyErr_SetString(PyExc_ValueError, "Can't convert a 0d array to a list"); > return NULL; > } > > return PyArray_ToList((PyObject *)self); > } > > static char doc_cast[] = "m.cast(t). Cast array m to type t. t can be either a string representing a typecode, or a python type object of type int, float, or complex."; > > PyObject *array_cast(PyArrayObject *self, PyObject *args) { > PyObject *op; > char typecode; > > if (!PyArg_ParseTuple(args, "O", &op)) return NULL; > > if (PyString_Check(op) && (PyString_Size(op) == 1)) { > return PyArray_Cast(self, PyString_AS_STRING((PyStringObject *)op)[0]); > } > > if (PyType_Check(op)) { > typecode = 'O'; > if (((PyTypeObject *)op) == &PyInt_Type) { > typecode = PyArray_LONG; > } > if (((PyTypeObject *)op) == &PyFloat_Type) { > typecode = PyArray_DOUBLE; > } > if (((PyTypeObject *)op) == &PyComplex_Type) { > typecode = PyArray_CDOUBLE; > } > return PyArray_Cast(self, typecode); > } > PyErr_SetString(PyExc_ValueError, > "type must be either a 1-length string, or a python type object"); > return NULL; > } > > static char doc_typecode[] = "m.typecode(). Return the single character code indicating the type of the elements of m."; > > PyObject *array_typecode(PyArrayObject *self, PyObject *args) { > if (!PyArg_ParseTuple(args, "")) return NULL; > > return PyString_FromStringAndSize(&(self->descr->type), 1); > } > > static char doc_itemsize[] = "m.itemsize(). Return the size in bytes of a single element of the array m."; > > PyObject *array_itemsize(PyArrayObject *self, PyObject *args) { > if (!PyArg_ParseTuple(args, "")) return NULL; > > return PyInt_FromLong(self->descr->elsize); > } > > static char doc_contiguous[] = "m.contiguous(). Return true if the memory used by the array m is contiguous. A non-contiguous array can be converted to a contiguous one by m.copy()."; > > PyObject *array_contiguous(PyArrayObject *self, PyObject *args) { > if (!PyArg_ParseTuple(args, "")) return NULL; > > return PyInt_FromLong(ISCONTIGUOUS(self)); > } > > static char doc_copy[] = "m.__copy__()."; > > PyObject *array_copy(PyArrayObject *self, PyObject *args) { > if (!PyArg_ParseTuple(args, "")) return NULL; > > return PyArray_Copy(self); > } > > int array_compare(PyArrayObject *self, PyObject *other) { > PyErr_SetString(PyExc_TypeError, > "Comparison of multiarray objects is not implemented."); > return -1; > } > > static char doc_init[] = "Array(sequence, copy=1, typecode=None) will return a new Array ExtensionClass instance formed from the given (potentially nested) sequence and of type given by typecode. If no typecode is given, then the type will be determined as the minimum type required to hold the objects in sequence.If copy is zero and the the sequence in of type array, then the sequence is not copied, i.e., the new array and the sequence share the same data. Note that keywords cannot be used and that the order of the last two arguments is reversed for that in the function array."; > > static PyObject *array_init(PyArrayObject *self, PyObject *args) { > PyObject *op=Py_None, *tpo=Py_None; > PyArrayObject *ret; > int sd; > PyArray_Descr *descr; > char *data, *tp=NULL; > int type, copy=1; > int *dimensions, *strides; > int flags=0; > int nd = 0; > > if(!PyArg_ParseTuple(args, "|OiO", &op, ©, &tpo)) return NULL; > > /* Have data */ > if (op != Py_None) { > if (tpo == Py_None) { > type = PyArray_NOTYPE; > } else { > tp = PyString_AsString(tpo); > if (tp[0] == 0) type = PyArray_NOTYPE; > else type = tp[0]; > } > > if (copy) { > if ((ret = (PyArrayObject *)PyArray_CopyFromObject(op, type, 0, 0)) == NULL) > return NULL; > } else { > if ((ret = (PyArrayObject *)PyArray_FromObject(op, type, 0, 0)) == NULL) > return NULL; > } > /* Copy ret to self. */ > self->data = ret->data; > self->dimensions = ret->dimensions; > self->strides = ret->strides; > self->nd = ret->nd; > self->descr= ret->descr; > > self->base = ret->base; > self->flags = ret->flags; > > ret->flags = 0; > Py_INCREF(Py_None); > return Py_None; > } > > /* Don't have any data */ > flags=CONTIGUOUS | OWN_DIMENSIONS | OWN_STRIDES; > data = NULL; > dimensions = strides = NULL; > type = PyArray_DOUBLE; > > if ((descr = PyArray_DescrFromType(type)) == NULL) return NULL; > sd = descr->elsize; > /* Make sure we're alligned on ints. */ > sd += sizeof(int) - sd%sizeof(int); > > if ((data = (char *)malloc(sd)) == NULL) { > PyErr_SetString(PyExc_MemoryError, "can't allocate memory for array"); > goto fail; > } > flags |= OWN_DATA; > if (flags & OWN_DATA) memset(data, 0, sd); > > self->data=data; > self->dimensions = dimensions; > self->strides = strides; > self->nd=nd; > self->descr=descr; > self->base = (PyObject *)NULL; > self->flags = flags; > Py_INCREF(Py_None); > return Py_None; > > fail: > if (flags & OWN_DATA) free(data); > return NULL; > } > > > > static struct PyMethodDef array_methods[] = { > {"tostring", (PyCFunction)array_tostring, 1, doc_tostring}, > {"tolist", (PyCFunction)array_tolist, 1, doc_tolist}, > {"byteswapped", (PyCFunction)array_byteswap, 1, doc_byteswapped}, > {"astype", (PyCFunction)array_cast, 1, doc_cast}, > {"typecode", (PyCFunction)array_typecode, 1, doc_typecode}, > {"itemsize", (PyCFunction)array_itemsize, 1, doc_itemsize}, > {"iscontiguous", (PyCFunction)array_contiguous, 1, doc_contiguous}, > {"__copy__", (PyCFunction)array_copy, 1, doc_copy}, > {"__init__", (PyCFunction)array_init, METH_VARARGS, doc_init}, > {NULL, NULL} /* sentinel */ > }; > > /* ---------- */ > static PyObject *array_getattr(PyArrayObject *self, char *name) { > PyArrayObject *ret; > > if (strcmp(name, "shape") == 0) { > PyObject *s, *o; > int i; > > if ((s=PyTuple_New(self->nd)) == NULL) return NULL; > > for(i=self->nd; --i >= 0;) { > if ((o=PyInt_FromLong(self->dimensions[i])) == NULL) return NULL; > if (PyTuple_SetItem(s,i,o) == -1) return NULL; > } > return s; > } > > if (strcmp(name, "real") == 0) { > if (self->descr->type_num == PyArray_CFLOAT || > self->descr->type_num == PyArray_CDOUBLE) { > ret = (PyArrayObject *)PyArray_FromDimsAndData(self->nd, > self->dimensions, > self->descr->type_num-2, > self->data); > if (ret == NULL) return NULL; > memcpy(ret->strides, self->strides, sizeof(int)*ret->nd); > ret->flags &= ~CONTIGUOUS; > Py_INCREF(self); > ret->base = (PyObject *)self; > return (PyObject *)ret; > } else { > ret = (PyArrayObject *)PyArray_FromDimsAndData(self->nd, > self->dimensions, > self->descr->type_num, > self->data); > if (ret == NULL) return NULL; > Py_INCREF(self); > ret->base = (PyObject *)self; > } > } > > if ((strcmp(name, "imaginary") == 0) || (strcmp(name, "imag") == 0)) > if (self->descr->type_num == PyArray_CFLOAT || > self->descr->type_num == PyArray_CDOUBLE) { > ret = (PyArrayObject *)PyArray_FromDimsAndData(self->nd, > self->dimensions, > self->descr->type_num-2, > self->data+self->descr->elsize/2); > if (ret == NULL) return NULL; > memcpy(ret->strides, self->strides, sizeof(int)*ret->nd); > ret->flags &= ~CONTIGUOUS; > Py_INCREF(self); > ret->base = (PyObject *)self; > return (PyObject *)ret; > } else { > PyErr_SetString(PyExc_ValueError, "No imaginary part to real array"); > return NULL; > } > } > > if (strcmp(name, "flat") == 0) { > int n; > n = SIZE(self); > if (!ISCONTIGUOUS(self)) { > PyErr_SetString(PyExc_ValueError, "flattened indexing only available for contiguous array"); > return NULL; > } > ret = (PyArrayObject *)PyArray_FromDimsAndDataAndDescr(1, &n, > self->descr, > self->data); > if (ret == NULL) return NULL; > Py_INCREF(self); > ret->base = (PyObject *)self; > return (PyObject *)ret; > } > > return Py_FindMethod(array_methods, (PyObject *)self, name); > } > > static int array_setattr(PyArrayObject *self, char *name, PyObject *op) { > PyArrayObject *ap; > int ret; > > if (strcmp(name, "shape") == 0) { > /* This can be made more efficient by copying code from array_reshape if needed */ > if ((ap = (PyArrayObject *)PyArray_Reshape(self, op)) == NULL) return -1; > if(self->flags & OWN_DIMENSIONS) free(self->dimensions); > self->dimensions = ap->dimensions; > if(self->flags & OWN_STRIDES) free(self->strides); > self->strides = ap->strides; > self->nd = ap->nd; > self->flags &= ~(OWN_DIMENSIONS | OWN_STRIDES); > self->flags |= ap->flags & (OWN_DIMENSIONS | OWN_STRIDES); > ap->flags &= ~(OWN_DIMENSIONS | OWN_STRIDES); > Py_DECREF(ap); > return 0; > } > > if (strcmp(name, "real") == 0) { > if (self->descr->type_num == PyArray_CFLOAT || > self->descr->type_num == PyArray_CDOUBLE) { > ap = (PyArrayObject *)PyArray_FromDimsAndData(self->nd, > self->dimensions, > self->descr->type_num-2, > self->data); > if (ap == NULL) return -1; > memcpy(ap->strides, self->strides, sizeof(int)*ap->nd); > ap->flags &= ~CONTIGUOUS; > ret = PyArray_CopyObject(ap, op); > Py_DECREF(ap); > return ret; > } else { > return PyArray_CopyObject(self, op); > } > } > > if ((strcmp(name, "imaginary") == 0) || (strcmp(name, "imag") == 0)) > if (self->descr->type_num == PyArray_CFLOAT || > self->descr->type_num == PyArray_CDOUBLE) { > ap = (PyArrayObject *)PyArray_FromDimsAndData(self->nd, > self->dimensions, > self->descr->type_num-2, > self->data+self->descr->elsize/2); > if (ap == NULL) return -1; > memcpy(ap->strides, self->strides, sizeof(int)*ap->nd); > ap->flags &= ~CONTIGUOUS; > ret = PyArray_CopyObject(ap, op); > Py_DECREF(ap); > return ret; > } else { > PyErr_SetString(PyExc_ValueError, "No imaginary part to real array"); > return -1; > } > } > > return PyDict_SetItemString(INSTANCE_DICT(self), name, op); > } > > static char Arraytype__doc__[] = > "A array object represents a multidimensional, homogeneous array of basic values. It has the folowing data members, m.shape (the size of each dimension in the array), m.itemsize (the size (in bytes) of each element of the array), and m.typecode (a character representing the type of the matrices elements). Matrices are sequence, mapping and numeric objects. Sequence indexing is similar to lists, with single indices returning a reference that points to the old matrices data, and slices returning by copy. A array is also allowed to be indexed by a sequence of sequences or ints. Each member of the sequence indexes the corresponding dimension of the array. If it is a sequence, it returns the elments along that dimension listed in the sequence, if a single number, it returns just that element (reducing the dimensionality of the returned array by 1. Numeric operations operate on matrices in an element-wise fashion."; > > > /* PyTypeObject PyArray_Type = { */ > PyExtensionClass PyArray_Type = { > PyObject_HEAD_INIT(0) > 0, /*ob_size*/ > "array", /*tp_name*/ > sizeof(PyArrayObject), /*tp_basicsize*/ > 0, /*tp_itemsize*/ > /* methods */ > (destructor)array_dealloc, /*tp_dealloc*/ > (printfunc)NULL, /*tp_print*/ > (getattrfunc)array_getattr, /*tp_getattr*/ > (setattrfunc)array_setattr, /*tp_setattr*/ > (cmpfunc)array_compare, /*tp_compare*/ > (reprfunc)array_repr, /*tp_repr*/ > &array_as_number, /*tp_as_number*/ > &array_as_sequence, /*tp_as_sequence*/ > &array_as_mapping, /*tp_as_mapping*/ > (hashfunc)0, /*tp_hash*/ > (ternaryfunc)0, /*tp_call*/ > (reprfunc)array_str, /*tp_str*/ > > /* Space for future expansion */ > 0L,0L,0L,0L, > Arraytype__doc__, /* Documentation string */ > METHOD_CHAIN(array_methods) /* Required by ExtensionClass */ > }; > > > /* The rest of this code is to build the right kind of array from a python */ > /* object. */ > > static int discover_depth(PyObject *s, int max, int stop_at_string) { > int d=0; > PyObject *e; > > if(max < 1) return -1; > > /*if(! PySequence_Check(s) || PyInstance_Check(s)) return 0; > /* > Relax this to allow class instances. Dangerous? */ > if(! PySequence_Check(s) || PySequence_Length(s) < 0) { > PyErr_Clear(); return 0; > } > if(PyString_Check(s)) return stop_at_string ? 0:1; > if (PySequence_Length(s) == 0) return 1; > > if ((e=PySequence_GetItem(s,0)) == NULL) return -1; > if(e!=s) > { > d=discover_depth(e,max-1, stop_at_string); > if(d >= 0) d++; > } > Py_DECREF(e); > return d; > } > > static int discover_dimensions(PyObject *s, int nd, int *d) { > PyObject *e; > int r, n, i, n_lower; > > n=PyObject_Length(s); > *d = n; > if(*d < 0) return -1; > if(nd <= 1) return 0; > n_lower = 0; > for(i=0; i if ((e=PySequence_GetItem(s,i)) == NULL) return -1; > r=discover_dimensions(e,nd-1,d+1); > Py_DECREF(e); > > if (r == -1) return -1; > if (d[1] > n_lower) n_lower = d[1]; > } > d[1] = n_lower; > > return 0; > } > > #ifndef max > #define max(a,b) (a)>(b)?(a):(b); > #endif > > int PyArray_ObjectType(PyObject *op, int minimum_type) { > int l; > PyObject *ip; > > if (minimum_type == -1) return -1; > > if (PyArray_Check(op)) return max((int)((PyArrayObject *)op)->descr->type_num, minimum_type); > > if (PyInstance_Check(op)) { > if (PyObject_HasAttrString(op, "__array__")) { > PyObject *ap, *arglist; > > arglist = Py_BuildValue("()"); > ap = PyObject_GetAttrString(op, "__array__"); > ip = PyEval_CallObject(ap, arglist); > Py_DECREF(arglist); > > return max(minimum_type, (int)((PyArrayObject *)ip)->descr->type_num); > } else { > if (PySequence_Length(op)<0) > PyErr_Clear(); > return (int)PyArray_OBJECT; > } > } > > if (PyString_Check(op)) { > return max(minimum_type, (int)PyArray_CHAR); > } > > if (PySequence_Check(op)) { > l = PyObject_Length(op); > if (l == 0 && minimum_type == 0) minimum_type = PyArray_LONG; > while (--l >= 0) { > ip = PySequence_GetItem(op, l); > minimum_type = PyArray_ObjectType(ip, minimum_type); > Py_DECREF(ip); > } > return minimum_type; > } > > if (PyInt_Check(op)) { > return max(minimum_type, (int)PyArray_LONG); > } else { > if (PyFloat_Check(op)) { > return max(minimum_type, (int)PyArray_DOUBLE); > } else { > if (PyComplex_Check(op)) { > return max(minimum_type, (int)PyArray_CDOUBLE); > } else { > return (int)PyArray_OBJECT; > } > } > } > } > > int Assign_Array(PyArrayObject *self, PyObject *v) { > PyObject *e; > int l, r; > > if (!PySequence_Check(v)) { > PyErr_SetString(PyExc_ValueError,"assignment from non-sequence"); > return -1; > } > > l=PyObject_Length(v); > if(l < 0) return -1; > > while(--l >= 0) > { > e=PySequence_GetItem(v,l); > if (e == NULL) return -1; > r = PySequence_SetItem((PyObject*)self,l,e); > Py_DECREF(e); > if(r == -1) return -1; > } > return 0; > } > > PyObject * > Array_FromSequence(PyObject *s, int type, int min_depth, int max_depth) > { > PyArrayObject *r; > int nd, *d; > > if (!PySequence_Check(s)) { > PyErr_SetString(PyExc_ValueError,"expect source sequence"); > return NULL; > } > if (!((nd=discover_depth(s,99, type == PyArray_OBJECT)) > 0)) { > PyErr_SetString(PyExc_ValueError, "invalid input sequence"); > return NULL; > } > if ((max_depth && nd > max_depth) || (min_depth && nd < min_depth)) { > PyErr_SetString(PyExc_ValueError, "invalid number of dimensions"); > return NULL; > } > if ((d=(int *)malloc(nd*sizeof(int))) == NULL) { > PyErr_SetString(PyExc_MemoryError,"out of memory"); > } > if(discover_dimensions(s,nd,d) == -1) > { > free(d); > return 0; > } > > if (type == PyArray_CHAR && nd > 0 && d[nd-1] == 1) { > nd = nd-1; > } > > r=(PyArrayObject*)PyArray_FromDims(nd,d,type); > free(d); > if(! r) return NULL; > if(Assign_Array(r,s) == -1) > { > Py_DECREF(r); > return NULL; > } > return (PyObject*)r; > } > > PyObject *PyArray_FromScalar(PyObject *op, int type) { > PyArrayObject *ret; > > if ((ret = (PyArrayObject *)PyArray_FromDims(0, NULL, type)) == NULL) > return NULL; > > ret->descr->setitem(op, ret->data); > > if (PyErr_Occurred()) { > array_dealloc(ret); > return NULL; > } else { > return (PyObject *)ret; > } > } > > PyObject *PyArray_Cast(PyArrayObject *mp, int type) { > PyArrayObject *rp, *tmp; > > if (mp->descr->type_num == PyArray_OBJECT) { > return PyArray_FromObject((PyObject *)mp, type, mp->nd, mp->nd); > } > > if ((tmp = PyArray_CONTIGUOUS(mp)) == NULL) return NULL; > > rp = (PyArrayObject *)PyArray_FromDims(tmp->nd, tmp->dimensions, type); > mp->descr->cast[(int)rp->descr->type_num](tmp->data, 1, rp->data, 1, SIZE(mp)); > > Py_DECREF(tmp); > return (PyObject *)rp; > } > > #define ByCopy 1 > #define Contiguous 2 > > PyObject *array_fromobject(PyObject *op_in, int type, int min_depth, int max_depth, int flags) { > PyObject *r, *op; > > r = NULL; > > if (!PyArray_Check(op_in) && PyObject_HasAttrString(op_in, "__array__")) { > PyObject *ap, *arglist; > > if (type == PyArray_NOTYPE) { > arglist = Py_BuildValue("()"); > } else { > arglist = Py_BuildValue("(c)", type); > } > ap = PyObject_GetAttrString(op_in, "__array__"); > op = PyEval_CallObject(ap, arglist); > Py_DECREF(ap); > Py_DECREF(arglist); > > if (op == NULL) return NULL; > } else { > op = op_in; > Py_INCREF(op); > } > > if (type == PyArray_NOTYPE) { > type = PyArray_ObjectType(op, 0); > } > > if (PyArray_Check(op) && (((PyArrayObject *)op)->descr->type_num != PyArray_OBJECT || > type == PyArray_OBJECT || type == 'O')) { > if ((((PyArrayObject *)op)->descr->type_num == type || > ((PyArrayObject *)op)->descr->type == type)) { > if ((flags & ByCopy) || (flags&Contiguous && !ISCONTIGUOUS((PyArrayObject *)op))) { > r = PyArray_Copy((PyArrayObject *)op); > } else { > Py_INCREF(op); > r = op; > } > } else { > if (type > PyArray_NTYPES) type = PyArray_DescrFromType(type)->type_num; > if (PyArray_CanCastSafely(((PyArrayObject *)op)->descr->type_num, type)) > r = PyArray_Cast((PyArrayObject *)op, type); > else { > PyErr_SetString(PyExc_TypeError, "Array can not be safely cast to required type"); > r = NULL; > } > } > } else { > r = Array_FromSequence(op, type, min_depth,max_depth); > if (r == NULL && min_depth <= 0) { > PyErr_Clear(); > r = PyArray_FromScalar(op, type); > } > } > Py_DECREF(op); > if (r == NULL) return NULL; > > if(!PyArray_Check(r)) { > PyErr_SetString(PyExc_ValueError, "Internal error array_fromobject not producing an array"); > return NULL; > } > if (min_depth != 0 && ((PyArrayObject *)r)->nd < min_depth) { > Py_DECREF(r); > PyErr_SetString(PyExc_ValueError, > "Object of too small depth for desired array"); > return NULL; > } > if (max_depth != 0 && ((PyArrayObject *)r)->nd > max_depth) { > Py_DECREF(r); > PyErr_SetString(PyExc_ValueError, > "Object too deep for desired array"); > return NULL; > } > return r; > } > > > > PyObject *PyArray_FromObject(PyObject *op, int type, int min_depth, int max_depth) { > return array_fromobject(op, type, min_depth, max_depth, 0); > } > PyObject *PyArray_ContiguousFromObject(PyObject *op, int type, int min_depth, int max_depth) { > return array_fromobject(op, type, min_depth, max_depth, Contiguous); > } > PyObject *PyArray_CopyFromObject(PyObject *op, int type, int min_depth, int max_depth) { > return array_fromobject(op, type, min_depth, max_depth, ByCopy); > } > > extern int PyArray_CanCastSafely(int fromtype, int totype) { > if (fromtype == totype) return 1; > if (totype == PyArray_OBJECT) return 1; > if (fromtype == PyArray_OBJECT) return 0; > > switch(fromtype) { > case PyArray_CHAR: > return 0; > case PyArray_UBYTE: > return (totype >= PyArray_SHORT); > case PyArray_SBYTE: > case PyArray_SHORT: > return (totype > fromtype); > case PyArray_INT: > case PyArray_LONG: > /*Allow Longs to be cast to Ints, not safe on 64-bit machines!*/ > return (totype >= PyArray_INT) && (totype != PyArray_FLOAT); > case PyArray_FLOAT: > return (totype > PyArray_FLOAT); > case PyArray_DOUBLE: > return (totype == PyArray_CDOUBLE); > case PyArray_CFLOAT: > return (totype == PyArray_CDOUBLE); > case PyArray_CDOUBLE: > return 0; > default: > return 0; > } > } > > extern int PyArray_As1D(PyObject **op, char **ptr, int *d1, int typecode) { > PyArrayObject *ap; > > if ((ap = (PyArrayObject *)PyArray_ContiguousFromObject(*op, typecode, 1, > 1)) == NULL) > return -1; > > *op = (PyObject *)ap; > *ptr = ap->data; > *d1 = ap->dimensions[0]; > return 0; > } > > extern int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, int typecode) { > PyArrayObject *ap; > int i, n, size; > char **data; > > if ((ap = (PyArrayObject *)PyArray_ContiguousFromObject(*op, typecode, 2, 2)) == NULL) > return -1; > > n = ap->dimensions[0]; > data = (char **)malloc(n*sizeof(char *)); > size = ap->descr->elsize * ap->dimensions[1]; > for(i=0; i data[i] = ap->data + i*ap->strides[0]; > } > *op = (PyObject *)ap; > *ptr = data; > *d1 = ap->dimensions[0]; > *d2 = ap->dimensions[1]; > return 0; > } > > > extern int PyArray_Free(PyObject *op, char *ptr) { > PyArrayObject *ap = (PyArrayObject *)op; > int i, n; > > if (ap->nd > 2) return -1; > if (ap->nd == 3) { > n = ap->dimensions[0]; > for (i=0; i free(((char **)ptr)[i]); > } > } > if (ap->nd >= 2) { > free(ptr); > } > Py_DECREF(ap); > return 0; > } > > extern PyObject * PyArray_Reshape(PyArrayObject *self, PyObject *shape) { > int i, n, s_original, i_unknown, s_known; > int *dimensions; > PyArrayObject *ret; > > if (!PyArray_ISCONTIGUOUS(self)) { > PyErr_SetString(PyExc_ValueError, "reshape only works on contiguous matrices"); > return NULL; > } > > if (PyArray_As1D(&shape, (char **)&dimensions, &n, PyArray_INT) == -1) return NULL; > > s_known = 1; > i_unknown = -1; > for(i=0; i if (dimensions[i] < 0) { > if (i_unknown == -1) { > i_unknown = i; > } else { > PyErr_SetString(PyExc_ValueError, "can only specify one unknown dimension"); > goto fail; > } > } else { > s_known *= dimensions[i]; > } > } > > s_original = PyArray_SIZE(self); > > if (i_unknown >= 0) { > if ((s_known == 0) || (s_original % s_known != 0)) { > PyErr_SetString(PyExc_ValueError, "total size of new array must be unchanged"); > goto fail; > } > dimensions[i_unknown] = s_original/s_known; > } else { > if (s_original != s_known) { > PyErr_SetString(PyExc_ValueError, "total size of new array must be unchanged"); > goto fail; > } > } > if ((ret = (PyArrayObject *)PyArray_FromDimsAndDataAndDescr(n, dimensions, > self->descr, > self->data)) == NULL) > goto fail; > > Py_INCREF(self); > ret->base = (PyObject *)self; > > PyArray_Free(shape, (char *)dimensions); > return (PyObject *)ret; > > fail: > PyArray_Free(shape, (char *)dimensions); > return NULL; > } > > extern PyObject *PyArray_Take(PyObject *self0, PyObject *indices0, int axis) { > PyArrayObject *self, *indices, *ret; > int nd, shape[MAX_DIMS]; > int i, j, chunk, n, m, max_item, tmp; > char *src, *dest; > > indices = ret = NULL; > self = (PyArrayObject *)PyArray_ContiguousFromObject(self0, PyArray_NOTYPE, 1, 0); > if (self == NULL) return NULL; > > if (axis < 0) axis = axis + self->nd; > if ((axis < 0) || (axis >= self->nd)) { > PyErr_SetString(PyExc_ValueError, "Invalid axis for this array"); > goto fail; > } > > indices = (PyArrayObject *)PyArray_ContiguousFromObject(indices0, PyArray_LONG, 1, 0); > if (indices == NULL) goto fail; > > n = m = chunk = 1; > nd = self->nd + indices->nd - 1; > for (i=0; i< nd; i++) { > if (i < axis) { > shape[i] = self->dimensions[i]; > n *= shape[i]; > } else { > if (i < axis+indices->nd) { > shape[i] = indices->dimensions[i-axis]; > m *= shape[i]; > } else { > shape[i] = self->dimensions[i-indices->nd+1]; > chunk *= shape[i]; > } > } > } > ret = (PyArrayObject *)PyArray_FromDims(nd, shape, self->descr->type_num); > > if (ret == NULL) goto fail; > > max_item = self->dimensions[axis]; > chunk = chunk * ret->descr->elsize; > src = self->data; > dest = ret->data; > > for(i=0; i for(j=0; j tmp = ((long *)(indices->data))[j]; > if (tmp < 0) tmp = tmp+max_item; > if ((tmp < 0) || (tmp >= max_item)) { > PyErr_SetString(PyExc_IndexError, "Index out of range for array"); > goto fail; > } > memcpy(dest, src+tmp*chunk, chunk); > dest += chunk; > } > src += chunk*max_item; > } > > PyArray_INCREF(ret); > > Py_XDECREF(indices); > Py_XDECREF(self); > > return (PyObject *)ret; > > > fail: > Py_XDECREF(ret); > Py_XDECREF(indices); > Py_XDECREF(self); > return NULL; > } > ---------------------------------------------------------------------------- ---- > #ifndef Py_ARRAYOBJECT_H > #define Py_ARRAYOBJECT_H > #ifdef __cplusplus > extern "C" { > #endif > > #ifndef _ARRAY_MODULE > #ifdef COMPILING_NUMPY > #include "ExtensionClassMacros.h" > #else > #include "Numeric/ExtensionClassMacros.h" > #endif > #endif > > #ifdef COMPILING_NUMPY > #include "ExtensionClass.h" > #else > #include "Numeric/ExtensionClass.h" > #endif > > > #define MAX_ELSIZE 16 > > enum PyArray_TYPES { PyArray_CHAR, PyArray_UBYTE, PyArray_SBYTE, > PyArray_SHORT, PyArray_INT, PyArray_LONG, > PyArray_FLOAT, PyArray_DOUBLE, > PyArray_CFLOAT, PyArray_CDOUBLE, > PyArray_OBJECT, > PyArray_NTYPES, PyArray_NOTYPE}; > > typedef void (PyArray_VectorUnaryFunc) Py_FPROTO((char *, int, char *, int, int)); > > typedef PyObject * (PyArray_GetItemFunc) Py_FPROTO((char *)); > typedef int (PyArray_SetItemFunc) Py_FPROTO((PyObject *, char *)); > > typedef struct { > PyArray_VectorUnaryFunc *cast[PyArray_NTYPES]; /* Functions to cast to */ > /* all other types */ > PyArray_GetItemFunc *getitem; > PyArray_SetItemFunc *setitem; > > int type_num, elsize; > char *one, *zero; > char type; > > } PyArray_Descr; > > #define CONTIGUOUS 1 > #define OWN_DIMENSIONS 2 > #define OWN_STRIDES 4 > #define OWN_DATA 8 > > typedef struct { > PyObject_HEAD > char *data; > int nd; > int *dimensions, *strides; > PyObject *base; > PyArray_Descr *descr; > int flags; > #ifndef NUMPY_NOEXTRA > PyObject* attributes; /* for user-defined information */ > #endif > } PyArrayObject; > > > /* > * C API > */ > > /* Type definitions */ > > #define PyArray_Type_NUM 0 > > /* Function definitions */ > > /* The following are not intended for use in user code, only needed by umath. */ > /* If you write your own match library, you might want this function. */ > #define PyArray_SetNumericOps_RET int > #define PyArray_SetNumericOps_PROTO Py_FPROTO((PyObject *)) > #define PyArray_SetNumericOps_NUM 1 > > #define PyArray_INCREF_RET int > #define PyArray_INCREF_PROTO Py_FPROTO((PyArrayObject *ap)) > #define PyArray_INCREF_NUM 2 > > #define PyArray_XDECREF_RET int > #define PyArray_XDECREF_PROTO Py_FPROTO((PyArrayObject *ap)) > #define PyArray_XDECREF_NUM 3 > > /* Export the array error object. Is this a good idea? */ > #define PyArrayError_RET PyObject * > #define PyArrayError_PROTO Py_FPROTO(()) > #define PyArrayError_NUM 4 > > /* Set the array print function to be a python function */ > #define PyArray_SetStringFunction_RET void > #define PyArray_SetStringFunction_PROTO Py_FPROTO((PyObject *op, int repr)) > #define PyArray_SetStringFunction_NUM 5 > > /* Get the PyArray_Descr structure for a typecode */ > #define PyArray_DescrFromType_RET PyArray_Descr * > #define PyArray_DescrFromType_PROTO Py_FPROTO((int)) > #define PyArray_DescrFromType_NUM 6 > > /* Cast an array to a different type */ > #define PyArray_Cast_RET PyObject * > #define PyArray_Cast_PROTO Py_FPROTO((PyArrayObject *, int)) > #define PyArray_Cast_NUM 7 > > /* Check the type coercion rules */ > #define PyArray_CanCastSafely_RET int > #define PyArray_CanCastSafely_PROTO Py_FPROTO((int fromtype, int totype)) > #define PyArray_CanCastSafely_NUM 8 > > /* Return the typecode to use for an object if it was an array */ > #define PyArray_ObjectType_RET int > #define PyArray_ObjectType_PROTO Py_FPROTO((PyObject *, int)) > #define PyArray_ObjectType_NUM 9 > > #define _PyArray_multiply_list_RET int > #define _PyArray_multiply_list_PROTO Py_FPROTO((int *lp, int n)) > #define _PyArray_multiply_list_NUM 10 > > > /* The following defines the C API for the array object for most users */ > > #define PyArray_SIZE(mp) (_PyArray_multiply_list((mp)->dimensions, (mp)->nd)) > #define PyArray_NBYTES(mp) ((mp)->descr->elsize * PyArray_SIZE(mp)) > /* Obviously this needs some work. */ > #define PyArray_ISCONTIGUOUS(m) ((m)->flags & CONTIGUOUS) > > > /* Return the size in number of items of an array */ > #define PyArray_Size_RET int > #define PyArray_Size_PROTO Py_FPROTO((PyObject *)) > #define PyArray_Size_NUM 11 > > > /* Array creation functions */ > /* new_array = PyArray_FromDims(n_dimensions, dimensions[n_dimensions], item_type); */ > #define PyArray_FromDims_RET PyObject * > #define PyArray_FromDims_PROTO Py_PROTO((int, int *, int)) > #define PyArray_FromDims_NUM 12 > > /* array_from_existing_data = PyArray_FromDimsAndData(n_dims, dims[n_dims], item_type, old_data); */ > /* WARNING: using PyArray_FromDimsAndData is not reccommended! It should only be used to refer to */ > /* global arrays that will never be freed (like FORTRAN common blocks). */ > #define PyArray_FromDimsAndData_RET PyObject * > #define PyArray_FromDimsAndData_PROTO Py_PROTO((int, int *, int, char *)) > #define PyArray_FromDimsAndData_NUM 13 > > /* Initialize from a python object. */ > > /* PyArray_ContiguousFromObject(object, typecode, min_dimensions, max_dimensions) */ > /* if max_dimensions = 0, then any number of dimensions are allowed. */ > /* If you want an exact number of dimensions, you should use max_dimensions */ > /* = min_dimensions. */ > > #define PyArray_ContiguousFromObject_RET PyObject * > #define PyArray_ContiguousFromObject_PROTO Py_FPROTO((PyObject *, int, int, int)) > #define PyArray_ContiguousFromObject_NUM 14 > > /* Same as contiguous, except guarantees a copy of the original data */ > #define PyArray_CopyFromObject_RET PyObject * > #define PyArray_CopyFromObject_PROTO Py_FPROTO((PyObject *, int, int, int)) > #define PyArray_CopyFromObject_NUM 15 > > /* Shouldn't be used unless you know what you're doing and are not scared by discontiguous arrays */ > #define PyArray_FromObject_RET PyObject * > #define PyArray_FromObject_PROTO Py_FPROTO((PyObject *, int, int, int)) > #define PyArray_FromObject_NUM 16 > > /* Return either an array, or if passed a 0d array return the appropriate python scalar */ > #define PyArray_Return_RET PyObject * > #define PyArray_Return_PROTO Py_FPROTO((PyArrayObject *)) > #define PyArray_Return_NUM 17 > > #define PyArray_Reshape_RET PyObject * > #define PyArray_Reshape_PROTO Py_FPROTO((PyArrayObject *ap, PyObject *shape)) > #define PyArray_Reshape_NUM 18 > > #define PyArray_Copy_RET PyObject * > #define PyArray_Copy_PROTO Py_FPROTO((PyArrayObject *ap)) > #define PyArray_Copy_NUM 19 > > #define PyArray_Take_RET PyObject * > #define PyArray_Take_PROTO Py_FPROTO((PyObject *ap, PyObject *items, int axis)) > #define PyArray_Take_NUM 20 > > /*Getting arrays in various useful forms. */ > #define PyArray_As1D_RET int > #define PyArray_As1D_PROTO Py_FPROTO((PyObject **op, char **ptr, int *d1, int typecode)) > #define PyArray_As1D_NUM 21 > > #define PyArray_As2D_RET int > #define PyArray_As2D_PROTO Py_FPROTO((PyObject **op, char ***ptr, int *d1, int *d2, int typecode)) > #define PyArray_As2D_NUM 22 > > #define PyArray_Free_RET int > #define PyArray_Free_PROTO Py_FPROTO((PyObject *op, char *ptr)) > #define PyArray_Free_NUM 23 > > /* array_from_existing_data = PyArray_FromDimsAndDataAndDescr(n_dims, dims[n_dims], descr, old_data); */ > /* WARNING: using PyArray_FromDimsAndDataAndDescr is not reccommended! It should only be used to refer to */ > /* global arrays that will never be freed (like FORTRAN common blocks). */ > #define PyArray_FromDimsAndDataAndDescr_RET PyObject * > #define PyArray_FromDimsAndDataAndDescr_PROTO Py_PROTO((int, int *, PyArray_Descr *, char *)) > #define PyArray_FromDimsAndDataAndDescr_NUM 24 > > /* Total number of C API pointers */ > #define PyArray_API_pointers 25 > > > #ifdef _ARRAY_MODULE > extern PyExtensionClass PyArray_Type; > > /* define PyArray_Check(op) ((op)->ob_type == &PyArray_Type) */ > #define PyArray_Check(op) (ExtensionClassSubclassInstance_Check((op), &(PyArray_Type))) > > > extern PyArray_SetNumericOps_RET PyArray_SetNumericOps \ > PyArray_SetNumericOps_PROTO; > extern PyArray_INCREF_RET PyArray_INCREF PyArray_INCREF_PROTO; > extern PyArray_XDECREF_RET PyArray_XDECREF PyArray_XDECREF_PROTO; > extern PyArrayError_RET PyArrayError PyArrayError_PROTO; > extern PyArray_SetStringFunction_RET PyArray_SetStringFunction \ > PyArray_SetStringFunction_PROTO; > extern PyArray_DescrFromType_RET PyArray_DescrFromType \ > PyArray_DescrFromType_PROTO; > extern PyArray_Cast_RET PyArray_Cast PyArray_Cast_PROTO; > extern PyArray_CanCastSafely_RET PyArray_CanCastSafely \ > PyArray_CanCastSafely_PROTO; > extern PyArray_ObjectType_RET PyArray_ObjectType PyArray_ObjectType_PROTO; > extern _PyArray_multiply_list_RET _PyArray_multiply_list \ > _PyArray_multiply_list_PROTO; > extern PyArray_Size_RET PyArray_Size PyArray_Size_PROTO; > extern PyArray_FromDims_RET PyArray_FromDims PyArray_FromDims_PROTO; > extern PyArray_FromDimsAndData_RET PyArray_FromDimsAndData \ > PyArray_FromDimsAndData_PROTO; > extern PyArray_FromDimsAndDataAndDescr_RET PyArray_FromDimsAndDataAndDescr \ > PyArray_FromDimsAndDataAndDescr_PROTO; > extern PyArray_ContiguousFromObject_RET PyArray_ContiguousFromObject \ > PyArray_ContiguousFromObject_PROTO; > extern PyArray_CopyFromObject_RET PyArray_CopyFromObject \ > PyArray_CopyFromObject_PROTO; > extern PyArray_FromObject_RET PyArray_FromObject PyArray_FromObject_PROTO; > extern PyArray_Return_RET PyArray_Return PyArray_Return_PROTO; > extern PyArray_Reshape_RET PyArray_Reshape PyArray_Reshape_PROTO; > extern PyArray_Copy_RET PyArray_Copy PyArray_Copy_PROTO; > extern PyArray_Take_RET PyArray_Take PyArray_Take_PROTO; > extern PyArray_As1D_RET PyArray_As1D PyArray_As1D_PROTO; > extern PyArray_As2D_RET PyArray_As2D PyArray_As2D_PROTO; > extern PyArray_Free_RET PyArray_Free PyArray_Free_PROTO; > > > #else > > /* C API address pointer */ > #if defined(NO_IMPORT) || defined(NO_IMPORT_ARRAY) > extern void **PyArray_API; > #else > void **PyArray_API; > #endif > > #define PyArray_Check(op) \ > MakeSureExtensionClassImported && ExtensionClassSubclassInstance_Check(op, (PyExtensionClass *)PyArray_API[PyArray_Type_NUM]) > #define PyArray_Type *(PyExtensionClass *)PyArray_API[PyArray_Type_NUM] > > #define PyArray_SetNumericOps \ > (*(PyArray_SetNumericOps_RET (*)PyArray_SetNumericOps_PROTO) \ > PyArray_API[PyArray_SetNumericOps_NUM]) > #define PyArray_INCREF \ > (*(PyArray_INCREF_RET (*)PyArray_INCREF_PROTO) \ > PyArray_API[PyArray_INCREF_NUM]) > #define PyArray_XDECREF \ > (*(PyArray_XDECREF_RET (*)PyArray_XDECREF_PROTO) \ > PyArray_API[PyArray_XDECREF_NUM]) > #define PyArrayError \ > (*(PyArrayError_RET (*)PyArrayError_PROTO) \ > PyArray_API[PyArrayError_NUM]) > #define PyArray_SetStringFunction \ > (*(PyArray_SetStringFunction_RET (*)PyArray_SetStringFunction_PROTO) \ > PyArray_API[PyArray_SetStringFunction_NUM]) > #define PyArray_DescrFromType \ > (*(PyArray_DescrFromType_RET (*)PyArray_DescrFromType_PROTO) \ > PyArray_API[PyArray_DescrFromType_NUM]) > #define PyArray_Cast \ > (*(PyArray_Cast_RET (*)PyArray_Cast_PROTO) \ > PyArray_API[PyArray_Cast_NUM]) > #define PyArray_CanCastSafely \ > (*(PyArray_CanCastSafely_RET (*)PyArray_CanCastSafely_PROTO) \ > PyArray_API[PyArray_CanCastSafely_NUM]) > #define PyArray_ObjectType \ > (*(PyArray_ObjectType_RET (*)PyArray_ObjectType_PROTO) \ > PyArray_API[PyArray_ObjectType_NUM]) > #define _PyArray_multiply_list \ > (*(_PyArray_multiply_list_RET (*)_PyArray_multiply_list_PROTO) \ > PyArray_API[_PyArray_multiply_list_NUM]) > #define PyArray_Size \ > (*(PyArray_Size_RET (*)PyArray_Size_PROTO) \ > PyArray_API[PyArray_Size_NUM]) > #define PyArray_FromDims \ > (*(PyArray_FromDims_RET (*)PyArray_FromDims_PROTO) \ > PyArray_API[PyArray_FromDims_NUM]) > #define PyArray_FromDimsAndData \ > (*(PyArray_FromDimsAndData_RET (*)PyArray_FromDimsAndData_PROTO) \ > PyArray_API[PyArray_FromDimsAndData_NUM]) > #define PyArray_FromDimsAndDataAndDescr \ > (*(PyArray_FromDimsAndDataAndDescr_RET (*)PyArray_FromDimsAndDataAndDescr_PROTO) \ > PyArray_API[PyArray_FromDimsAndDataAndDescr_NUM]) > #define PyArray_ContiguousFromObject \ > (*(PyArray_ContiguousFromObject_RET (*)PyArray_ContiguousFromObject_PROTO) \ > PyArray_API[PyArray_ContiguousFromObject_NUM]) > #define PyArray_CopyFromObject \ > (*(PyArray_CopyFromObject_RET (*)PyArray_CopyFromObject_PROTO) \ > PyArray_API[PyArray_CopyFromObject_NUM]) > #define PyArray_FromObject \ > (*(PyArray_FromObject_RET (*)PyArray_FromObject_PROTO) \ > PyArray_API[PyArray_FromObject_NUM]) > #define PyArray_Return \ > (*(PyArray_Return_RET (*)PyArray_Return_PROTO) \ > PyArray_API[PyArray_Return_NUM]) > #define PyArray_Reshape \ > (*(PyArray_Reshape_RET (*)PyArray_Reshape_PROTO) \ > PyArray_API[PyArray_Reshape_NUM]) > #define PyArray_Copy \ > (*(PyArray_Copy_RET (*)PyArray_Copy_PROTO) \ > PyArray_API[PyArray_Copy_NUM]) > #define PyArray_Take \ > (*(PyArray_Take_RET (*)PyArray_Take_PROTO) \ > PyArray_API[PyArray_Take_NUM]) > #define PyArray_As1D \ > (*(PyArray_As1D_RET (*)PyArray_As1D_PROTO) \ > PyArray_API[PyArray_As1D_NUM]) > #define PyArray_As2D \ > (*(PyArray_As2D_RET (*)PyArray_As2D_PROTO) \ > PyArray_API[PyArray_As2D_NUM]) > #define PyArray_Free \ > (*(PyArray_Free_RET (*)PyArray_Free_PROTO) \ > PyArray_API[PyArray_Free_NUM]) > > #define import_array() \ > { \ > PyObject *numpy = PyImport_ImportModule("_numpy"); \ > if (numpy != NULL) { \ > PyObject *module_dict = PyModule_GetDict(numpy); \ > PyObject *c_api_object = PyDict_GetItemString(module_dict, "_ARRAY_API"); \ > if (PyCObject_Check(c_api_object)) { \ > PyArray_API = (void **)PyCObject_AsVoidPtr(c_api_object); \ > } \ > } \ > } > > #endif > > #ifdef __cplusplus > } > #endif > #endif /* !Py_ARRAYOBJECT_H */ > From v_pascucci@hotmail.com Tue Aug 17 19:40:29 1999 From: v_pascucci@hotmail.com (Valerio Pascucci) Date: Tue, 17 Aug 1999 13:40:29 CDT Subject: [Matrix-SIG] Resampling a 3D function Message-ID: <19990817184029.2177.qmail@hotmail.com> I have a 3D (discrete) function represented as a 3D array of floats of dimensions (l,m,n). I would like to resample it at a different resolution (l1,m1,n1). Is there any function that allows to do the resampling directly (like the PIL resize function would do for 2D images?) thanks Valerio _______________________________________________________________ Get Free Email and Do More On The Web. Visit http://www.msn.com From S.I.Reynolds@cs.bham.ac.uk Wed Aug 18 12:12:59 1999 From: S.I.Reynolds@cs.bham.ac.uk (Stuart Reynolds) Date: Wed, 18 Aug 1999 12:12:59 +0100 Subject: [Matrix-SIG] Deepcopying numeric array Message-ID: <37BA953B.23C5@cs.bham.ac.uk> Does anyone know why the Numeric arrays can't be deepcopied? Is it just an omission? Cheers Stuart >>> from Numeric import * >>> a = array([1,2,3]) >>> a array([1, 2, 3]) >>> from copy import * >>> deepcopy(a) Traceback (innermost last): File "", line 1, in ? File "/bham/ums/common/pd/packages/Python/lib/python1.5/copy.py", line 147, in deepcopy raise error, \ copy.error: un-deep-copyable object of type >>> -- Office: G8 School of Computer Science, The University of Birmingham, Edgbaston, Birmingham, B15 2TT. UK. tel: +44-(0)-121-414-7969 mailto:sir@cs.bham.ac.uk http://www.cs.bham.ac.uk/~sir/ From da@ski.org Thu Aug 19 05:54:20 1999 From: da@ski.org (David Ascher) Date: Wed, 18 Aug 1999 21:54:20 -0700 (Pacific Daylight Time) Subject: [Matrix-SIG] Re: Deepcopying numeric array In-Reply-To: <37BB8989.7BA08CA3@compaq.com> Message-ID: On Thu, 19 Aug 1999, Greg Ewing wrote: > Stuart Reynolds wrote: > > > > Does anyone know why the Numeric arrays can't be deepcopied? Is it just > > an omission? That's an omission. With release 12 BETA, I added a __deepcopy__ method in a single line: in: class Array(base): add: def __deepcopy__(self, memo): return array(self, copy=1) This won't work for releases 1-11, though -- I don't have a fix for those releases. (btw, I don't understand what one is supposed to do w/ the memo argument...) > Not only that, but I just found out a few minutes > ago that they're un-comparable and un-hashable! > Is there any good reason for these restrictions? > Are they on hold awaiting Rich Comparisons or > something? Uncomparability is due to the fact that comparison of arrays right now is impossible to do meaningfully. Meaningful comparison will have to wait for rich comparisons, yes. In early version of NumPy, the comparison returned the result of the comparison between the pointer values =). The current behavior is better. Unhashability is a direct consequence of the mutability, just as in the case of dictionaries & lists. --david ascher From hinsen@cnrs-orleans.fr Tue Aug 24 09:36:28 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Tue, 24 Aug 1999 10:36:28 +0200 Subject: [Matrix-SIG] Resampling a 3D function In-Reply-To: <19990817184029.2177.qmail@hotmail.com> (v_pascucci@hotmail.com) References: <19990817184029.2177.qmail@hotmail.com> Message-ID: <199908240836.KAA13085@chinon.cnrs-orleans.fr> > I have a 3D (discrete) function represented as a 3D array of > floats of dimensions (l,m,n). > I would like to resample it at a different resolution (l1,m1,n1). > Is there any function that allows to do the resampling directly > (like the PIL resize function would do for 2D images?) Not in Numeric at least, nor anywhere I know of. The closest I know is the module Functions.Interpolation in my ScientificPython package (http://starship.python.net/crew/hinsen/scientific.html). It provides a callable function object that interpolates between points on a grid. Adding a resampling operation should be straightforward. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From hinsen@cnrs-orleans.fr Tue Aug 24 09:42:37 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Tue, 24 Aug 1999 10:42:37 +0200 Subject: [Matrix-SIG] Modules: Shared Lib In-Reply-To: (message from Patrick Dahiroc on Tue, 17 Aug 1999 09:41:30 -0400 (EDT)) References: Message-ID: <199908240842.KAA13109@chinon.cnrs-orleans.fr> > i'm having problems making a shared object for a python extension module > that i wrote. below is the steps i took to make the module. (i just > followed the steps in the GCC-HOWTO) I suggest you do it "the Python way", as described in the "Extending and Embedding" manual: http://www.python.org/doc/current/ext/building-on-unix.html -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From r.hooft@euromail.net Wed Aug 25 13:18:43 1999 From: r.hooft@euromail.net (Rob Hooft) Date: Wed, 25 Aug 1999 14:18:43 +0200 (MZT) Subject: [Matrix-SIG] "Sorting" a large array?! Message-ID: <14275.57123.542027.549203@octopus.chem.uu.nl> In my application I have a series (upto hundreds) of 2D arrays (625x576). I am interested in one-but-the-lowest or two-but-the-lowest element at each location of the 2D arrays. Currently I have programmed this like: class NthLowestSink(imagesink.AccumulatingSink): n=3 # Which element are we interested in. n=1 calculates the "minimum" mem=3 def Frame(self,data): data=data.astype('f') if not hasattr(self,'arrdata'): self.arrdata=data[Numeric.NewAxis,:,:] else: self.arrdata=Numeric.concatenate((self.arrdata,data[Numeric.NewAxis,:,:])) if len(self.arrdata)>=self.mem: self.arrdata=Numeric.sort(self.arrdata,axis=0)[0:self.n] def FinalEnd(self): self.arrdata=Numeric.sort(self.arrdata,axis=0)[0:self.n] self.data=self.arrdata[self.n-1,:,:] To simplify: for n=1, this just does a Numeric.minimum.reduce along the frames. Obviously I do not have enough memory to do everything at once. Instead, each time a new frame is added, the array is sorted on the frame-axis (axis=0) and the lowest element is kept. If n>1, more than one element needs to be kept. To save a bit of time, the "mem" variable is added. If mem is raised, this reduces the number of sort operations, speeding up the process at the expense of memory usage. I have 2 questions: 1) Am I missing something obvious that might speed this up? It is quite slow like this.... 2) The "concatenate" operation currently duplicates memory completely. Slow and expensive. Is there a better way? Thanks for any hints. Rob -- ===== R.Hooft@EuroMail.net http://www.xs4all.nl/~hooft/rob/ ===== ===== R&D, Nonius BV, Delft http://www.nonius.nl/ ===== ===== PGPid 0xFA19277D ========================== Use Linux! ========= From r.hooft@euromail.net Wed Aug 25 13:36:18 1999 From: r.hooft@euromail.net (Rob Hooft) Date: Wed, 25 Aug 1999 14:36:18 +0200 (MZT) Subject: [Matrix-SIG] "Sorting" a large array?! In-Reply-To: <14275.57123.542027.549203@octopus.chem.uu.nl> References: <14275.57123.542027.549203@octopus.chem.uu.nl> Message-ID: <14275.58178.872193.550358@octopus.chem.uu.nl> >>>>> "RH" == Rob Hooft writes: RH> 2) The "concatenate" operation currently duplicates memory RH> completely. Slow and expensive. Is there a better way? Hmmm.... Writing such a message definitely clears ones mind. I managed to avoid the concatenate: def Frame(self,data): data=data.astype('f') if not hasattr(self,'arrdata'): self.arrdata=Numeric.zeros((self.mem+1,)+data.shape,'f') self.arrindex=0 self.arrdata[self.arrindex]=data self.arrindex=self.arrindex+1 if self.arrindex>=self.mem+1: self.arrdata=Numeric.sort(self.arrdata,axis=0) self.arrindex=self.n def FinalEnd(self): self.arrdata=Numeric.sort(self.arrdata[:self.arrindex],axis=0) self.data=self.arrdata[self.n-1] del self.arrdata Any other help is still appreciated. Regards, -- ===== R.Hooft@EuroMail.net http://www.xs4all.nl/~hooft/rob/ ===== ===== R&D, Nonius BV, Delft http://www.nonius.nl/ ===== ===== PGPid 0xFA19277D ========================== Use Linux! ========= From hinsen@cnrs-orleans.fr Wed Aug 25 18:16:28 1999 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed, 25 Aug 1999 19:16:28 +0200 Subject: [Matrix-SIG] "Sorting" a large array?! In-Reply-To: <14275.58178.872193.550358@octopus.chem.uu.nl> (r.hooft@euromail.net) References: <14275.57123.542027.549203@octopus.chem.uu.nl> <14275.58178.872193.550358@octopus.chem.uu.nl> Message-ID: <199908251716.TAA32228@chinon.cnrs-orleans.fr> > Any other help is still appreciated. It all depends on how large you can afford to make self.mem. Suppose you can make it significantly larger than self.n, then the following strategy might be better: Whenever arrdata is "full", locate the self.n lowest values at each array position, pack them into the first self.n slots, and discard everything else, making space for adding new frames. For locating the lowest values, try the following: 1) Calculate the minima with minimum.reduce. 2) Create a mask that discards all values equal to the minimum from a subsequent search (this requires some restriction on the data range in your arrays), and use minimum.reduce again to find the second-lowest values. 3) Continue up to self.n lowest values. Good luck, Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ------------------------------------------------------------------------------- From pearu@ioc.ee Fri Aug 27 17:47:03 1999 From: pearu@ioc.ee (Pearu Peterson) Date: Fri, 27 Aug 1999 19:47:03 +0300 (EETDST) Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions Message-ID: Travis and others interested in the subject, I am in a process of rewriting the fortran (in future, also C) to python interface generator (f2py.py) and I am looking for opinions on the following question: How the Python user should see the interfaced fortran routine? I will be more specific in below: ** Consider the interface of fortran (77,90) subroutine `sub`: subroutine sub(a,b,c,t) integer intent(in) :: a integer intent(out) :: b integer intent(inout) :: c integer intent(temporary) :: t end subroutine sub where argument `a` is considered as an input parameter, `b` is a result of `sub`, `c` is both input and output parameter, and `t` is a temporary (work space) variable (this is quite common in fortran 77 routines when work space is required for the routine). ** Now say that the interface generator generates (and compiles) Py/C API module `test` containing function `sub`. As I see it, there are at least three possible ways to call `sub` and interpret its arguments and results: 1) test.sub(a,b,c,t) --- that is in python `sub` has similar definition as in Fortran. This is advantage if interfacing automatically large fortran libraries like lapack and others: almost no need to write documentation as it is available for fortran routines. Disadvantage is that in this case all out and inout arguments must be Numeric arrays, even for scalars and this might be confusing. Also, the user must supply the work-space variable `t`, though for the end python user it should be irrelevant. 2) test.sub(a,b,c) --- this is a kind of reduced form of 1): no work-space variable in argument list (the work space is allocated automatically in the Py/C API interface). There are fortran routines that take, say, 20 arguments, and say, half of them, are work-space variables. So, for these cases reducing the number of arguments may seem to be an advantage, but the user may be even more confused if in/out/inout/temporary arguments for fortran routine are mixed, and as a result, the fortran documentation does not hold for interfaced python function. Disadvantage of hiding workspace variables is that they are allocated/disallocated in each call to function which is not good for performance. 3) b,c=test.sub(a,b) --- here all output of `sub` is returned by the functions `sub` as a tuple. The advantage of this approach is that here it is much clearer which arguments are input and which output. As an disadvantage, the fortran documentation does not hold. Above I have pointed out that in cases 2) and 3) the fortran documentations will not hold and I have considered this as a disadvantage. But is this an argument for python user who is also novice in fortran? Another point is that if making an interface to some fortran library, the end user needs not to specify all the arguments that fortran routine initially has: there should be (almost always) a higher level interface written in python that deals with, say, specifying arguments (to fortran routine) representing the length of vectors and so reducing the number of arguments in python. In conclusion, interfaced fortran routines need python specific documentation anyway. That's why I have proposed options 2) and 3). Any thoughts, suggestions are very welcome as I need to decide in which direction I should proceed. All given ways are possible in terms of optional flags, but which way should be default? With regards, Pearu From jhauser@ifm.uni-kiel.de Fri Aug 27 18:54:26 1999 From: jhauser@ifm.uni-kiel.de (jhauser@ifm.uni-kiel.de) Date: Fri, 27 Aug 1999 19:54:26 +0200 (CEST) Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions In-Reply-To: References: Message-ID: <14278.53131.35635.499125@lisboa> If you say additional flags, do you mean flags during the extension_module creation or at runtime? If at creation time, I would say, if one does see a need to write additional wrapper code in Python he can choose the option flag to produce a raw-binding (1). For the rest I like (3) as the default. One question I have, how would common blocks be handled. If these are wraped into workspace fields on the python side than there needs to be a way to use them as arguments. But I don't know if you think already about common-blocks. 0.02 __Janko From pearu@ioc.ee Fri Aug 27 19:32:28 1999 From: pearu@ioc.ee (Pearu Peterson) Date: Fri, 27 Aug 1999 21:32:28 +0300 (EETDST) Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions In-Reply-To: <14278.53131.35635.499125@lisboa> Message-ID: On Fri, 27 Aug 1999 jhauser@ifm.uni-kiel.de wrote: > If you say additional flags, do you mean flags during the > extension_module creation or at runtime? > > If at creation time, I would say, if one does see a need to write > additional wrapper code in Python he can choose the option flag to > produce a raw-binding (1). This is what I meant. > > For the rest I like (3) as the default. One question I have, how would > common blocks be handled. If these are wraped into workspace fields on > the python side than there needs to be a way to use them as arguments. > > But I don't know if you think already about common-blocks. Yes, I have thought about common blocks and my new code supports them as well. Common blocks need not be in the arguments of the functions. I have thinking of interacting with them directly, with additional get/set functions in the module. For example, subroutine sub integer i,j common /cblock1/ i,j j=i end In python import test test.cblock1_i_set(5) test.sub() print test.cblock1_j_get() would print '5' If you can think of more natural way in seeing common blocks from python, I would like to know as the above is just first solution that I thought of. Thanks, Pearu From pas@scansoft.com Fri Aug 27 21:40:13 1999 From: pas@scansoft.com (Perry A. Stoll) Date: Fri, 27 Aug 1999 16:40:13 -0400 Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking foropinions Message-ID: <01bef0cc$5cc065a0$3e3e79cf@nara.scansoft.com> Pearu wrote: > subroutine sub > integer i,j > common /cblock1/ i,j > j=i > end >In python > import test > test.cblock1_i_set(5) > test.sub() > print test.cblock1_j_get() >would print '5' > >If you can think of more natural way in seeing common blocks from python, >I would like to know as the above is just first solution that I >thought of. How about creating a CommonBlock object that uses __getattr__, __setattr__ to check for the appropriate attributes. Then the above could look like: > import test > test.cblock1.i = 5 > print test.cblock1.j 5 -Perry From amullhau@zen-pharaohs.com Sat Aug 28 16:57:42 1999 From: amullhau@zen-pharaohs.com (Andrew P. Mullhaupt) Date: Sat, 28 Aug 1999 11:57:42 -0400 Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions References: Message-ID: <048b01bef16e$3c21d7a0$99a0720a@amullhau> From: Pearu Peterson > > Travis and others interested in the subject, > > I am in a process of rewriting the fortran (in future, also C) to python > interface generator (f2py.py) and I am looking for opinions on the > following question: > How the Python user should see the interfaced fortran routine? > I will be more specific in below: > > ** Consider the interface of fortran (77,90) subroutine `sub`: > subroutine sub(a,b,c,t) > integer intent(in) :: a > integer intent(out) :: b > integer intent(inout) :: c > integer intent(temporary) :: t > end subroutine sub > where argument `a` is considered as an input parameter, `b` is a result of > `sub`, `c` is both input and output parameter, and `t` is a temporary > (work space) variable (this is quite common in fortran 77 routines when > work space is required for the routine) intent(temporary) doesn't appear to be standard f90, f95, or proposed f2k. I think you actually get better results by _not_ declaring the intent of such a variable and using the compiler's options for what to do about automatic variables. This is probably why intent(temporary) isn't standard and may never be. > Also, the user must supply the work-space > variable `t`, though for the end python user it should be irrelevant. There are _huge_ performance issues surrounding the allocation of temporaries. > 2) test.sub(a,b,c) --- this is a kind of reduced form of 1): no work-space > variable in argument list (the work space is allocated automatically in > the Py/C API interface). Bad idea. The programmer wants the choice of allocation method. > Disadvantage of hiding workspace variables is that they are > allocated/disallocated in each call to function which is not good for > performance. Hideous for performance. Do not do this. A lot of S experience has proven this. S copies values, no matter how large before passing them to an external routine, and this puts a high minimum on the amount of work a routine has to do to overcome before it makes sense to call out. One of the reasons people have been bailing out of S-plus and Matlab in favor of Numerical Python is that although Numerical Python has worse indexing and graphics, it is far friendlier to calling out. In this case, the workaround would be to wrap the F77 code in another language to handle the temporary, but then they wouldn't be using the Fortran-to-Python interface generator. Plus, there is the "save" attribute. > 3) b,c=test.sub(a,b) --- here all output of `sub` is returned by the > functions `sub` as a tuple. The advantage of this approach is that here it > is much clearer which arguments are input and which output. As an > disadvantage, the fortran documentation does not hold. You really have to let the Fortran documentation go. In cases of Fortran 77 (which is now twice obsoleted by F90 and F95) a good deal of the Fortran documentation relates to complicated subprogram interfaces which no longer make sense in modern languages. When people write F90 and F95 wrappers to F77 libraries they document the interface to the wrapper. > In conclusion, interfaced fortran routines need python specific > documentation anyway. Yes. > Any thoughts, suggestions are very welcome as I need to decide in which > direction I should proceed. Keep in mind that Fortran 77 is no longer a standard language. F90 isn't either. F95 obsoleted them both. Although compiler vendors are still writing stuff that can compile F77, some features of F77 actually were obsoleted and more have been deprecated in F95, and the upcoming F2000 standard may change some more. The way to proceed is to move toward adding F95 compatibility. The interfaces are infinitely cleaner, and many of the painful issues you will have with F77 will go away. >. All given ways are possible in terms of > optional flags, but which way should be default? Since the newer Fortran variants reduce the appearance of temporaries in interfaces, the default should not take into account storage for temporaries. People who want automatic allocation of temporaries (rarely appropriate) should have to ask for it by setting a flag. Later, Andrew Mullhaupt From dubois1@llnl.gov Sat Aug 28 17:57:54 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Sat, 28 Aug 1999 09:57:54 -0700 Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions References: <048b01bef16e$3c21d7a0$99a0720a@amullhau> Message-ID: <000d01bef176$78a27420$f4160218@plstn1.sfba.home.com> > > intent(temporary) doesn't appear to be standard f90, f95, or proposed f2k. > > I think you actually get better results by _not_ declaring the intent of > such a variable and using the compiler's options for what to do about > automatic variables. This is probably why intent(temporary) isn't standard > and may never be. > intent(temporary) is an option in Pyfort, not Fortran. It indicates that the user WANTS this array created by the tool. If the user doesn't want it, don't use it. The point is to allow a simpler calling sequence for the Python user. Again, if you don't think that is a good idea, you simply declare all arrays as inout and Pyfort doesn't touch them. From amullhau@zen-pharaohs.com Sun Aug 29 00:32:00 1999 From: amullhau@zen-pharaohs.com (Andrew P. Mullhaupt) Date: Sat, 28 Aug 1999 19:32:00 -0400 Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions References: <048b01bef16e$3c21d7a0$99a0720a@amullhau> <000d01bef176$78a27420$f4160218@plstn1.sfba.home.com> Message-ID: <00bc01bef1af$1c6c8f40$99a0720a@amullhau> > intent(temporary) is an option in Pyfort, not Fortran. It indicates that the > user WANTS this array > created by the tool. If the user doesn't want it, don't use it. Oh, OK. Sorry for the misunderstanding. I thought this might be a situation in which the Fortran code would result in automatic allocation by the interface. The more things you can stop things from allocating memory so you can do it yourself the better your life is. > The point is to allow a simpler calling sequence for the Python user. Again, > if you don't think that is a good idea, you simply declare all arrays as > inout and Pyfort doesn't touch them. No problem. Later, Andrew Mullhaupt From dubois1@llnl.gov Sun Aug 29 05:10:23 1999 From: dubois1@llnl.gov (Paul F. Dubois) Date: Sat, 28 Aug 1999 21:10:23 -0700 Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions References: <048b01bef16e$3c21d7a0$99a0720a@amullhau> <000d01bef176$78a27420$f4160218@plstn1.sfba.home.com> <00bc01bef1af$1c6c8f40$99a0720a@amullhau> Message-ID: <000701bef1d4$6adbc0e0$f4160218@plstn1.sfba.home.com> > Oh, OK. Sorry for the misunderstanding. I thought this might be a situation > in which the Fortran code would result in automatic allocation by the > interface. The more things you can stop things from allocating memory so you > can do it yourself the better your life is. > No, you didn't misunderstand. If you declare a temporary array, the interface does the allocation/deallocation of the array rather than force the user to supply it. If you are calling something in a loop and it doesn't do a lot of work and a bunch of other ifs, it might be important not to do this. However, I disagree with the notion that the more things I can do myself the better my life is. (:->. It is worth noting that nothing says you cannot build two Python interfaces to the same Fortran routine. Just put them in different modules, say module fastbutclumsytouse and module normal. From amullhau@zen-pharaohs.com Sun Aug 29 17:23:30 1999 From: amullhau@zen-pharaohs.com (Andrew P. Mullhaupt) Date: Sun, 29 Aug 1999 12:23:30 -0400 Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions References: <048b01bef16e$3c21d7a0$99a0720a@amullhau> <000d01bef176$78a27420$f4160218@plstn1.sfba.home.com> <00bc01bef1af$1c6c8f40$99a0720a@amullhau> <000701bef1d4$6adbc0e0$f4160218@plstn1.sfba.home.com> Message-ID: <052501bef23a$ea51b1c0$99a0720a@amullhau> > > Oh, OK. Sorry for the misunderstanding. I thought this might be a > situation > > in which the Fortran code would result in automatic allocation by the > > interface. The more things you can stop things from allocating memory so > you > > can do it yourself the better your life is. > > > No, you didn't misunderstand. If you declare a temporary array, the > interface does the allocation/deallocation of the array rather than force > the user to supply it. If that is in fact the case it's hideous. Of course, it's only hideous for someone who declares an array as intent(temporary), which does not occur in any Fortran standard, so it won't affect me. > If you are calling something in a loop and it doesn't do a lot of work and a > bunch of other ifs, it might be important not to do this. That is exactly the kind of thing that makes extending a good interpreter important. A lot of times, you don't want to give up the flexibility of the interpreter, but the interpreter, usually for reasons of design choice, cannot provide the necessary performance. So you have to call out. When you call out, it's easiest to wrap up only that part which cannot be provided by the interpreter - typically an inner loop or other simple performance critical stretch of code, and only wrap that. There is a lot of "let's wrap the whole library function" which is a somewhat different situation, but then you end up with things that do not really integrate well with the rest of the interpreter that you call. A perfect example is what to do with Lapack. In truth the thing to do is to wrap up some parts of the BLAS and then translate the higher level algorithms into Python. People with Fortran 90 have realized this which is why Lapack90 is not a translation of Lapack. A good deal of the BLAS can be junked too, since you no longer need to call _axpy when you can write y = beta * y + alpha * matmul(a, x) in Fortran90, which does the same thing. Note here that this is also a Fortran90 implementation of _gemv as well. Note that there is still a performance issue depending on the vendor's implementation of matmul, but most of them are essentially calling _gemm or _gemv as the case may be. That _is_ suboptimal, but you can fix that by writing your own array function mymatmul, etc. > However, I disagree with the notion that the more things I can do myself the > better my life is. (:->. Let's put it another way. The more often I can stop other people from calling malloc or using space on the stack the better my life is. Both can be needlessly harmful ways of using memory, used mainly because most programmers are unaware of alternatives. > It is worth noting that nothing says you cannot build two Python interfaces > to the same Fortran routine. Just put them in different modules, say module > fastbutclumsytouse and module normal. Actually, it's more like once the fast one is out there "normal" gets left behind. The reason is that the fast one will be no harder to use, and will make sense to use more often. It's kind of like why Python is gradually being preferred to other, more functional interpreters as more people discover it. Later, Andrew Mullhaupt From godzilla@netmeg.net (Les Schaffer) Sun Aug 29 19:10:51 1999 From: godzilla@netmeg.net (Les Schaffer) (Les Schaffer) Date: Sun, 29 Aug 1999 14:10:51 -0400 (EDT) Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions In-Reply-To: <052501bef23a$ea51b1c0$99a0720a@amullhau> References: <048b01bef16e$3c21d7a0$99a0720a@amullhau> <000d01bef176$78a27420$f4160218@plstn1.sfba.home.com> <00bc01bef1af$1c6c8f40$99a0720a@amullhau> <000701bef1d4$6adbc0e0$f4160218@plstn1.sfba.home.com> <052501bef23a$ea51b1c0$99a0720a@amullhau> Message-ID: <14281.30635.355630.30623@gargle.gargle.HOWL> andrew said: > Let's put it another way. The more often I can stop other people > from calling malloc or using space on the stack the better my life > is. Both can be needlessly harmful ways of using memory, used mainly > because most programmers are unaware of alternatives. can you or someone else explain what this means? when is it bad to use malloc? what alternatives? les -- ____ Les Schaffer ___| --->> Engineering R&D <<--- Theoretical & Applied Mechanics | Designspring, Inc. Center for Radiophysics & Space Research | http://www.designspring.com/ Cornell Univ. schaffer@tam.cornell.edu | les@designspring.com From amullhau@zen-pharaohs.com Sun Aug 29 21:41:22 1999 From: amullhau@zen-pharaohs.com (Andrew P. Mullhaupt) Date: Sun, 29 Aug 1999 16:41:22 -0400 Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions References: <048b01bef16e$3c21d7a0$99a0720a@amullhau> <000d01bef176$78a27420$f4160218@plstn1.sfba.home.com> <00bc01bef1af$1c6c8f40$99a0720a@amullhau> <000701bef1d4$6adbc0e0$f4160218@plstn1.sfba.home.com> <052501bef23a$ea51b1c0$99a0720a@amullhau> <000701bef240$6d90fdc0$f4160218@plstn1.sfba.home.com> Message-ID: <059c01bef25e$ee8fc640$99a0720a@amullhau> > almost always the case that you would not want to use this > feature. Obviously we are dealing with two different experience bases. My > views are based on fifteen years of being in charge of a Fortran-extendible > interpreter with hundreds of customers; some of those customers had needs > similar to yours. My experience is based on thirty years of programming including eleven years of _not_ being in charge of Fortran-extendible interpreters, although a spent some reasonable fraction of that time in contact with, and contributing to the development of the fastest interpreter known, as well as being consulted authors of several others. As a result I know several ways to do most things interpreters do, and have seen the difference between the "right" ways, the "wrong" ways, and the "usual" ways. As far as I know I am the only person to have had access to source for both the fastest and slowest commercial interpreters for array based languages. In the case of the really fast one, there were many nontrivial cases where due to the careful attention to implementation of the interpreter primitives, it was possible to write code which would outperform Fortran or C code written in a "normally" optimized way. The reason for this was that it made sense to apply an extremely exigent approach to optimizing primitives because they were guaranteed to be reused many times as opposed to the time one could spend optimizing Fortran or C code. Later, Andrew Mullhaupt From milgram@cgpp.com Sun Aug 29 23:01:10 1999 From: milgram@cgpp.com (J. Milgram) Date: Sun, 29 Aug 1999 18:01:10 -0400 Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions In-Reply-To: Your message of "Sat, 28 Aug 1999 11:57:42 EDT." <048b01bef16e$3c21d7a0$99a0720a@amullhau> Message-ID: <199908292201.SAA05784@cgpp.com> > From: "Andrew P. Mullhaupt" > Subject: Re: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions > > > Any thoughts, suggestions are very welcome as I need to decide in which > > direction I should proceed. > > Keep in mind that Fortran 77 is no longer a standard language. Not to take issue with this, but I think what you actually mean is that F77 is obsolete. It's still a standard language, even if it has been formally disowned by ANSI (I don't know for a fact that it has been, but in any event). There's still a widely-available document that defines the language, it hasn't changed since 1978, and it's unlikely to ever change. The very definition of standard. I only mention this because lots of people (e.g. me) still work with Fortran 77 because of (1) legacy code (2) availability of free compilers (g77 and f2c/gcc) and (3) it's *very* portable if you stick to the standard. So I'd discourage any development directions that involve abandoning F77 compatibility at any level. Which I don't think this discussion was really about, but still. regards, Judah From turner@blueskystudios.com Mon Aug 30 16:58:14 1999 From: turner@blueskystudios.com (John A. Turner) Date: Mon, 30 Aug 1999 11:58:14 -0400 (EDT) Subject: [Matrix-SIG] Fortran-to-python-interface-generator: looking for opinions In-Reply-To: <048b01bef16e$3c21d7a0$99a0720a@amullhau> References: <048b01bef16e$3c21d7a0$99a0720a@amullhau> Message-ID: <14282.43542.129979.611915@waterloo.blueskyprod.com> >>>>> "APM" == Andrew P Mullhaupt writes: APM> Keep in mind that Fortran 77 is no longer a standard APM> language. F90 isn't either. F95 obsoleted them both. Although APM> compiler vendors are still writing stuff that can compile F77, APM> some features of F77 actually were obsoleted and more have been APM> deprecated in F95, and the upcoming F2000 standard may change APM> some more. APM> APM> The way to proceed is to move toward adding F95 APM> compatibility. The interfaces are infinitely cleaner, and many of APM> the painful issues you will have with F77 will go away. I am in violent agreement with this sentiment. The reality is that time-honored legacy F77 like LAPACK will be with us for a while, but the more quickly people stop writing new code in F77 the better it will be for everyone. Modern Fortran is just a far superior language. The fact that there is no open source Fortran compiler (g77 is not a Fortran compiler, since F95 is Fortran, F77 is not any more) is unfortunate, but there was no open source F77 compiler for years either. -- John A. Turner, Ph.D. Senior Research Associate Blue Sky Studios http://www.blueskystudios.com/ One South Road, Harrison, NY 10528 http://www.lanl.gov/home/turner/ Phone: (914) 381-8400 http://john.turner.org/ Info on Blue Sky's fully computer-generated film _Bunny_, winner of the 1998 Academy Award in the Short Animated Film category: http://bunny.blueskystudios.com/