From c9603085@zeus.hud.ac.uk Mon Aug 4 12:03:28 1997 From: c9603085@zeus.hud.ac.uk (MISS. K.L.COLLIER) Date: Mon, 4 Aug 1997 12:03:28 +0100 Subject: [MATRIX-SIG] complex conjugate Message-ID: <199708041103.MAA10528@artemis.csm> Hi, I have an array of complex elements (after performing FFT2d) and wish to find the complex conjugate. Is there a function for doing this? Karen Collier c9603085@zeus.hud.ac.uk _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From holz@zib.de Mon Aug 4 13:18:57 1997 From: holz@zib.de (Martin Holz) Date: Mon, 4 Aug 1997 14:18:57 +0200 Subject: [MATRIX-SIG] complex conjugate In-Reply-To: <199708041103.MAA10528@artemis.csm> References: <199708041103.MAA10528@artemis.csm> Message-ID: <199708041218.OAA08269@num15.zib.de> MISS. K.L.COLLIER writes: > Hi, > I have an array of complex elements (after performing FFT2d) and wish to find > the complex conjugate. > Is there a function for doing this? > Yes, there is ufunc in Numeric, named conjugate. y = Numeric.conjugate(x) should do it. Always try the obvious first in python :-). Tschuess Martin _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From chase@att.com Wed Aug 6 18:09:49 1997 From: chase@att.com (Chris Chase) Date: Wed, 6 Aug 97 13:09:49 EDT Subject: [MATRIX-SIG] Prototype array with gather/scatter behavior Message-ID: <9708061709.AA14116@hoccson.ho.att.com> I updated a module I was playing around with last year that implements what I call general product indexing. The module is ProdArray.py. ProdArray.py depends on the Numeric module. It implements an array object that allows subscripts that are scalars, slices, integer sequences (e.g. arrays), NewAxis or Ellipsis. This provides for general scatter/gather behavior. "get" subscripting always produces a reference to the original data unless a scalar (rank-0) is the result. "set" subscripting works as you would expect it. However, when using a ProdArray instance where data elements are referenced more than once, than the behavior of how items are set is not guaranteed, i.e., a = ProdArray([1,2])[[0,0,1,1],] a.asarray() a[:] = [5,6,7,8] a a.asarray() the current implementation results in the original data being set to 6 and 8. The code is not efficient, particularly when setting values via subscripting (just a big loop). There are probably bugs. But I thought this would be a useful prototype. But changing the current Numeric module to have this behavior would require a lot of recoding. Forgive the programming style and lack of extensive documentation. See the docstring for the ProdArray class (of special interest are the resize and reshape methods). Enjoy, Chris chase@att.com -- ProdArray.py included below -- from Numeric import * import types import operator version = "1.0" _arrayType = type(array(0)) # documentation notwithstanding, Numeric.array() does not accept # None as a value for typecode. def _array(a, typecode=None): if typecode == None: return array(a) else: return array(a, typecode) def _projectSlice(s, n, stride=1): # ProdArray helper function for converting slice objects into # projectedSlice objects plus offset. # project a slice into length=n, stride=stride start=0 sequence. # This does the clipping of the slice as done in traditional python way. if n == 0: return 0, projectedSlice(0,1) # if s != types.SliceType: # raise TypeError, 'Slice required' step = s.step if step == None: step = 1 if step > 0: dir = 1 else: dir = -1 start = s.start if start == None: if dir > 0: start = 0 else: start = n-1 if start < 0: start = start + n stop = s.stop if stop == None: if dir > 0: stop = n else: stop = -1 else: if stop < 0: stop = stop + n # do clipping of slice to dimension modulo dir. # I don't like this but it is the current implementation. # It would be much more useful in numerical processing to # generate an exception for slices outside the valid # range as is done when indexing with scalars and take(). if dir > 0: if start < 0: start = start % dir if start > 0: start = dir - start stop = min(n, stop) else: if start >= n: start = (start - (n-1)) % dir if start < 0: start = dir - start start = (n-1) + start stop = max(-1, stop) len = max((stop-(start-step)-dir)/step, 0) # if len == 0: stop = start # else: stop = start+(len-1)*step+dir return start*stride, projectedSlice(len,step*stride) def _substituteEllipsis(index_in, rank): # number of actual indexes present nind = sum(map(lambda i: i != NewAxis and i != Ellipsis, index_in)) # remove Ellipsis replacing with slice ":" z = map(lambda i: i == Ellipsis, index_in) ie = nonzero(z) if len(ie) > 1: raise IndexError, "At most one Ellipsis allowed in indexing." if len(ie) == 1: ie = ie[0] nskip = rank-nind index = list(index_in[0:ie])+[slice(None)]*nskip+list(index_in[ie+1:]) return index else: if nind != rank: raise IndexError, "Incorrect number of indexes." return index_in def _indexArrayCopy(a,index): # a helper function that combines NumPy array indexing behavior # and take() for a generalized indexing scheme. # Used for indexing array indexes in ProdArray.__getitem__() # Indexes a as an array with a sequence of indexes producing a copy # index is a sequence of: integer sequence, scalar, NewAxis, slice, Ellipsis # All indexes perform as with normal indexing, except sequence # indexes have the behavior of take() b = asarray(a) r = len(b.shape) index = _substituteEllipsis(index, r) axis = r-1 all = (slice(None),)*r # apply indexes in reverse order for i in range(len(index)-1,-1,-1): idx = index[i] a_idx = asarray(idx) if len(a_idx.shape) == 0: if idx == NewAxis: b = b[all[0:axis+1]+(idx,Ellipsis)] else: # slices and scalar/rank-0 indexes b = b[all[0:axis]+(a_idx[0],Ellipsis)] else: # sequence indexes b = take(b, a_idx, axis) if idx != NewAxis: axis = axis-1 return array(b) class _projectedSlice: # a simplified slice-like object for by ProdArray representation. # just a stride and a length. Offsets for _projectedSlice's are # always assumed to be zero since ProdArray keeps track of a # single offset for the representation. def __init__(self, length, stride=1): self.stride = stride self.shape = [length] self.length = length self.__array__ = self.asarray def asarray(self,type='i'): if self.stride != 0: return arange(0,self.length*self.stride,self.stride,type) else: return zeros([self.length],type) def __len__(self): return self.length def __getslice__(self, start, stop): return self[start:stop:] def __getitem__(self, i): # return a list containing offset and optionally a # _projectedSlice or a contiguous array # check i for slice if type(i) == types.SliceType: return _projectSlice(i,self.length,self.stride) if type(i) == _projectedSliceType: if i.stride == 0: # convention: 0 stride means replication of first element return 0, projectedSlice(i.length,0) else: return 0, projectedSlice(min([(self.length-1)/abs(i.stride)+1,i.length]),i.stride*self.stride) b = asarray(i) # check i for scalar/rank-0 which result in a scalar return, # i.e. just an offset. if len(b.shape) == 0: b = b[0] if b < 0: b=b+self.length if b < 0 or b >= self.length: raise IndexError, "Subscript out of range" return b*self.stride else: # not a scalar # python behavior b = b + less(b,0)*self.length # error on out of range if logical_or.reduce(ravel(logical_or(less(b,0),greater_equal(b,self.length)))): raise IndexError, "Subscript out of range" return [0, b*self.stride] def __repr__(self): return ("projectedSlice(%d,%d)" % (self.length,self.stride)) def __str__(self): return self.__repr__() _projectedSliceType = type(projectedSlice(1)) def _indirectIndexFromDims(dims): # just a helper function if len(dims) == 0: return [] strides = [1] # last dimension varies fastest for n in range(len(dims)-1,0,-1): strides.append(dims[n]*strides[-1]) strides.reverse() index = map(_projectedSlice,dims,strides) return index # indexing a ProdArray returns a ProdArray containing a reference to # the original data unless a rank-0 array results in which case # indexing returns a scalar. # copy=true - always produces a copy of data # copy=false - will not produce a copy if data is a contiguous array, # instead uses a reference to data class ProdArray: """Prototype array class providing complete product indexing behavior. Indexes can be scalars, slices, integer sequences, NewAxis or Ellipsis. Subscripting a ProdArray produces a reference to the original data. Both "get" and "set" type subscripting is supported. Public Instances: shape - array shape Additional methods: asarray() - returns a copy of object as a Numeric array object. getIndirectIndex() - returns an array that can be used as an indirect index. Subscripting data with this index produces the same data as self. For example, a=arange(10) b=ProdArray(a).reshape([2,5])[:,1:3] i=b.getIndirectIndex() c=take(a,i) # same as b.asarray(). 'c' contains a copy of the same array represented by 'b'. The usefulness would be in using 'i' to index some different data with the same size as 'a'. resize(dims) - resize self to new shape given by 'dims'. Each axis in the shape of self is expanded (via circular repetition of the data) or truncated. New axes are added to the right end of the original shape. The resize representation is very efficient. Any original size 1 axis or new axis is represented by a special _projectedSlice instance with zero stride. reshape(dims,copy=0) - reshapes self to the new shape given by 'dims'. Has same restrictions as reshape() from Numeric module. If 'copy' is false a reference to the original data is returned. Otherwise a Numeric array object is returned with a copy of the data. """ def __init__(self, data, index=None, offset=0, copy=1): # need to do argument checking # only integer self.offset = offset self.array = asarray(data) if index != None: # only a sequence of _projectedSlice or Array. # don't need to check that the index vectors are valid # since any indexing or asarray() accesses will generate # an error self.index = index else: # get shape before flattening self.index = _indirectIndexFromDims(self.array.shape) if copy or not self.array.iscontiguous(): self.array = array(data).flat else: self.array = self.array.flat # Interface for Numeric array objects: __array__ contains a # function that returns the object as a Numeric array self.__array__ = self.asarray self.shape = [] for x in self.index: l = map(None,x.shape) self.shape = self.shape+l def __repr__(self): return 'ProdArray(%s,%s,%s)' % (repr(self.array),repr(self.index),repr(self.offset)) def __getslice__(self, start, stop): return self[start:stop:] def __len__(self): if len(self.shape) == 0: return 0 return self.shape[0] def __getitem__(self, index_in, as_scalar=1): if not operator.isSequenceType(index_in): # scalar or slice by itself index = (index_in, Ellipsis) else: index = index_in r = len(self.shape) # this won't allow rank-0 arrays to be subscripted index = _substituteEllipsis(index, r) v2 = [] i = 0 # index j = 0 # self.index offset = self.offset while i < len(index): if index[i] == NewAxis: v2.append(projectedSlice(1,1)) i = i+1 elif type(self.index[j]) == _arrayType: tidx = [] k = 0 while k < len(self.index[j].shape): tidx.append(index[i]) if index[i] != NewAxis: k = k+1 i = i+1 # subscript the array v2.append(_indexArrayCopy(self.index[j],tidx)) j = j+1 else: # index a slice tidx = self.index[j][index[i]] try: len(tidx) v2.append(tidx[1]) offset = offset+tidx[0] except: #scalar offset = offset+tidx # check for scalar result i = i+1 j = j+1 if v2 or not as_scalar: return ProdArray(self.array, index=v2, offset=offset, copy=0) return self.array[offset] def reshape(self,shape,copy=0): if copy: return reshape(self) else: index = reshape(self.getIndirectIndex().flat,shape) return ProdArray(self.array,index,copy=0) def __setslice__(self,i,j,x): self[i:j:] = x def __setitem__(self,i,x): d = self.__getitem__(i,0) x = asarray(x) try: # check frames aligned - size 1 dimensions are broadcast # The frames are check by aligning their right ends (rather # than from their left ends) for k in range(-len(x.shape),0): if x.shape[k] != 1 and x.shape[k] != d.shape[k]: raise ValueError, "frames not aligned at source index %d" % k except IndexError: raise ValueError, "Source rank too large" x = ProdArray(x).resize(d.shape).asarray().flat idx = d.getIndirectIndex().flat for k in range(len(idx)): d.array[idx[k]] = x[k] def resize(self, dims): idx = [] n = len(self.shape) r = len(dims) x = self.__getitem__((Ellipsis,)+(NewAxis,)*(r-n),0) s = x.shape for i in range(len(dims)): if s[i] == 1: idx.append(projectedSlice(dims[i],0)) elif s[i] >= dims[i]: idx.append(slice(0,dims[i],1)) else: nc = ((dims[i]-1)/s[i]+1) idx.append(concatenate((arange(s[i]),)*nc)[0:dims[i]]) return x.__getitem__(idx,0) def getIndirectIndex(self): # generate an indirect index vector return asarray(reduce(add.outer,self.index,0)+self.offset) def asarray(self, type=None): # take() does not work for scalars or rank-0 indexes return asarray(_indexArrayCopy(self.array,[self.getIndirectIndex()]),type) # I don't know why I need this since __array__ is defined, but # arithmetic doesn't work without it. I don't understand the # coersion rules def __coerce__(self, x): return self.asarray(), x _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Wed Aug 6 19:19:30 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Wed, 6 Aug 1997 14:19:30 -0400 Subject: code generators? Re: [MATRIX-SIG] Prototype array with gather/scatter behavior Message-ID: <199708061815.OAA07944@dante.mh.lucent.com> > The code is not efficient, particularly when setting values via > subscripting (just a big loop). There are probably bugs. But I > thought this would be a useful prototype. But changing the current > Numeric module to have this behavior would require a lot of recoding. Great stuff! Thanks! BTW: my impression was that much of the Numeric C code is generated by python code generators. Is this (still) true? If so, are the generators available? Might make a C-implementation of a scatter-gather (or alternative) C level array easier. (looking at umathmodule with SBYTE_greater, LONG_greater, ... it looks suspiciously like an expanded macro...) -- Aaron Watters _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Aug 8 11:11:15 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 8 Aug 1997 12:11:15 +0200 Subject: [MATRIX-SIG] two newbie questions In-Reply-To: (message from Kelvin Lee on Wed, 23 Jul 1997 18:49:02 -0700 (PDT)) Message-ID: <199708081011.MAA21444@lmspc1.ibs.fr> > 1) How much memory does an NxN Float16 matrix use? 16*N*N plus a small constant. > 2) What's the most efficient way of constructing a "correlation-like" > matrix of a given function for a data set? eg. using, say, RMSE instead > of r? > > To show you where I'm at, I'm stuck with: > > def correlation_like(X, function): > R = zeros((len(X), len(X)), Float16) > for j in range(len(X)): > for i in range(len(X)): > R[i,j] = function(X[i], X[j]) > return R If your function can deal with arrays, it's very simple: R = function(X[:, Numeric.NewAxis], X) Note that any function that uses only arithmetic and the math functions from Numeric can deal with arrays. Example: def function(Xi, Xj): return Numeric.sqrt(Xi-Xj) (I know that this is not a particularly useful example...). -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Aug 8 11:14:08 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 8 Aug 1997 12:14:08 +0200 Subject: [MATRIX-SIG] abs vs absolute In-Reply-To: (message from Jean-Bernard ADDOR on Thu, 24 Jul 1997 11:17:26 -0400 (EDT)) Message-ID: <199708081014.MAA21455@lmspc1.ibs.fr> > What is the difference between: > > abs(a) and absolute(a) ? There should be no difference for arrays, but for other data types there may be differences. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Fri Aug 8 11:22:36 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Fri, 8 Aug 1997 12:22:36 +0200 Subject: [MATRIX-SIG] PyArrayObjects In-Reply-To: (message from Dale Bertrand on Tue, 29 Jul 1997 17:30:24 -0400 (EDT)) Message-ID: <199708081022.MAA21480@lmspc1.ibs.fr> > Unfortunately, it seems to me that PyArray_ContiguousFromObject makes a copy of > the PyObject which I _must_ return to see my changes. I would like my c functio > n to simply access and modify the array data without making a copy. Essentially there are two easy options and one complicated one: 1) Make a copy of the array and return the copy instead of modifying the original. 2) Restrict the function to contiguous arrays (and let it check its argument!). 3) Write some complicated code to modify an arbitrary array. If the C function is supposed to be a general library routine, I recommend the first solution; it's always messy to deal with functions that modify a matrix. If, on the other hand, the C routine is just the low-level work horse for an algorithm implemented mainly in Python, the second solution may be better, and you might even know that the array must be contiguous. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Mon Aug 11 01:19:01 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Sun, 10 Aug 1997 19:19:01 -0500 (CDT) Subject: [MATRIX-SIG] Bug report - seg-fault In-Reply-To: <9708061709.AA14116@hoccson.ho.att.com> Message-ID: I recently found the following anti-social behaviour when multiplying arrays of zero length on one dimension. Python seg-faults when the second dimension is mismatched by 4 or 5 (it depends...). Can anyone else reproduce this? I haven't tried to track it down in the C code, but maybe I'll get a chance to try to next week... >>> from Numeric import * >>> ones((0,1))*ones((0,2)) zeros((0, 2), 'l') >>> ones((0,1))*ones((0,3)) zeros((0, 3), 'l') >>> ones((0,1))*ones((0,4)) Segmentation fault (core dumped) ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jhauser@ifm.uni-kiel.de Mon Aug 11 11:26:53 1997 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Mon, 11 Aug 1997 12:26:53 +0200 (CEST) Subject: [MATRIX-SIG] Bug report - seg-fault In-Reply-To: References: <9708061709.AA14116@hoccson.ho.att.com> Message-ID: Here the same with the llnl-distribution on a linux-box (2.0.30). Also if these commands are executed from a script. __Janko _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jbaddor@phy.ulaval.ca Mon Aug 11 14:00:39 1997 From: jbaddor@phy.ulaval.ca (Jean-Bernard ADDOR) Date: Mon, 11 Aug 1997 09:00:39 -0400 (EDT) Subject: [MATRIX-SIG] Bug report - seg-fault In-Reply-To: Message-ID: Hy ! It seems to give the expected result on sunos. Python 1.4 (Apr 12 1997) [GCC 2.7.2] Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> from Numeric import * >>> ones((0,1))*ones((0,2)) zeros((0, 2), 'l') >>> ones((0,1))*ones((0,3)) zeros((0, 3), 'l') >>> ones((0,1))*ones((0,4)) zeros((0, 4), 'l') >>> A bientot, Jean-Bernard On Sun, 10 Aug 1997, Timothy A. Hochberg wrote: > > I recently found the following anti-social behaviour when multiplying > arrays of zero length on one dimension. Python seg-faults when the second > dimension is mismatched by 4 or 5 (it depends...). Can anyone else > reproduce this? I haven't tried to track it down in the C code, but maybe > I'll get a chance to try to next week... > > >>> from Numeric import * > >>> ones((0,1))*ones((0,2)) > zeros((0, 2), 'l') > >>> ones((0,1))*ones((0,3)) > zeros((0, 3), 'l') > >>> ones((0,1))*ones((0,4)) > Segmentation fault (core dumped) > > > ____ > /im > > +------------------------------------------------+ > |Tim Hochberg Research Assistant | > |hochberg wwa.com University of Illinois | > | Electrical Engineering | > +------------------------------------------------+ > > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Mon Aug 11 15:03:06 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Mon, 11 Aug 1997 10:03:06 -0400 Subject: [MATRIX-SIG] Bug report - seg-fault Message-ID: <199708111410.KAA00848@dante.mh.lucent.com> Win95, no problem either, I think. Strange. ---------- > From: Jean-Bernard ADDOR > To: Timothy A. Hochberg > Cc: matrix-sig@python.org > Subject: Re: [MATRIX-SIG] Bug report - seg-fault > Date: Monday, August 11, 1997 9:00 AM > > Hy ! > > It seems to give the expected result on sunos. > > Python 1.4 (Apr 12 1997) [GCC 2.7.2] > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam > >>> from Numeric import * > >>> ones((0,1))*ones((0,2)) > zeros((0, 2), 'l') > >>> ones((0,1))*ones((0,3)) > zeros((0, 3), 'l') > >>> ones((0,1))*ones((0,4)) > zeros((0, 4), 'l') > >>> > > > A bientot, > > Jean-Bernard _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Timothy A. Hochberg" Message-ID: I think I've found a fix for the problem, although I admit to not fully understanding it. Here's what I've figured out: a) It never happens for sizes < 4 (e.g., zeros((0,3)) * zeros((0,3)) always works). This probably varies from machine to machine. b) For larger sizes, it doesn't always fail. I'd be interested in hearing from those of you for whom it didn't crash before as to whether it crashes if you repeat zeros((0,1000)) * zeros((0,1000)) several times. c) The problem arises because some of the stride lengths in a zero size array will be longer than the memory allocated for the array (which turns out to be one int). Where exactly this causes a problem I'm not sure, but it's not suprising. d) There seem to be two ways to fix this: 1) allocate extra memory - this is wasteful, but safe; and it's what I did in the patch I included below. 2) Make all the strides have zero length for these zeros size arrays. This wastes no space but might have consequences that are too horrible to imagine. Anyway, included below is a patch that fixes the problem by allocating product(max(1,dimension[i]) space for the array rather than product(dimension[i]) space. ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ *** arrayobject.c Wed Aug 6 09:45:50 1997 --- arrayobject.c.old Wed Aug 6 09:46:45 1997 *************** *** 313,324 **** PyErr_SetString(PyExc_ValueError, "negative dimensions are not allowed"); goto fail; } ! /* ! This may waste some space, but it seems to be ! (unsuprisingly) unhealthy to allow strides that are ! longer than sd. ! */ ! sd *= dimensions[i] ? dimensions[i] : 1; } /* Make sure we're alligned on ints. */ --- 313,319 ---- PyErr_SetString(PyExc_ValueError, "negative dimensions are not allowed"); goto fail; } ! sd *= dimensions[i]; } /* Make sure we're alligned on ints. */ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@maigret.cog.brown.edu Tue Aug 12 01:03:36 1997 From: da@maigret.cog.brown.edu (David Ascher) Date: Mon, 11 Aug 1997 20:03:36 -0400 (EDT) Subject: [MATRIX-SIG] printing buglet? Message-ID: Why the extra spaces in: >>> arange(5)[NewAxis,:] array([ [0, 1, 2, 3, 4]]) >>> print arange(5)[NewAxis,:] [ [0 1 2 3 4]] They seem unnecessary. --da _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From holz@zib.de Tue Aug 12 13:21:00 1997 From: holz@zib.de (Martin Holz) Date: Tue, 12 Aug 1997 14:21:00 +0200 Subject: [MATRIX-SIG] Bug report - seg-fault In-Reply-To: References: Message-ID: <199708121221.OAA14023@num15.zib.de> Timothy A. Hochberg writes: > > I think I've found a fix for the problem, although I admit to not fully > understanding it. Here's what I've figured out: > > a) It never happens for sizes < 4 (e.g., zeros((0,3)) * zeros((0,3)) > always works). This probably varies from machine to machine. > > b) For larger sizes, it doesn't always fail. I'd be interested in hearing > from those of you for whom it didn't crash before as to whether it > crashes if you repeat zeros((0,1000)) * zeros((0,1000)) several times. You are right. Python does not crash at ones((0,1))*ones((0,4), but crashes, if I call zeros((0,1000)) * zeros((0,1000)). I use Solaris 2.5 and gcc 2.7.2.1 Tschuess Martin _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From carlosm@moet.cs.colorado.edu Tue Aug 12 17:25:13 1997 From: carlosm@moet.cs.colorado.edu (Carlos Maltzahn) Date: Tue, 12 Aug 1997 10:25:13 -0600 (MDT) Subject: [MATRIX-SIG] Bug report - seg-fault In-Reply-To: <199708121221.OAA14023@num15.zib.de> Message-ID: On Tue, 12 Aug 1997, Martin Holz wrote: > Timothy A. Hochberg writes: > > > > I think I've found a fix for the problem, although I admit to not fully > > understanding it. Here's what I've figured out: > > > > a) It never happens for sizes < 4 (e.g., zeros((0,3)) * zeros((0,3)) > > always works). This probably varies from machine to machine. > > > > b) For larger sizes, it doesn't always fail. I'd be interested in hearing > > from those of you for whom it didn't crash before as to whether it > > crashes if you repeat zeros((0,1000)) * zeros((0,1000)) several times. > > You are right. Python does not crash at ones((0,1))*ones((0,4), but > crashes, if I call zeros((0,1000)) * zeros((0,1000)). > I use Solaris 2.5 and gcc 2.7.2.1 > > Tschuess > Martin Same with Digital Unix 4.0A and its cc compiler. The patch fixed it. Carlos _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Jens.Decker@chemie.uni-regensburg.de Thu Aug 14 10:14:06 1997 From: Jens.Decker@chemie.uni-regensburg.de (Jens Decker) Date: Thu, 14 Aug 1997 11:14:06 +0200 Subject: [MATRIX-SIG] SWIG, Numeric Python Message-ID: <199708140914.LAA28715@rchsg6.chemie.uni-regensburg.de> Hello! Last Friday I began using Python for my data analysis work and in the meantime I've managed to interface my Fortran Code and a few NAG-Library Routines (nonlinear least square fitting, numerical integrations) with this wonderful language and the nice numeric package. For my further programming it would be useful to automate the interfacing. Unfortunately I didn't find any example with helper functions for SWIG and numeric python datatypes. Are there any packages around which I could look at? Any code fragment showing how to wrap double arrays and things like that would be useful for me to get an idea. Yours, Jens Decker ___________________________________________________________ * Jens Decker * * Universitaet Regensburg * * Institut fuer Physikalische und Theoretische Chemie * * Lehrst. Prof. Dick * * Universitaetstr. 31 * * 93059 Regensburg * * * * Tel: +49-941/943-4473, -4458 Fax: -4488 Room: 22.2.14 * * Email: Jens.Decker@chemie.uni-regensburg.de * ___________________________________________________________ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From michaelb@gold.net.au Fri Aug 15 09:54:00 1997 From: michaelb@gold.net.au (Michael Bell) Date: Fri, 15 Aug 97 08:54 WST Subject: [MATRIX-SIG] SWIG, Numeric Python In-Reply-To: <199708140914.LAA28715@rchsg6.chemie.uni-regensburg.de> (Jens.Decker@chemie.uni-regensburg.de) Message-ID: Hi Jens, You wrote: > Last Friday I began using Python for my data analysis work and in the meantime > I've managed to interface my Fortran Code and a few NAG-Library Routines > (nonlinear least square fitting, numerical integrations) with this wonderful > language and the nice numeric package. For my further programming it would > be useful to automate the interfacing. Unfortunately I didn't find any example > with helper functions for SWIG and numeric python datatypes. Are there any > packages around which I could look at? Any code fragment showing how to > wrap double arrays and things like that would be useful for me to get an > idea. I use Python for data analysis, and I've recently wrapped some Fortran code using SWIG. It's not pretty, but I've managed to get the job done that I needed to. Here is the first part of my SWIG interface file which wrapped the PGPLOT graphing library. I've included the typemaps which handle two dimensional NumPy arrays. I hope this is what you are looking for. Remember, I said the code wasn't pretty! cheers, Michael -- Michael Bell Kalgoorlie Consolidated Gold Mines, Western Australia "To know the laws that govern the winds, and to know that you know them, will give you an easy mind on your voyage around the world; otherwise you may tremble at the appearance of every cloud." Joshua Slocum ==================== %module pgplot %{ #include "cpgplot.h" #include "arrayobject.h" %} typedef int Logical; %typemap(python, in) float* IN_2D { PyObject *obj; PyArrayObject *arr; int i, j; unsigned char *ptr; /* Check that obj is really an 2D array of bytes */ if (!PyArray_Check($source)) { PyErr_SetString(PyExc_TypeError,"First argument is not an array"); return NULL; } /* check type (could also use arr->descr->type_num) */ if (PyArray_ObjectType($source,0) != PyArray_FLOAT) { PyErr_SetString(PyExc_TypeError,"Incorrect array type: we need an array of FLOAT16"); return NULL; } arr = (PyArrayObject *)$source; if (arr->nd != 2) { /* we are really strict ! */ PyErr_SetString(PyExc_TypeError,"Incorrect number of dims: we want a 2d array"); return NULL; } $target = (float *)arr->data; } %typemap(python, in) float* IN_1D { PyObject *obj; PyArrayObject *arr; int i, j; unsigned char *ptr; /* Check that obj is really a 1D array of bytes */ if (!PyArray_Check($source)) { PyErr_SetString(PyExc_TypeError,"First argument is not an array"); return NULL; } /* check type (could also use arr->descr->type_num) */ if (PyArray_ObjectType($source,0) != PyArray_FLOAT) { PyErr_SetString(PyExc_TypeError,"Incorrect array type: we need an array of FLOAT16"); return NULL; } arr = (PyArrayObject *)$source; if (arr->nd != 1) { /* we are really strict ! */ PyErr_SetString(PyExc_TypeError,"Incorrect number of dims: we want a 13d array"); return NULL; } $target = (float *)arr->data; } Qaddafi plutonium Semtex Peking KGB Ortega Panama terrorist quiche BATF Nazi Soviet Uzi Ft. Bragg NSA _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From michaelb@gold.net.au Fri Aug 15 13:12:00 1997 From: michaelb@gold.net.au (Michael Bell) Date: Fri, 15 Aug 97 12:12 WST Subject: [MATRIX-SIG] SWIG, Numeric Python In-Reply-To: <199708150256.TAA05391@curlew.geog.ubc.ca> (message from Phil Austin on Thu, 14 Aug 1997 19:56:10 -0700) Message-ID: Phil Austin wrote: > >>>>> "Michael" == Michael Bell writes: > > Michael> I use Python for data analysis, and I've recently wrapped > Michael> some Fortran code using SWIG. It's not pretty, but I've > Michael> managed to get the job done that I needed to. > > Would you be willing to share the rest of the interface? I have > some astronomer friends that I'd like to convert over to Python. > > Regards, Phil My earlier post was just supposed to go to Jens, but I forgot to delete the "CC:" line. I didn't want to fill everyone's mbox. I've put my wrapper for the pgplot library into http://gold.net.au/~michaelb/pgplot4python.tgz if anyone would like to look at it. I have no great pretensions to professional programming skills, and make no promises about the code. Most of it has been cut and pasted from elsewhere. But it worked for me. Earlier this year someone else mentioned that they had wrapped pgplot into Python with SWIG, but I don't have the details. They may have done a far more complete and correct job than me. cheers, -- Michael Bell Kalgoorlie Consolidated Gold Mines, Western Australia "To know the laws that govern the winds, and to know that you know them, will give you an easy mind on your voyage around the world; otherwise you may tremble at the appearance of every cloud." Joshua Slocum FSF Kennedy Uzi Ft. Bragg nuclear smuggle radar [Hello to all my fans in domestic surveillance] Ortega strategic domestic disruption cracking Treasury fissionable Khaddafi _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From stjohn@gav.gat.com Fri Aug 15 18:40:25 1997 From: stjohn@gav.gat.com (Holger St. John) Date: Fri, 15 Aug 1997 09:40:25 -0800 Subject: [MATRIX-SIG] Python/Fortran interface Message-ID: <199708151637.MAA00691@python.org> A couple of messages over the past two days have indicated that there is some interest in calling Fortran from Python. A project that I am working on will require this ability for its success. I would appreciate any info that this group might have that relates to creating such an interface . Apparently FIDL (Fortran interface Definition Language) ,mentioned once in Com. Physics, has not happened ? The idea is to create steerable (ie programable) applications that do heavy duty number crunching in Fortran and are controlled by a Python script. (recoding the Fortran in C++ is not an option). Holger St. John General Atomics _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jbaddor@phy.ulaval.ca Fri Aug 15 22:26:10 1997 From: jbaddor@phy.ulaval.ca (Jean-Bernard ADDOR) Date: Fri, 15 Aug 1997 17:26:10 -0400 (EDT) Subject: [MATRIX-SIG] array([0,1,2,3]).reverse() In-Reply-To: <199708151637.MAA00691@python.org> Message-ID: Hy! How to obtain an array in the reverse order ? If i have array([0,1,2,3]) I want array([3,2,1,0]). A bientot, Jean-Bernard _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Sat Aug 16 01:02:05 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Fri, 15 Aug 1997 19:02:05 -0500 (CDT) Subject: [MATRIX-SIG] array([0,1,2,3]).reverse() In-Reply-To: Message-ID: On Fri, 15 Aug 1997, Jean-Bernard ADDOR wrote: > Hy! > > How to obtain an array in the reverse order ? > If i have array([0,1,2,3]) > I want array([3,2,1,0]). Try: a = array([0,1,2,3]) b = a[::-1] b # should give [3,2,1,0] ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Fri Aug 15 19:59:58 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Fri, 15 Aug 1997 14:59:58 -0400 Subject: [MATRIX-SIG] Python/Fortran interface Message-ID: <199708151855.OAA05893@dante.mh.lucent.com> Chastise me severely on evidence of stupidity! It seems to me that the first step is to know how to call Fortran from C: then the standard methods for wrapping C-functions and objects apply. (see the std refs or either English Python book.) As for mapping Fortran arrays (or other structures) into NumPy arrays, that might be trickier, and I'd love to look into it more (maybe in a few weeks). Who else knows more? Please let us know of success or problems. Brag with simple example code if you get lucky! A simple example of mapping a 2d fortran array of doubles into/out-of Numeric would be great. Is there such a beasty? Dr. Zane? -- Aaron Watters ---------- > From: Holger St. John > To: matrix-sig@python.org > Subject: [MATRIX-SIG] Python/Fortran interface > Date: Friday, August 15, 1997 1:40 PM > > A couple of messages over the past two days have indicated > that there is some interest in calling Fortran from Python. > A project that I am working on will require this ability for its > success.... _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Sat Aug 16 10:45:12 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Sat, 16 Aug 1997 11:45:12 +0200 Subject: [MATRIX-SIG] Python/Fortran interface In-Reply-To: <199708151637.MAA00691@python.org> (stjohn@gav.gat.com) Message-ID: <199708160945.LAA20498@lmspc1.ibs.fr> > A couple of messages over the past two days have indicated > that there is some interest in calling Fortran from Python. > A project that I am working on will require this ability for its > success. I would appreciate any info that this group might have > that relates to creating such an interface . Apparently There is no problem at all in calling Fortran from Python; in fact, it requires no more than the ability to call Fortran from C, since Python is written in C. The point of FIDL (as I understand it) was to automatize interface generation in a similar way as SWIG does for C. That is certainly important to deal with a large number of routines, but if you need just a few, manual wrapping should pose no major problems. To simplify C-Fortran interfacing, you could use the CFORTRAN package developed at DESY. I am using the Fortran version of LAPACK with the NumPy module LinearAlgebra, and I have also called a few of my own Fortran subroutines from Python, so it is definitely possible. Depending on your platform, it could be tricky to put Fortran code into a dynamic library due to library problems (and lack of documentation), but I don't see any problem for static linking. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Sat Aug 16 10:47:49 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Sat, 16 Aug 1997 11:47:49 +0200 Subject: [MATRIX-SIG] Python/Fortran interface In-Reply-To: <199708151855.OAA05893@dante.mh.lucent.com> (arw@dante.mh.lucent.com) Message-ID: <199708160947.LAA20524@lmspc1.ibs.fr> > As for mapping Fortran arrays (or other structures) > into NumPy arrays, that might be trickier, and > I'd love to look into it more (maybe in a few weeks). > Who else knows more? No problem. Look at the LAPACK modules in NumPy as an example. They are usually used with f2c-translated versions of LAPACK, but that doesn't change the interface at all. In fact, on many platforms you can simply link with the Fortran version instead. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From michaelb@gold.net.au Mon Aug 18 13:57:03 1997 From: michaelb@gold.net.au (Michael Bell) Date: Mon, 18 Aug 1997 20:57:03 +0800 (WST) Subject: [MATRIX-SIG] NumPy arrays of python objects Message-ID: The NumPy doc.html, and the Numeric Tutorial, both mention arrays of Python objects, but with little elaboration. Can someone point me to, or send my some, sample code making use of these beasts? I'd really like to make an array of lists, but I can't see how to. I can get by with a list of list of lists, but it's clumsy. Does the array of objects feature truly exist? -- Michael Bell Kalgoorlie, Western Australia. Cocaine nuclear munitions Serbian CIA North Korea KGB bomb ammunition Ortega Marxist South Africa Rule Psix DES quiche _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Mon Aug 18 14:28:35 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Mon, 18 Aug 1997 09:28:35 -0400 Subject: numpy FAQ (wizard) Re: [MATRIX-SIG] array([0,1,2,3]).reverse() Message-ID: <199708181351.JAA08951@dante.mh.lucent.com> Looks like we need a NumPy FAQ somewhere. Someone should look into setting up Guido's excellent FAQ wizard for this purpose somewhere. This would allow all of us to add our own entries. -- Aaron Watters ps: /F, if you are listening, have you had any luck with this? I haven't tried, I'm ashamed to admit. ---------- > From: Timothy A. Hochberg > To: Jean-Bernard ADDOR > Cc: matrix-sig@python.org > Subject: Re: [MATRIX-SIG] array([0,1,2,3]).reverse() > Date: Friday, August 15, 1997 8:02 PM > > On Fri, 15 Aug 1997, Jean-Bernard ADDOR wrote: > > > Hy! > > > > How to obtain an array in the reverse order ? > > If i have array([0,1,2,3]) > > I want array([3,2,1,0]). > > Try: > > a = array([0,1,2,3]) > b = a[::-1] > b # should give [3,2,1,0] > > ____ > /im > > +------------------------------------------------+ > |Tim Hochberg Research Assistant | > |hochberg wwa.com University of Illinois | > | Electrical Engineering | > +------------------------------------------------+ > > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From motteler@laura.llnl.gov Mon Aug 18 15:39:59 1997 From: motteler@laura.llnl.gov (Zane C. Motteler) Date: Mon, 18 Aug 1997 07:39:59 -0700 (PDT) Subject: [MATRIX-SIG] Python/Fortran interface In-Reply-To: <199708151855.OAA05893@dante.mh.lucent.com> Message-ID: Aaron et al, On Fri, 15 Aug 1997, Aaron Watters wrote: >Chastise me severely on evidence of stupidity! > >It seems to me that the first step is to know how >to call Fortran from C: then the standard methods >for wrapping C-functions and objects apply. >(see the std refs or either English Python book.) > >As for mapping Fortran arrays (or other structures) >into NumPy arrays, that might be trickier, and >I'd love to look into it more (maybe in a few weeks). >Who else knows more? FORTRAN arrays are stored in column-major order whereas (contiguous) NumPy arrays are stored in row-major order. Thus to communicate a NumPy array to FORTRAN, make it contiguous and take its transpose. >Please let us know of success or problems. >Brag with simple example code if you get lucky! >A simple example of mapping a 2d fortran array >of doubles into/out-of Numeric would be great. >Is there such a beasty? Dr. Zane? We have a Perl script (appropriately called Fcc) which automates the process of calling C from FORTRAN for many common platforms. It does so by creating wrappers for the C functions, which convert the FORTRAN arguments into what C expects. This is just the reverse of what you asked for, but much of the code could be used to go in the opposite direction. Certainly the ideas we used are generally applicable. I can make the script and its man page available on our ftp site if people are interested. Konrad mentions CFORTRAN. We used to use this; basically it consists of a header file which creates huge macros to do the job of argument conversion. We found it to be very difficult to use, particularly while debugging, because of the humongous multi- line macros and the impenetrable code that they generate. Zane ----------------------------------------------------------------------------- Zane C. Motteler, Ph. D. ___________ALSO:_________________ Computer Scientist | Professor Emeritus of Computer| Lawrence Livermore National Laboratory | Science and Engineering | P O Box 808, L-472 | California Polytechnic State | Livermore, CA 94551-9900 | University, San Luis Obispo | 510/423-2143 --------------------------------- FAX 510/423-9969 zcm@llnl.gov or motteler@laura.llnl.gov or zmottel@calpoly.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Daniel.Michelson@smhi.se Mon Aug 18 15:40:23 1997 From: Daniel.Michelson@smhi.se (Daniel Michelson) Date: Mon, 18 Aug 1997 16:40:23 +0200 Subject: numpy FAQ (wizard) Re: [MATRIX-SIG] array([0,1,2,3]).reverse() References: <199708181351.JAA08951@dante.mh.lucent.com> Message-ID: <33F85ED7.2001@smhi.se> Hej, > > Try: > > > > a =3D array([0,1,2,3]) > > b =3D a[::-1] > > b # should give [3,2,1,0] It does for me: >>> a =3D array([0,1,2,3]) >>> b =3D a[::-1] >>> b 3 2 1 0 >>> a 0 1 2 3 best, -d +------------------+--------------------------------------------------+ | | Daniel B. Michelson | | /\ |\ /| | | | | Swedish Meteorological and Hydrological Institute| | \ | \/ | |__| | | S-601 76 Norrk=F6ping, Sweden |= | \ | | | | | | Telephone: +46 11 - 15 84 94 | | \/ | | | | | | Telefax: +46 11 - 17 02 07 | | | Internet: Daniel.Michelson@smhi.se | +------------------+--------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From Timothy A. Hochberg" Message-ID: On Mon, 18 Aug 1997, Michael Bell wrote: > The NumPy doc.html, and the Numeric Tutorial, both mention arrays of > Python objects, but with little elaboration. > > Can someone point me to, or send my some, sample code making use of > these beasts? I've never used them "for real", but maybe the following will be enough to get you started: from Numeric import * # Make some empty object arrays. l1 = zeros((3,), PyObject) l2 = zeros((3,), PyObject) # Tha data we'll be using d1 = ([1,2,3], ["a", 4, 5], [None, 0 , "spam"]) d2 = ([1,5,3], ["a", 6, 5], [None, 0 , "spam"]) # Objects are hard to initialize, so we'll use a loop. # l1[:] = d1 doesn't work (try it and see why...) for i in range(3): l1[i] = d1[i] l2[i] = d2[i] print l1 print l2 # This should probably work but doesn't ... ##print equal(l1, l2) ##print less(l1, l2) print l1 + l2 print argsort(l1) > > I'd really like to make an array of lists, but I can't see how to. > > I can get by with a list of list of lists, but it's clumsy. Does the > array of objects feature truly exist? It exists - but it appears that some of the functionality that is available for numeric types is missing for PyObjects. Oddly 'abs' works which seems like it would be of limited utility for PyObjects, while 'less', which would be more useful, does not. I suspect some of the PyObject utility is missing because no one has really needed it, but maybe there is another reason. ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From djc@lct.com Mon Aug 18 16:06:12 1997 From: djc@lct.com (Duncan Child) Date: Mon, 18 Aug 97 10:06:12 CDT Subject: [MATRIX-SIG] Python/Fortran interface Message-ID: <9708181506.AA25477@ lct.com> > > It seems to me that the first step is to know how > to call Fortran from C: then the standard methods > for wrapping C-functions and objects apply. > (see the std refs or either English Python book.) > The above approach has also worked for me. I have since read about modulator.py which may help produce the boiler plate C code that you will need. See page 630 of Mark Lutz's book for more details. Regards Duncan _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From motteler@laura.llnl.gov Mon Aug 18 16:05:34 1997 From: motteler@laura.llnl.gov (Zane C. Motteler) Date: Mon, 18 Aug 1997 08:05:34 -0700 (PDT) Subject: [MATRIX-SIG] Python/Fortran interface In-Reply-To: <199708181454.KAA17439@dante.mh.lucent.com> Message-ID: Aaron, On Mon, 18 Aug 1997, Aaron Watters wrote: >yes please, and archive the url on matrix-sig. > -- Aaron Watters Done! Our anonymous ftp site is ftp-icf.llnl.gov. You will want to go to subdirectory /ftp/pub/basis. The text-only files are Fcc (the perl script) and Fcc.1 (the man page); or, if you want a tar file, Fcc.tar; and if you want a gzipped file, Fcc.tar.gz. Zane ----------------------------------------------------------------------------- Zane C. Motteler, Ph. D. ___________ALSO:_________________ Computer Scientist | Professor Emeritus of Computer| Lawrence Livermore National Laboratory | Science and Engineering | P O Box 808, L-472 | California Polytechnic State | Livermore, CA 94551-9900 | University, San Luis Obispo | 510/423-2143 --------------------------------- FAX 510/423-9969 zcm@llnl.gov or motteler@laura.llnl.gov or zmottel@calpoly.edu _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From krodgers@tdyryan.com Mon Aug 18 16:36:13 1997 From: krodgers@tdyryan.com (Kevin Rodgers) Date: Mon, 18 Aug 1997 08:36:13 -0700 Subject: [MATRIX-SIG] Python/Fortran interface In-Reply-To: References: <199708151855.OAA05893@dante.mh.lucent.com> Message-ID: <3.0.1.32.19970818083613.005253d0@gate.tdyryan.com> At 07:39 AM 8/18/97 -0700, motteler@laura.llnl.gov wrote: >FORTRAN arrays are stored in column-major order whereas >(contiguous) NumPy arrays are stored in row-major order. >Thus to communicate a NumPy array to FORTRAN, make it >contiguous and take its transpose. > This may seem a minor nit, but "row-major" and "column-major" seem to imply two-dimensional arrays only; it would be more correct to say, "Fortran stores arrays contiguously with the leftmost index varying fastest and the rightmost index varying slowest; C is just the opposite." This generalizes to arrays of more than two dimensions, which is one of the real advantages of NumPy over Matlab or Xmath (yes, I know that Matlab 5 has "true" multidimensional arrays, and Xmath (of which I am a devoted user) has some specialized arrays of more than two dimensions). So plllleeeeezzzzzeeeee let's not restrict ourselves to two-d, linear algebra stuff here! On the more general subject of extending/embedding/interfacing NumPy with whatever, does anybody have a reasonably straightforward and concise explanation of how NumPy arrays are stored (i.e., an annotated structure definition)? I've looked through the source, plus a fairly substantial library that has NumPy interfaces (PyOpenGL), and I've been able to piece it together somewhat, but it still has a bit of the fumbling-in-the-pea-soup feeling to it. Thanks in advance . . . ---------------------------------------------------------------------- Kevin Rodgers Teledyne Ryan Aeronautical krodgers@tdyryan.com "This one goes up to eleven." -- Nigel Tufnel ---------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Aug 18 18:27:45 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 18 Aug 1997 19:27:45 +0200 Subject: [MATRIX-SIG] NumPy arrays of python objects In-Reply-To: (message from Michael Bell on Mon, 18 Aug 1997 20:57:03 +0800 (WST)) Message-ID: <199708181727.TAA21448@lmspc1.ibs.fr> > The NumPy doc.html, and the Numeric Tutorial, both mention arrays of > Python objects, but with little elaboration. > > Can someone point me to, or send my some, sample code making use of > these beasts? Tim has done that, so I'll just add one comment: it is pretty easy to create arrays of anything using the standard array constructor, *except* for arrays of sequence objects (e.g. arrays of lists). The reason is simply that the array constructor assumes that any level of sequence object in its input should become a dimension of the array. That means that the only way to have an array of lists is to generate first an appropriate array of "general objects" with whatever elements you like, and then replace the elements by the lists using indexed assignment. I don't think there's a way out - you can't have one syntax stand for two meanings! Commenting Tim's message: > It exists - but it appears that some of the functionality that is > available for numeric types is missing for PyObjects. Oddly 'abs' works > which seems like it would be of limited utility for PyObjects, while > 'less', which would be more useful, does not. I suspect some of the > PyObject utility is missing because no one has really needed it, but maybe > there is another reason. I strongly suspect the reason is ease of implementation! ;-) "less" is implemented as a ufunc in module umath, just like all the mathematical functions. The ufunc's are optimized functions for fast array operations, but they work only on the standard numerical array types, not on general object arrays. "abs", on the other hand, is a built-in Python function that can easily be made to work for any type. When you apply abs() to a general object array, it is simply mapped to the elements. So which functions *are* available on general object arrays? All those for which Python provides an extension interface, i.e. the basic arithmetic functions plus some oddities like abs. And of course all structural array functions that don't care about the values of the elements. As you may guess, I have used general object arrays in real life; they work well and are quite useful, although only in rather special circumstances. For example, they are very nice for passing data to C routines. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Aug 18 18:34:43 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 18 Aug 1997 19:34:43 +0200 Subject: [MATRIX-SIG] Python/Fortran interface In-Reply-To: <3.0.1.32.19970818083613.005253d0@gate.tdyryan.com> (message from Kevin Rodgers on Mon, 18 Aug 1997 08:36:13 -0700) Message-ID: <199708181734.TAA21466@lmspc1.ibs.fr> > On the more general subject of extending/embedding/interfacing NumPy with > whatever, does anybody have a reasonably straightforward and concise > explanation of how NumPy arrays are stored (i.e., an annotated structure > definition)? I've looked through the source, plus a fairly substantial It's a bit messy if you want all the details. In C code, I tend to limit myself to contiguous arrays, and then life becomes easy. Here's what you need to know: Suppose that a is declared as PyArrayObject *. Then we have: type: a->descr->type (see PyArray_TYPES in arrayobject.h) number of dimensions: a->nd length of first dimension: a->dimensions[0] length of second dimension: a->dimensions[1] ... length of last dimension: a->dimensions[a->nd-1] data: a->data And the data is arranged exactly as in an equally dimensioned static C array. Note that a->data is declared as char *; you have to cast it to the type of pointer appropriate for the data before accessing it. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Mon Aug 18 18:36:59 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Mon, 18 Aug 1997 19:36:59 +0200 Subject: [MATRIX-SIG] Python/Fortran interface In-Reply-To: (motteler@laura.llnl.gov) Message-ID: <199708181736.TAA21474@lmspc1.ibs.fr> > Konrad mentions CFORTRAN. We used to use this; > basically it consists of a header file which creates > huge macros to do the job of argument conversion. We > found it to be very difficult to use, particularly > while debugging, because of the humongous multi- > line macros and the impenetrable code that they > generate. I have to agree. In fact, I don't use it myself anymore, writing an interface manually in the extremely few cases where I still need to interface C and Fortran libraries. Fortunately most of my recent work involves only symmetric matrices! :-) Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@maigret.cog.brown.edu Mon Aug 18 19:35:34 1997 From: da@maigret.cog.brown.edu (David Ascher) Date: Mon, 18 Aug 1997 14:35:34 -0400 (EDT) Subject: numpy FAQ (wizard) Re: [MATRIX-SIG] array([0,1,2,3]).reverse() In-Reply-To: <199708181351.JAA08951@dante.mh.lucent.com> Message-ID: On Mon, 18 Aug 1997, Aaron Watters wrote: > Looks like we need a NumPy FAQ somewhere. > > Someone should look into setting up Guido's > excellent FAQ wizard for this purpose somewhere. > This would allow all of us to add our own entries. > -- Aaron Watters I have a NumPY faq wizard working, and will install it on the starship ASAP -- When it's up, I'll let this group know what the editing password is. Cheers --da _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From michaelb@gold.net.au Tue Aug 19 10:12:00 1997 From: michaelb@gold.net.au (Michael Bell) Date: Tue, 19 Aug 97 09:12 WST Subject: [MATRIX-SIG] NumPy arrays of python objects In-Reply-To: <199708181727.TAA21448@lmspc1.ibs.fr> (message from Konrad Hinsen on Mon, 18 Aug 1997 19:27:45 +0200) Message-ID: Thank you Tim, thank you Konrad. I'm now generating arrays of lists nicely. I'm sure I tried exactly the same things last night without success, but maybe it was just late. I have two other questions though: I don't have a "PyObject" in my NumPy - I guess that's because I'm using a late beta version, not the most up to date? Also, I'm unable to do a "from Numeric import *" - I need to do a "import ni, Numeric", or "import ni; from Numeric.Core import *". Is this also a matter of upgrading my version? What happened to using the ni module? -- Michael Bell Kalgoorlie Consolidated Gold Mines, Western Australia "To know the laws that govern the winds, and to know that you know them, will give you an easy mind on your voyage around the world; otherwise you may tremble at the appearance of every cloud." Joshua Slocum NORAD Peking Delta Force Uzi Panama Mossad North Korea kibo $400 million in gold bullion colonel DES Kennedy FBI fissionable plutonium _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Aug 19 09:55:37 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 19 Aug 1997 10:55:37 +0200 Subject: [MATRIX-SIG] NumPy arrays of python objects In-Reply-To: (message from Michael Bell on Tue, 19 Aug 97 09:12 WST) Message-ID: <199708190855.KAA23641@lmspc1.ibs.fr> > I have two other questions though: I don't have a "PyObject" in my > NumPy - I guess that's because I'm using a late beta version, not the > most up to date? Right. You can always write 'O' of course (that's the value of Numeric.PyObject), but I strongly recommend updating... > Also, I'm unable to do a "from Numeric import *" - I need to do a > "import ni, Numeric", or "import ni; from Numeric.Core import *". > > Is this also a matter of upgrading my version? What happened to using > the ni module? Your version must be very old! ni hasn't been used for quite a while. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hyoon@nyptsrv1.etsd.ml.com Tue Aug 19 14:28:19 1997 From: hyoon@nyptsrv1.etsd.ml.com (Hoon Yoon - IPT Quant) Date: Tue, 19 Aug 1997 09:28:19 -0400 Subject: [MATRIX-SIG] Extending NumPy Message-ID: <199708191328.JAA11015@ptq3.etsd> Hello, I am finally embarking on extending Python with C programs and at this point I am trying to push in Lists of strings and get back lists of strings and Numerical arrays back from a C function. Unfortunately, I am having problems finding examples of these codes in C. I guess the following from gdmodule does show how to kinda pass in a list or tuple into a C prog, but I am lost on how to do this for string arrays (multiple) and get back string arrays and most of all Numeric Arrays: ***************************************************************** Nice example of how to do this with number tuple Can I use **points to do this for string tuple and args1, args2, args3? ***************************************************************** static PyObject *image_lines(self, args) imageobject *self; PyObject *args; { PyObject *point, *points, *bit; int color, i, x,y,ox,oy; if (!PyArg_ParseTuple(args, "O!i", &PyTuple_Type, &points, &color)) { PyErr_Clear(); if (PyArg_ParseTuple(args, "O!i", &PyList_Type, &points, &color)) points = PyList_AsTuple(points); else return NULL; } point = PyTuple_GET_ITEM(points,0); ox = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); oy = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); for (i=1; iimagedata, X(ox), Y(oy), X(x), Y(y), color); ox=x;oy=y; } Py_INCREF(Py_None); return Py_None; } ********************************************************************** Can anyone show me some example of these extentions, I've mentioned here? At least I would like to get back list in list [[1,2,3],[4,5,6]] type of thing, which I could easily convert back to a Num Array. Much appreciated, Hoon, _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Tue Aug 19 15:25:59 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Tue, 19 Aug 1997 16:25:59 +0200 Subject: [MATRIX-SIG] Extending NumPy In-Reply-To: <199708191328.JAA11015@ptq3.etsd> (hyoon@nyptsrv1.etsd.ml.com) Message-ID: <199708191425.QAA25222@lmspc1.ibs.fr> > I am finally embarking on extending Python with C programs and at > this point I am trying to push in Lists of strings and get back > lists of strings and Numerical arrays back from a C function. For an example of an extension module that uses arrays extensively see my netCDF interface at http://starship.skyport.net/crew/hinsen/netcdf.html Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Aug 19 16:39:48 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 19 Aug 1997 11:39:48 -0400 Subject: [MATRIX-SIG] Extending NumPy Message-ID: <199708191550.LAA16438@dante.mh.lucent.com> If you can deal with a bit of overhead, maybe you should use the abstract object interface (thanks again Jim Fulton!) to extract Python objects from sequences and get their values. Look at abstract.h. Here, I think, you could pass any Python sequence down to the C interface and extract the elements. For Numeric this might add some unneeded malloc overhead when an int or float is extracted from a NumPy array, and that will slow down the code, but it would almost certainly be the easiest thing to do. In the reverse direction you can create and populate a listobject, and have Python convert it to an array. Again, not the most space/time efficient approach, but it'll work and soon. Look at listobject.h. Careful with refcounts, of course. Life is compromise. -- Aaron Watters ---------- > From: Hoon Yoon - IPT Quant > To: matrix-sig@python.org > Subject: [MATRIX-SIG] Extending NumPy > Date: Tuesday, August 19, 1997 9:28 AM > > Hello, > > I am finally embarking on extending Python with C programs and at > this point I am trying to push in Lists of strings and get back > lists of strings and Numerical arrays back from a C function. > Unfortunately, I am having problems finding examples of these > codes in C. I guess the following from gdmodule does show how > to kinda pass in a list or tuple into a C prog, but I am lost on > how to do this for string arrays (multiple) and get back string > arrays and most of all Numeric Arrays: > > ***************************************************************** > Nice example of how to do this with number tuple > Can I use **points to do this for string tuple and args1, args2, args3? > ***************************************************************** > static PyObject *image_lines(self, args) > imageobject *self; > PyObject *args; > { > PyObject *point, *points, *bit; > int color, i, x,y,ox,oy; > > if (!PyArg_ParseTuple(args, "O!i", &PyTuple_Type, &points, &color)) > { > PyErr_Clear(); > if (PyArg_ParseTuple(args, "O!i", &PyList_Type, &points, &color)) > points = PyList_AsTuple(points); > else return NULL; > } > > point = PyTuple_GET_ITEM(points,0); > ox = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > oy = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > for (i=1; i { > point = PyTuple_GET_ITEM(points,i); > x = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > y = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > gdImageLine(self->imagedata, X(ox), Y(oy), X(x), Y(y), color); > ox=x;oy=y; > } > > Py_INCREF(Py_None); > return Py_None; > } > ********************************************************************** > Can anyone show me some example of these extentions, I've mentioned > here? At least I would like to get back list in list > [[1,2,3],[4,5,6]] type of thing, which I could easily convert back > to a Num Array. > > Much appreciated, > > Hoon, > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Aug 19 16:39:48 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 19 Aug 1997 11:39:48 -0400 Subject: [MATRIX-SIG] Extending NumPy Message-ID: <199708191552.LAA16520@dante.mh.lucent.com> If you can deal with a bit of overhead, maybe you should use the abstract object interface (thanks again Jim Fulton!) to extract Python objects from sequences and get their values. Look at abstract.h. Here, I think, you could pass any Python sequence down to the C interface and extract the elements. For Numeric this might add some unneeded malloc overhead when an int or float is extracted from a NumPy array, and that will slow down the code, but it would almost certainly be the easiest thing to do. In the reverse direction you can create and populate a listobject, and have Python convert it to an array. Again, not the most space/time efficient approach, but it'll work and soon. Look at listobject.h. Careful with refcounts, of course. Life is compromise. -- Aaron Watters ---------- > From: Hoon Yoon - IPT Quant > To: matrix-sig@python.org > Subject: [MATRIX-SIG] Extending NumPy > Date: Tuesday, August 19, 1997 9:28 AM > > Hello, > > I am finally embarking on extending Python with C programs and at > this point I am trying to push in Lists of strings and get back > lists of strings and Numerical arrays back from a C function. > Unfortunately, I am having problems finding examples of these > codes in C. I guess the following from gdmodule does show how > to kinda pass in a list or tuple into a C prog, but I am lost on > how to do this for string arrays (multiple) and get back string > arrays and most of all Numeric Arrays: > > ***************************************************************** > Nice example of how to do this with number tuple > Can I use **points to do this for string tuple and args1, args2, args3? > ***************************************************************** > static PyObject *image_lines(self, args) > imageobject *self; > PyObject *args; > { > PyObject *point, *points, *bit; > int color, i, x,y,ox,oy; > > if (!PyArg_ParseTuple(args, "O!i", &PyTuple_Type, &points, &color)) > { > PyErr_Clear(); > if (PyArg_ParseTuple(args, "O!i", &PyList_Type, &points, &color)) > points = PyList_AsTuple(points); > else return NULL; > } > > point = PyTuple_GET_ITEM(points,0); > ox = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > oy = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > for (i=1; i { > point = PyTuple_GET_ITEM(points,i); > x = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > y = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > gdImageLine(self->imagedata, X(ox), Y(oy), X(x), Y(y), color); > ox=x;oy=y; > } > > Py_INCREF(Py_None); > return Py_None; > } > ********************************************************************** > Can anyone show me some example of these extentions, I've mentioned > here? At least I would like to get back list in list > [[1,2,3],[4,5,6]] type of thing, which I could easily convert back > to a Num Array. > > Much appreciated, > > Hoon, > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Aug 19 16:39:48 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 19 Aug 1997 11:39:48 -0400 Subject: [MATRIX-SIG] Extending NumPy Message-ID: <199708191556.LAA16768@dante.mh.lucent.com> If you can deal with a bit of overhead, maybe you should use the abstract object interface (thanks again Jim Fulton!) to extract Python objects from sequences and get their values. Look at abstract.h. Here, I think, you could pass any Python sequence down to the C interface and extract the elements. For Numeric this might add some unneeded malloc overhead when an int or float is extracted from a NumPy array, and that will slow down the code, but it would almost certainly be the easiest thing to do. In the reverse direction you can create and populate a listobject, and have Python convert it to an array. Again, not the most space/time efficient approach, but it'll work and soon. Look at listobject.h. Careful with refcounts, of course. Life is compromise. -- Aaron Watters ---------- > From: Hoon Yoon - IPT Quant > To: matrix-sig@python.org > Subject: [MATRIX-SIG] Extending NumPy > Date: Tuesday, August 19, 1997 9:28 AM > > Hello, > > I am finally embarking on extending Python with C programs and at > this point I am trying to push in Lists of strings and get back > lists of strings and Numerical arrays back from a C function. > Unfortunately, I am having problems finding examples of these > codes in C. I guess the following from gdmodule does show how > to kinda pass in a list or tuple into a C prog, but I am lost on > how to do this for string arrays (multiple) and get back string > arrays and most of all Numeric Arrays: > > ***************************************************************** > Nice example of how to do this with number tuple > Can I use **points to do this for string tuple and args1, args2, args3? > ***************************************************************** > static PyObject *image_lines(self, args) > imageobject *self; > PyObject *args; > { > PyObject *point, *points, *bit; > int color, i, x,y,ox,oy; > > if (!PyArg_ParseTuple(args, "O!i", &PyTuple_Type, &points, &color)) > { > PyErr_Clear(); > if (PyArg_ParseTuple(args, "O!i", &PyList_Type, &points, &color)) > points = PyList_AsTuple(points); > else return NULL; > } > > point = PyTuple_GET_ITEM(points,0); > ox = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > oy = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > for (i=1; i { > point = PyTuple_GET_ITEM(points,i); > x = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > y = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > gdImageLine(self->imagedata, X(ox), Y(oy), X(x), Y(y), color); > ox=x;oy=y; > } > > Py_INCREF(Py_None); > return Py_None; > } > ********************************************************************** > Can anyone show me some example of these extentions, I've mentioned > here? At least I would like to get back list in list > [[1,2,3],[4,5,6]] type of thing, which I could easily convert back > to a Num Array. > > Much appreciated, > > Hoon, > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Aug 19 16:39:48 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 19 Aug 1997 11:39:48 -0400 Subject: [MATRIX-SIG] Extending NumPy Message-ID: <199708191556.LAA16800@dante.mh.lucent.com> If you can deal with a bit of overhead, maybe you should use the abstract object interface (thanks again Jim Fulton!) to extract Python objects from sequences and get their values. Look at abstract.h. Here, I think, you could pass any Python sequence down to the C interface and extract the elements. For Numeric this might add some unneeded malloc overhead when an int or float is extracted from a NumPy array, and that will slow down the code, but it would almost certainly be the easiest thing to do. In the reverse direction you can create and populate a listobject, and have Python convert it to an array. Again, not the most space/time efficient approach, but it'll work and soon. Look at listobject.h. Careful with refcounts, of course. Life is compromise. -- Aaron Watters ---------- > From: Hoon Yoon - IPT Quant > To: matrix-sig@python.org > Subject: [MATRIX-SIG] Extending NumPy > Date: Tuesday, August 19, 1997 9:28 AM > > Hello, > > I am finally embarking on extending Python with C programs and at > this point I am trying to push in Lists of strings and get back > lists of strings and Numerical arrays back from a C function. > Unfortunately, I am having problems finding examples of these > codes in C. I guess the following from gdmodule does show how > to kinda pass in a list or tuple into a C prog, but I am lost on > how to do this for string arrays (multiple) and get back string > arrays and most of all Numeric Arrays: > > ***************************************************************** > Nice example of how to do this with number tuple > Can I use **points to do this for string tuple and args1, args2, args3? > ***************************************************************** > static PyObject *image_lines(self, args) > imageobject *self; > PyObject *args; > { > PyObject *point, *points, *bit; > int color, i, x,y,ox,oy; > > if (!PyArg_ParseTuple(args, "O!i", &PyTuple_Type, &points, &color)) > { > PyErr_Clear(); > if (PyArg_ParseTuple(args, "O!i", &PyList_Type, &points, &color)) > points = PyList_AsTuple(points); > else return NULL; > } > > point = PyTuple_GET_ITEM(points,0); > ox = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > oy = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > for (i=1; i { > point = PyTuple_GET_ITEM(points,i); > x = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > y = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > gdImageLine(self->imagedata, X(ox), Y(oy), X(x), Y(y), color); > ox=x;oy=y; > } > > Py_INCREF(Py_None); > return Py_None; > } > ********************************************************************** > Can anyone show me some example of these extentions, I've mentioned > here? At least I would like to get back list in list > [[1,2,3],[4,5,6]] type of thing, which I could easily convert back > to a Num Array. > > Much appreciated, > > Hoon, > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Aug 19 16:39:48 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 19 Aug 1997 11:39:48 -0400 Subject: [MATRIX-SIG] Extending NumPy Message-ID: <199708191552.LAA16539@dante.mh.lucent.com> If you can deal with a bit of overhead, maybe you should use the abstract object interface (thanks again Jim Fulton!) to extract Python objects from sequences and get their values. Look at abstract.h. Here, I think, you could pass any Python sequence down to the C interface and extract the elements. For Numeric this might add some unneeded malloc overhead when an int or float is extracted from a NumPy array, and that will slow down the code, but it would almost certainly be the easiest thing to do. In the reverse direction you can create and populate a listobject, and have Python convert it to an array. Again, not the most space/time efficient approach, but it'll work and soon. Look at listobject.h. Careful with refcounts, of course. Life is compromise. -- Aaron Watters ---------- > From: Hoon Yoon - IPT Quant > To: matrix-sig@python.org > Subject: [MATRIX-SIG] Extending NumPy > Date: Tuesday, August 19, 1997 9:28 AM > > Hello, > > I am finally embarking on extending Python with C programs and at > this point I am trying to push in Lists of strings and get back > lists of strings and Numerical arrays back from a C function. > Unfortunately, I am having problems finding examples of these > codes in C. I guess the following from gdmodule does show how > to kinda pass in a list or tuple into a C prog, but I am lost on > how to do this for string arrays (multiple) and get back string > arrays and most of all Numeric Arrays: > > ***************************************************************** > Nice example of how to do this with number tuple > Can I use **points to do this for string tuple and args1, args2, args3? > ***************************************************************** > static PyObject *image_lines(self, args) > imageobject *self; > PyObject *args; > { > PyObject *point, *points, *bit; > int color, i, x,y,ox,oy; > > if (!PyArg_ParseTuple(args, "O!i", &PyTuple_Type, &points, &color)) > { > PyErr_Clear(); > if (PyArg_ParseTuple(args, "O!i", &PyList_Type, &points, &color)) > points = PyList_AsTuple(points); > else return NULL; > } > > point = PyTuple_GET_ITEM(points,0); > ox = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > oy = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > for (i=1; i { > point = PyTuple_GET_ITEM(points,i); > x = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > y = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > gdImageLine(self->imagedata, X(ox), Y(oy), X(x), Y(y), color); > ox=x;oy=y; > } > > Py_INCREF(Py_None); > return Py_None; > } > ********************************************************************** > Can anyone show me some example of these extentions, I've mentioned > here? At least I would like to get back list in list > [[1,2,3],[4,5,6]] type of thing, which I could easily convert back > to a Num Array. > > Much appreciated, > > Hoon, > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Aug 19 16:39:48 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 19 Aug 1997 11:39:48 -0400 Subject: [MATRIX-SIG] Extending NumPy Message-ID: <199708191535.LAA15229@dante.mh.lucent.com> If you can deal with a bit of overhead, maybe you should use the abstract object interface (thanks again Jim Fulton!) to extract Python objects from sequences and get their values. Look at abstract.h. Here, I think, you could pass any Python sequence down to the C interface and extract the elements. For Numeric this might add some unneeded malloc overhead when an int or float is extracted from a NumPy array, and that will slow down the code, but it would almost certainly be the easiest thing to do. In the reverse direction you can create and populate a listobject, and have Python convert it to an array. Again, not the most space/time efficient approach, but it'll work and soon. Look at listobject.h. Careful with refcounts, of course. Life is compromise. -- Aaron Watters ---------- > From: Hoon Yoon - IPT Quant > To: matrix-sig@python.org > Subject: [MATRIX-SIG] Extending NumPy > Date: Tuesday, August 19, 1997 9:28 AM > > Hello, > > I am finally embarking on extending Python with C programs and at > this point I am trying to push in Lists of strings and get back > lists of strings and Numerical arrays back from a C function. > Unfortunately, I am having problems finding examples of these > codes in C. I guess the following from gdmodule does show how > to kinda pass in a list or tuple into a C prog, but I am lost on > how to do this for string arrays (multiple) and get back string > arrays and most of all Numeric Arrays: > > ***************************************************************** > Nice example of how to do this with number tuple > Can I use **points to do this for string tuple and args1, args2, args3? > ***************************************************************** > static PyObject *image_lines(self, args) > imageobject *self; > PyObject *args; > { > PyObject *point, *points, *bit; > int color, i, x,y,ox,oy; > > if (!PyArg_ParseTuple(args, "O!i", &PyTuple_Type, &points, &color)) > { > PyErr_Clear(); > if (PyArg_ParseTuple(args, "O!i", &PyList_Type, &points, &color)) > points = PyList_AsTuple(points); > else return NULL; > } > > point = PyTuple_GET_ITEM(points,0); > ox = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > oy = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > for (i=1; i { > point = PyTuple_GET_ITEM(points,i); > x = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,0)); > y = PyInt_AS_LONG((PyIntObject *)PyTuple_GET_ITEM(point,1)); > gdImageLine(self->imagedata, X(ox), Y(oy), X(x), Y(y), color); > ox=x;oy=y; > } > > Py_INCREF(Py_None); > return Py_None; > } > ********************************************************************** > Can anyone show me some example of these extentions, I've mentioned > here? At least I would like to get back list in list > [[1,2,3],[4,5,6]] type of thing, which I could easily convert back > to a Num Array. > > Much appreciated, > > Hoon, > > _______________ > MATRIX-SIG - SIG on Matrix Math for Python > > send messages to: matrix-sig@python.org > administrivia to: matrix-sig-request@python.org > _______________ > _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From arw@dante.mh.lucent.com Tue Aug 19 17:59:35 1997 From: arw@dante.mh.lucent.com (Aaron Watters) Date: Tue, 19 Aug 1997 12:59:35 -0400 Subject: sorry! Re: [MATRIX-SIG] Extending NumPy Message-ID: <199708191655.MAA29292@dante.mh.lucent.com> mail server problems. (blush). ---------- > From: Aaron Watters > To: Hoon Yoon - IPT Quant ; matrix-sig@python.org > Subject: Re: [MATRIX-SIG] Extending NumPy > Date: Tuesday, August 19, 1997 11:39 AM _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Wed Aug 20 09:51:50 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Wed, 20 Aug 1997 10:51:50 +0200 Subject: [MATRIX-SIG] Extending NumPy In-Reply-To: <199708191556.LAA16768@dante.mh.lucent.com> (arw@dante.mh.lucent.com) Message-ID: <199708200851.KAA04069@lmspc2.ibs.fr> > If you can deal with a bit of overhead, maybe > you should use the abstract object interface > (thanks again Jim Fulton!) to extract Python > objects from sequences and get their values. For values passed into a function there is some merit to this approach, but for large amounts of data it is not very efficient, and it's certainly more complicated to code. If you want to accept an arbitrary sequence object and convert it to an array, just call PyArray_ContiguousFromObject(). That's more or less what Numeric.array() is at the Python level. You get a contiguous array that is easy to work on. For returning data, constructing an array is probably much easier than creating any other type of sequence object. Just call PyArray_FromDims(), which creates a contiguous array, and fill it with your data. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Wed Aug 20 15:21:19 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Wed, 20 Aug 1997 09:21:19 -0500 (CDT) Subject: [MATRIX-SIG] Bug report (and patch) Complex to PyObject In-Reply-To: Message-ID: There's a bug in the Complex->PyObject casting routine (For both CFLOAT and CDOUBLE). Here's an example: >>> arange(5) + 0j array([ 0.+0.j, 1.+0.j, 2.+0.j, 3.+0.j, 4.+0.j]) >>> _.astype(PyObject) array([0j , 1j , (1+0j) , 2j , (2+0j) ],'O') >>> A patch to arraytypes.c to fix this is included below. Basically, the complex array was being stepped by 1 double (or float) instead of two, so the result was all messed up. ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ *** NumPy/arraytypes.c Sat Aug 16 08:36:09 1997 --- NumPy.old/arraytypes.c Fri Mar 21 15:09:59 1997 *************** *** 474,480 **** {int i; for(i=0;i<2*n;i++,ip+=ipstep,op+=opstep) {*op = (double)*ip;}} static void CFLOAT_to_OBJECT(float *ip, int ipstep, PyObject **op, int opstep, int n) ! {int i; for(i=0;i Message-ID: Konrad said it would be hard, so I had to do it... 1) I added a compare function compares python arrays element by element. For example compare([1,2,4], [4,2,1]) returns array([ -1, 0, 1]). This works with arrays of PyObjects as well as the numerical types. 2) I extended less, less_equal, etc. to work with arrays of PyObjects. Of course the number of times I've had to compare PyObjects is, well, zero. However, compare might be more useful. For instance sign(x) can be implemented as compare(x,0). And besides, this is what revealed the complex to PyObject conversion bug, so it wasn't totally useless... Anyway, if anyone is interested in this, the patches are available on my starship page: http://starship.skyport.net/~hochberg ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From da@maigret.cog.brown.edu Sun Aug 24 03:54:50 1997 From: da@maigret.cog.brown.edu (David Ascher) Date: Sat, 23 Aug 1997 22:54:50 -0400 (EDT) Subject: [MATRIX-SIG] NumPy FAQ Message-ID: Well, I set it up, but I really don't have time right now to add entries to it. So I suggest that whoever is interested feel free to access the budding NumPy FAQ and add entries to their heart's content. When my personal dust settles down a bit, I'll do a bit of filtering and reorganization. URL: http://starship.skyport.net/~da/numpyfaqw.cgi Use the "add a faq entry" button to add entries, that way you see all the categories I started out with. I pretty much randomly chose: 1. General information and availability 2. Simple things, really 3. Advanced topics, still in Python 4. Integration with C, FORTRAN, C++, etc. 5. Graphics, Plotting, etc. 6. Other things Aaron suggested the password of Euler, but I overrode him and chose "Hadamard", without the quotes. All changes are logged. FYI: The cookies are so that you don't have to type your name and email and password everytime. --da _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Fri Aug 29 18:40:34 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Fri, 29 Aug 1997 12:40:34 -0500 (CDT) Subject: [MATRIX-SIG] FancyArray In-Reply-To: Message-ID: If anyone's interested in playing with it I've put together a FancyArray class derived from UserArray that allows indexing with sequences. >>> a = FancyArray(zeros((10,10))) >>> a[(1,4), (5,6)] = 5 >>> a[(2,9), 1:5] = 9 >>> a FancyArray([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 5, 5, 0, 0, 0], [0, 9, 9, 9, 9, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 5, 5, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 9, 9, 9, 9, 0, 0, 0, 0, 0]]) Addition, etc, all work. However, there are deinitely still some bugs, some of which are my fault, and some of which have to do with UserArray and its interaction with the builtin functions. In particular, reshape doesn't work right for FancyArray or UserArray. So, don't look at this as a finished product, consider it a proof of concept type thang'. If enough people are interested, this could be polished into a useful type for those who yearn for S-like indexing (not that I've ever used S). It does this by keeping a data array (shared between different slices of the same matrix) and a mask array. It should be not too slow since all the loops are still in C. The one exception is that I needed to fake a 'give' (opposite of take) function using map, and I don't know how fast that is, but is could be easily be rewritten in C. Anyway, enjoy. If this looks interesting let me know. ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ """FancyArray is a class that provides fancy indexing.""" import sys, UserArray, operator from Numeric import * from types import * from indexing import * def give(fro, to, indices, axis=0): """Give 'to' the elements of 'fro' at 'indices'.""" map(operator.setitem, [to]*len(fro), indices, fro) class FancyArray(UserArray.UserArray): """An array-like object that supports "fancy" indexing. The index is either a tuple of subindinces or a single subindex. The subindexes can have one of four forms: 1. An integer. 2. A slice object (or i:j:k). 3. An ellipses object (or ...). 4. A NewAxis (None) object. 5. A sequence of integers. The first four types of subindices are available with normal arrays, it is this last type which is new. If a single, sequence, subindex is used, the sequence must not be a tuple as that would be interpreted as a tuple of integer subindices. Examples: * A[(1,2,3), (0,2,4)] => A[1:4:2,0:5:2]. * A[[1,4,7,0] => The {A[1], A[4], A[7], A[0]} """ def __init__(self, array, typecode=None, mask=None): """Create a FancyArray.""" try: data = asarray(array.data, typecode) mask = asarray(array.mask) except: try: data = reshape(asarray(array, typecode), (-1,)) if mask == None: mask = reshape(arange(len(data)), shape(array)) except: raise ValueError, "Bad initializer." self.data = data self.mask = mask self.shape = shape(mask) self.typecode = self.data.typecode() self.name = string.split(str(self.__class__))[1] def __getattr__(self, key): """Fake the presence of an array attribute.""" if key == "array": return take(self.data, self.mask) else: raise AttributeError, key def __float__(self): """Return the floating point representation of self.""" # This could be put in UserArray instead - it would be better?! # __int__ (?), etc. should probably also be added to UserArray. return self.__class__(self, Float) def _return(self, mask): """Return a new FancyArray from a mask.""" if len(shape(mask)) == 0: self.data[mask] else: return self.__class__(self.data, mask=mask) def _rc(self, array): """Return a new FancyArray from an array.""" if len(shape(array)) == 0: return array else: return self.__class__(array) def _make_mask(self, indices): if type(indices) != TupleType: indices = (indices,) mask = self.mask skip = 0 ellip = 0 theslice = slice(None, None, None) for (axis, index) in indexed(indices): itype = type(index) if itype == SliceType: mask = mask[[theslice]*skip+[index]] skip = skip + 1 elif itype == IntType: mask = mask[[theslice]*skip+[index]] elif itype is NoneType: mask = mask[[theslice]*skip+[index]] skip = skip + 2 elif itype == EllipsisType: skip = len(shape(mask)) - (len(indices) - axis) + 1 else: # Try a sequence. try: mask = take(mask, index, skip) skip = skip + 1 except: raise IndexError, "Bad index of type %s" % itype return mask def __getitem__(self, indices): """Get an item from an array using generalized indexing.""" return self._return(self._make_mask(indices)) def __getslice__(self, i, j): """Get a slice from array.""" return self._return(self.mask[i:j]) def __setitem__(self, indices, value): """Set an item in array using generalized indexing.""" mask = self._make_mask(indices) if shape(value) != shape(mask): raise IndexError, "Matrices not alligned for copy" give(reshape(value, (-1,)), self.data, reshape(mask, (-1,))) def __setslice__(self, i, j, value): """Set a slice in the array.""" give(reshape(value, (-1,)), self.data, reshape(mask[i:j], (-1))) def print_result(c,d): try: if alltrue(equal(c,d).flat) or \ (type(c) != InstanceType and c.__class__ != FancyArray): print "OK" else: print "FAILED!" except: print print "ERROR IN TEST SUITE" print "Was processing:" print shape(c) print shape(d) def test(): """Test suite.""" a0 = ones((5,5,5)) a = FancyArray(a0) b0 = arange(5.) b = FancyArray(b0) for op in (add, subtract, multiply, divide): print "Checking", `op`, c = op(a0,b0) d = op(a, b) print_result(c,d) for expr, equiv in (("b[(range(5))]", "b0[0:5]"), ("b[(range(0,5,2))]", "b0[0:5:2]"), ("a[:,(range(0,5,2))]", "a0[:,0:5:2]"), ("a[:,:,(range(0,5,2))]", "a0[:,:,0:5:2]"), ("a[...,(range(0,5,2))]", "a0[...,0:5:2]"), ("a[NewAxis,...][...,(range(0,5,2))]", "a0[NewAxis,...][...,0:5:2]"), ): print "Checking", expr, c = eval(expr) d = eval(equiv) print_result(c,d) for expr, equiv in \ (("c[range(5)] = arange(5)[:,NewAxis,NewAxis] + zeros(shape(c[range(5)]))", "d[0:5] = arange(5)[:,NewAxis,NewAxis] + zeros(shape(d[0:5]))"), ("c[(range(0,5,2))]=arange(0,5,2)[:,NewAxis,NewAxis]+zeros(shape(c[(range(0,5,2))])) ", "d[0:5:2] = arange(0,5,2)[:,NewAxis,NewAxis]+zeros(shape(c[(range(0,5,2))]))"), ("c[:,(range(0,5,2))] = arange(0,5,2)[NewAxis,:,NewAxis]+zeros(shape(c[:,(range(0,5,2))]))", "d[:,0:5:2] = arange(0,5,2)[NewAxis,:,NewAxis]+zeros(shape(c[:,(range(0,5,2))]))"), ("c[:,:,(range(0,5,2))] = arange(0,5,2)[NewAxis,NewAxis,:]+zeros(shape(c[:,:,(range(0,5,2))]))", "d[:,:,0:5:2] = arange(0,5,2)[NewAxis,NewAxis,:]+zeros(shape(c[:,:,(range(0,5,2))]))"), ("c[...,(range(0,5,2))] = arange(0,5,2)[NewAxis,NewAxis,:]+zeros(shape(c[:,:,(range(0,5,2))]))", "d[...,0:5:2] = arange(0,5,2)[NewAxis,NewAxis,:]+zeros(shape(c[:,:,(range(0,5,2))]))"), ("c[NewAxis,...][...,(range(0,5,2))] = arange(0,5,2)[NewAxis,NewAxis,NewAxis,:] + zeros(shape(c[NewAxis,...][...,(range(0,5,2))]))", "d[NewAxis,...][...,0:5:2] = arange(0,5,2)[NewAxis,NewAxis,NewAxis,:]+ zeros(shape(c[NewAxis,...][...,(range(0,5,2))]))"), ): c = FancyArray(zeros((10,10,10), Float)) d = zeros((10,10,10), Float) print "Checking", expr[:30], "...", exec (expr) exec (equiv) print_result(c,d) if __name__ == "__main__": test() ~ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jhauser@ifm.uni-kiel.de Fri Aug 29 21:30:47 1997 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Fri, 29 Aug 1997 22:30:47 +0200 (CEST) Subject: [MATRIX-SIG] FancyArray Message-ID: Hi, I haven't played with it yet, but here is a solution to the reshape problem. First, reshape does function here, but returns not an UserArray object. >>> a UserArray([0, 1, 2, 3, 4, 5, 6, 7, 8]) >>> reshape(a,(3,3)) array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) But according to the docs of reshape this is a behavior, which can be expected in some sense, because reshape returns a copy. reshape is also not an builtin function, so I see no way to deal with it in the class itself. But the docs emphasize, that one should use a.shape=(3,3) for inplace reshaping. This can be handled by the class with following method: def __setattr__(self,att,value): if att == 'shape': self.__dict__['shape']=value self.array.shape=value else: self.__dict__[att]=value >>> c=myUserArray.UserArray(range(9)) >>> c UserArray [0 1 2 3 4 5 6 7 8] >>> c.shape=(3,3) >>> c UserArray [[0 1 2] [3 4 5] [6 7 8]] >>> If there are some more drawbacks or wishful things for the UserArray class, please mention it. I see many useful way to use UserArray, so this class should be as powerful as possible. __Janko _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Fri Aug 29 23:15:11 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Fri, 29 Aug 1997 17:15:11 -0500 (CDT) Subject: [MATRIX-SIG] FancyArray In-Reply-To: Message-ID: On Fri, 29 Aug 1997, Janko Hauser wrote: > Hi, I haven't played with it yet, but here is a solution to the > reshape problem. > First, reshape does function here, but returns not an UserArray > object. > > >>> a > UserArray([0, 1, 2, 3, 4, 5, 6, 7, 8]) > >>> reshape(a,(3,3)) > array([[0, 1, 2], > [3, 4, 5], > [6, 7, 8]]) > > But according to the docs of reshape this is a behavior, which can be > expected in some sense, because reshape returns a copy. Yes, but I'd still argue that the natural thing to get back is a copy of the same type as what you put in. Reshaping a UserArray should really return a UserArray. Or maybe not. It doesn't bother me that reshaping a tuple gives me an array. Hmmmm.... Now I'm thinking you might want both. Numeric.reshape which has the current behaviour and UserArray.reshape which preserves the class of the reshaped object. > reshape is > also not an builtin function, so I see no way to deal with it in the > class itself. The way to deal with this is to shadow the functions from multiarray with python functions in order to "add extra intelligence to the basic C functions" (from Numeric.py). In order to fix reshape, for instance, one could use something like: def reshape(a, shape): if type(a) === InstanceType: return a.__class__(reshape(a,shape)) else: return reshape(a, shape) A bunch of these shadow functions would need to be constructed, but they're all essentially the same. As I mentioned, UserArray would probably be a good place for them. > But the docs emphasize, that one should use > a.shape=(3,3) for inplace reshaping. This can be handled by the class > with following method: > > def __setattr__(self,att,value): > if att == 'shape': > self.__dict__['shape']=value > self.array.shape=value > else: > self.__dict__[att]=value This fix looks good for UserArray. And I could do something similar for FancyArray. To be honest, I'd completely forgotten that setting shape reshaped the arrray. I think I would raise an exception when attributes other than shape get set though. > > > >>> c=myUserArray.UserArray(range(9)) > >>> c > UserArray > [0 1 2 3 4 5 6 7 8] > >>> c.shape=(3,3) > >>> c > UserArray > [[0 1 2] > [3 4 5] > [6 7 8]] > >>> > > > If there are some more drawbacks or wishful things for the UserArray > class, please mention it. I see many useful way to use UserArray, so > this class should be as powerful as possible. Well I haven't used UserArray much yet, so my list is pretty short. * Write shadow functions (I just thought of a clever way to do that.) * Figure out what __float__ is doing there. As far as I can tell float never suceeds for UserArrays or Normal arrays, so what is it doing there? * The indexing on UserArray isn't quite right - sometimes is returns a copy when it should be returning a reference. This is fixed in FancyArray though. Has anyone used UserArray much? Have you had any problems with it? Successes? I'm mostly just curious... ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Sat Aug 30 14:27:33 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Sat, 30 Aug 1997 15:27:33 +0200 Subject: [MATRIX-SIG] FancyArray In-Reply-To: (hochberg@wwa.com) Message-ID: <199708301327.PAA32667@lmspc1.ibs.fr> > Yes, but I'd still argue that the natural thing to get back is a copy of > the same type as what you put in. Reshaping a UserArray should really > return a UserArray. Sure. But what about more complicated situations? What should add(a, b) return if a and b are instances of different subclasses of UserArray? > The way to deal with this is to shadow the functions from multiarray with > python functions in order to "add extra intelligence to the basic C > functions" (from Numeric.py). In order to fix reshape, for instance, one Which adds overhead to every function call. It would be better to implement a good strategy right in the C code - but first there must be a good strategy. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hinsen@ibs.ibs.fr Sat Aug 30 14:31:29 1997 From: hinsen@ibs.ibs.fr (Konrad Hinsen) Date: Sat, 30 Aug 1997 15:31:29 +0200 Subject: [MATRIX-SIG] Weird bug Message-ID: <199708301331.PAA32723@lmspc1.ibs.fr> I have hit the following bug which looks really dangerous to me - wrong results in a perfectly ordinary situation: >>> from Numeric import * >>> a = array([[1,2],[3,4]]) >>> a + transpose(a) array([[2, 5], [5, 8]]) >>> add(a, transpose(a)) array([[2, 5], [5, 8]]) >>> add(a, transpose(a), a) array([[2, 5], [8, 8]]) >>> a array([[2, 5], [8, 8]]) I won't have any time soon to check what's going on, but watch out for this if you use the three-argument arithmetic functions. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@ibs.ibs.fr Laboratoire de Dynamique Moleculaire | Tel.: +33-4.76.88.99.28 Institut de Biologie Structurale | Fax: +33-4.76.88.54.94 41, av. des Martyrs | Deutsch/Esperanto/English/ 38027 Grenoble Cedex 1, France | Nederlands/Francais ------------------------------------------------------------------------------- _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From jhauser@ifm.uni-kiel.de Sat Aug 30 17:15:36 1997 From: jhauser@ifm.uni-kiel.de (Janko Hauser) Date: Sat, 30 Aug 1997 18:15:36 +0200 (CEST) Subject: [MATRIX-SIG] Weird bug In-Reply-To: <199708301331.PAA32723@lmspc1.ibs.fr> References: <199708301331.PAA32723@lmspc1.ibs.fr> Message-ID: Here is a variation wich exhibits not this bug. >>> a = array([[1,2],[3,4]]) >>> b=zeros(a.shape) >>> add(a, transpose(a), b) array([[2, 5], [5, 8]]) >>> One additional note. I find it very strange to write in a program: b=add(a, transpose(a), b) although I understand the mechanismen why this must be done. But is there a way around this? __Janko _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hugunin@CNRI.Reston.Va.US Sat Aug 30 17:59:09 1997 From: hugunin@CNRI.Reston.Va.US (Jim Hugunin) Date: Sat, 30 Aug 1997 12:59:09 -0400 Subject: [MATRIX-SIG] Weird bug References: <199708301331.PAA32723@lmspc1.ibs.fr> Message-ID: <3408515D.F0EEE522@cnri.reston.va.us> I understand exactly what's going on here, and I'd argue that in fact it is NOT a bug. The problem is related to the choice early on for things like transpose(a) to point to the same memory as the original 'a' does. Naive users are strongly discouraged from using the 3-argument numeric functions and it is exactly this sort of behavior that is behind these warnings. If you realize that 'a' and transpose(a) both point to the same region of memory this "bug" makes perfect sense. In fact, there's no alternative way of implementing things that would make this "bug" go away without making 3-argument numeric functions use a temporary array for storing their result, which would completely eliminate their usefulness. -Jim Konrad Hinsen wrote: > > I have hit the following bug which looks really dangerous to me - wrong > results in a perfectly ordinary situation: > > >>> from Numeric import * > >>> a = array([[1,2],[3,4]]) > >>> a + transpose(a) > array([[2, 5], > [5, 8]]) > >>> add(a, transpose(a)) > array([[2, 5], > [5, 8]]) > >>> add(a, transpose(a), a) > array([[2, 5], > [8, 8]]) > >>> a > array([[2, 5], > [8, 8]]) > > I won't have any time soon to check what's going on, but watch out > for this if you use the three-argument arithmetic functions. _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________ From hochberg@wwa.com Sat Aug 30 22:21:59 1997 From: hochberg@wwa.com (Timothy A. Hochberg) Date: Sat, 30 Aug 1997 16:21:59 -0500 (CDT) Subject: [MATRIX-SIG] FancyArray In-Reply-To: <199708301327.PAA32667@lmspc1.ibs.fr> Message-ID: TH = Tim Hochberg KH = Konrad Hinsen On Sat, 30 Aug 1997, Konrad Hinsen wrote: [TH sez' reshaping a UserArray should return a UserArray] To which KH replies: > Sure. But what about more complicated situations? What > should add(a, b) return if a and b are instances of different subclasses > of UserArray? Well the best rule I can come up with is, given a set of arguments that are in a direct line of descent such as (array->UserArray->SomeSpecializedArray), the return type should be of the type that is lowest in the inheritance chain. Mixed array types that were not in a direct line of descent would raise an error. For example, given the following class structure: array--->UserArray-+->SpecializedArray1--->SubSpecializedArray1 | +->SpecializedArray2 take(array, SubSpecializedArray1) returns a SubSpecializedArray1 take(UserArray, SpecializedArray2) returns a SpecializedArray2 take(SpecializedArray1, SpecializedArray2) raises an error. Implementing this, even in Python let alone C, seems like it might be somewhat of a pain. However, it would only need to be implemented once (say as a function called figure_out_return_type(*args)). This could then be called by each function that needs to know the return type. [TH sez' shadow functions would be a good thing, but KH says:] > Which adds overhead to every function call. It would be better to > implement a good strategy right in the C code - but first there must > be a good strategy. That is why I was suggesting that these functions go in UserArray at least for the part of their lifecycle where they are written in Python. People who are using UserArray are allready working with enough overhead that I doubt they'd notice the difference. Also, a good chunk of the Numeric functions are actually shadow functions in Numeric. Should these be migrated to C? ____ /im +------------------------------------------------+ |Tim Hochberg Research Assistant | |hochberg wwa.com University of Illinois | | Electrical Engineering | +------------------------------------------------+ _______________ MATRIX-SIG - SIG on Matrix Math for Python send messages to: matrix-sig@python.org administrivia to: matrix-sig-request@python.org _______________