[C++-sig] numeric.hpp: copying raw data

[Philip Austin]

(Paul, I've cc'd you in case you aren't following the c++-sig.
Briefly, Dave has implemented a boost wrapper for both Numeric and
numarray that avoids the need to load the C api, using
PyObject_CallObject.  Since Numeric allows a cast to 
PyArrayObject*, this provides access to the data pointer
and to all of Numeric except PyArray_FromDimsAndData.  I
can't figure out how to make this work for numarray,
because the only way to get at the void* data pointer is
through the get_info function call.  Am I missing something?)

Dave, a couple of points extracted from below:

 > Anyway, about the above code: is there a way to do the equivalent of
 > 
 >     NA_Empty(1, &n, tInt32)
 > 
 > from Python?

There are two calls (zeros() and ones()) that are missing from your
interface that would make this work for Numeric (but not from
numarray, unless I'm missing something).  They will be slower than
NA_empty, since they initialize the arrays, but I don't think many
users will notice.  As I mention above, this still excludes the
constructors that use existing C data without a copy:
PyArray_FromDimsAndData in Numeric, and NA_New in numarray.  
These calls are especially useful for embedding, when there's
a Fortran main program that's allocating the memory.


 > Lastly, though I could be wrong since I'm no Numeric expert, is there any
 > point to constructing numeric::array in the last line? I mean, don't you
 > already have one? It seems you could just return obj.

We use some of the member functions (resizing, etc.) later in the
program, so it seemed clearer to keep it a numarray.

Regards, Phil

David Abrahams writes:
 > 
 > ----- Original Message -----
 > From: "Philip Austin" <paustin at eos.ubc.ca>
 > 
 > 
 > >
 > > Hi, we've been playing around with numeric.hpp and it's working well.
 > > I'm uncertain, however, about the best way to move data from C to
 > > python using numeric::array -- it looks like users with large arrays
 > coming
 > > from C need to bite the bullet and call import_libnumarray()?  I
 > > wanted to check to make sure we aren't missing something before we
 > > start implementing our own functions, like:
 > >
 > > numeric::array ptrToNum(int* data, int n=0){
 > >   NDInfo info;
 > >   object obj(detail::new_reference(NA_Empty(1, &n, tInt32)));
 > >   get_info(&obj, &info);
 > >   memcpy(info.data, data, 4*n);
 > >   return numeric::array(obj, "Int", n, false);
 > > }
 > >
 > > Regards, Phil Austin
 > 
 > The truth about the numeric support is this:
 > 
 > I discussed what to do with Paul Dubois, and he said, roughly, "what we
 > want is something that can be used as an argument to a wrapped function,
 > and which will match anything that you can construct a numarray with". I
 > countered with, "what about something that only matches numarrays -- you
 > can always use python::object if you want to match anything, and then you
 > can construct the numarray with that? It would be more consistent with the
 > other type wrappers". He said "great!" and we had a deal. We never
 > discussed the things we'd need to do to actually make it useful ;-)
 > 
 > Anyway, about the above code: is there a way to do the equivalent of
 > 
 >     NA_Empty(1, &n, tInt32)
 > 
 > from Python?
 > 
 > If so, you should be able to construct a numeric::array that way, right?
 > In that case, if you needed to touch the underlying data you could always
 > use its ptr() function instead of relying on implementation detail::s.
 > 
 > Lastly, though I could be wrong since I'm no Numeric expert, is there any
 > point to constructing numeric::array in the last line? I mean, don't you
 > already have one? It seems you could just return obj.
 > 
 > -Dave
 > 
 > -----------------------------------------------------------
 >            David Abrahams * Boost Consulting
 > dave at boost-consulting.com * http://www.boost-consulting.com
 > 
 > 
 > _______________________________________________
 > C++-sig mailing list
 > C++-sig at python.org
 > http://mail.python.org/mailman/listinfo/c++-sig