[Numpy-discussion] Numeric array's actual data address?

[RJ]

Thanks Todd, Travis,

Yes, Travis, I _was_ using ctypes' memmove() with numarray to great benefit. I had also understood that the address of the numarray data was stable.

Originally, I was trying to move data from a hardware A/D driver DLL call to Python as quickly as possible and did some benchmarking, so as per Thomas Heller's suggestion on the ctypes list I used memmove():
http://sourceforge.net/mailarchive/forum.php?thread_id=6166311&forum_id=24606
 from ctypes import *
 import array 
 src = array.array("i", range(32))
 dst = (c_int * 32)()
 memmove(dst, *src.buffer_info())
memmove() is orders of magnitude faster than map() or assignments.
I then tested and found numarray faster for the rest of the tasks (FFT etc.) http://sourceforge.net/mailarchive/forum.php?thread_id=6205658&forum_id=24606
and so parsed the info() output and plugged the data[0] address into the call.

Thomas Heller (in the above) suggested he make a mod to ctypes to accept Python objects which implement the buffer interface as function parameters, which would allow Numeric use once implemented.
I like numarray's breadth of methods so I used it at the time, but in another try at speed-squeezing yesterday afternoon I found Numeric's FFT and subtraction to be >30% faster in this case, so I switched that code over (this increase includes the use of map() with Numeric to read the A/D in another thread). Using memmove() with Numeric would speed up the reader thread once again.

At 08:06 AM 2/1/2005 -0500, Todd Miller wrote:
>> 
>> What is the actual address of the first element? 
>
>In C,  look at a->data.

I had read the Numeric API and looked at the PyObject structure, as Travis then suggested, but my question then is: if the offset from the "array object at 0x..." (object address value) to the array[0] address is not fixed and must be read from the pointer in the PyObject structure, can we get that pointer's value directly from Python or ctypes? 
ctypes pointer() "allows" poking in and directly reading memory: 
"It is also possible to use indexes different from 0, but you must know what you're doing when you use this: You access or change arbitrary memory locations when you do this."
http://starship.python.net/crew/theller/ctypes/tutorial.html
So, could I use ctypes' pointer(offset) to read the structure's pointer-to-data[0]-value and then use it in memmove()? 
I'll cross-post this to ctypes-users...

>I think the "fragile data pointer" is generally useful information,  but
>not completely dependable so I gave it the garish name it has.
>Comments?

It's quite reasonable. 
Is the data pointer value really often changed for contiguous arrays in small stand-alone Python apps? If so, I may stick with map() to avoid having to re-read the pointer before each data buffer transfer.

If anyone is interested I could post some relevant code snips from the office...

Thanks for the help,
Ray
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20050201/74d07089/attachment-0001.html>