[SciPy-User] [SciPy-user] numpy and C

[tinauser]

Dear David,
thanks for your suggestions. I have however some doubts, probably coming
from my unexpertice in C.

David Baddeley wrote:
> 
> If your description holds, what you're doing is allocating a block of
> memory (with PyArray_SimpleNew),
> then changing the pointer so that it points to your camera buffer, 
>  
that's right


David Baddeley wrote:
> 
> without ever using the memory you allocated. The original memory allocated
> with PyArray_SimpleNew will get leaked at this point. When Python comes to
> garbage collect your array, the camera buffer will be dealloced instead of
> the original block of memory. This sounds all BAD!!! 
> 
This I can't understand. I allocate only once (at initiation time) a PyArray
(2bytes);at running time I'm just updating the value of the data pointer
each time I want to get a frame.Python is always going to use this PyArray,
that is always at the same address,and look for the data in a different
section of the buffer,according to the update value of the "data" field. Am
I missing something?



David Baddeley wrote:
> 
> I have a feeling that PyArray_SimpleNew also sets the reference count to 1
> so there's no need to incref it (although you'd be well advised to check
> up on this). If this is the case, increfing effectively ensures that the
> array will never be garbage collected and creates a memory leak.
> 
I'll check that


David Baddeley wrote:
> 
> depending on how the data gets from the camera into the buffer you've got
> a few options - is it a preallocated buffer which gets constantly
> refreshed by the camera, or is it a buffer allocated on the fly to hold
> the results of a command such as camera_get_frame(*buffer).
> 
it is the first. The buffer is preallocated and the command is 
camera_get_frame(*frame). This command gives me the pointer to the frame
(which is within the preallocated buffer)


David Baddeley wrote:
> 
> If it's the first you could either ...
> 
> Use PyArray_SimpleNewFromData on your camera buffer, with the caveat that
> the values in the resulting array will be constantly refreshed from the
> camera.
> 
> or, use memcopy to copy the contents of the buffer to your newly allocated
> (with PyArray_SimpleNew) array - this way the python array won't change as
> the camera takes another frame. This also has the advantage that the c
> code doesn't need to worry about whether python is still using the
> original buffer before deleting it.
> 
I don't think I can use the first solution ;I'm using a buffer because while
I need to record all the frames, I can accept to miss some frames for
painting the wiget. Therefore, when I'm asking for a frame, the recording
camera is locking the frame and I can use that memory without limitation of
time. 
I avoided to use memcopy because I thought was quite slow with respect to
just pass a pointer.

Is there a way to check if I'm really leaking memory?
Thank you again

Lorenzo


David Baddeley wrote:
> 
> If it's the second the buffer contents won't be changing with time and I'd
> either use PyArray_SimpleNewFromData, or preferably, as this means you can
> let python handle the garbage collection for the frame, use
> PyArray_SimpleNew to allocate an array and pass the data pointer of this
> array to your camera_get_frame(*buffer) method. If you are stuck with a
> pre-allocated array and want to keep the python an c memory management as
> separate as possible, you could also use the memcopy route.
> 
> cheers,
> David
> 
> 
> 
> ----- Original Message ----
> From: tinauser <tinauser at libero.it>
> To: scipy-user at scipy.org
> Sent: Thu, 10 June, 2010 2:35:09 AM
> Subject: Re: [SciPy-User] [SciPy-user] numpy and C
> 
> 
> Dear Charles,
> 
> thanks again for the replies.
> Why do you say that is difficoult to free memory?
> What I do is to allocate the memory(pyincref) before calling the Python
> script. The Python script uses then a timer to call a C function to which
> the allocated PyArrayObject (created with PyArray SimpleNew) is passed. In
> C, the pointer of the PyArray is assigned to a pointer that points to a
> sort
> of data buffer that is filled from a camera. The data buffer is allocated
> elsewhere.
> When the python GUI is closed, I just decref my PyArrayObject, that I'm
> basically using just to pass pointer values. 
> 
> 
> 
> Charles R Harris wrote:
>> 
>> On Wed, Jun 9, 2010 at 7:46 AM, Charles R Harris
>> <charlesr.harris at gmail.com>wrote:
>> 
>>>
>>>
>>> On Wed, Jun 9, 2010 at 5:38 AM, tinauser <tinauser at libero.it> wrote:
>>>
>>>>
>>>> Dear Charles,
>>>> thanks for the reply.
>>>> The part of code causing the problem was exactly this
>>>>
>>>> Pymatout_img->data= cam_frame->data;
>>>> where Pymatout is a PyArrayObject and cam_frame is a structure having a
>>>> pointer to undefined char data.
>>>>
>>>> The code works all right if I recast in this way
>>>>
>>>> Pymatout_img->data= (char*)cam_frame->data;
>>>>
>>>> I'm not sure if this is allowed;I guessed it works because even if
>>>> Pymatout_img->data is always a pointer to char, the PyArrayObject looks
>>>> in
>>>> ->descr->type_num to see what is the data type.
>>>>
>>>>
>>> Numpy uses char* all over the place and later casts to the needed type,
>>> it's the old way of doing void*. So your explicit cast is fine. For some
>>> compilers, gcc for example, you also need to use a compiler flag to let
>>> the
>>> compiler know that you are going to do such things. In gcc the flag is
>>> -fno-strict-aliasing but I don't think you need to worry about this in
>>> VC.
>>>
>>> <snip>
>>>
>>>
>> That said, managing the data in this way can be problematic as you need
>> to
>> track alignment and worry about freeing of memory. You might want to look
>> at
>> PyArray SimpleNewFromData.
>> 
>> Chuck
>> 
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>> 
>> 
> 
> -- 
> View this message in context:
> http://old.nabble.com/numpy-and-C-tp28767579p28831237.html
> Sent from the Scipy-User mailing list archive at Nabble.com.
> 
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> 
> 
> 
>       
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> 
> 

-- 
View this message in context: http://old.nabble.com/numpy-and-C-tp28767579p28854111.html
Sent from the Scipy-User mailing list archive at Nabble.com.