How properly manage memory of this PyObject* array?? (C extension)

[John Machin]

On 14/07/2006 4:16 PM, seberino at spawar.navy.mil wrote:
>> It's quite simple, really: You malloc it, you free it.
> 
> John - I hope you don't mind that I really want to make sure I
> understand your
> good wisdom in this area by asking for clarification....
> 
> ASSUMPTIONS:
> 
> 1.  As long as we properly handle the reference counting of PyObjects
> then
>      memory management is taken care of for us.  (i.e. You don't ever
>      use the 'free' command explicitly on a PyObject* but let Python
> garbage collector
>      do the final freeing.)

That's correct: You didn't malloc it, you don't free it.

BTW, free is a function, not a "command".

> 
> 2.  All malloc'd stuff in C must be freed with free command.  (Python
> garbage
>     collector is limited to PyObjects.  It won't free non-Python stuff
> for us
>     like int arrays, char* strings, etc.)
> 

Correct.


> 
> Now it would appear that if you **malloc an array of PyObjects**
> (call it FOO[]) then you have an ambiguity....

No, not at all; there is no such action as "malloc an array of 
PyObjects". Firstly, malloc does not produce an array of anything; it 
allocates a chunk of memory and returns the address of the start of the 
chunk. secondly, what you stuffing into that memory is not Python 
objects but addresses of (a.k.a. pointers to) Python objects.

> 
> The PyObject elements will be freed *for us* eventually by garbage
> collector.
> Hence, we can't ever do 'free(FOO); ' because we don't know when
> garbage collector will free FOO[0], FOO[1], FOO[2], etc.

Let's try reductio ad adsurdum on that one. Suppose that instead of 
filling in a malloced chunk of memory, you had stored those gizmoids in 
local variables foo0, foo1, foo2, etc. Using your reasoning: we can't 
ever return from our function (which frees up the stack memory 
containing foo0 etc) because we don't know when garbage collector will 
free foo0 etc. Avoiding that problem would require a Python/C API 
function with a name like Py_AwaitGarbageCollection() ... but there 
isn't one.

HTH,
John