releasing memory to malloc

[Bryan Olson]

Paul Rubin wrote:
>     Is there any way to get Python to release memory back to the
>     C allocator? I'm currently running a script that goes through
>     the following steps:
> 
>     1) Creates a very large number of Python objects to produce
>     a relatively small data structure that sits in a C extension.
>     The Python objects consume quite a bit of memory.
> 
>     2) Releases all the Python objects.
> 
>     3) Invokes a function of said C extension for further
>     processing. This step needs as much memory as possible.
>     ...
> 
>    I happen to have the code for the C library in question, but I
>    don't think this is the way to go in general.  If there's a way to
>    get Python to give memory back to the C allocator I can avoid
>    touching the library at all.
> 
> A cave-man approach might be to fork a new process after step 1, pass
> the small data structure to it, and have the old process exit
> (releasing all its memory back to the OS).  The new process then
> carries out the remaining steps.

I think I see what you're doing, but fork() after step 1 will
create a child process with the same memory allocated.

I think it would make more sense to do step 1 in a subprocess.
Use the subprocess module or one of the older popen()s to create
a process that builds the target object, pickles it and pipes
it back to the main process, then exits.


-- 
--Bryan