Upper memory limit

Siegfried Gonzi siegfried.gonzi at kfunigraz.ac.at
Tue May 14 06:28:30 EDT 2002


Is there an upper memory limit which Python cannot handle?

I have a calculation (external C function included with the help of
SWIG; external Fortran function included with the help of F2PY; some pre
processed Lisp files; a dozen of binary satellite data files). One run
of the code takes about 1 hour. The execution time is not the problem!
Memory is not a problem too, but: it consumes after 1 hour 200 MB of
RAM. If I start the calculation in the DOS shell I can see that Python
gives memory back to the OS (I got the advice to start the calculation
from the DOS line), but this is only for 2 runs the case. If I start the
calculation for a third time I have to see that Python wreaks havoc. A
calculation would then take many, many hours due to the Python memory
management and at the end the system is down and I would have to restart
it (even on a newer laptop rebooting is not very quick and hence
anoying). 

I need some advice for my further strategy  (I am about to write my
simulations in Fortran 90/95 and use Python only for simple plotting
jobs):

a) Has anybody ever experienced the same problem?; I am not angry with
the execution time I am sick of the poor memory management behavior.

b) Is there semantically speaking a difference between automatic memory
managament and garbage collection?

c) Dose Python's garbage collector just simply collect and dispose all
the memory junks at the end of the calculation (and hence once) or does
Python collect garbage and at the same time removing old memory? Maybe
Windows's task manager is faulty but when I observe the beforementioned
calculation/simulation I can see that Python memory goes up to about
200MB, but I have a calculation of the scheme:


function for_a_better world( ... ):
	a = call_here_function( ... )
	b = call_here_function( ... )
	c = call_here_function( ... )
	...
return erg

Note: a,b,c,... essentially call the *same* function but with some minor
changes in the parameter list. Why doesn't Python delete the old memory
when it comes back from "call_here_function(...)"? It seems that all the
used memory in "call_here_function" remains  garbage.

I tried to use  del statements inside of call_here_function() but the
scenario goes even worse then. 

d) Has anybody experience with Python on Linux or free BSD? I would be
ready to install Linux in order to get a better Python behavior.


S. Gonzi



More information about the Python-list mailing list