Upper memory limit

[Siegfried Gonzi]

Chris Barker wrote:

> I highly suspect that your memory leak is in your C or Fortran
> functions. Python DOES NOT handle memory mangement for extenal
> functions. If your C or Fortran functions create Python objects and
> don't DECREF them properly, you can create a memory leak there. I have
> had that problem. It is pretty easy to test for. before and after
> calling your functions, do a memory use check. You could do this by hand
> by putting a input() statement or something in to halt the code, or you
> can write a little function that makes a system call to check how much
> memory Python is using. I have dsone this on Linux, I have no idea how
> on Windows.

I have the clue now I think.

I wrote (for the sake of testing) a Fortran 95 program in order to
extract floating point values from a very large file (10MB). The scheme
of the file is as follows:

12.233,NIL,22.334,NIL
1.23,0.002,0.0,NIL
NIL,1223.334,33.34,4445
...

I write the array (400 000 of lines) into a Numeric array (NIL becomes
-1.0). I repeat the task 10 times in a Python function. The Fortran 95
(F compiler from imagine1) takes on my laptop under Windows XP about 5
minutes for 10 reading cycles. The Fortran version collects about 133MB
of RAM (I use also some pointers in the Fortran version). After reading
10 times the file Windows XP is very okay, and I do not have to reboot
the system. This all on my Windows XP laptop.

The same task now  on my 1000 MHz Celeron, 256MB, laptop takes
theoretically about 30 minutes; but practically it never ends. Beginning
at the 10. reading cycle the system wreaks havoc (the first 9 readings
are okay and every file access takes 60MB RAM; here the RAM consumption
from one cycle to the next is constant; the Fortran version collects for
every cycle and adds them up:  10 times 13MB).

I know the above situation: Is it a memory leak? Windows XP shit?

I did the same on my new installed Linux on my stationary machine
(Pentium III 450MHz, 128MB RAM). Python takes on this machine 2 times
longer than on my 1000Mhz Celeron (For comparison I write here the ideal
situation if Python had stopped in 30 minutes on my Celeron laptop).
After completing the task the system goes on normal.

I did the same test on the same stationary machine but now under Windows
NT 4.0 and Python. On Windows NT 4.0 the Python file reading function
takes exactly the same amount of time than the version under Linux *and*
after completing the task nothing happens and the system remains
*stable*.

There are only 2 explanations: Windows XP (NT 5.1) is shit or the
combination Windows XP and laptop is not optimal.

Normally I have Windows NT 4.0 running on my stationary machine for
weeks without ever experiencing a blue screen. 

S. Gonzi