memory error

Bart Nessux bart_nessux at hotmail.com
Wed Jun 2 09:11:13 EDT 2004


def windows():
    import os
    excludes = ['hiberfil.sys', 'ipnathlp.dll', 'helpctr.exe', 'etc', 
'etc', 'etc']
    size_list = []
    for root, dirs, files in os.walk('/'):
       total = [x for x in files if x not in excludes]
       for t in total:
          s = file(os.path.join(root,t))
          size = s.read()
          size_list.append(size)
          s.close()

windows()

The above function crashes with a memory error on Windows XP Pro at the 
'size = s.read()' line. Page File usage (normally ~120 MB) will rise to 
300+ MB and pythonw.exe will consume about 200 MB of actual ram right 
before the crash. The machine has 512 MB of ram and is doing nothing 
else while running the script.

I've written the script several ways, all with the same result. I've 
noticed that a binary read 'rb' consumes almost twice as much physical 
memory and causes the crash to happen quicker, but that's about it.

Initially, I wanted to use Python to open every file on the system (that 
could be opened) and read the contents so I'd know the size of the file 
and then add all of the reads to a list that I'd sum up. Basically 
attempt to add up all bytes on the machine's disk drive.

Any ideas on what I'm doing wrong or suggestions on how to do this 
differently?

Thanks,

Bart



More information about the Python-list mailing list