memory error

[fishboy]

On Wed, 02 Jun 2004 09:11:13 -0400, Bart Nessux
<bart_nessux at hotmail.com> wrote:

>def windows():
>    import os
>    excludes = ['hiberfil.sys', 'ipnathlp.dll', 'helpctr.exe', 'etc', 
>'etc', 'etc']
>    size_list = []
>    for root, dirs, files in os.walk('/'):
>       total = [x for x in files if x not in excludes]
>       for t in total:
>          s = file(os.path.join(root,t))
>          size = s.read()
>          size_list.append(size)
>          s.close()
>
>windows()

Yeah, what the other guys said about os.stat and os.path.getsize.
Also, if you really want to read the actual file into memory, just get
small chunks and add those up.

Like (untested):

numberofbytes = 0
CHUNKSIZE = 4096
for root,dirs, files in os.walk('/'):
    for name in files:
        if name not in excludes:
	f = file(os.path.join(root,name))
	while 1:
	    s = f.read(CHUNKSIZE)
	    if not s:
	        f.close()
	        break
	    numberofbytes += len(s)

this way you never have more than 4k of data in memory at once.
(well it might be 8k, I dont know enought about the internals to tell
you when the previous 's' is garbage collected.)

><{{{*>