Using Gzip with as little as possible memory usage

[Johann Hibschman]

Tw  writes:

> Hi,
> I need to compress a file using the gzip module. Could I do this
> completly without reading each chunk of the file and writing, etc. Is
> there a compress(in, out)-kinda thing?

Hrm.  I guess "os.system('gzip file')" is too simple?

> PS! My reason for asking this is that when I try to use it the old
> fashion way, like :

> while 1:
> 	buff = in.read(1024)
> 	if not buff:break
> 	out.write(buff)
> 	# etc

> it crashes. the code is called inside a larger object, and it the
> error comes from inside the gzip module. Testing the same thing
> outside my object works fine. No errors. Inside, no way. I hoped there
> was some way of bypassing the whole thing, calling the library
> directly with an input and an output file. 

You know, python *is* deterministic, rather unlike Monty Python.  What
you described just shouldn't be.  Are you sure you're doing the *same*
thing inside the large object?

-- 
Johann Hibschman                           johann at physics.berkeley.edu