[SciPy-user] read/write compressed files

[Dominik Szczerba]

I got it (partially) working, but am not sure about optimality. In
particular, will fromstring copy memory into the array or decompress in
place? I think the former (how else would it know the size, and tell()
will be slow), but please correct me if I am wrong.

import gzip
fh = gzip.GzipFile("test.dat.gz", 'rb');
#ps = zeros(256*256) - will it help?
ps = fromstring(fh.read(), 'd')
ps.shape = (256,256)
fh.close()
fp = open('test.dat', 'wb')
io.numpyio.fwrite(fp, ps.size, ps)
fp.close()

- Dominik

Dominik Szczerba wrote:
> That works very well for ascii files, but I failed to figure out about
> binary data...
> Thanks for any hints,
> - Dominik
> 
> Anne Archibald wrote:
>> On 20/06/07, Dominik Szczerba <domi at vision.ee.ethz.ch> wrote:
>>> Yes, I know it, but it does not return a scipy array, does it?
>>> Can I achieve it without copying memory? (I have huge arrays to process)
>> If the bz2 module will provide a file-like object, scipy.read_array
>> can read from that.
>>
>> Anne
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://projects.scipy.org/mailman/listinfo/scipy-user
> 

-- 
Dominik Szczerba, Ph.D.
Computer Vision Lab CH-8092 Zurich
http://www.vision.ee.ethz.ch/~domi