Numpy array to gzip file

[Sean Davis]

On Jun 11, 12:42 pm, "drobi... at gmail.com" <drobi... at gmail.com> wrote:
> On Jun 11, 9:17 am, Sean Davis <seand... at gmail.com> wrote:
>
>
>
> > I have a set of numpy arrays which I would like to save to a gzip
> > file.  Here is an example without gzip:
>
> > b=numpy.ones(1000000,dtype=numpy.uint8)
> > a=numpy.zeros(1000000,dtype=numpy.uint8)
> > fd = file('test.dat','wb')
> > a.tofile(fd)
> > b.tofile(fd)
> > fd.close()
>
> > This works fine.  However, this does not:
>
> > fd = gzip.open('test.dat','wb')
> > a.tofile(fd)
>
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> > IOError: first argument must be a string or open file
>
> > In the bigger picture, I want to be able to write multiple numpy
> > arrays with some metadata to a binary file for very fast reading, and
> > these arrays are pretty compressible (strings of small integers), so I
> > can probably benefit in speed and file size by gzipping.
>
> > Thanks,
> > Sean
>
> Use
>    fd.write(a)

That seems to work fine.  Just to add to the answer a bit, one can
then use:

b=numpy.frombuffer(fd.read(),dtype=numpy.uint8)

to get the array back as a numpy uint8 array.

Thanks for the help.

Sean