[Numpy-discussion] Reading from binary file with memmap, with offset
Julian Taylor
jtaylor.debian at googlemail.com
Thu Oct 10 08:57:39 EDT 2013
you have to use the offset keyword argument of np.memmap, else it will
always start from the beginning of the file
>>> np.memmap(fd, dtype="float32", mode="r", offset=offset)
On Thu, Oct 10, 2013 at 2:43 PM, Andreas Hilboll <lists at hilboll.de> wrote:
> Hi,
>
> I have a problem using memmap correctly. I need to read a data file
> which consists of an ASCII header and appended binary single precision
> floating point values. memmap complains that the "Size of available data
> is not a multiple of the data-type size." But as far as I can tell, the
> size *doas* match the data-type size.
>
> The problem is illustrated by the following code:
>
> ---8<-------
>
> url = "http://www.iup.uni-bremen.de/~hilboll/download/20120204.XD4_N2"
> localfile = "np_memmap.dat"
>
> import os
> import urllib
>
> import numpy as np
>
>
> # download data file
> if not os.access(localfile, os.R_OK):
> urllib.urlretrieve(url, localfile)
>
> with open(localfile, "r") as fd:
> # read offset from first line of file
> offset = int(fd.readlines()[0].split()[-2])
> # jump to begin of data block
> fd.seek(offset)
> # read until EOF
> blob = fd.read()
> print("Size of data blob [bytes]: {}".format(len(blob)))
> print("This can actually be divided by 4: {} / 4.0 = {}".format(
> len(blob), len(blob) / 4.0))
> # go back to begin of data block
> fd.seek(offset)
> print("But it cannot be loaded as np.memmap with dtype float32:")
> np.memmap(fd, dtype="float32", mode="r")
>
> ---8<-------
>
> Any help is greatly appreciated :)
>
> -- Andreas.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20131010/581b333d/attachment.html>
More information about the NumPy-Discussion
mailing list