[Numpy-discussion] Saving and loading a structured array from a TEXT file

[Derek Homeier]

On 23 Jan 2012, at 21:15, Emmanuel Mayssat wrote:

> Is there a way to save a structured array in a text file?
> My problem is not so much in the saving procedure, but rather in the
> 'reloading' procedure.
> See below
> 
> 
> In [3]: import numpy as np
> 
> In [4]: r = np.ones(3,dtype=[('name', '|S5'), ('foo', '<i8'), ('bar', '<f8')])
> 
> In [5]: r.tofile('toto.txt',sep='\n')
> 
> bash-4.2$ cat toto.txt
> ('1', 1, 1.0)
> ('1', 1, 1.0)
> ('1', 1, 1.0)
> 
> In [7]: r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype)
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> /home/cls1fs/clseng/10/<ipython-input-7-b07ba265ede7> in <module>()
> ----> 1 r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype)
> 
> ValueError: Unable to read character files of that array type

I think most of the np.fromfile functionality works for binary input; for reading text 
input np.loadtxt and np.genfromtxt are the (currently) recommended functions. 
It is bit tricky to read the format generated by tofile() in the above example, but 
the following should work:

cnv =  {0: lambda s: s.lstrip('('), -1: lambda s: s.rstrip(')')}
r2 = np.loadtxt('toto.txt', delimiter=',', converters=cnv, dtype=r.dtype)

Generally loadtxt works more smoothly together with savetxt, but the latter unfortunately 
does not offer an easy way to save structured arrays (note to self and others currently 
working on npyio: definitely room for improvement!).

HTH,
							Derek