file data => array(s)

[Eric]

On Dec 14, 4:59 pm, Dave Angel <d... at davea.name> wrote:
> On 12/14/2011 05:20 PM, Eric wrote:
>
>
>
>
>
>
>
> > I'm trying to read some file data into a set of arrays.  The file data
> > is just four columns of numbers, like so:
>
> >     1.2    2.2   3.3  0.5
> >     0.1   0.2    1.0  10.1
> >     ... and so on
>
> > I'd like to read this into four arrays, one array for each column.
> > Alternatively, I guess something like this is okay too:
>
> >     [[1.2, 2.2, 3.3, 0.5], [0.1, 0.2, 1.0, 10.1], ... and so on]
>
> > I came up with the following for the four array option:
>
> >     file = open(fileName, 'r')
> >     for line in file.readlines():
>
> The readlines() call is a waste of time/space.  file is already an
> iterator that'll return lines for you.
>
>
>
>
>
>
>
> >        d1, e1, d2, e2 = map(float, line.split())
> >        data1.append(d1)  # where data1, err1, data2, err2 are init-ed
> > as empty lists
> >        err1.append(e1)
> >        data2.append(d2)
> >        err2.append(e2)
> >     file.close()
>
> > But somehow it doesn't seem very python-esque (I'm thinking there's a
> > more elegant and succinct way to do it in python).  I've also tried
> > replacing the above "map" line with:
>
> >        d = d + map(float, line.split())  # where d is initialized as d
> > = []
>
> > But all I get is one long flat list, not what I want.
>
> > So is the map and append method the best I can do or is there a
> > slicker way?
>
> > One more thing, no numpy.  Nothing against numpy but I'm curious to
> > see what can be done with just the box stock python install.
>
> > TIA,
> > eric
>
> When I see a problem like this, I turn to zip().  It's got some powerful
> uses when rows and columns need inverting.
>
> I didn't try it on an actual file, but the following works:
> linedata =    [[1.2, 2.2, 3.3, 0.5], [0.1, 0.2, 1.0, 10.1] ]
>
> data, err1, data2, err2 = zip(*linedata)
>
> print data
> print err1
> print data2
> print err2
>
> So you could try (untested)
>
> file = open(filename, "r")
> linedata = [ map(float, line) for line in file]
> data, err1, data2, err2 = zip(*linedata)
> file.close()
>
> DaveA

Neat.  This is what I had in mind for a python-esque solution.  Only
thing is "map(float,line)" should be "map(float,line.split()).  Looks
like it should be easy enough to weed out any funky data sets because
between map() and zip() it's fairly picky about the amount and type of
data.

Finally, the input files I'll be using for real aren't just four
columns of data.  The beginning of the file may have comments
(optional) and will have two lines of text to identify the data.
Maybe I can still do it w/o readlines.

Thanks,
eric