[SciPy-dev] Reading in arrays.

[eric jones]

> As eric pointed out reading arrays of ASCII data is actually quite easy
> (and fairly sophisticated).   It is available thanks to the support of
> Agilent who has given some money for the development of SciPy.

Yes, and we still need to acknowledge their support on the site. : |  It
will happen, I've only recently paid SciPy serious attention again.

>
> Reading ascii data can be as simple as
>
> a = io.read_array('test.dat')   # you can pass a string object already.
>
> to read in space-separated data as an array of doubles.
>
> Eric,  Why on earth would you reach for Excel now that you have it this
> easy?  (Except for maybe getting better graphics on windows --- if you
> have an X-server you can already do quite a bit).

I'm hoping we'll get to the point where I won't, but it takes takes maybe 4
button clicks and 10 seconds to do it.  I think you can make the argument
that they are about equivalent effort now for some things.  Its just
whatever your used to...  On the other hand the Excel import tool does
things visually so you can see how your settings are affecting the division
of data.  Also, it handles files of mixed text/numbers just fine.  For
instance, I take Python profiling outputs and put them in Excel (actually
cut/paste, but the interface is the same).  It is trivial then to use
Excel's nice sorting features to sort on the different columns.  You can do
this directly in Python also, it just isn't as quick for me.  I'm sure
others have equivalently nice ways of doing things, this is just my approach
and it is very much facilitated by Excel's grid layout and import tools.
Use the easiest tool for the tasks... and easiest is sometimes training
dependent.

Again, I'm not of the either/or opinion.  I want both.  I can forsee have a
more flexible visual (GUI) import tool that was Excel like in functionality.
This way we have the best of both worlds -- the feedback/flexibility
afforded by a GUI along with the current command line/scripting interface.

> I'm not sure I agree with Eric that this is "buried" way to deep.
>
> io.read_array does not seem very deep to me.

Ahhh.  I was looking at 0.1, not the CVS which doesn't automatically import
array_import.  The CVS version does have it promoted one level to living in
io.  I like that fine.

see ya,
eric

>
>
> There are many options which can make the reading in easier.
>
> >>> help(io.read_array)
>  read_array(fileobject, separator=None, columns=None, comment='#',
lines=None,
>             atype='d', linesep='\n', rowsize=10000, missing=0)
>
> Return an array represented ascii_formatted data in |fileobject|.
>
> Inputs:
>
>   fileobject -- An open file object or a string for a valid filename.
>   separator -- a string or a tuple of strings to indicate the column
>                separators.  If the length of the string tuple is less
>                than the total number of columns, then the last separator
>                is assumed to be the separator for the rest of the columns.
>   columns -- a tuple of integers and range-tuples which describe the
>              columns to read from the file.  A negative entry in the
>              last column specifies the negative skip value to the end.
>              Example:  columns=(1, 4, (5, 9), (11, 15, 3), 17, -2)
>                      will read [1,4,5,6,7,8,11,14,17,18,20,22,...]
>   lines   -- a tuple with the same structure as columns which indicates
>              the lines to read.
>   comment -- the comment character (line will be ignored even if it is
>              specified by the lines tuple)
>   linesep -- separator between rows.
>   missing -- value to insert in array when conversion to number fails.
>   atype -- the typecode of the output array.
>   rowsize -- the allocation row size (array grows by this amount as
>            data is read in).
>
>
> Adding wizards to make this more "interactive" may be useful especially
> for beginners.
>
> But, the functionality is there.
>
> -Travis
>
>
>
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-dev
>