Newbie array question

[Fernando Perez]

Neale wrote:

> My first task with Python is to scan multiple small text files and
> "batch together" those records with a "5" or "6" in column 2, and then
> save as a new file. The records are 256 characters. I know it sounds
> like a homework problem, but it's not.
> 
> My math brain wants to dump all the records into a big 2-D array and
> sort on the second column. Then "compress out" all the records whose
> second character isn't "5" or "6". Easy to say.
> 
> Is this the right strategy for using Python? Does it even have a
> "compress out" function?

There's obviously a million ways to do this, but I (math person also) would do
it with Numeric (http://www.pfdubois.com/numpy).  It will efficiently do the
filtering, and if you are interested in numerical work in python, you might as
well get started with the proper tool right away.

If you are not interested in numerical libraries for the future, then Numeric
may be overkill and a simple line-oriented loop will do just fine.  Note
however that the Numeric code will likely be much faster for large datasets
(assuming you don't start swapping) as all loops are done internally by fast C
code.

Cheers,

f