[AstroPy] reading one line from many small fits files

[Erik Bray]

On 07/31/2012 12:13 PM, Derek Homeier wrote:
> indeed the question was about tables, I just still had the discussion about automatic
> scaling and the potential type conversion coming with it in mind. And the docs mention
>
> `FITS_rec` is the data part of a table HDU's data part.  This is a
> layer over the `recarray`, so we can deal with scaled columns.
>
> so I was wondering if some automatic scaling features might have been added
> (or be in testing) in 3.1 svn.
>
> Cheers,
> 						Derek

PyFITS has had scaling of table columns with TSCAL and TZERO for a long 
time--to my knowledge nothing has changed there.  (For what it's worth, 
the in-progress changelog for 3.1 can be viewed here: 
http://pyfits.readthedocs.org/en/latest/appendix/changelog.html#unreleased)

Are you saying there is a noticeable performance degradation from a 
previous version of PyFITS?  What version were you using before?

There was at one time a severe performance issue with tables containing 
string columns, but that should be fixed.  It's possible there is another?

Though I will say that row-based access in PyFITS has always been rather 
inefficient--it was originally designed for more efficient column 
access.  Though this is an artifact of using Numpy's recarray class in 
the background.  There are plans to eventually replace this with the 
Astropy package's Table implementation.

Erik