[AstroPy] A replacement for the FITS format.

[Christoph Deil]

On Jul 22, 2013, at 4:25 PM, Erik Bray <embray at stsci.edu> wrote:

> On 07/19/2013 02:21 PM, Demitri Muna wrote:
>> Hello,
>> 
>> See introduction in the email "The trouble with FITS". The Google document I created for this is here:
>> 
>> http://bit.ly/1dKcKih
>> 
>> Cheers,
>> Demitri
> 
> I think that the last bullet point could use clarification.  The way I might 
> word this is that it should be possible to order table data in row-order or 
> column-order depending on how it is most likely to be accessed.  This is of 
> course a determination that would have to be made by the user, and sometimes 
> there is no right answer (but where there is a right answer it should be 
> possible to do).
> 

For reference, the last bullet point is:
• FITS tables and compressed image are stored inefficiently.  To read a column of data (e.g. for plotting purposes) requires a pointer jumping from byte to byte once per entry.  Columns need to be stored in a cohesive chunk. [4]

In case you hadn't seen this … there is "column-oriented FITS" to address this issue:
http://www.star.bris.ac.uk/~mbt/topcat/sun253/outColfits.html

I haven't done any benchmarking if this hack results in fast column-oriented I/O for tools other than STILS / TOPCAT, though.

Christoph