[AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6

[Wolfgang Kerzendorf]

Hey guys,

I would have it the other way round: Always raises an exception - except when specifically asked not to, then it will just warn.
I think the algorithms in fitsverify are there to ensure the integrity of the data. If it doesn't comply - the integrity of the data can not be guaranteed (in this case, the integrity is fine, it's just a bad implementation of the fits standard on the telescopes end afaict). Other data formats do similar things: zip files do not decompress if the CRC doesn't match. 

My 2 cents,
   Wolfgang


On 2012-04-03, at 11:42 AM, Thomas Robitaille wrote:

> Would it be worth having a pedantic=True/False flag similar to that in
> Mike's VO module, that would default to False (and would produce
> warnings as has been mentioned), but if turned to True would raise
> Exception if the FITS file does not conform to the standard?
> 
> Cheers,
> Tom
> 
> On 3 April 2012 16:09, Erik Bray <embray at stsci.edu> wrote:
>> On 04/03/2012 09:53 AM, Phil Hodge wrote:
>>> On 04/02/2012 07:48 PM, Eddy Barratt wrote:
>>>> I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.
>>>> 
>>>> Here is the error message:
>>>> 
>>>> ...
>>> 
>>> Here is the output from fverify for your file:
>>> 
>>>                         FVERIFY V4.0.0 (CFITSIO V2.470)
>>>                         -------------------------------
>>> 
>>> HEASARC conventions are being checked.
>>> 
>>> File: 00000075.fits
>>> 
>>> 1 Header-Data Units in this file.
>>> 
>>> =================== HDU 1: Primary Array ===================
>>> 
>>> *** Error:   Byte #1 in Card#33 is a null(\0).
>>> *** Error:   Keyword #6, BSCALE: lower-case exponent d or e is illegal
>>> in value
>>>                +1.000000000000e+000.
>>> *** Error:   Keyword #7, BZERO: lower-case exponent d or e is illegal in
>>> value
>>>                +3.276800000000e+004.
>>> *** Error:   Keyword #9, ORIGIN: Value and Comment not separated by a "/".
>>> *** Error:   Keyword #11, FOCALLEN: lower-case exponent d or e is illegal in
>>>                value +1.550000000000e+002.
>>> *** Error:   Keyword #12, APERTURE: lower-case exponent d or e is illegal in
>>>                value +0.000000000000e+000.
>>> *** Error:   Keyword #22, TEMPERAT: lower-case exponent d or e is illegal in
>>>                value -2.041762134545e+001.
>>> *** Error:   Keyword #24, E-GAIN: lower-case exponent d or e is illegal in
>>>                value +1.430000000000e+000.
>>> *** Error:   Keyword #25, XPIXSZ: lower-case exponent d or e is illegal in
>>>                value +6.800000000000e-003.
>>> *** Error:   Keyword #26, YPIXSZ: lower-case exponent d or e is illegal in
>>>                value +6.800000000000e-003.
>>> *** Error:   Keyword #29, EXPOSURE: lower-case exponent d or e is illegal in
>>>                value +3.000000000000e+002.
>>> *** Error:   The header fill area is not totally filled with blanks.
>>> 
>>>    32 header keywords
>>> 
>>>    16-bit integer pixels,  2 axes (2184 x 1472),
>>> 
>>> ++++++++++++++++++++++ Error Summary  ++++++++++++++++++++++
>>> 
>>>    HDU#  Name (version)       Type             Warnings  Errors
>>>    1                          Primary Array    0         12
>>> 
>>> **** Verification found 0 warning(s) and 12 error(s). ****
>> 
>> Specifically, the problem in this file that PyFITS is breaking on is
>> that the header block is not filled with spaces.  According to the FITS
>> standard any extra bytes at the end of a header block must be spaces,
>> and not nulls as is the case in this file.
>> 
>> I'm not sure what PyFITS should do here.  Should it treat a file like
>> this as an error, or should it just produce a warning?  Given that other
>> tools (including earlier PyFITS versions) seem to be accepting of this,
>> I'm leaning toward the latter.  There's no reason PyFITS can't ignore
>> the standard here and read the file anyways.  That said, it's still
>> malformatted...
>> 
>> Erik
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy