[AstroPy] pyfits: checksum/datasum not both required in fits files

[John K. Parejko]

Replying to my self on this with a further question:

What is the correct datasum/checksum for unsigned int data with BZERO=37268, BSCALE=1? Should it be calculated on the original bits, or on the scaled values?

I ask, because it appears that pyfits and IDL's fits_test_checksum[1] seem to disagree on the correct checksums for unsigned in data. Also, pyfits transparently removes BZERO/BSCALE when writing files if "uint=True" is specified on open(). 

Essentially, the interaction between uint, scale_back, do_not_scale_image_data, DATASUM/CHECKSUM, and BZERO/BSCALE is very unclear to me. Could someone provide some guidance on this?

John

1: http://idlastro.gsfc.nasa.gov/ftp/pro/fits/fits_test_checksum.pro

On 4Sep 2013, at 18:00, John K. Parejko wrote:

> Hello,
> 
> It looks like the fits checksum proposal does not require both DATASUM and CHECKSUM in the header (2nd paragraph of the PDF):
> 
> http://fits.gsfc.nasa.gov/registry/checksum.html
> 
> verify_checksun in pyfits 3.1.2 does not calculate a matching checksum if "CHECKSUM" is present, but "DATASUM" is not, because it assumes datasum is 0. If I pre-compute datasum and pass that to hdu._calculate_checksum(), I do get the matching checksum.
> 
> This problem came up because we have some data written with nom.tam.fits v1.06.0, which only writes CHECKSUM, not DATASUM. We've hacked around it when reading the data with pyfits, but it'd be nice to not need that hack. Unfortunately, the correct behavior seems unspecified in the proposal, in the case where CHECKSUM is present but DATASUM is not.
> 
> Should pyfits relax this requirement, and attempt to pre-compute the data checksum before calling _calculate_checksum(), instead of assuming 0?
> 
> John