[AstroPy] Reading FITS uncompressed headers performance

[Jiří Nádvorník]

Aha, so maybe I misunderstood how "less" works with the files - it displays
the header correctly even for the .bz2 file. I thought it behaves so
because the header is not compressed, only data is.

I'm attaching the SDSS image FITS link
<https://dr12.sdss.org/sas/dr12/boss/photoObj/frames/301/1894/1/frame-u-001894-1-0103.fits.bz2>
to
the .bz2 file so it's clear what I'm talking about.

BR,

Jiri

čt 14. 10. 2021 v 12:17 odesílatel Evert Rol <evert.rol at gmail.com> napsal:

> Bzip doesn't care about the file format: it compresses the whole file (and
> needs the complete file for that information). It probably divides things
> up in compressed blocks, but is not aware of FITS headers and the like, and
> such blocks will not adhere to the file structure. Possibly, `pyfits.open`
> can read and uncompress the relevant blocks for the first (primary) header
> and not uncompress anything further, but I'm not aware of that
> functionality.
> So as far as I'm aware, you can't do what you want: you'll need to
> uncompress the whole file first, before you can read just the primary
> header.
>
> What you probably want, is to just compress the data section of the FITS
> HDUs. For that, you'll need to save the file with pyfits, with the
> compression algorithm specified for each HDU. See
> https://docs.astropy.org/en/stable/io/fits/usage/unfamiliar.html#astropy-io-fits-compressedimagedata .
> The compressed algorithms are described at
> https://docs.astropy.org/en/stable/io/fits/api/images.html#astropy.io.fits.CompImageHDU .
> Be aware that some truncation happens with floating point data, as also
> described in the latter link.
>
> The advantage of this is that the header is also readable with other
> tools: it is just plain text (as normal for FITS), and it's only the data
> that is compressed.
>
> There may be some external tools that can do this easily for you from the
> command line (perhaps even some of the astropy tools), instead of having to
> script this yourself, but I don't know such tools.
>
>
>
> On 14 Oct 2021, at 10:44, Jiří Nádvorník <nadvornik.ji at gmail.com> wrote:
>
> Hi all,
>
> I am a little confused about the performance of astropy.io.fits when
> trying to open just the primary header of a compressed (.bz2) FITS file.
> The header is uncompressed plain text, correctly ended by the END tag.
>
> Is there some setting that I need to use to parse the header without
> needing to decompress the whole file?
>
> Thank you,
>
> Jiri N.
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/astropy/attachments/20211014/c1f091d7/attachment-0001.html>