From fhessma at uni-goettingen.de Tue Oct 10 06:28:18 2023 From: fhessma at uni-goettingen.de (Hessman, Frederic) Date: Tue, 10 Oct 2023 10:28:18 +0000 Subject: [AstroPy] dropped blanks in FITS string header cards References: Message-ID: I'm parsing IRAF multispec headers by hand (specutils has problems with my files ??) and came across this apparent bug: multispec stores lots and lots of dispersion info in a long string instead of putting it into a reasonable set of cards (trying to use less 8-track tape?). The string is so long that it has to be distributed over several cards. That means that a string containing a list of numbers is split into sub-strings and placed in adjacent cards. The spaces in the strings are important - they separate the string representations of numbers - but they appear at random positions in the sub-strings. While parsing a FITS file with a header containing multispec info, I couldn't extract all of the information because the concatenated dispersion info was missing an internal space used to separate two numbers. Here's an example of the effect: first I set the values of the cards as strings: >>> WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 -8.060336767' >>> WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 6375.15322418564 589.9' >>> WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 -0.0423755076218827 ' >>> WAT2_004 = '0.00126765958623805"' When I print them out, everything is OK, of course: >>> for key in ['WAT2_001','WAT2_002','WAT2_003','WAT2_004'] : ... val = globals()[key] ... print (f'{key} = \'{val}\'') WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 -8.060336767' WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 6375.15322418564 589.9' WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 -0.0423755076218827 ' WAT2_004 = '0.00126765958623805"' So far, so good. Note that multispec uses " to define an internal string within the concatenated information, which means that there's a single " in the middle of the 1st card and a 2nd one at the end of the 4th card. Note also that there are several numbers that have been split between cards. Now, put this info into a HDU and see what happens: >>> hdu = fits.PrimaryHDU(np.arange(100.0)) >>> hdr = hdu.header >>> for key in ['WAT2_001','WAT2_002','WAT2_003','WAT2_004'] : ... hdr[key] = globals()[key] ... print (f'{key} = \'{hdr[key]}\'') WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 -8.060336767' WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 6375.15322418564 589.9' WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 -0.0423755076218827' WAT2_004 = '0.00126765958623805"' i.e. the space at the end of the 3rd card is dropped, later creating the unparseable number "-0.04237550762188270.00126765958623805" in the concatenated string. This is a random error in practice, since it depends upon having a space appear in the original string at the problematic position. If one sticks in other content, it always happens whenever the string has a trailing blank, no matter what position or length. Looks like a bug. Using Python 3.9.11 under OSX 12.6.1 and astropy 5.3.3 Rick -------------- next part -------------- An HTML attachment was scrubbed... URL: From pweilbacher at aip.de Tue Oct 10 06:55:38 2023 From: pweilbacher at aip.de (Peter Weilbacher) Date: Tue, 10 Oct 2023 12:55:38 +0200 Subject: [AstroPy] dropped blanks in FITS string header cards In-Reply-To: References: Message-ID: Hi Rick, in the FITS Standard (v4, Sect. 4.2.1.1 Single-record string keywords) I read "Leading spaces are significant; trailing spaces are not." So I guess that AstroPy is correct here and that IRAF constructed invalid FITS headers. Can you maybe find out if you need an extra trailing space by looking for decimal dots before and after a "break"? ?? Peter. On 10.10.23 12:28, Hessman, Frederic wrote: > I'm parsing IRAF multispec headers by hand (specutils has problems > with my files ??) and came across this apparent bug: > > multispec stores lots and lots of dispersion info in a long string > instead of putting it into a reasonable set of cards (trying to use > less 8-track tape?). ?The string is so long that it has to be > distributed over several cards. ?That means that a string containing a > list of numbers is split into sub-strings and placed in adjacent > cards. ?The spaces in the strings are important - they separate the > string representations of numbers - but they appear at random > positions in the sub-strings. > > While parsing a FITS file with a header containing multispec info, I > couldn't extract all of the information because the concatenated > dispersion info was missing an internal space used to separate two > numbers. > > Here's an example of the effect: ?first I set the values of the cards > as strings: > > >>> WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 > -8.060336767' > >>> WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 > 6375.15322418564 589.9' > >>> WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 > -0.0423755076218827 ' > >>> WAT2_004 = '0.00126765958623805"' > > When I print them out, everything is OK, of course: > > >>> for key in ['WAT2_001','WAT2_002','WAT2_003','WAT2_004'] : > ... ? ??val = globals()[key] > ... ??print (f'{key} = \'{val}\'') > WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 > -8.060336767' > WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 > 6375.15322418564 589.9' > WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 > -0.0423755076218827 ' > WAT2_004 = '0.00126765958623805"' > > So far, so good. ?Note that multispec uses " to define an internal > string within the concatenated information, which means that there's a > single " in the middle of the 1st card and a 2nd one at the end of the > 4th card. ?Note also that there are several numbers that have been > split between cards. > > Now, put this info into a HDU and see what happens: > > >>> hdu = fits.PrimaryHDU(np.arange(100.0)) > >>> hdr = hdu.header > >>> forkey in['WAT2_001','WAT2_002','WAT2_003','WAT2_004'] : > ... ?hdr[key] = globals()[key] > ... print(f'{key} = \'{hdr[key]}\'') > WAT2_001 = 'wtype=multispec spec1 = "1 1 2 30. 3.4938694016091 1989 > -8.060336767' > WAT2_002 = '2905E-5 315.89 325.25 1. 0. 2 6 14.65 1972.02 > 6375.15322418564 589.9' > WAT2_003 = '46425975261 -0.582739656692844 1.02948926318171 > -0.0423755076218827' > WAT2_004 = '0.00126765958623805"' > > i.e. the space at the end of the 3rd card is dropped, later creating > the unparseable number "-0.04237550762188270.00126765958623805" in the > concatenated string. ?This is a random error in practice, since it > depends upon having a space appear in the original string at the > problematic position. > > If one sticks in other content, it always happens whenever the string > has a trailing blank, no matter what position or length. > > Looks like a bug. ?Using Python 3.9.11 under OSX 12.6.1 and astropy 5.3.3 > > Rick > > > _______________________________________________ > AstroPy mailing list > AstroPy at python.org > https://mail.python.org/mailman/listinfo/astropy -- Dr. Peter M. Weilbacherhttps://www.aip.de/Members/pweilbacher Phone +49 331 74 99-667 encryption key ID 7D6B4AA0 ------------------------------------------------------------------------ Leibniz-Institut f?r Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam Vorstand: Prof. Dr. Matthias Steinmetz, Wolfram Rosenbach Stiftung b?rgerlichen Rechts, Stiftungsverz. Brandenburg: 26 742-00/7026 -------------- next part -------------- An HTML attachment was scrubbed... URL: