[AstroPy] Changing table column from float64 to float32

[John K. Parejko]

For the record, float32 is only around 10mas precision for ra/dec degrees. For many modern datasets, that’s comparable with the nominal centroiding accuracy (e.g. LSST expects ~10mas repeatability for bright sources). 64bit floats are absolutely the correct data type for sky coordinates these days.

John

> On 12Jul 2022, at 03:11, Ivan Valtchanov <ivvv68 at gmail.com> wrote:
> 
> Thanks all,
> 
> Yes, I managed to do the change of float64 to float32 columns using astropy.table.Table. 
> 
> It's funny how people keep float64 columns in huge catalogues for something that's perfectly well stored with float32, e.g. RA, Dec, LII, BII, MJD, etc... It's true that RAM is cheap nowadays but this laziness puzzles me a lot (/rant_over).
> 
> Cheers,
> Ivan
> 
> 
> On Tue, 12 Jul 2022 at 10:21, Simon Conseil <simon at sconseil.fr> wrote:
> Indeed the BinTableHDU object doesn't have a method to easily replace a
> column (which is defined as float64 by the format='D' attribute in your
> example, so data assigned to the column will always be recast to
> float64).
> 
> One option would be to use table_hdu.columns.del_col and
> table_hdu.columns.add_col but this isn't the most practical. 
> 
> In [3]: a = table_hdu.data['a'].astype('float32')
> 
> In [4]: cols = table_hdu.columns.del_col('a')
> 
> In [6]: cols = table_hdu.columns.add_col(fits.Column(name='a', array=a,
> format='E'))
> 
> In [8]: table_hdu = fits.BinTableHDU.from_columns(cols)
> 
> In [9]: table_hdu.columns
> Out[9]: 
> ColDefs(
>     name = 'b'; format = 'E'
>     name = 'a'; format = 'E'
> )
> 
> The easiest here would be to use astropy.table.Table:
> 
> In [3]: t = Table.read(table_hdu)
> 
> In [4]: t['a'] = t['a'].astype('float32')
> 
> In [5]: table_hdu = fits.table_to_hdu(t)
> 
> In [6]: table_hdu.columns
> Out[6]: 
> ColDefs(
>     name = 'a'; format = 'E'
>     name = 'b'; format = 'E'
> )
> 
> 
> Simon
> 
> On Tue, 2022-07-12 at 00:13 +0000, Homeier, Derek wrote:
> > On 11 Jul 2022, at 11:14 pm, Michael Brewer <brewer at astro.umass.edu>
> > wrote:
> > > 
> > > Please try this:
> > > 
> > > table_hdu.replace_column('a',
> > > table_hdu.data['a'].astype(np.single))
> > > 
> > > Michael
> > > 
> > replace_column is a [astropy.table.]Table method; this won’t work on
> > a FITS BinTable –
> > I think the most direct way is indeed to recreate the HDU from the
> > appropriately cast
> > Columns as outlined by Peter. Note that the HDU also contains the
> > corresponding header
> > information characterising the data section formats, so direct
> > manipulation of the data
> > parts is limited for a reason.
> > > 
> > > On 7/11/22 12:17 PM, Peter Weilbacher wrote:
> > > > Hi Ivan,
> > > > 
> > > > I have no particular insight into this, but it seems to me that
> > > > since
> > > > you created the Column objects with a specific format, assigning
> > > > any
> > > > data to it will cause it to be casted to match that. (Otherwise
> > > > the
> > > > underlying memory buffer does not match.)
> > > > 
> > > > Instead, I guess you could create a new column with the desired
> > > > type and
> > > > format and recreate the table using that:
> > > > 
> > > > c1_single = fits.Column(name='a',
> > > > array=table_hdu.data['a'].astype(np.single), format='E')
> > > > table_hdu2 = fits.BinTableHDU.from_columns(c1_single +
> > > > table_hdu.columns[1:])
> > > > 
> > > > (It appears that you can even remove the .astype() from this and
> > > > it
> > > > still works.)
> > 
> > Yes, the `format=‘E’` specifier will already recast to the respective
> > FITS (= Fortran) format.
> > 
> > Cheers
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at python.org
> > https://mail.python.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy

--
*************************
John Parejko
parejkoj at uw.edu
Rubin Observatory, DIRAC Fellow
http://staff.washington.edu/parejkoj/
Department of Physics and Astronomy
University of Washington
Seattle, WA
**************************