[AstroPy] Reading/writing tables

[Aldcroft, Thomas]

This should be fixed for astropy 4.3.  :-)  In
https://github.com/astropy/astropy/pull/11155, I added support for writing
out and reading back tables that have multidimensional columns such as the
`R[2]` here.

This uses the ECSV format to include the necessary meta-data to stitch the
table back together automatically, but one can always use any old CSV
reader and deal with the columns manually.

- Tom

On Tue, Dec 8, 2020 at 5:19 AM Derek Homeier <
derek at astro.physik.uni-goettingen.de> wrote:

> On 8 Dec 2020, at 9:08 am, yyang <yanbin.yang at obspm.fr> wrote:
> >
> > Could it be linked to the your duplicated column named “R” ?  see  a
> screen shot enclosed.
> > Cheers
> > Yanbin
> > <PastedGraphic-1.png>
> >
> >> On 8 Dec 2020, at 08:58, Judith Irwin <irwinja at queensu.ca> wrote:
> >>
> >> Could someone help with this problem? I want to read a fits table and
> write out a .csv file.  Here is what I'm doing:
> >>
> >> >>> import astropy
> >> >>> from astropy.table import Table
> >> >>> from astropy.io import ascii
> >> >>> t=Table.read('merged_0.3-7_newerr_src.fits')
> >> >>> print t
> >>       RA           DEC            RA_ERR           DEC_ERR
> X       ... 797_OFFAXS_ANG WEIGHT_OFFAXS      WANG_ERR         POS_ERR
> >>      deg           deg             deg               deg
>  pix      ...     arcmin         arcmin          arcmin           arcmin
> >> ------------- ------------- ----------------- -----------------
> ------------- ... -------------- -------------- ----------------
> ----------------
> >> 190.491776756 32.4809974381 7.81833679753e-06  4.8707159408e-06
> 4071.66367713 ...    2.958585048    2.958585048 0.00652461345853
> 0.00654797997354
> >> 190.490783106 32.4956819235 2.04948640032e-05 3.86556461152e-05
> 4077.8 ...  2.07617011778  2.07617011778 0.00490486335967 0.00556319682508
> >> (I'm only showing the first rows of the table)
> >> >>> ascii.write(t, 'Xsources_new.csv', format='csv',overwrite=True)
>
> It is not duplicated, but has itself a dtype or shape (2, ), i.e. the
> column represents a 2-dim array of shape (72, 2).
>
> >>> t['SHAPE’, 'R’, 'ROTANG’, 'PSFRATIO']
> <Table length=72>
>   SHAPE                R [2]                ROTANG
>                         pix                  deg
>   str10               float32              float32
> ---------- ------------------------------ —————
> ellipse             4.603549 .. 2.7749648  154.52849
> ellipse             2.7087543 .. 1.141337   98.67701
> ellipse             3.5766919 .. 2.232861  140.08878
>
> That is in fact the problem, as structured columns like this are not
> supported by the (default) fast parser,
> and in fact only incompletely supported in CSV format.
> A quick workaround therefore would be to disable the fast parser:
>
> >>> t.write('merged_0.3-7_newerr_src.csv', format='ascii.csv',
> fast_writer=False)
>
> But this will represent your column ‘R’ as a string, so if you want to
> read the table back in from csv
>
> >>> Table.read('merged_0.3-7_newerr_src.csv')['R']
> <Column name='R' dtype='str30' length=72>
>          4.603549 .. 2.7749648
>          2.7087543 .. 1.141337
>
> it will no longer be directly accessible as the original array.
>
> To preserve them in a usable format, I suggest to change the table to a
> “flattened” column format
> by replacing it with one column for each subcolumn, e.g.:
>
> >>> t.add_columns([t['R'][:,0], t['R'][:,1]], indexes=t.index_column('R’)
> + np.zeros(2), names=['R0', 'R1'])
> >>> t.remove_column('R')
> >>> t.write('merged_0.3-7_newer_src.csv', format=‘ascii.csv’,
> overwrite=True)
>
> I also recommend to upgrade your installation to a Python 3-based version,
> as Astropy 2.0.x is really old..;-)
>
> But this issue as such still persists in the current version and is a bug,
> since the writer should
> report a clearer error message (or automatically switch to the
> ‘fast_writer’=False version),
> thanks for reporting this!
>
> HTH
>                                                 Derek
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/astropy/attachments/20201208/19db3a0d/attachment.html>