[AstroPy] Question re. astropy.table

[Brian York]

On 2016/02/25 18:59 , "Aldcroft, Thomas" <aldcroft at head.cfa.harvard.edu>
wrote:

>Hi Brian,
>
>
>If your input file an ASCII data table then please see:
>   https://github.com/astropy/astropy/issues/3334
>
>
>
>There is a code snippet there that should work to read in a large table
>in chunks.

I've tried this, but I keep running into issues. The table is an ASCII
IPAC format, and running the above code (with format='ipac') always gets
me the following exception:

Traceback (most recent call last):
  File 
"/Users/york/ssbvirt/ssbdev-osx/lib/python2.7/site-packages/celery/app/trac
e.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/Users/york/JWST/pandeia/simulator.py", line 87, in __call__
    return TaskBase.__call__(self, *args, **kwargs)
  File 
"/Users/york/ssbvirt/ssbdev-osx/lib/python2.7/site-packages/celery/app/trac
e.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "/Users/york/JWST/pandeia/simulator.py", line 173, in observation
    observed_catalogues.extend(obs.addCatalogue(cat))
  File 
"/Users/york/JWST/pandeia/stips/stips/observation_module/observation_module
.py", line 184, in addCatalogue
    cats = self.instrument.addCatalogue(catalogue)
  File "/Users/york/JWST/pandeia/stips/stips/instruments/instrument.py",
line 218, in addCatalogue
    cats.append(detector.addCatalogue(cat))
  File "/Users/york/JWST/pandeia/stips/stips/astro_image/astro_image.py",
line 348, in addCatalogue
    for i, t in enumerate(read_table(cat)):
  File "/Users/york/JWST/pandeia/stips/stips/utilities/utilities.py", line
109, in read_table
    chunk = ascii.read(lines, format=format, guess=False)
  File 
"/usr/stsci/ssbdev/python/lib/python2.7/site-packages/astropy/io/ascii/ui.p
y", line 339, in read
    dat = reader.read(table)
  File 
"/usr/stsci/ssbdev/python/lib/python2.7/site-packages/astropy/io/ascii/core
.py", line 1045, in read
    self.header.get_cols(self.lines)
  File 
"/usr/stsci/ssbdev/python/lib/python2.7/site-packages/astropy/io/ascii/ipac
.py", line 180, in get_cols
    raise ValueError('At least one header line beginning and ending with '
ValueError: At least one header line beginning and ending with delimiter
required


I have no idea of what's causing this (for reference, the opening lines of
the table are below, and this table opens just fine if I do a
Table.read(format='ascii.ipac') (except that it takes up ~4GB of memory
for a typical stellar population):

\ Internal Format Catalogue
\ 
\ Parameters:
\ 
\type="internal"
\filter="F115W"
\ 
|               RA|               DEC|          FLUX| TYPE|   N|  Re|
Phi|Ratio|    ID| Notes|
|           double|            double|        double| char|char|char|char|
char|  long|  char|
|                 |                  |              |     |    |    |    |
    |      |      |
|             null|              null|          null| null|null|null|null|
null|  null|  null|
 8.35744567067e-09  6.07703574397e-09  1.77083404307 point  N/A  N/A  N/A
 N/A      1   None 
 1.82673672356e-10 -2.22541481358e-11  4.16120404729 point  N/A  N/A  N/A
 N/A      2   None 
 1.67512697711e-08 -1.68243630672e-08  2.70787939571 point  N/A  N/A  N/A
 N/A      3   None 


If anyone has any idea of how I'm using the code wrong, it would be much
appreciated.

	-Brian