[AstroPy] Error in vo on Mac OS 10.6.8, Python 2.7

Michael Droettboom mdroe at stsci.edu
Mon Jul 11 08:35:30 EDT 2011 

This is very helpful.  It's an invalid VOTable file.  kxml accepts it 
because it's just a generic XML editor and doesn't know about VOTable 
datatype/arraysize specifications. Looking at the TOPCAT source code, it 
appears that the "value" attribute isn't parsed until the value is 
actually requested.  So simply loading the file doesn't exercise the 
value parser for PARAM elements.  However, I suspect (this is untested) 
calling getObject() on that ParamElement would raise an exception.

That said, this could probably be loosened in the vo library.  If 
"pedantic" is False, we could just raise a warning (rather than an 
exception) and fill as many array elements as we can.  In this case, the 
arraysize would still be "0", and no values would be set, but at least 
it would continue to parse the rest of the file.

Cheers,
Mike

On 07/11/2011 05:25 AM, Jean-Baptiste Marquette wrote:
>
> Hi Mike,
>
> The declaration looks like:
>
> <PARAM name="Weight_Thresh" datatype="float" arraysize="0" 
> ucd="instr.sensitivity;obs.param" value="0"/>
>
> Let me know if you wish me to put a complete table on my ftp.
>
> Cheers
> Jean-Baptiste
>
>
>
>> Sorry I'm just getting to this message now...  (This strangely ended 
>> up being classified as spam...)
>>
>> I'd be curious to see what the arraysize declaration looks like.  
>> It's possible that TOPCAT and KXML aren't really parsing the PARAM 
>> elements, but just storing them to spit them out verbatim again, so 
>> they aren't really "deeply" parsing the value.
>>
>> Cheers,
>> Mike
>>
>> On 07/05/2011 03:47 AM, Jean-Baptiste Marquette wrote:
>>> Hi Micheal,
>>>
>>> I got the point: a wrong arraysize declaration of a PARAM in 
>>> SExtractor while creating the VOTables. I just informed Emmanuel 
>>> Bertin, and have to sed these 201,000 catalogues to correct them.
>>> But it is strange that both TOPCAT and KXML Editor load the tables 
>>> without complaining...
>>>
>>> Début du message réexpédié :
>>>
>>>> *De : *Jean-Baptiste Marquette <marquett at iap.fr 
>>>> <mailto:marquett at iap.fr>>
>>>> *Date : *4 juillet 2011 10:46:01 HAEC
>>>> *À : *Michael Droettboom <mdroe at stsci.edu <mailto:mdroe at stsci.edu>>
>>>> *Objet : **Réexp : [AstroPy] Error in vo on Mac OS 10.6.8, Python 2.7*
>>>>
>>>> Hi Michael,
>>>>
>>>> I investigated a bit further on the issue below. I installed KXML 
>>>> Editor on my Mac Pro, which reads my table seamlessly, as TOPCAT 
>>>> did. So I think this is an actual bug in vo. Should I open a ticket ?
>>>>
>>>> Cheers,
>>>> Jean-Baptiste
>>>>
>>>> Début du message réexpédié :
>>>>
>>>>> *De : *Jean-Baptiste Marquette <MARQUETT at IAP.FR 
>>>>> <mailto:MARQUETT at IAP.FR>>
>>>>> *Date : *30 juin 2011 19:46:10 HAEC
>>>>> *À : *astropy at scipy.org <mailto:astropy at scipy.org>
>>>>> *Objet : **[AstroPy] Error in vo on Mac OS 10.6.8, Python 2.7*
>>>>>
>>>>> Dear vo gurus,
>>>>>
>>>>> I use vo through ATpy on VOTables which are well read by TOPCAT. 
>>>>> Thus I didn't find any violation of VO specifications.
>>>>>
>>>>> The traceback is:
>>>>>
>>>>> bs300.date processing ...
>>>>> /Volumes/pepperland/erosdata/sexcatcalib/bs/bs300/bs3001/bs3001_6f2079.cat:3:0: 
>>>>> W20: No version number specified in file. Assuming 1.0
>>>>> Traceback (most recent call last):
>>>>>  File "/Users/marquett/workspace/EpochsCat/src/PutEpochs.py", line 
>>>>> 28, in <module>
>>>>>    Table = atpy.Table(Cat, type='vo', pedantic=False, tid=2)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/atpy/basetable.py", 
>>>>> line 167, in __init__
>>>>>    self.read(*args, **kwargs)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/atpy/basetable.py", 
>>>>> line 213, in read
>>>>>    atpy._readers[table_type](self, *args, **kwargs)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/atpy/votable.py", 
>>>>> line 86, in read
>>>>>    votable = parse(filename, pedantic=pedantic)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/table.py", 
>>>>> line 103, in parse
>>>>>    return tree.VOTableFile(config=config, pos=(1, 
>>>>> 1)).parse(iterator, config)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 2963, in parse
>>>>>    iterator, tag, data, config, pos)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 2897, in _add_resource
>>>>>    resource.parse(self, iterator, config)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 2732, in parse
>>>>>    tag_mapping.get(tag, self._add_unknown_tag)(iterator, tag, 
>>>>> data, config, pos)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 2710, in _add_resource
>>>>>    resource.parse(self._votable, iterator, config)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 2732, in parse
>>>>>    tag_mapping.get(tag, self._add_unknown_tag)(iterator, tag, 
>>>>> data, config, pos)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 2710, in _add_resource
>>>>>    resource.parse(self._votable, iterator, config)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 2732, in parse
>>>>>    tag_mapping.get(tag, self._add_unknown_tag)(iterator, tag, 
>>>>> data, config, pos)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 2698, in _add_param
>>>>>    param = Param(self._votable, config=config, pos=pos, **data)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 1232, in __init__
>>>>>    id=id, config=config, pos=pos)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 915, in __init__
>>>>>    self._setup(config)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 1251, in _setup
>>>>>    self.value = self._value
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", 
>>>>> line 1239, in _set_value
>>>>>    self._value = self.converter.parse(value, self._config, 
>>>>> self._pos)[0]
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/converters.py", 
>>>>> line 433, in parse
>>>>>    config, pos)
>>>>>  File 
>>>>> "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/voexceptions.py", 
>>>>> line 109, in vo_raise
>>>>>    raise exc(message)
>>>>> vo.voexceptions.VOTableSpecError: 
>>>>> /Volumes/pepperland/erosdata/sexcatcalib/bs/bs300/bs3001/bs3001_6f2079.cat:87041:3: 
>>>>> E02: Incorrect number of elements in array, expected 0, got 1
>>>>>
>>>>> As I said this table is well loaded by TOPCAT, and was created by 
>>>>> Emmanuel Bertin's SExtractor. It is 20 Mb large.
>>>>> 87041 (if it is a rank) is far beyond the index bound.
>>>>> If necessary, I can provide the table in my ftp, together with my 
>>>>> python script.
>>>>> I checked that the issue remains the same on other tables of the 
>>>>> same type.
>>>>>
>>>>> Thanks for your help,
>>>>>
>>>>> Jean-Baptiste Marquette
>>>>> _______________________________________________
>>>>> AstroPy mailing list
>>>>> AstroPy at scipy.org <mailto:AstroPy at scipy.org>
>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>>
>>>
>>> ====================================================================
>>> Bien cordialement / Very truly yours / Mit freundlichen Gruessen,
>>> Jean-Baptiste Marquette
>>> Institut d'Astrophysique de Paris
>>> CNRS - UMR 7095
>>> Université Pierre & Marie Curie
>>> 98bis Bd Arago
>>> 75014 Paris - France
>>> Tel +33 (0)1 4432 8196
>>> Fax +33 (0)1 4432 8001
>>> Web : http://www2.iap.fr/users/marquett
>>> ====================================================================
>>>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110711/a1fb427c/attachment.html>

More information about the AstroPy mailing list