From eddybarratt1 at yahoo.co.uk  Mon Apr  2 20:09:44 2012
From: eddybarratt1 at yahoo.co.uk (Eddy Barratt)
Date: Tue, 3 Apr 2012 01:09:44 +0100 (BST)
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
Message-ID: <1333411784.92293.YahooMailNeo@web29503.mail.ird.yahoo.com>

I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.


Here is the error message:

>>> pyfits.open('00000064.DARK.fits')
Unexpected extra padding at the end of the file. ?This padding may not be preserved when saving changes.
Error validating header for HDU #0 (note: PyFITS uses zero-based indexing).
? ? Header size is not multiple of 2880: 2560
There may be extra bytes after the last HDU or the file is corrupted.
Traceback (most recent call last):
? File "<stdin>", line 1, in <module>
? File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/util.py", line 206, in _with_extensions_wrapper
? ? return func(*args, **kwargs)
? File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/hdu/hdulist.py", line 103, in fitsopen
? ? return HDUList.fromfile(name, mode, memmap, **kwargs)
? File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/hdu/hdulist.py", line 275, in fromfile
? ? raise IOError('Empty or corrupt FITS file')
IOError: Empty or corrupt FITS file

I'm new to PyFits, I'd appreciate any?guidance?at all on this issue. I tried to attach the file to this e-mail but it seems that's not allowed, if anyone thinks they may be able to assist please let me know and I can e-mail the file.

Thanks. Eddy 


From aldcroft at head.cfa.harvard.edu  Tue Apr  3 08:59:48 2012
From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft)
Date: Tue, 3 Apr 2012 08:59:48 -0400
Subject: [AstroPy] Python installation and ecosystem tutorial update
Message-ID: <CAGsQ-Nu7MYDmbtFSFjjH6_7c0ieMsketh_fd7nYciRYBn9yzNQ@mail.gmail.com>

In preparation for a 2 hour workshop at CfA today I've updated the
Python4Astronomers tutorial section on installing scientific Python
and understanding the Python package ecosystem:

  http://python4astronomers.github.com/installation/installation.html

As always I'm interested in different perspectives and feedback.  I
hope someday there will be one clearly superior solution for
installation, but we're not there yet.

You can see recent commits here:

  https://github.com/python4astronomers/python4astronomers/commits/master

Thanks,
Tom


From eddybarratt1 at yahoo.co.uk  Mon Apr  2 19:48:30 2012
From: eddybarratt1 at yahoo.co.uk (Eddy Barratt)
Date: Tue, 3 Apr 2012 00:48:30 +0100 (BST)
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
Message-ID: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>

I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.

Here is the error message:

>>> pyfits.open('00000064.DARK.fits')
Unexpected extra padding at the end of the file. ?This padding may not be preserved when saving changes.
Error validating header for HDU #0 (note: PyFITS uses zero-based indexing).
? ? Header size is not multiple of 2880: 2560
There may be extra bytes after the last HDU or the file is corrupted.
Traceback (most recent call last):
? File "<stdin>", line 1, in <module>
? File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/util.py", line 206, in _with_extensions_wrapper
? ? return func(*args, **kwargs)
? File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/hdu/hdulist.py", line 103, in fitsopen
? ? return HDUList.fromfile(name, mode, memmap, **kwargs)
? File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/hdu/hdulist.py", line 275, in fromfile
? ? raise IOError('Empty or corrupt FITS file')
IOError: Empty or corrupt FITS file

I'm new to PyFits, I'd appreciate any?guidance?at all on this issue.

Thanks. Eddy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 00000075.fits
Type: application/octet-stream
Size: 6433920 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20120403/4844b156/attachment.obj>

From mrdavis at stsci.edu  Tue Apr  3 09:36:50 2012
From: mrdavis at stsci.edu (Matt Davis)
Date: Tue, 3 Apr 2012 13:36:50 +0000
Subject: [AstroPy] SciPy 2012
In-Reply-To: <4F735E42.3040601@stsci.edu>
References: <4F735A23.9040402@gemini.edu> <4F735E42.3040601@stsci.edu>
Message-ID: <7C5DDE37-954A-496B-9C15-A093FDF4D72B@stsci.edu>

I'm with Erik, waiting to see what the tutorials are. A Pandas tutorial might be nice. But if we could do a significant AstroPy sprint that would definitely be an enticement.

- Matt

On Mar 28, 2012, at 2:53 PM, Erik Bray wrote:

> On 03/28/2012 02:36 PM, James Turner wrote:
>> Hi everyone,
>> 
>> I'm just wondering who is thinking of attending SciPy in Austin this
>> July 16-21:
>> 
>>    http://conference.scipy.org/scipy2012/
>> 
>> I've been asked by the organizers to help out, but am just figuring out
>> what my plans are, since it clashes with a couple of things. If there's
>> enough interest, I would propose an AstroPy sprint during the last
>> couple of days, where people (including me) can get started contributing
>> and focus on one or two areas of missing functionality, compared with
>> eg. IRAF or IDL.
>> 
>> I gather the next AstroPy co-ordination meeting is likely to happen
>> separately elsewhere, due to people's limited availability during the
>> conference season, but SciPy is a good venue for wider exposure and for
>> participants to learn about other Python tools (eg. NumPy, Matplotlib,
>> GPU libraries, symbolic mathematics, you name it...).
>> 
>> Cheers,
>> 
>> James.
> 
> I'm probably going to hold off until I see what tutorials get 
> accepted--the tutorials were kind of the highlight of SciPy for me last 
> year.  I don't recall getting too much out of the talks.  Not that there 
> was anything wrong with them, or that there weren't some very 
> interesting ones. Just that there were only a handful I felt that I took 
> much away from that's of immediate use to me.  But that's just me.
> 
> That said, I don't mean to sound down on SciPy. And an Astropy sprint 
> would be a great incentive.  I can't believe it's already been nearly a 
> year since last time...
> 
> Erik
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From hodge at stsci.edu  Tue Apr  3 09:53:54 2012
From: hodge at stsci.edu (Phil Hodge)
Date: Tue, 3 Apr 2012 09:53:54 -0400
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
Message-ID: <4F7B00F2.2060607@stsci.edu>

On 04/02/2012 07:48 PM, Eddy Barratt wrote:
> I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.
>
> Here is the error message:
>
> ...

Here is the output from fverify for your file:

                       FVERIFY V4.0.0 (CFITSIO V2.470)
                       -------------------------------

HEASARC conventions are being checked.

File: 00000075.fits

1 Header-Data Units in this file.

=================== HDU 1: Primary Array ===================

*** Error:   Byte #1 in Card#33 is a null(\0).
*** Error:   Keyword #6, BSCALE: lower-case exponent d or e is illegal 
in value
              +1.000000000000e+000.
*** Error:   Keyword #7, BZERO: lower-case exponent d or e is illegal in 
value
              +3.276800000000e+004.
*** Error:   Keyword #9, ORIGIN: Value and Comment not separated by a "/".
*** Error:   Keyword #11, FOCALLEN: lower-case exponent d or e is illegal in
              value +1.550000000000e+002.
*** Error:   Keyword #12, APERTURE: lower-case exponent d or e is illegal in
              value +0.000000000000e+000.
*** Error:   Keyword #22, TEMPERAT: lower-case exponent d or e is illegal in
              value -2.041762134545e+001.
*** Error:   Keyword #24, E-GAIN: lower-case exponent d or e is illegal in
              value +1.430000000000e+000.
*** Error:   Keyword #25, XPIXSZ: lower-case exponent d or e is illegal in
              value +6.800000000000e-003.
*** Error:   Keyword #26, YPIXSZ: lower-case exponent d or e is illegal in
              value +6.800000000000e-003.
*** Error:   Keyword #29, EXPOSURE: lower-case exponent d or e is illegal in
              value +3.000000000000e+002.
*** Error:   The header fill area is not totally filled with blanks.

  32 header keywords

  16-bit integer pixels,  2 axes (2184 x 1472),

++++++++++++++++++++++ Error Summary  ++++++++++++++++++++++

  HDU#  Name (version)       Type             Warnings  Errors
  1                          Primary Array    0         12

**** Verification found 0 warning(s) and 12 error(s). ****


From embray at stsci.edu  Tue Apr  3 10:09:23 2012
From: embray at stsci.edu (Erik Bray)
Date: Tue, 3 Apr 2012 10:09:23 -0400
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <4F7B00F2.2060607@stsci.edu>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu>
Message-ID: <4F7B0493.1070104@stsci.edu>

On 04/03/2012 09:53 AM, Phil Hodge wrote:
> On 04/02/2012 07:48 PM, Eddy Barratt wrote:
>> I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.
>>
>> Here is the error message:
>>
>> ...
>
> Here is the output from fverify for your file:
>
>                         FVERIFY V4.0.0 (CFITSIO V2.470)
>                         -------------------------------
>
> HEASARC conventions are being checked.
>
> File: 00000075.fits
>
> 1 Header-Data Units in this file.
>
> =================== HDU 1: Primary Array ===================
>
> *** Error:   Byte #1 in Card#33 is a null(\0).
> *** Error:   Keyword #6, BSCALE: lower-case exponent d or e is illegal
> in value
>                +1.000000000000e+000.
> *** Error:   Keyword #7, BZERO: lower-case exponent d or e is illegal in
> value
>                +3.276800000000e+004.
> *** Error:   Keyword #9, ORIGIN: Value and Comment not separated by a "/".
> *** Error:   Keyword #11, FOCALLEN: lower-case exponent d or e is illegal in
>                value +1.550000000000e+002.
> *** Error:   Keyword #12, APERTURE: lower-case exponent d or e is illegal in
>                value +0.000000000000e+000.
> *** Error:   Keyword #22, TEMPERAT: lower-case exponent d or e is illegal in
>                value -2.041762134545e+001.
> *** Error:   Keyword #24, E-GAIN: lower-case exponent d or e is illegal in
>                value +1.430000000000e+000.
> *** Error:   Keyword #25, XPIXSZ: lower-case exponent d or e is illegal in
>                value +6.800000000000e-003.
> *** Error:   Keyword #26, YPIXSZ: lower-case exponent d or e is illegal in
>                value +6.800000000000e-003.
> *** Error:   Keyword #29, EXPOSURE: lower-case exponent d or e is illegal in
>                value +3.000000000000e+002.
> *** Error:   The header fill area is not totally filled with blanks.
>
>    32 header keywords
>
>    16-bit integer pixels,  2 axes (2184 x 1472),
>
> ++++++++++++++++++++++ Error Summary  ++++++++++++++++++++++
>
>    HDU#  Name (version)       Type             Warnings  Errors
>    1                          Primary Array    0         12
>
> **** Verification found 0 warning(s) and 12 error(s). ****

Specifically, the problem in this file that PyFITS is breaking on is 
that the header block is not filled with spaces.  According to the FITS 
standard any extra bytes at the end of a header block must be spaces, 
and not nulls as is the case in this file.

I'm not sure what PyFITS should do here.  Should it treat a file like 
this as an error, or should it just produce a warning?  Given that other 
tools (including earlier PyFITS versions) seem to be accepting of this, 
I'm leaning toward the latter.  There's no reason PyFITS can't ignore 
the standard here and read the file anyways.  That said, it's still 
malformatted...

Erik


From perry at stsci.edu  Tue Apr  3 10:18:14 2012
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 3 Apr 2012 10:18:14 -0400
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <4F7B0493.1070104@stsci.edu>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
Message-ID: <B53036EE-41C3-41C8-8E8D-70CFB1F1E589@stsci.edu>

I generally think that if the interpretation is unambiguous (and for  
nulls, presumably spaces are intended, or at least, cause no change),  
that doing an implicit correction and giving a warning is the right  
thing to do.

Perry

On Apr 3, 2012, at 10:09 AM, Erik Bray wrote:

> On 04/03/2012 09:53 AM, Phil Hodge wrote:
>> On 04/02/2012 07:48 PM, Eddy Barratt wrote:
>>> I have a number of fits files that I cannot open in PyFits 3.0.6,  
>>> though a colleague using 2.4 can open them. PyFits does work on  
>>> other fits files on my computer. I would have assumed that the  
>>> problem is with the files, but they do open using the older  
>>> version and my colleagues using IDL and matlab have managed to  
>>> open them too, so perhaps the issue is with PyFits instead. I've  
>>> attached one of the files, they all have the same error message.
>>>
>>> Here is the error message:
>>>
>>> ...
>>
>> Here is the output from fverify for your file:
>>
>>                        FVERIFY V4.0.0 (CFITSIO V2.470)
>>                        -------------------------------
>>
>> HEASARC conventions are being checked.
>>
>> File: 00000075.fits
>>
>> 1 Header-Data Units in this file.
>>
>> =================== HDU 1: Primary Array ===================
>>
>> *** Error:   Byte #1 in Card#33 is a null(\0).
>> *** Error:   Keyword #6, BSCALE: lower-case exponent d or e is  
>> illegal
>> in value
>>               +1.000000000000e+000.
>> *** Error:   Keyword #7, BZERO: lower-case exponent d or e is  
>> illegal in
>> value
>>               +3.276800000000e+004.
>> *** Error:   Keyword #9, ORIGIN: Value and Comment not separated by  
>> a "/".
>> *** Error:   Keyword #11, FOCALLEN: lower-case exponent d or e is  
>> illegal in
>>               value +1.550000000000e+002.
>> *** Error:   Keyword #12, APERTURE: lower-case exponent d or e is  
>> illegal in
>>               value +0.000000000000e+000.
>> *** Error:   Keyword #22, TEMPERAT: lower-case exponent d or e is  
>> illegal in
>>               value -2.041762134545e+001.
>> *** Error:   Keyword #24, E-GAIN: lower-case exponent d or e is  
>> illegal in
>>               value +1.430000000000e+000.
>> *** Error:   Keyword #25, XPIXSZ: lower-case exponent d or e is  
>> illegal in
>>               value +6.800000000000e-003.
>> *** Error:   Keyword #26, YPIXSZ: lower-case exponent d or e is  
>> illegal in
>>               value +6.800000000000e-003.
>> *** Error:   Keyword #29, EXPOSURE: lower-case exponent d or e is  
>> illegal in
>>               value +3.000000000000e+002.
>> *** Error:   The header fill area is not totally filled with blanks.
>>
>>   32 header keywords
>>
>>   16-bit integer pixels,  2 axes (2184 x 1472),
>>
>> ++++++++++++++++++++++ Error Summary  ++++++++++++++++++++++
>>
>>   HDU#  Name (version)       Type             Warnings  Errors
>>   1                          Primary Array    0         12
>>
>> **** Verification found 0 warning(s) and 12 error(s). ****
>
> Specifically, the problem in this file that PyFITS is breaking on is
> that the header block is not filled with spaces.  According to the  
> FITS
> standard any extra bytes at the end of a header block must be spaces,
> and not nulls as is the case in this file.
>
> I'm not sure what PyFITS should do here.  Should it treat a file like
> this as an error, or should it just produce a warning?  Given that  
> other
> tools (including earlier PyFITS versions) seem to be accepting of  
> this,
> I'm leaning toward the latter.  There's no reason PyFITS can't ignore
> the standard here and read the file anyways.  That said, it's still
> malformatted...
>
> Erik
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From hodge at stsci.edu  Tue Apr  3 10:26:26 2012
From: hodge at stsci.edu (Phil Hodge)
Date: Tue, 3 Apr 2012 10:26:26 -0400
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <4F7B0493.1070104@stsci.edu>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
Message-ID: <4F7B0892.5030203@stsci.edu>

On 04/03/2012 10:09 AM, Erik Bray wrote:
> Specifically, the problem in this file that PyFITS is breaking on is
> that the header block is not filled with spaces.  According to the FITS
> standard any extra bytes at the end of a header block must be spaces,
> and not nulls as is the case in this file.
>
> I'm not sure what PyFITS should do here.  Should it treat a file like
> this as an error, or should it just produce a warning?  Given that other
> tools (including earlier PyFITS versions) seem to be accepting of this,
> I'm leaning toward the latter.  There's no reason PyFITS can't ignore
> the standard here and read the file anyways.  That said, it's still
> malformatted...

I'm strongly in favor of allowing the user to open the file unless it's 
totally incomprehensible.  If you can't open it you can't fix it.  In 
order to open a really messed-up file, it would be reasonable to require 
setting a flag, of course.

I could read this file using catfits (in iraf/stsdas), and that let me 
see why fverify complained about the ORIGIN keyword, which was useful:

ORIGIN  = 'Miner's View Observatory'

There's a quote within the quoted string, so the value was interpreted 
as "Miner".

Phil


From taro at ap.smu.ca  Tue Apr  3 11:28:37 2012
From: taro at ap.smu.ca (Taro Sato)
Date: Tue, 3 Apr 2012 12:28:37 -0300
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <B53036EE-41C3-41C8-8E8D-70CFB1F1E589@stsci.edu>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
	<B53036EE-41C3-41C8-8E8D-70CFB1F1E589@stsci.edu>
Message-ID: <CA+ABecwNrPX4MQ_UR7Denc3fGedrfDFwwK-g6eVtT5aacoNinA@mail.gmail.com>

On Tue, Apr 3, 2012 at 11:18 AM, Perry Greenfield <perry at stsci.edu> wrote:
> I generally think that if the interpretation is unambiguous (and for
> nulls, presumably spaces are intended, or at least, cause no change),
> that doing an implicit correction and giving a warning is the right
> thing to do.
>
> Perry


I like this approach as well.  Warning can be turned off if a user
wishes and when necessary.  There are malformed files that are useful,
and preventing their use just because they don't stick to the right
principles would not win PyFITS more users.



-- 
Taro Sato
Department of Astronomy & Physics
Saint Mary's University? ? ? ? ? ?? email: taro at ap.smu.ca
Halifax, NS B3H 3C3? ? ? ? ? ? ? ?? phone: (902)420-5018
Canada? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? web: http://ap.smu.ca/~taro


From thomas.robitaille at gmail.com  Tue Apr  3 11:42:17 2012
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 3 Apr 2012 17:42:17 +0200
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <4F7B0493.1070104@stsci.edu>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
Message-ID: <CAGMHX_3LxT5FZOQvgabmW+mXSxNs01WjUNuH_XCD=FRx=-VHZg@mail.gmail.com>

Would it be worth having a pedantic=True/False flag similar to that in
Mike's VO module, that would default to False (and would produce
warnings as has been mentioned), but if turned to True would raise
Exception if the FITS file does not conform to the standard?

Cheers,
Tom

On 3 April 2012 16:09, Erik Bray <embray at stsci.edu> wrote:
> On 04/03/2012 09:53 AM, Phil Hodge wrote:
>> On 04/02/2012 07:48 PM, Eddy Barratt wrote:
>>> I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.
>>>
>>> Here is the error message:
>>>
>>> ...
>>
>> Here is the output from fverify for your file:
>>
>> ? ? ? ? ? ? ? ? ? ? ? ? FVERIFY V4.0.0 (CFITSIO V2.470)
>> ? ? ? ? ? ? ? ? ? ? ? ? -------------------------------
>>
>> HEASARC conventions are being checked.
>>
>> File: 00000075.fits
>>
>> 1 Header-Data Units in this file.
>>
>> =================== HDU 1: Primary Array ===================
>>
>> *** Error: ? Byte #1 in Card#33 is a null(\0).
>> *** Error: ? Keyword #6, BSCALE: lower-case exponent d or e is illegal
>> in value
>> ? ? ? ? ? ? ? ?+1.000000000000e+000.
>> *** Error: ? Keyword #7, BZERO: lower-case exponent d or e is illegal in
>> value
>> ? ? ? ? ? ? ? ?+3.276800000000e+004.
>> *** Error: ? Keyword #9, ORIGIN: Value and Comment not separated by a "/".
>> *** Error: ? Keyword #11, FOCALLEN: lower-case exponent d or e is illegal in
>> ? ? ? ? ? ? ? ?value +1.550000000000e+002.
>> *** Error: ? Keyword #12, APERTURE: lower-case exponent d or e is illegal in
>> ? ? ? ? ? ? ? ?value +0.000000000000e+000.
>> *** Error: ? Keyword #22, TEMPERAT: lower-case exponent d or e is illegal in
>> ? ? ? ? ? ? ? ?value -2.041762134545e+001.
>> *** Error: ? Keyword #24, E-GAIN: lower-case exponent d or e is illegal in
>> ? ? ? ? ? ? ? ?value +1.430000000000e+000.
>> *** Error: ? Keyword #25, XPIXSZ: lower-case exponent d or e is illegal in
>> ? ? ? ? ? ? ? ?value +6.800000000000e-003.
>> *** Error: ? Keyword #26, YPIXSZ: lower-case exponent d or e is illegal in
>> ? ? ? ? ? ? ? ?value +6.800000000000e-003.
>> *** Error: ? Keyword #29, EXPOSURE: lower-case exponent d or e is illegal in
>> ? ? ? ? ? ? ? ?value +3.000000000000e+002.
>> *** Error: ? The header fill area is not totally filled with blanks.
>>
>> ? ?32 header keywords
>>
>> ? ?16-bit integer pixels, ?2 axes (2184 x 1472),
>>
>> ++++++++++++++++++++++ Error Summary ?++++++++++++++++++++++
>>
>> ? ?HDU# ?Name (version) ? ? ? Type ? ? ? ? ? ? Warnings ?Errors
>> ? ?1 ? ? ? ? ? ? ? ? ? ? ? ? ?Primary Array ? ?0 ? ? ? ? 12
>>
>> **** Verification found 0 warning(s) and 12 error(s). ****
>
> Specifically, the problem in this file that PyFITS is breaking on is
> that the header block is not filled with spaces. ?According to the FITS
> standard any extra bytes at the end of a header block must be spaces,
> and not nulls as is the case in this file.
>
> I'm not sure what PyFITS should do here. ?Should it treat a file like
> this as an error, or should it just produce a warning? ?Given that other
> tools (including earlier PyFITS versions) seem to be accepting of this,
> I'm leaning toward the latter. ?There's no reason PyFITS can't ignore
> the standard here and read the file anyways. ?That said, it's still
> malformatted...
>
> Erik
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From wkerzendorf at gmail.com  Tue Apr  3 12:07:52 2012
From: wkerzendorf at gmail.com (Wolfgang Kerzendorf)
Date: Tue, 3 Apr 2012 12:07:52 -0400
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <CAGMHX_3LxT5FZOQvgabmW+mXSxNs01WjUNuH_XCD=FRx=-VHZg@mail.gmail.com>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
	<CAGMHX_3LxT5FZOQvgabmW+mXSxNs01WjUNuH_XCD=FRx=-VHZg@mail.gmail.com>
Message-ID: <05B608C0-4108-427A-9BA8-A47F2815045B@gmail.com>

Hey guys,

I would have it the other way round: Always raises an exception - except when specifically asked not to, then it will just warn.
I think the algorithms in fitsverify are there to ensure the integrity of the data. If it doesn't comply - the integrity of the data can not be guaranteed (in this case, the integrity is fine, it's just a bad implementation of the fits standard on the telescopes end afaict). Other data formats do similar things: zip files do not decompress if the CRC doesn't match. 

My 2 cents,
   Wolfgang


On 2012-04-03, at 11:42 AM, Thomas Robitaille wrote:

> Would it be worth having a pedantic=True/False flag similar to that in
> Mike's VO module, that would default to False (and would produce
> warnings as has been mentioned), but if turned to True would raise
> Exception if the FITS file does not conform to the standard?
> 
> Cheers,
> Tom
> 
> On 3 April 2012 16:09, Erik Bray <embray at stsci.edu> wrote:
>> On 04/03/2012 09:53 AM, Phil Hodge wrote:
>>> On 04/02/2012 07:48 PM, Eddy Barratt wrote:
>>>> I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.
>>>> 
>>>> Here is the error message:
>>>> 
>>>> ...
>>> 
>>> Here is the output from fverify for your file:
>>> 
>>>                         FVERIFY V4.0.0 (CFITSIO V2.470)
>>>                         -------------------------------
>>> 
>>> HEASARC conventions are being checked.
>>> 
>>> File: 00000075.fits
>>> 
>>> 1 Header-Data Units in this file.
>>> 
>>> =================== HDU 1: Primary Array ===================
>>> 
>>> *** Error:   Byte #1 in Card#33 is a null(\0).
>>> *** Error:   Keyword #6, BSCALE: lower-case exponent d or e is illegal
>>> in value
>>>                +1.000000000000e+000.
>>> *** Error:   Keyword #7, BZERO: lower-case exponent d or e is illegal in
>>> value
>>>                +3.276800000000e+004.
>>> *** Error:   Keyword #9, ORIGIN: Value and Comment not separated by a "/".
>>> *** Error:   Keyword #11, FOCALLEN: lower-case exponent d or e is illegal in
>>>                value +1.550000000000e+002.
>>> *** Error:   Keyword #12, APERTURE: lower-case exponent d or e is illegal in
>>>                value +0.000000000000e+000.
>>> *** Error:   Keyword #22, TEMPERAT: lower-case exponent d or e is illegal in
>>>                value -2.041762134545e+001.
>>> *** Error:   Keyword #24, E-GAIN: lower-case exponent d or e is illegal in
>>>                value +1.430000000000e+000.
>>> *** Error:   Keyword #25, XPIXSZ: lower-case exponent d or e is illegal in
>>>                value +6.800000000000e-003.
>>> *** Error:   Keyword #26, YPIXSZ: lower-case exponent d or e is illegal in
>>>                value +6.800000000000e-003.
>>> *** Error:   Keyword #29, EXPOSURE: lower-case exponent d or e is illegal in
>>>                value +3.000000000000e+002.
>>> *** Error:   The header fill area is not totally filled with blanks.
>>> 
>>>    32 header keywords
>>> 
>>>    16-bit integer pixels,  2 axes (2184 x 1472),
>>> 
>>> ++++++++++++++++++++++ Error Summary  ++++++++++++++++++++++
>>> 
>>>    HDU#  Name (version)       Type             Warnings  Errors
>>>    1                          Primary Array    0         12
>>> 
>>> **** Verification found 0 warning(s) and 12 error(s). ****
>> 
>> Specifically, the problem in this file that PyFITS is breaking on is
>> that the header block is not filled with spaces.  According to the FITS
>> standard any extra bytes at the end of a header block must be spaces,
>> and not nulls as is the case in this file.
>> 
>> I'm not sure what PyFITS should do here.  Should it treat a file like
>> this as an error, or should it just produce a warning?  Given that other
>> tools (including earlier PyFITS versions) seem to be accepting of this,
>> I'm leaning toward the latter.  There's no reason PyFITS can't ignore
>> the standard here and read the file anyways.  That said, it's still
>> malformatted...
>> 
>> Erik
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From eddybarratt1 at yahoo.co.uk  Tue Apr  3 12:37:06 2012
From: eddybarratt1 at yahoo.co.uk (Eddy Barratt)
Date: Tue, 3 Apr 2012 17:37:06 +0100 (BST)
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <05B608C0-4108-427A-9BA8-A47F2815045B@gmail.com>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
	<CAGMHX_3LxT5FZOQvgabmW+mXSxNs01WjUNuH_XCD=FRx=-VHZg@mail.gmail.com>
	<05B608C0-4108-427A-9BA8-A47F2815045B@gmail.com>
Message-ID: <1333471026.30961.YahooMailNeo@web29503.mail.ird.yahoo.com>

Thanks a lot guys. So for now I need to try to get the files reproduced with a proper header? Spaces rather than nulls, and no apostrophe in the Origin field?

Also, I found Pyfits v2.4, and installed it, and the files still didn't load. So I'm not sure how my colleague managed to open them, but perhaps it's not a capability that Pyfits lost when it was updated, as my first post suggested.

Thanks, Eddy




----- Original Message -----
From: Wolfgang Kerzendorf <wkerzendorf at gmail.com>
To: astropy at scipy.org
Cc: 
Sent: Tuesday, 3 April 2012, 10:07
Subject: Re: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6

Hey guys,

I would have it the other way round: Always raises an exception - except when specifically asked not to, then it will just warn.
I think the algorithms in fitsverify are there to ensure the integrity of the data. If it doesn't comply - the integrity of the data can not be guaranteed (in this case, the integrity is fine, it's just a bad implementation of the fits standard on the telescopes end afaict). Other data formats do similar things: zip files do not decompress if the CRC doesn't match. 

My 2 cents,
?  Wolfgang


On 2012-04-03, at 11:42 AM, Thomas Robitaille wrote:

> Would it be worth having a pedantic=True/False flag similar to that in
> Mike's VO module, that would default to False (and would produce
> warnings as has been mentioned), but if turned to True would raise
> Exception if the FITS file does not conform to the standard?
> 
> Cheers,
> Tom
> 
> On 3 April 2012 16:09, Erik Bray <embray at stsci.edu> wrote:
>> On 04/03/2012 09:53 AM, Phil Hodge wrote:
>>> On 04/02/2012 07:48 PM, Eddy Barratt wrote:
>>>> I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.
>>>> 
>>>> Here is the error message:
>>>> 
>>>> ...
>>> 
>>> Here is the output from fverify for your file:
>>> 
>>>? ? ? ? ? ? ? ? ? ? ? ?  FVERIFY V4.0.0 (CFITSIO V2.470)
>>>? ? ? ? ? ? ? ? ? ? ? ?  -------------------------------
>>> 
>>> HEASARC conventions are being checked.
>>> 
>>> File: 00000075.fits
>>> 
>>> 1 Header-Data Units in this file.
>>> 
>>> =================== HDU 1: Primary Array ===================
>>> 
>>> *** Error:?  Byte #1 in Card#33 is a null(\0).
>>> *** Error:?  Keyword #6, BSCALE: lower-case exponent d or e is illegal
>>> in value
>>>? ? ? ? ? ? ? ? +1.000000000000e+000.
>>> *** Error:?  Keyword #7, BZERO: lower-case exponent d or e is illegal in
>>> value
>>>? ? ? ? ? ? ? ? +3.276800000000e+004.
>>> *** Error:?  Keyword #9, ORIGIN: Value and Comment not separated by a "/".
>>> *** Error:?  Keyword #11, FOCALLEN: lower-case exponent d or e is illegal in
>>>? ? ? ? ? ? ? ? value +1.550000000000e+002.
>>> *** Error:?  Keyword #12, APERTURE: lower-case exponent d or e is illegal in
>>>? ? ? ? ? ? ? ? value +0.000000000000e+000.
>>> *** Error:?  Keyword #22, TEMPERAT: lower-case exponent d or e is illegal in
>>>? ? ? ? ? ? ? ? value -2.041762134545e+001.
>>> *** Error:?  Keyword #24, E-GAIN: lower-case exponent d or e is illegal in
>>>? ? ? ? ? ? ? ? value +1.430000000000e+000.
>>> *** Error:?  Keyword #25, XPIXSZ: lower-case exponent d or e is illegal in
>>>? ? ? ? ? ? ? ? value +6.800000000000e-003.
>>> *** Error:?  Keyword #26, YPIXSZ: lower-case exponent d or e is illegal in
>>>? ? ? ? ? ? ? ? value +6.800000000000e-003.
>>> *** Error:?  Keyword #29, EXPOSURE: lower-case exponent d or e is illegal in
>>>? ? ? ? ? ? ? ? value +3.000000000000e+002.
>>> *** Error:?  The header fill area is not totally filled with blanks.
>>> 
>>>? ? 32 header keywords
>>> 
>>>? ? 16-bit integer pixels,? 2 axes (2184 x 1472),
>>> 
>>> ++++++++++++++++++++++ Error Summary? ++++++++++++++++++++++
>>> 
>>>? ? HDU#? Name (version)? ? ?  Type? ? ? ? ? ?  Warnings? Errors
>>>? ? 1? ? ? ? ? ? ? ? ? ? ? ? ? Primary Array? ? 0? ? ? ?  12
>>> 
>>> **** Verification found 0 warning(s) and 12 error(s). ****
>> 
>> Specifically, the problem in this file that PyFITS is breaking on is
>> that the header block is not filled with spaces.? According to the FITS
>> standard any extra bytes at the end of a header block must be spaces,
>> and not nulls as is the case in this file.
>> 
>> I'm not sure what PyFITS should do here.? Should it treat a file like
>> this as an error, or should it just produce a warning?? Given that other
>> tools (including earlier PyFITS versions) seem to be accepting of this,
>> I'm leaning toward the latter.? There's no reason PyFITS can't ignore
>> the standard here and read the file anyways.? That said, it's still
>> malformatted...
>> 
>> Erik
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

_______________________________________________
AstroPy mailing list
AstroPy at scipy.org
http://mail.scipy.org/mailman/listinfo/astropy



From rowen at uw.edu  Tue Apr  3 12:47:21 2012
From: rowen at uw.edu (Russell Owen)
Date: Tue, 3 Apr 2012 09:47:21 -0700
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <05B608C0-4108-427A-9BA8-A47F2815045B@gmail.com>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
	<CAGMHX_3LxT5FZOQvgabmW+mXSxNs01WjUNuH_XCD=FRx=-VHZg@mail.gmail.com>
	<05B608C0-4108-427A-9BA8-A47F2815045B@gmail.com>
Message-ID: <A36ECAFE-4802-4371-A7DB-971A7C3D0D2D@uw.edu>

I see no use for an exception in the normal course of affairs.

I think Postel's Robusness Principal applies: "be conservative in what you send, liberal in what you accept". (<http://en.wikipedia.org/wiki/Jon_Postel#Postel.27s_Law>).

However, it would be useful to be able to detect benign errors. For this I favor a state variable that the user can test after reading a file. That eliminates a mode switch (e.g. the pedantic flag). But a pedantic flag would certainly work.

-- Russell

On Apr 3, 2012, at 9:07 AM, Wolfgang Kerzendorf wrote:

> Hey guys,
> 
> I would have it the other way round: Always raises an exception - except when specifically asked not to, then it will just warn.
> I think the algorithms in fitsverify are there to ensure the integrity of the data. If it doesn't comply - the integrity of the data can not be guaranteed (in this case, the integrity is fine, it's just a bad implementation of the fits standard on the telescopes end afaict). Other data formats do similar things: zip files do not decompress if the CRC doesn't match. 
> 
> My 2 cents,
>   Wolfgang
> 
> 
> On 2012-04-03, at 11:42 AM, Thomas Robitaille wrote:
> 
>> Would it be worth having a pedantic=True/False flag similar to that in
>> Mike's VO module, that would default to False (and would produce
>> warnings as has been mentioned), but if turned to True would raise
>> Exception if the FITS file does not conform to the standard?
>> 
>> Cheers,
>> Tom
>> 
>> On 3 April 2012 16:09, Erik Bray <embray at stsci.edu> wrote:
>>> On 04/03/2012 09:53 AM, Phil Hodge wrote:
>>>> On 04/02/2012 07:48 PM, Eddy Barratt wrote:
>>>>> I have a number of fits files that I cannot open in PyFits 3.0.6, though a colleague using 2.4 can open them. PyFits does work on other fits files on my computer. I would have assumed that the problem is with the files, but they do open using the older version and my colleagues using IDL and matlab have managed to open them too, so perhaps the issue is with PyFits instead. I've attached one of the files, they all have the same error message.
>>>>> 
>>>>> Here is the error message:
>>>>> 
>>>>> ...
>>>> 
>>>> Here is the output from fverify for your file:
>>>> 
>>>>                        FVERIFY V4.0.0 (CFITSIO V2.470)
>>>>                        -------------------------------
>>>> 
>>>> HEASARC conventions are being checked.
>>>> 
>>>> File: 00000075.fits
>>>> 
>>>> 1 Header-Data Units in this file.
>>>> 
>>>> =================== HDU 1: Primary Array ===================
>>>> 
>>>> *** Error:   Byte #1 in Card#33 is a null(\0).
>>>> *** Error:   Keyword #6, BSCALE: lower-case exponent d or e is illegal
>>>> in value
>>>>               +1.000000000000e+000.
>>>> *** Error:   Keyword #7, BZERO: lower-case exponent d or e is illegal in
>>>> value
>>>>               +3.276800000000e+004.
>>>> *** Error:   Keyword #9, ORIGIN: Value and Comment not separated by a "/".
>>>> *** Error:   Keyword #11, FOCALLEN: lower-case exponent d or e is illegal in
>>>>               value +1.550000000000e+002.
>>>> *** Error:   Keyword #12, APERTURE: lower-case exponent d or e is illegal in
>>>>               value +0.000000000000e+000.
>>>> *** Error:   Keyword #22, TEMPERAT: lower-case exponent d or e is illegal in
>>>>               value -2.041762134545e+001.
>>>> *** Error:   Keyword #24, E-GAIN: lower-case exponent d or e is illegal in
>>>>               value +1.430000000000e+000.
>>>> *** Error:   Keyword #25, XPIXSZ: lower-case exponent d or e is illegal in
>>>>               value +6.800000000000e-003.
>>>> *** Error:   Keyword #26, YPIXSZ: lower-case exponent d or e is illegal in
>>>>               value +6.800000000000e-003.
>>>> *** Error:   Keyword #29, EXPOSURE: lower-case exponent d or e is illegal in
>>>>               value +3.000000000000e+002.
>>>> *** Error:   The header fill area is not totally filled with blanks.
>>>> 
>>>>   32 header keywords
>>>> 
>>>>   16-bit integer pixels,  2 axes (2184 x 1472),
>>>> 
>>>> ++++++++++++++++++++++ Error Summary  ++++++++++++++++++++++
>>>> 
>>>>   HDU#  Name (version)       Type             Warnings  Errors
>>>>   1                          Primary Array    0         12
>>>> 
>>>> **** Verification found 0 warning(s) and 12 error(s). ****
>>> 
>>> Specifically, the problem in this file that PyFITS is breaking on is
>>> that the header block is not filled with spaces.  According to the FITS
>>> standard any extra bytes at the end of a header block must be spaces,
>>> and not nulls as is the case in this file.
>>> 
>>> I'm not sure what PyFITS should do here.  Should it treat a file like
>>> this as an error, or should it just produce a warning?  Given that other
>>> tools (including earlier PyFITS versions) seem to be accepting of this,
>>> I'm leaning toward the latter.  There's no reason PyFITS can't ignore
>>> the standard here and read the file anyways.  That said, it's still
>>> malformatted...
>>> 
>>> Erik
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From embray at stsci.edu  Tue Apr  3 13:20:41 2012
From: embray at stsci.edu (Erik Bray)
Date: Tue, 3 Apr 2012 13:20:41 -0400
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <05B608C0-4108-427A-9BA8-A47F2815045B@gmail.com>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
	<CAGMHX_3LxT5FZOQvgabmW+mXSxNs01WjUNuH_XCD=FRx=-VHZg@mail.gmail.com>
	<05B608C0-4108-427A-9BA8-A47F2815045B@gmail.com>
Message-ID: <4F7B3169.9000002@stsci.edu>

On 04/03/2012 12:07 PM, Wolfgang Kerzendorf wrote:
> Hey guys,
>
> I would have it the other way round: Always raises an exception - except when specifically asked not to, then it will just warn.
> I think the algorithms in fitsverify are there to ensure the integrity of the data. If it doesn't comply - the integrity of the data can not be guaranteed (in this case, the integrity is fine, it's just a bad implementation of the fits standard on the telescopes end afaict). Other data formats do similar things: zip files do not decompress if the CRC doesn't match.
>
> My 2 cents,
>     Wolfgang
>

Part of me agrees with this in principle, but in practice it just 
wouldn't work.  It's like how part of me wishes that whenever a web 
browser hits some malformed or invalid HTML it would just throw up an 
error and refuse to render the page.

But it's a good thing that web browsers *don't* do that, as such 
behavior would have severely hindered the adoption and growth of the 
web.  I think that FITS, for better or for worse, needs to be treated a 
little like HTML in that respect.

In particular, it shouldn't throw up errors when it's fairly unambiguous 
what was *meant*.  There's absolutely nothing about the "spaces for 
padding" requirement that affects parsing of the header--as long as the 
header ends on the right alignment that's all that matters.

I already have a fix in git (and subversion as soon as a I `git svn 
dcommit`) for this issue.  So I'll try to slip that in to the 3.0.7 
release, which I hope to get done today or tomorrow.

Incidentally, PyFITS has no problem with the invalid apostrophe in the 
origin keyword.  Maybe it should say something about that and consider 
fixing it...

Erik


From embray at stsci.edu  Tue Apr  3 13:32:02 2012
From: embray at stsci.edu (Erik Bray)
Date: Tue, 3 Apr 2012 13:32:02 -0400
Subject: [AstroPy] Opening file in PyFits, works in v2.4, not in v3.0.6
In-Reply-To: <4F7B3169.9000002@stsci.edu>
References: <1333410510.96927.YahooMailNeo@web29502.mail.ird.yahoo.com>
	<4F7B00F2.2060607@stsci.edu> <4F7B0493.1070104@stsci.edu>
	<CAGMHX_3LxT5FZOQvgabmW+mXSxNs01WjUNuH_XCD=FRx=-VHZg@mail.gmail.com>
	<05B608C0-4108-427A-9BA8-A47F2815045B@gmail.com>
	<4F7B3169.9000002@stsci.edu>
Message-ID: <4F7B3412.6040500@stsci.edu>

On 04/03/2012 01:20 PM, Erik Bray wrote:
> Part of me agrees with this in principle, but in practice it just
> wouldn't work.  It's like how part of me wishes that whenever a web
> browser hits some malformed or invalid HTML it would just throw up an
> error and refuse to render the page.
>
> But it's a good thing that web browsers *don't* do that, as such
> behavior would have severely hindered the adoption and growth of the
> web.  I think that FITS, for better or for worse, needs to be treated a
> little like HTML in that respect.
>
> In particular, it shouldn't throw up errors when it's fairly unambiguous
> what was *meant*.  There's absolutely nothing about the "spaces for
> padding" requirement that affects parsing of the header--as long as the
> header ends on the right alignment that's all that matters.
>
> I already have a fix in git (and subversion as soon as a I `git svn
> dcommit`) for this issue.  So I'll try to slip that in to the 3.0.7
> release, which I hope to get done today or tomorrow.
>
> Incidentally, PyFITS has no problem with the invalid apostrophe in the
> origin keyword.  Maybe it should say something about that and consider
> fixing it...

Though I also just found that PyFITS does have some problems with the 
malformatted float values in the header that might be worth investigating.

Erik



From embray at stsci.edu  Tue Apr  3 18:15:10 2012
From: embray at stsci.edu (Erik Bray)
Date: Tue, 3 Apr 2012 18:15:10 -0400
Subject: [AstroPy] Python installation and ecosystem tutorial update
In-Reply-To: <CAGsQ-Nu7MYDmbtFSFjjH6_7c0ieMsketh_fd7nYciRYBn9yzNQ@mail.gmail.com>
References: <CAGsQ-Nu7MYDmbtFSFjjH6_7c0ieMsketh_fd7nYciRYBn9yzNQ@mail.gmail.com>
Message-ID: <4F7B766E.9000009@stsci.edu>

On 04/03/2012 08:59 AM, Tom Aldcroft wrote:
> In preparation for a 2 hour workshop at CfA today I've updated the
> Python4Astronomers tutorial section on installing scientific Python
> and understanding the Python package ecosystem:
>
>    http://python4astronomers.github.com/installation/installation.html
>
> As always I'm interested in different perspectives and feedback.  I
> hope someday there will be one clearly superior solution for
> installation, but we're not there yet.
>
> You can see recent commits here:
>
>    https://github.com/python4astronomers/python4astronomers/commits/master
>
> Thanks,
> Tom

Very nice writeup!

At some point I need to add to that a small tutorial on creating Python 
distributions and developing/testing with virtualenv and setup.py develop.

Not that there isn't other documentation out there, but a lot of it is 
outdated or incomplete, and doesn't address all the issues that might 
come up in packaging science software.

It's far too often that I *still* see people doing things like

sudo cp foo.py /usr/lib/python2.7/site-packages/

Yargh!

Erik



From embray at stsci.edu  Tue Apr 10 17:55:19 2012
From: embray at stsci.edu (Erik Bray)
Date: Tue, 10 Apr 2012 17:55:19 -0400
Subject: [AstroPy] [ANN] PyFITS 3.0.7 Released
Message-ID: <4F84AC47.2040603@stsci.edu>

I'm happy to announce the release of PyFITS 3.0.7.  This is a bug fix
release--most of the issues fixed are relatively uncommon corner cases. 
  In particular, this release fixed several bugs in the (previously 
neglected) support for Random Groups HDUs.

See http://www.stsci.edu/resources/software_hardware/pyfits/Download for
download and installation information.

See http://www.stsci.edu/resources/software_hardware/pyfits/release for
the full changelog.

Big thanks to everyone who submitted bug reports and thus helped 
improved PyFITS' overall stability.

Erik


From embray at stsci.edu  Tue Apr 10 18:14:04 2012
From: embray at stsci.edu (Erik Bray)
Date: Tue, 10 Apr 2012 18:14:04 -0400
Subject: [AstroPy] [ANN] PyFITS 3.0.7 Released
In-Reply-To: <4F84AC47.2040603@stsci.edu>
References: <4F84AC47.2040603@stsci.edu>
Message-ID: <4F84B0AC.3060000@stsci.edu>

Sorry, jumped the gun a bit on that release announcement.  Still waiting 
on IT to push my updates to the PyFITS web site.  In the meantime you 
can still view the changelog and download links on PyPI:

http://pypi.python.org/pypi/pyfits/3.0.7

Erik

On 04/10/2012 05:55 PM, Erik Bray wrote:
> I'm happy to announce the release of PyFITS 3.0.7.  This is a bug fix
> release--most of the issues fixed are relatively uncommon corner cases.
>    In particular, this release fixed several bugs in the (previously
> neglected) support for Random Groups HDUs.
>
> See http://www.stsci.edu/resources/software_hardware/pyfits/Download for
> download and installation information.
>
> See http://www.stsci.edu/resources/software_hardware/pyfits/release for
> the full changelog.
>
> Big thanks to everyone who submitted bug reports and thus helped
> improved PyFITS' overall stability.
>
> Erik
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From wkerzendorf at gmail.com  Thu Apr 12 18:12:25 2012
From: wkerzendorf at gmail.com (Wolfgang Kerzendorf)
Date: Fri, 13 Apr 2012 00:12:25 +0200
Subject: [AstroPy] got centroid?
Message-ID: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>

Hey all,

I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.

Cheers
    Wolfgang

From adrianmpw at gmail.com  Thu Apr 12 18:26:15 2012
From: adrianmpw at gmail.com (Adrian Price-Whelan)
Date: Thu, 12 Apr 2012 18:26:15 -0400
Subject: [AstroPy] got centroid?
In-Reply-To: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
References: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
Message-ID: <B031701C-7F2E-49A0-9EE7-E34B48CFF246@gmail.com>

This may not help you because it's not so smart, but to zeroth order, you can 
do something like this:

import scipy.ndimage as snd
labels, num = snd.label(imageData > (2.*sigma), np.ones((3,3)))
starCoordinates = snd.center_of_mass(imageData, labels, range(1,num+1))

Obviously it's making some nasty assumptions about the image, but it's at 
least a place to start. 

- Adrian

On Apr 12, 2012, at 6:12 PM, Wolfgang Kerzendorf wrote:

> Hey all,
> 
> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.
> 
> Cheers
>    Wolfgang
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

--
Adrian Price-Whelan
Department of Astronomy
Columbia University



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20120412/84a3d6fb/attachment.html>

From mperrin at stsci.edu  Thu Apr 12 18:45:37 2012
From: mperrin at stsci.edu (Marshall Perrin)
Date: Thu, 12 Apr 2012 22:45:37 +0000
Subject: [AstroPy] got centroid?
In-Reply-To: <B031701C-7F2E-49A0-9EE7-E34B48CFF246@gmail.com>
References: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
	<B031701C-7F2E-49A0-9EE7-E34B48CFF246@gmail.com>
Message-ID: <75348548-6DAA-4B3E-9916-3F173CCDA6A7@stsci.edu>

On Apr 12, 2012, at 6:12 PM, Wolfgang Kerzendorf wrote:
I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.


This is not in a standard library but should be easily installable:  My webbpsf package (for optical simulations of JWST) includes an implementation of the unbiased floating window centroid algorithm adopted for target acquisitions with JWST.  (Note that this algorithm may be considered more smart than quick...)

You can get the entire webbpsf package from http://www.stsci.edu/jwst/software/webbpsf, but as that's overkill for this purpose I attach here just the one relevant file.

Cheers,

 - Marshall





On Apr 12, 2012, at 6:26 PM, Adrian Price-Whelan wrote:

This may not help you because it's not so smart, but to zeroth order, you can
do something like this:

import scipy.ndimage as snd
labels, num = snd.label(imageData > (2.*sigma), np.ones((3,3)))
starCoordinates = snd.center_of_mass(imageData, labels, range(1,num+1))

Obviously it's making some nasty assumptions about the image, but it's at
least a place to start.

- Adrian

On Apr 12, 2012, at 6:12 PM, Wolfgang Kerzendorf wrote:

Hey all,

I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.

Cheers
   Wolfgang
_______________________________________________
AstroPy mailing list
AstroPy at scipy.org<mailto:AstroPy at scipy.org>
http://mail.scipy.org/mailman/listinfo/astropy

--
Adrian Price-Whelan
Department of Astronomy
Columbia University



_______________________________________________
AstroPy mailing list
AstroPy at scipy.org<mailto:AstroPy at scipy.org>
http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20120412/e7081407/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fwcentroid.py
Type: text/x-python-script
Size: 8654 bytes
Desc: fwcentroid.py
URL: <http://mail.python.org/pipermail/astropy/attachments/20120412/e7081407/attachment.bin>

From rowen at uw.edu  Thu Apr 12 19:08:15 2012
From: rowen at uw.edu (Russell Owen)
Date: Thu, 12 Apr 2012 16:08:15 -0700
Subject: [AstroPy] got centroid?
In-Reply-To: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
References: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
Message-ID: <4AE697B4-FDF8-4EFC-8BB4-A2A0DC641481@uw.edu>

On Apr 12, 2012, at 3:12 PM, Wolfgang Kerzendorf wrote:

> Hey all,
> 
> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.

One option is PyGuide: <http://www.astro.washington.edu/users/rowen/PyGuide/Manual.html>
Centroiding is done by hunting for the point of maximum symmetry. This was designed by jim Gunn to be robust for images with missing data (e.g. for guiding using spectrograph slitviewer camera images or coherent guide bundle images). The package includes a star finder. I have no idea if it will be fast enough for you.

-- Russell

From llg at lpnhe.in2p3.fr  Fri Apr 13 03:36:23 2012
From: llg at lpnhe.in2p3.fr (Laurent Le Guillou)
Date: Fri, 13 Apr 2012 09:36:23 +0200
Subject: [AstroPy] got centroid?
In-Reply-To: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
References: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
Message-ID: <4F87D777.5090503@lpnhe.in2p3.fr>


Hi Wolfgang,

A long time ago I wrote a small SExtractor python wrapper.
You may find the package here :

http://supernovae.in2p3.fr/~llg/SExtractor/sextractor.tgz

I hope it may help, as SExtractor is fast.

Cheers,

Laurent


Le 13/04/2012 00:12, Wolfgang Kerzendorf a ?crit :
> Hey all,
> 
> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.
> 
> Cheers
>     Wolfgang
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From nicolas.gruel at gmail.com  Fri Apr 13 03:50:54 2012
From: nicolas.gruel at gmail.com (Nicolas Gruel)
Date: Fri, 13 Apr 2012 09:50:54 +0200
Subject: [AstroPy] got centroid?
In-Reply-To: <4F87D777.5090503@lpnhe.in2p3.fr>
References: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
	<4F87D777.5090503@lpnhe.in2p3.fr>
Message-ID: <CAPzb=+eDQfzsCD5oAqCoA3qhGKhi7NQSdZ3BqcLJc1L2YE5XVg@mail.gmail.com>

Hi,

I also wrote one some times ago and it is more or less maintained. I should
go back to it in a near futur.

You can download it:

https://gitorious.org/pysextractor

I am not saying that it is better than Laurent's one but the multiplication
of SExtractor wrapper is a good indication that one should be provide by
astropy.

Take care,

Nicolas


On Fri, Apr 13, 2012 at 9:36 AM, Laurent Le Guillou <llg at lpnhe.in2p3.fr>wrote:

>
> Hi Wolfgang,
>
> A long time ago I wrote a small SExtractor python wrapper.
> You may find the package here :
>
> http://supernovae.in2p3.fr/~llg/SExtractor/sextractor.tgz
>
> I hope it may help, as SExtractor is fast.
>
> Cheers,
>
> Laurent
>
>
> Le 13/04/2012 00:12, Wolfgang Kerzendorf a ?crit :
> > Hey all,
> >
> > I just wanted to know if anyone has a quick and smart centroid script in
> any of the python libraries (easily installable). Even better would be star
> finding algorithms like sextractor.
> >
> > Cheers
> >     Wolfgang
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org
> > http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20120413/d086fea6/attachment.html>

From erik.tollerud at gmail.com  Fri Apr 13 04:11:31 2012
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Fri, 13 Apr 2012 01:11:31 -0700
Subject: [AstroPy] got centroid?
In-Reply-To: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
References: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
Message-ID: <CAARXUwVq9AWMKor7rWm1jO=euy6D7pEHQwztmwCpDejqYCnVNA@mail.gmail.com>

Clearly this is something that should be in astropy, given the number
of implementations that already exist...

But to add yet another, astropysics has `astropysics.utils.centroid`
and `astropysics.utils.moments`.  And like Laurent, I've also written
an SExtractor wrapper that's in `astropysics.phot.SExtractor`.


On Thu, Apr 12, 2012 at 3:12 PM, Wolfgang Kerzendorf
<wkerzendorf at gmail.com> wrote:
> Hey all,
>
> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.
>
> Cheers
> ? ?Wolfgang
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



-- 
Erik Tollerud


From wkerzendorf at gmail.com  Fri Apr 13 06:18:01 2012
From: wkerzendorf at gmail.com (Wolfgang Kerzendorf)
Date: Fri, 13 Apr 2012 12:18:01 +0200
Subject: [AstroPy] got centroid?
In-Reply-To: <CAARXUwVq9AWMKor7rWm1jO=euy6D7pEHQwztmwCpDejqYCnVNA@mail.gmail.com>
References: <93C70650-22AD-4BE1-9D4B-64A896FEB099@gmail.com>
	<CAARXUwVq9AWMKor7rWm1jO=euy6D7pEHQwztmwCpDejqYCnVNA@mail.gmail.com>
Message-ID: <362D4AF6-05A9-4DAA-9F01-951AD071D1EF@gmail.com>

Hey all,

Thanks for all the suggestions. I'll give some a try, it's just for aligning data cubes.

But my email had another positive effect: We are collecting algorithms for astropy. So even though, I think the mentioned things are good enough for me, you should keep posting things so we can use this thread as future reference for centroiding. 


Cheers
   Wolfgang
On 2012-04-13, at 10:11 AM, Erik Tollerud wrote:

> Clearly this is something that should be in astropy, given the number
> of implementations that already exist...
> 
> But to add yet another, astropysics has `astropysics.utils.centroid`
> and `astropysics.utils.moments`.  And like Laurent, I've also written
> an SExtractor wrapper that's in `astropysics.phot.SExtractor`.
> 
> 
> On Thu, Apr 12, 2012 at 3:12 PM, Wolfgang Kerzendorf
> <wkerzendorf at gmail.com> wrote:
>> Hey all,
>> 
>> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.
>> 
>> Cheers
>>    Wolfgang
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> 
> -- 
> Erik Tollerud



From jh at physics.ucf.edu  Fri Apr 13 08:40:45 2012
From: jh at physics.ucf.edu (Joe Harrington)
Date: Fri, 13 Apr 2012 08:40:45 -0400
Subject: [AstroPy] got centroid?
In-Reply-To: <mailman.2370.1334302346.1208.astropy@scipy.org>
	(astropy-request@scipy.org)
Message-ID: <wl8lilz3iuq.fsf@physics.ucf.edu>

Hi Wolfgang,

> I just wanted to know if anyone has a quick and smart centroid script
> in any of the python libraries (easily installable). Even better would
> be star finding algorithms like sextractor.

A PhD student here, Nate Lust, has spent considerable time evaluating
centering (not centroiding) algorithms.  The quickest and worst is the
usual center-of-light algorithm.  He has a fast Python/C implementation
of the Gunn-Owen least-asymmetry algorithm that he may be willing to
share.  The other easy one is simply fitting a Gaussian + constant
offset to the inner part of the PSF.  Nate is working on a paper that
evaluates these vs. real and synthetic data.  He and Kevin Stevenson
published preliminary results in the Supplementary Information to:

Stevenson, K. B., J. Harrington, S. Nymeyer, N. Madhusudhan, S. Seager,
W. C. Bowman, R. Hardy, D. Deming, E. Rauscher, and N. Lust
2010. Possible thermochemical disequilibrium in the atmosphere of the
exoplanet GJ 436b. Nature 464, 1161?1164.

The short version is that least asymmetry is the most accurate and takes
the longest to calculate.  Gaussian is most precise but can be biased
away from the true answer if the star drifts a lot within the pixel,
especially if the images are underresolved and the center is nearer to a
pixel boundary than a pixel center.  Oddly enough, for our project we
preferred the Gaussian.  Being consistent (i.e., having the aperture
always be off by 0.05 pixels or so) was better than being correct with a
larger scatter.  We consistently get 0.01-pixel precision with the
Gaussian fit on Spitzer data.

Of course, if you have an accurate and 100x-overresolved version of your
PSF, then PSF shifting, rebinning, and fitting will always win out.  But
the PSF needs to be very accurate for this to work.

--jh--


From jerome_caron_astro at ymail.com  Fri Apr 13 09:36:32 2012
From: jerome_caron_astro at ymail.com (Jerome Caron)
Date: Fri, 13 Apr 2012 14:36:32 +0100 (BST)
Subject: [AstroPy] Re :  got centroid?
In-Reply-To: <wl8lilz3iuq.fsf@physics.ucf.edu>
References: <mailman.2370.1334302346.1208.astropy@scipy.org>
	(astropy-request@scipy.org) <wl8lilz3iuq.fsf@physics.ucf.edu>
Message-ID: <1334324192.34677.YahooMailNeo@web24609.mail.ird.yahoo.com>

Dear list,
?
I found an interesting algorithm for calculating star centroids in the following paper,?appendix B.
http://lanl.arxiv.org/pdf/astro-ph/9907229v1.pdfN. Kaiser, G. Wilson, G. Luppino, H. Dahle, "A photometric study of the supercluster MS0302 with the UH8K CCD camera: image processing and object catalogs", arXiv 16 Jul 1999.
It is easy to program and fast, so I use it for very large number of stars. I tested it without smoothing the image and found it is more stable than a gaussian fit (much less failures), which surprised me. One needs to add a test to make sure the second derivative does not vanish, but this does not happen so often.
Jerome Caron
www.aspylib.com

________________________________
De?: Joe Harrington <jh at physics.ucf.edu>
??: astropy at scipy.org 
Cc?: Nate Lust <natelust at linux.com>; jh at physics.ucf.edu 
Envoy? le : Vendredi 13 avril 2012 14h40
Objet?: Re: [AstroPy] got centroid?

Hi Wolfgang,

> I just wanted to know if anyone has a quick and smart centroid script
> in any of the python libraries (easily installable). Even better would
> be star finding algorithms like sextractor.

A PhD student here, Nate Lust, has spent considerable time evaluating
centering (not centroiding) algorithms.? The quickest and worst is the
usual center-of-light algorithm.? He has a fast Python/C implementation
of the Gunn-Owen least-asymmetry algorithm that he may be willing to
share.? The other easy one is simply fitting a Gaussian + constant
offset to the inner part of the PSF.? Nate is working on a paper that
evaluates these vs. real and synthetic data.? He and Kevin Stevenson
published preliminary results in the Supplementary Information to:

Stevenson, K. B., J. Harrington, S. Nymeyer, N. Madhusudhan, S. Seager,
W. C. Bowman, R. Hardy, D. Deming, E. Rauscher, and N. Lust
2010. Possible thermochemical disequilibrium in the atmosphere of the
exoplanet GJ 436b. Nature 464, 1161?1164.

The short version is that least asymmetry is the most accurate and takes
the longest to calculate.? Gaussian is most precise but can be biased
away from the true answer if the star drifts a lot within the pixel,
especially if the images are underresolved and the center is nearer to a
pixel boundary than a pixel center.? Oddly enough, for our project we
preferred the Gaussian.? Being consistent (i.e., having the aperture
always be off by 0.05 pixels or so) was better than being correct with a
larger scatter.? We consistently get 0.01-pixel precision with the
Gaussian fit on Spitzer data.

Of course, if you have an accurate and 100x-overresolved version of your
PSF, then PSF shifting, rebinning, and fitting will always win out.? But
the PSF needs to be very accurate for this to work.

--jh--
_______________________________________________
AstroPy mailing list
AstroPy at scipy.org
http://mail.scipy.org/mailman/listinfo/astropy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20120413/73cf7122/attachment.html>

From hack at stsci.edu  Fri Apr 13 11:19:35 2012
From: hack at stsci.edu (Warren J. Hack)
Date: Fri, 13 Apr 2012 11:19:35 -0400
Subject: [AstroPy] got centroid?
Message-ID: <4F884407.4000100@stsci.edu>

Wolfgang,

There is a star-finding routine in the soon-to-be-officially-released 
drizzlepac package available from STScI.  This routine can be run on its 
own (it is all in Python after all), but a stand-alone interface to run 
just the star finding has not been documented yet.  It is just a single 
function call that requires a numpy array of the image and a few 
numerical parameters and I can point you to examples where it is run 
within a larger task if you contact me directly. Anyone familiar with 
DAOFIND from IRAF will be able to use this code easily.  If you want to 
check it out, you can get this code in pre-release beta form (in binary 
for Mac or source for other platforms) from:

     http://stsdas.stsci.edu/irafx

We have found that it can provide RMS on point sources of <0.1 pixels 
(and seen 0.03 pix RMS in some cases), but your mileage may vary.

Warren Hack
Science Software Branch
STScI

> Hey all,
>
> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.
>
> Cheers
>      Wolfgang


From wkerzendorf at gmail.com  Fri Apr 13 11:47:38 2012
From: wkerzendorf at gmail.com (Wolfgang Kerzendorf)
Date: Fri, 13 Apr 2012 17:47:38 +0200
Subject: [AstroPy] got centroid?
In-Reply-To: <4F884407.4000100@stsci.edu>
References: <4F884407.4000100@stsci.edu>
Message-ID: <F3E5DA50-795D-4F58-88C8-95625267D41D@gmail.com>

Hey all,

Thanks for the many suggestions, there seem to be quite a few gems among it. For future reference I have put the suggestions on the astropy wiki (very quick and dirty for now), but hopefully that will help us in the long run when looking for these (only seemingly) simple algorithms to center on an object. Feel free to edit and add to it: https://github.com/astropy/astropy/wiki/2ddata_centroid

I also created a root page for 2ddata utilities at : https://github.com/astropy/astropy/wiki/2ddata_util

Cheers
   Wolfgang

On 2012-04-13, at 5:19 PM, Warren J. Hack wrote:

> Wolfgang,
> 
> There is a star-finding routine in the soon-to-be-officially-released 
> drizzlepac package available from STScI.  This routine can be run on its 
> own (it is all in Python after all), but a stand-alone interface to run 
> just the star finding has not been documented yet.  It is just a single 
> function call that requires a numpy array of the image and a few 
> numerical parameters and I can point you to examples where it is run 
> within a larger task if you contact me directly. Anyone familiar with 
> DAOFIND from IRAF will be able to use this code easily.  If you want to 
> check it out, you can get this code in pre-release beta form (in binary 
> for Mac or source for other platforms) from:
> 
>     http://stsdas.stsci.edu/irafx
> 
> We have found that it can provide RMS on point sources of <0.1 pixels 
> (and seen 0.03 pix RMS in some cases), but your mileage may vary.
> 
> Warren Hack
> Science Software Branch
> STScI
> 
>> Hey all,
>> 
>> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.
>> 
>> Cheers
>>     Wolfgang
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20120413/5abc3648/attachment.html>

From wkerzendorf at gmail.com  Fri Apr 13 12:42:46 2012
From: wkerzendorf at gmail.com (Wolfgang Kerzendorf)
Date: Fri, 13 Apr 2012 18:42:46 +0200
Subject: [AstroPy] Extracting Stellar Spectra from IFU cubes
Message-ID: <52E66EAD-88A5-4E1F-A7B4-5CE8202B455F@gmail.com>

Hey all,

After the success with my centroid question, I thought I post this to the list as well. I am trying to extract stellar spectra from more or less bad IFU data. I have written very simple software that extracts a aperture and uses a sky annulus to get the skyline spectra and subtract one from the other (not very sophisticated, I know). Has anyone done this in a smarter way? 

Cheers
   Wolfgang

From klabrie at gemini.edu  Fri Apr 13 16:18:28 2012
From: klabrie at gemini.edu (Kathleen Labrie)
Date: Fri, 13 Apr 2012 10:18:28 -1000
Subject: [AstroPy] got centroid?
In-Reply-To: <4F884407.4000100@stsci.edu>
References: <4F884407.4000100@stsci.edu>
Message-ID: <594CC6C0-B46A-4213-A564-B67B4DAE421D@gemini.edu>

Hi Warren,

Is this function also providing flux estimates and fwhm measurements?

Kathleen


On Apr 13, 2012, at 5:19 AM, Warren J. Hack wrote:

> Wolfgang,
> 
> There is a star-finding routine in the soon-to-be-officially-released 
> drizzlepac package available from STScI.  This routine can be run on its 
> own (it is all in Python after all), but a stand-alone interface to run 
> just the star finding has not been documented yet.  It is just a single 
> function call that requires a numpy array of the image and a few 
> numerical parameters and I can point you to examples where it is run 
> within a larger task if you contact me directly. Anyone familiar with 
> DAOFIND from IRAF will be able to use this code easily.  If you want to 
> check it out, you can get this code in pre-release beta form (in binary 
> for Mac or source for other platforms) from:
> 
>     http://stsdas.stsci.edu/irafx
> 
> We have found that it can provide RMS on point sources of <0.1 pixels 
> (and seen 0.03 pix RMS in some cases), but your mileage may vary.
> 
> Warren Hack
> Science Software Branch
> STScI
> 
>> Hey all,
>> 
>> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.
>> 
>> Cheers
>>     Wolfgang
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From mperrin at stsci.edu  Thu Apr 19 13:50:04 2012
From: mperrin at stsci.edu (Marshall Perrin)
Date: Thu, 19 Apr 2012 17:50:04 +0000
Subject: [AstroPy] got flux-conservative rebinning?
In-Reply-To: <F3E5DA50-795D-4F58-88C8-95625267D41D@gmail.com>
References: <4F884407.4000100@stsci.edu>
	<F3E5DA50-795D-4F58-88C8-95625267D41D@gmail.com>
Message-ID: <A9082BDE-2A8E-4B9D-9A67-8B8521B45CB1@stsci.edu>



While we're fishing for useful algorithms, does anyone have a convenient implementation of flux-conservative rebinning? The image interpolations in  scipy.ndimage are not conservative.  Montage can do this, of course, but that seems like overkill (and adds a lot of interface complexity) if I just want to scale an image by 75% while conserving energy...  Thanks in advance,

 - Marshall






On Apr 13, 2012, at 11:47 AM, Wolfgang Kerzendorf wrote:

Hey all,

Thanks for the many suggestions, there seem to be quite a few gems among it. For future reference I have put the suggestions on the astropy wiki (very quick and dirty for now), but hopefully that will help us in the long run when looking for these (only seemingly) simple algorithms to center on an object. Feel free to edit and add to it: https://github.com/astropy/astropy/wiki/2ddata_centroid

I also created a root page for 2ddata utilities at : https://github.com/astropy/astropy/wiki/2ddata_util

Cheers
   Wolfgang

On 2012-04-13, at 5:19 PM, Warren J. Hack wrote:

Wolfgang,

There is a star-finding routine in the soon-to-be-officially-released
drizzlepac package available from STScI.  This routine can be run on its
own (it is all in Python after all), but a stand-alone interface to run
just the star finding has not been documented yet.  It is just a single
function call that requires a numpy array of the image and a few
numerical parameters and I can point you to examples where it is run
within a larger task if you contact me directly. Anyone familiar with
DAOFIND from IRAF will be able to use this code easily.  If you want to
check it out, you can get this code in pre-release beta form (in binary
for Mac or source for other platforms) from:

    http://stsdas.stsci.edu/irafx

We have found that it can provide RMS on point sources of <0.1 pixels
(and seen 0.03 pix RMS in some cases), but your mileage may vary.

Warren Hack
Science Software Branch
STScI

Hey all,

I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.

Cheers
    Wolfgang
_______________________________________________
AstroPy mailing list
AstroPy at scipy.org<mailto:AstroPy at scipy.org>
http://mail.scipy.org/mailman/listinfo/astropy

_______________________________________________
AstroPy mailing list
AstroPy at scipy.org<mailto:AstroPy at scipy.org>
http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20120419/9fd53657/attachment.html>

From sammyniemi2010 at gmail.com  Thu Apr 19 15:37:15 2012
From: sammyniemi2010 at gmail.com (Sami-Matias Niemi)
Date: Thu, 19 Apr 2012 20:37:15 +0100
Subject: [AstroPy] got flux-conservative rebinning?
In-Reply-To: <A9082BDE-2A8E-4B9D-9A67-8B8521B45CB1@stsci.edu>
References: <4F884407.4000100@stsci.edu>
	<F3E5DA50-795D-4F58-88C8-95625267D41D@gmail.com>
	<A9082BDE-2A8E-4B9D-9A67-8B8521B45CB1@stsci.edu>
Message-ID: <5D081DE7-A4A3-43B2-972F-13D416AD1E74@gmail.com>

Hi Marshall,

The enclosed Python file might contain a function (frebin) that allows you to do what you are looking for (no guarantee that the functionality is correct). It's a very simple dummy solution and you cannot input the zoom factor as percentage or anything fancy like that. Some kind of documentation is here:
http://sammyniemi.com/SamPy/SamPy.image.manipulation.frebin.html


Cheers,
Sami






On 19 Apr 2012, at 18:50, Marshall Perrin wrote:

> 
> 
> While we're fishing for useful algorithms, does anyone have a convenient implementation of flux-conservative rebinning? The image interpolations in  scipy.ndimage are not conservative.  Montage can do this, of course, but that seems like overkill (and adds a lot of interface complexity) if I just want to scale an image by 75% while conserving energy...  Thanks in advance, 
> 
>  - Marshall
> 
> 
> 
> 
> 
> 
> On Apr 13, 2012, at 11:47 AM, Wolfgang Kerzendorf wrote:
> 
>> Hey all,
>> 
>> Thanks for the many suggestions, there seem to be quite a few gems among it. For future reference I have put the suggestions on the astropy wiki (very quick and dirty for now), but hopefully that will help us in the long run when looking for these (only seemingly) simple algorithms to center on an object. Feel free to edit and add to it: https://github.com/astropy/astropy/wiki/2ddata_centroid
>> 
>> I also created a root page for 2ddata utilities at : https://github.com/astropy/astropy/wiki/2ddata_util
>> 
>> Cheers
>>    Wolfgang
>> 
>> On 2012-04-13, at 5:19 PM, Warren J. Hack wrote:
>> 
>>> Wolfgang,
>>> 
>>> There is a star-finding routine in the soon-to-be-officially-released 
>>> drizzlepac package available from STScI.  This routine can be run on its 
>>> own (it is all in Python after all), but a stand-alone interface to run 
>>> just the star finding has not been documented yet.  It is just a single 
>>> function call that requires a numpy array of the image and a few 
>>> numerical parameters and I can point you to examples where it is run 
>>> within a larger task if you contact me directly. Anyone familiar with 
>>> DAOFIND from IRAF will be able to use this code easily.  If you want to 
>>> check it out, you can get this code in pre-release beta form (in binary 
>>> for Mac or source for other platforms) from:
>>> 
>>>     http://stsdas.stsci.edu/irafx
>>> 
>>> We have found that it can provide RMS on point sources of <0.1 pixels 
>>> (and seen 0.03 pix RMS in some cases), but your mileage may vary.
>>> 
>>> Warren Hack
>>> Science Software Branch
>>> STScI
>>> 
>>>> Hey all,
>>>> 
>>>> I just wanted to know if anyone has a quick and smart centroid script in any of the python libraries (easily installable). Even better would be star finding algorithms like sextractor.
>>>> 
>>>> Cheers
>>>>     Wolfgang
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>> 
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: manipulation.py
Type: text/x-python-script
Size: 5058 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20120419/22953502/attachment.bin>

From d.berry at jach.hawaii.edu  Thu Apr 19 15:38:02 2012
From: d.berry at jach.hawaii.edu (David Berry)
Date: Thu, 19 Apr 2012 20:38:02 +0100
Subject: [AstroPy] got flux-conservative rebinning?
In-Reply-To: <CAALafn7ap2g8HXUvfHveCboQKootdsL49wiAyQhgHuZecZsWaQ@mail.gmail.com>
References: <4F884407.4000100@stsci.edu>
	<F3E5DA50-795D-4F58-88C8-95625267D41D@gmail.com>
	<A9082BDE-2A8E-4B9D-9A67-8B8521B45CB1@stsci.edu>
	<CAALafn7ap2g8HXUvfHveCboQKootdsL49wiAyQhgHuZecZsWaQ@mail.gmail.com>
Message-ID: <CAALafn5o8O2MrF3yDJqhrH4=WwdRh9-jEP4n7L7ht5sD9LfFWg@mail.gmail.com>

On 19 April 2012 20:36, David Berry <d.berry at jach.hawaii.edu> wrote:
> On 19 April 2012 18:50, Marshall Perrin <mperrin at stsci.edu> wrote:
>>
>>
>> While we're fishing for useful algorithms, does anyone have a convenient
>> implementation of flux-conservative rebinning? The image interpolations in
>> ?scipy.ndimage are not conservative. ?Montage can do this, of course, but
>> that seems like overkill (and adds a lot of interface complexity) if I just
>> want to scale an image by 75% while conserving energy... ?Thanks in
>> advance,
>
> pyast (https://github.com/timj/starlink-pyast) can do this. For instance:
>
> import starlink.Ast
>
> # ?Create an object describing a 2D transformation that zoom by a
> factor of 0.7 about the origin of a 2M


Arghh!  A finger slip caused my message to be sent prematurely. The
rest will follow shortly :-)

David


From d.berry at jach.hawaii.edu  Thu Apr 19 15:36:45 2012
From: d.berry at jach.hawaii.edu (David Berry)
Date: Thu, 19 Apr 2012 20:36:45 +0100
Subject: [AstroPy] got flux-conservative rebinning?
In-Reply-To: <A9082BDE-2A8E-4B9D-9A67-8B8521B45CB1@stsci.edu>
References: <4F884407.4000100@stsci.edu>
	<F3E5DA50-795D-4F58-88C8-95625267D41D@gmail.com>
	<A9082BDE-2A8E-4B9D-9A67-8B8521B45CB1@stsci.edu>
Message-ID: <CAALafn7ap2g8HXUvfHveCboQKootdsL49wiAyQhgHuZecZsWaQ@mail.gmail.com>

On 19 April 2012 18:50, Marshall Perrin <mperrin at stsci.edu> wrote:
>
>
> While we're fishing for useful algorithms, does anyone have a convenient
> implementation of flux-conservative rebinning? The image interpolations in
> ?scipy.ndimage are not conservative. ?Montage can do this, of course, but
> that seems like overkill (and adds a lot of interface complexity) if I just
> want to scale an image by 75% while conserving energy... ?Thanks in
> advance,

pyast (https://github.com/timj/starlink-pyast) can do this. For instance:

import starlink.Ast

#  Create an object describing a 2D transformation that zoom by a
factor of 0.7 about the origin of a 2M


From d.berry at jach.hawaii.edu  Thu Apr 19 16:42:49 2012
From: d.berry at jach.hawaii.edu (David Berry)
Date: Thu, 19 Apr 2012 21:42:49 +0100
Subject: [AstroPy] got flux-conservative rebinning?
In-Reply-To: <A9082BDE-2A8E-4B9D-9A67-8B8521B45CB1@stsci.edu>
References: <4F884407.4000100@stsci.edu>
	<F3E5DA50-795D-4F58-88C8-95625267D41D@gmail.com>
	<A9082BDE-2A8E-4B9D-9A67-8B8521B45CB1@stsci.edu>
Message-ID: <CAALafn65mJhDNi7fkg-C6JBEovQpcxxirgS-_mGfqDMu2H6UQA@mail.gmail.com>

On 19 April 2012 18:50, Marshall Perrin <mperrin at stsci.edu> wrote:
>
>
> While we're fishing for useful algorithms, does anyone have a convenient
> implementation of flux-conservative rebinning? The image interpolations in
> ?scipy.ndimage are not conservative. ?Montage can do this, of course, but
> that seems like overkill (and adds a lot of interface complexity) if I just
> want to scale an image by 75% while conserving energy... ?Thanks in
> advance,
>
> ?- Marshall
>

The pyast package (https://github.com/timj/starlink-pyast) can do
this. Specifically, see the rebin
(http://dsberry.github.com/starlink/node1.html#Rebin) and resample
(http://dsberry.github.com/starlink/node1.html#Resample) methods. For
instance:

import starlink.Ast

# Create an object describing the mathematical transformation between
the position
# of a value in the input array and its corresponding position in the
output array. Here,
# a simple mapping that zooms by a factor of 0.75 about the origin of
a 2D coordinate
# system. AST can do create many other forms of mapping, including compound
# mappings formed by combining simpler mappings.
zoommap = starlink.Ast.ZoomMap( 2, 0.75 )

# Create the 2D array to use as a test of the rebinning.
data_in = numpy.array(   .......    )

# Paste the elements of this array into another array, using "zoommap" to
# describe the transformation from the position of each sample in the input
# array to its position in the output array. u/lbnd_in gives the
dimensions of the input
# array to be rebinned. Divide each input value up linearly between
# the neighbouring output values (other schemes are available too). Any input
#  values equal to starlink.AST.BAD are ignored (useful for flagging
missing data).
# The output array is created and returned as the method value, and
has dimensions
# given by lbnd_out/ubnd_out.

out = zoommap.rebin( 0.5, lbnd_in, ubnd_in, data_in, None, starlink.Ast.LINEAR,
                                       starlink.Ast.USEBAD , 0.0, 100,
starlink.AST.BAD, lbnd_out,
                                       ubnd_out, lbnd, ubnd )


If an array holding the variance of each input value is available,
these methods can also calculate the variance of each output value.

David


From fabricio at ferrari.pro.br  Fri Apr 20 14:20:21 2012
From: fabricio at ferrari.pro.br (Fabricio Ferrari)
Date: Fri, 20 Apr 2012 15:20:21 -0300
Subject: [AstroPy] AstroPy Digest, Vol 68, Issue 10
In-Reply-To: <mailman.1.1334941202.15648.astropy@scipy.org>
References: <mailman.1.1334941202.15648.astropy@scipy.org>
Message-ID: <CAKzpUfYaXt9WcHrmDhy2-65fP6CqEffOePsccVnR6NgJs1u=rg@mail.gmail.com>

Hi Marshall,

just divide by the zooming factor squared.

Fabricio
..-. ..-.
Fabricio Ferrari? ? ?? [www.ferrari.pro.br]
IMEF -? FURG
Rio Grande, RS
Brasil

> While we're fishing for useful algorithms, does anyone have a convenient implementation of flux-conservative rebinning? The image interpolations in ?scipy.ndimage are not conservative. ?Montage can do this, of course, but that seems like overkill (and adds a lot of interface complexity) if I just want to scale an image by 75% while conserving energy... ?Thanks in advance,
>
> ?- Marshall


From J.P.Terlouw at astro.rug.nl  Sat Apr 21 08:32:46 2012
From: J.P.Terlouw at astro.rug.nl (Hans Terlouw)
Date: Sat, 21 Apr 2012 14:32:46 +0200
Subject: [AstroPy] Kapteyn Package 2.2 released
Message-ID: <4F92A8EE.7040706@astro.rug.nl>

We would like to announce the release of version 2.2 of the Kapteyn
Package, a collection of Python modules developed by the computer group
of the Kapteyn Astronomical Institute, University of Groningen, The
Netherlands.


New features:

- A class for solving non-linear least squares problems using the
   Levenberg-Marquardt technique. It is based on the implementation in C
   of Craig Markwardt?s MPFIT routine. Besides reference documentation,
   an extensive tutorial is provided.

- Many smaller improvements and some bugfixes.


Other features:

- Classes for the handling of spatial and spectral coordinates, WCS
   projections and transformations between different sky systems.
   For WCS, Dr. Mark Calabretta's WCSLIB 4.13.3 is used. WCSLIB is
   integrated in the package and need not be installed separately.

- Tools for writing small applications for the display and interactive
   inspection of (FITS) data and for the creation of plots with world
   coordinate information, including all-sky plots. A class for plotting
   shapes like ellipses, rectangles and regular polygons. Support for
   RGB-coded images.

- Utilities for use with Matplotlib such as obtaining coordinate
   information from plots and interactively modifiable colormaps.

- A class for the efficient reading, writing and manipulating of simple
   table-like structures in text files.

- A function for estimating Gaussian components in a profile. These
   estimates could subsequently be used as initial estimates for a least
   squares fit.


Currently supported architectures are Linux, Apple Mac OS X and
Microsoft Windows.

Extensive documentation, including tutorial and background information,
as well as download and installation instructions can be found at
http://www.astro.rug.nl/software/kapteyn/


-- 
Hans Terlouw
Kapteyn Astronomical Institute
University of Groningen
Postbus 800
NL-9700 AV Groningen
The Netherlands

Phone: +31-(0)50-3634068/73
Fax:   +31-(0)50-3636100


From jturner at gemini.edu  Thu Apr 26 20:15:01 2012
From: jturner at gemini.edu (James Turner)
Date: Thu, 26 Apr 2012 21:15:01 -0300
Subject: [AstroPy] Fwd: SciPy 2012 - Registration and deadlines
Message-ID: <4F99E505.1090701@gemini.edu>

If you'd like to submit an abstract for this year's SciPy in Austin
in July, note that the general deadline is coming up on Monday. The
meeting includes an astronomy mini-symposium that relevant talks
will be considered for.

Cheers,

James.


-------- Original Message --------
Subject: SciPy 2012 - Registration and deadlines
Date: Thu, 26 Apr 2012 16:39:57 +0000
From: SciPy 2012 Organizers <scipy2012 at scipy.org>
Reply-To: SciPy 2012 Organizers <scipy2012 at scipy.org>
To: <jturner at gemini.edu>

Dear all,

(Sorry if you receive this announcement multiple times.)

Registration for SciPy 2012, the eleventh annual Conference on Scientific
Computing with Python, is open! Go to
     https://conference.scipy.org/scipy2012/register/index.php

We would like to remind you that the submissions for talks, posters and
tutorials are open until April 30th, which is just around the corner.
For more information see:

     http://conference.scipy.org/scipy2012/tutorials.php
     http://conference.scipy.org/scipy2012/talks/index.php

For talks or posters, all we need is an abstract.  Tutorials require more
significant preparation.  If you are preparing a tutorial, please send a
brief note to Jonathan Rocher (jrocher at enthought.com) to indicate
your intent.

We look forward to seeing many of you this summer.

Kind regards,

The SciPy 2012 organizers
scipy2012 at scipy.org
==============================================
You are receiving this email because you subscribed to the mailing list or 
registered for the SciPy 2010 or SciPy 2011 conference in Austin, TX.

Unsubscribe jturner at gemini.edu from this list:
http://enthought.us1.list-manage.com/unsubscribe?u=e91b4574d5d1709a9dc4f7ab7&id=069dcb6ee4&e=0c2c22aae8&c=6d4089dda9

Our mailing address is:
Enthought, Inc.
515 Congress Ave.
Austin, TX 78701

Our telephone:
512 536 1057

Forward this email to a friend:
http://us1.forward-to-friend1.com/forward?u=e91b4574d5d1709a9dc4f7ab7&id=6d4089dda9&e=0c2c22aae8

Update your profile:
http://enthought.us1.list-manage.com/profile?u=e91b4574d5d1709a9dc4f7ab7&id=069dcb6ee4&e=0c2c22aae8


From chanley at gmail.com  Thu Apr 26 20:43:42 2012
From: chanley at gmail.com (Christopher Hanley)
Date: Thu, 26 Apr 2012 20:43:42 -0400
Subject: [AstroPy] Fwd: SciPy 2012 - Registration and deadlines
In-Reply-To: <4F99E505.1090701@gemini.edu>
References: <4F99E505.1090701@gemini.edu>
Message-ID: <CADeni3v0EXqUmJKcQuetxiuKvjfWt3_greXRFqR42Pf0Cv3+LA@mail.gmail.com>

What dates will the mini-symposium be held?



On Thu, Apr 26, 2012 at 8:15 PM, James Turner <jturner at gemini.edu> wrote:

> If you'd like to submit an abstract for this year's SciPy in Austin
> in July, note that the general deadline is coming up on Monday. The
> meeting includes an astronomy mini-symposium that relevant talks
> will be considered for.
>
> Cheers,
>
> James.
>
>
> -------- Original Message --------
> Subject: SciPy 2012 - Registration and deadlines
> Date: Thu, 26 Apr 2012 16:39:57 +0000
> From: SciPy 2012 Organizers <scipy2012 at scipy.org>
> Reply-To: SciPy 2012 Organizers <scipy2012 at scipy.org>
> To: <jturner at gemini.edu>
>
> Dear all,
>
> (Sorry if you receive this announcement multiple times.)
>
> Registration for SciPy 2012, the eleventh annual Conference on Scientific
> Computing with Python, is open! Go to
>     https://conference.scipy.org/scipy2012/register/index.php
>
> We would like to remind you that the submissions for talks, posters and
> tutorials are open until April 30th, which is just around the corner.
> For more information see:
>
>     http://conference.scipy.org/scipy2012/tutorials.php
>     http://conference.scipy.org/scipy2012/talks/index.php
>
> For talks or posters, all we need is an abstract.  Tutorials require more
> significant preparation.  If you are preparing a tutorial, please send a
> brief note to Jonathan Rocher (jrocher at enthought.com) to indicate
> your intent.
>
> We look forward to seeing many of you this summer.
>
> Kind regards,
>
> The SciPy 2012 organizers
> scipy2012 at scipy.org
> ==============================================
> You are receiving this email because you subscribed to the mailing list or
> registered for the SciPy 2010 or SciPy 2011 conference in Austin, TX.
>
> Unsubscribe jturner at gemini.edu from this list:
>
> http://enthought.us1.list-manage.com/unsubscribe?u=e91b4574d5d1709a9dc4f7ab7&id=069dcb6ee4&e=0c2c22aae8&c=6d4089dda9
>
> Our mailing address is:
> Enthought, Inc.
> 515 Congress Ave.
> Austin, TX 78701
>
> Our telephone:
> 512 536 1057
>
> Forward this email to a friend:
>
> http://us1.forward-to-friend1.com/forward?u=e91b4574d5d1709a9dc4f7ab7&id=6d4089dda9&e=0c2c22aae8
>
> Update your profile:
>
> http://enthought.us1.list-manage.com/profile?u=e91b4574d5d1709a9dc4f7ab7&id=069dcb6ee4&e=0c2c22aae8
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20120426/305c3852/attachment.html>

From jturner at gemini.edu  Fri Apr 27 09:54:37 2012
From: jturner at gemini.edu (James Turner)
Date: Fri, 27 Apr 2012 10:54:37 -0300
Subject: [AstroPy] Fwd: SciPy 2012 - Registration and deadlines
In-Reply-To: <CADeni3v0EXqUmJKcQuetxiuKvjfWt3_greXRFqR42Pf0Cv3+LA@mail.gmail.com>
References: <4F99E505.1090701@gemini.edu>
	<CADeni3v0EXqUmJKcQuetxiuKvjfWt3_greXRFqR42Pf0Cv3+LA@mail.gmail.com>
Message-ID: <4F9AA51D.7030009@gemini.edu>

> What dates will the mini-symposium be held?

They said either on Wednesday or Thursday evening. Not sure which yet.

Cheers,

James.


From aldcroft at head.cfa.harvard.edu  Fri Apr 27 11:04:43 2012
From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft)
Date: Fri, 27 Apr 2012 11:04:43 -0400
Subject: [AstroPy] Fwd: SciPy 2012 - Registration and deadlines
In-Reply-To: <4F9AA51D.7030009@gemini.edu>
References: <4F99E505.1090701@gemini.edu>
	<CADeni3v0EXqUmJKcQuetxiuKvjfWt3_greXRFqR42Pf0Cv3+LA@mail.gmail.com>
	<4F9AA51D.7030009@gemini.edu>
Message-ID: <CAGsQ-NsCsfCLNReJXj3TBMh6-KhE2o2QvToJCJ1_cKESAo843w@mail.gmail.com>

Have any others definitely decided to go or not go?  I'm still on the
fence, but the mini-symposium certainly pushes me toward the go side.
It would be good for the AstroPy project to be represented with people
and a talk/poster.

- Tom

On Fri, Apr 27, 2012 at 9:54 AM, James Turner <jturner at gemini.edu> wrote:
>> What dates will the mini-symposium be held?
>
> They said either on Wednesday or Thursday evening. Not sure which yet.
>
> Cheers,
>
> James.
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From matthewturk at gmail.com  Fri Apr 27 11:07:00 2012
From: matthewturk at gmail.com (Matthew Turk)
Date: Fri, 27 Apr 2012 11:07:00 -0400
Subject: [AstroPy] Fwd: SciPy 2012 - Registration and deadlines
In-Reply-To: <CAGsQ-NsCsfCLNReJXj3TBMh6-KhE2o2QvToJCJ1_cKESAo843w@mail.gmail.com>
References: <4F99E505.1090701@gemini.edu>
	<CADeni3v0EXqUmJKcQuetxiuKvjfWt3_greXRFqR42Pf0Cv3+LA@mail.gmail.com>
	<4F9AA51D.7030009@gemini.edu>
	<CAGsQ-NsCsfCLNReJXj3TBMh6-KhE2o2QvToJCJ1_cKESAo843w@mail.gmail.com>
Message-ID: <CALO3=5G4uXChrT1Y9uMLip4xM4L+yRNytK6X0eiQu1KwY-3kZA@mail.gmail.com>

I'll be there, but probably coming in Tuesday night and leaving
Thursday evening.

On Fri, Apr 27, 2012 at 11:04 AM, Tom Aldcroft
<aldcroft at head.cfa.harvard.edu> wrote:
> Have any others definitely decided to go or not go? ?I'm still on the
> fence, but the mini-symposium certainly pushes me toward the go side.
> It would be good for the AstroPy project to be represented with people
> and a talk/poster.
>
> - Tom
>
> On Fri, Apr 27, 2012 at 9:54 AM, James Turner <jturner at gemini.edu> wrote:
>>> What dates will the mini-symposium be held?
>>
>> They said either on Wednesday or Thursday evening. Not sure which yet.
>>
>> Cheers,
>>
>> James.
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From embray at stsci.edu  Fri Apr 27 11:08:58 2012
From: embray at stsci.edu (Erik Bray)
Date: Fri, 27 Apr 2012 11:08:58 -0400
Subject: [AstroPy] Fwd: SciPy 2012 - Registration and deadlines
In-Reply-To: <CAGsQ-NsCsfCLNReJXj3TBMh6-KhE2o2QvToJCJ1_cKESAo843w@mail.gmail.com>
References: <4F99E505.1090701@gemini.edu>
	<CADeni3v0EXqUmJKcQuetxiuKvjfWt3_greXRFqR42Pf0Cv3+LA@mail.gmail.com>
	<4F9AA51D.7030009@gemini.edu>
	<CAGsQ-NsCsfCLNReJXj3TBMh6-KhE2o2QvToJCJ1_cKESAo843w@mail.gmail.com>
Message-ID: <4F9AB68A.6060704@stsci.edu>

I haven't gotten any sort of confirmation whether or not I'll be able to 
go.  But in addition to this mini-symposium I would really like to have 
an Astropy sprint for a couple days to get as many of the pending 0.1 
issues hammered out as possible.  I'd even be willing to help organize it :)

Erik B.

On 04/27/2012 11:04 AM, Tom Aldcroft wrote:
> Have any others definitely decided to go or not go?  I'm still on the
> fence, but the mini-symposium certainly pushes me toward the go side.
> It would be good for the AstroPy project to be represented with people
> and a talk/poster.
>
> - Tom
>
> On Fri, Apr 27, 2012 at 9:54 AM, James Turner<jturner at gemini.edu>  wrote:
>>> What dates will the mini-symposium be held?
>>
>> They said either on Wednesday or Thursday evening. Not sure which yet.
>>
>> Cheers,
>>
>> James.


From jturner at gemini.edu  Fri Apr 27 12:10:49 2012
From: jturner at gemini.edu (James Turner)
Date: Fri, 27 Apr 2012 13:10:49 -0300
Subject: [AstroPy] Fwd: SciPy 2012 - Registration and deadlines
In-Reply-To: <CAGsQ-NsCsfCLNReJXj3TBMh6-KhE2o2QvToJCJ1_cKESAo843w@mail.gmail.com>
References: <4F99E505.1090701@gemini.edu>	<CADeni3v0EXqUmJKcQuetxiuKvjfWt3_greXRFqR42Pf0Cv3+LA@mail.gmail.com>	<4F9AA51D.7030009@gemini.edu>
	<CAGsQ-NsCsfCLNReJXj3TBMh6-KhE2o2QvToJCJ1_cKESAo843w@mail.gmail.com>
Message-ID: <4F9AC509.5050506@gemini.edu>

> Have any others definitely decided to go or not go?  I'm still on the
> fence, but the mini-symposium certainly pushes me toward the go side.
> It would be good for the AstroPy project to be represented with people
> and a talk/poster.

I'd certainly like to see an AstroPy talk in the mini-symposium (which
I have now agreed to organize). Erik T. was figuring out (a month ago)
whether he can go for the main meeting and I'm not sure about Perry
(he's usually there but hasn't commented this time), but really the
more AstroPy people the merrier and if people want to discuss specific
aspects of the project then maybe it would be appropriate to have more
than one talk (but leaving room for other astronomy-related projects,
of course).

Cheers,

James.


From jturner at gemini.edu  Fri Apr 27 12:38:36 2012
From: jturner at gemini.edu (James Turner)
Date: Fri, 27 Apr 2012 13:38:36 -0300
Subject: [AstroPy] Fwd: SciPy 2012 - Registration and deadlines
In-Reply-To: <4F9AC509.5050506@gemini.edu>
References: <4F99E505.1090701@gemini.edu>	<CADeni3v0EXqUmJKcQuetxiuKvjfWt3_greXRFqR42Pf0Cv3+LA@mail.gmail.com>	<4F9AA51D.7030009@gemini.edu>	<CAGsQ-NsCsfCLNReJXj3TBMh6-KhE2o2QvToJCJ1_cKESAo843w@mail.gmail.com>
	<4F9AC509.5050506@gemini.edu>
Message-ID: <4F9ACB8C.7040800@gemini.edu>

Vicki also suggested a panel discussion, which does seem a good idea
if we have enough of the right people. There isn't loads of time
(1.5hrs or maybe longer on the Thursday if necessary) but I think it
would be good to finish the session with that.

Cheers,

James.


From martin.raue at desy.de  Mon Apr 30 09:45:38 2012
From: martin.raue at desy.de (Martin Raue)
Date: Mon, 30 Apr 2012 15:45:38 +0200
Subject: [AstroPy] Writing unsigned int with pyfits
Message-ID: <03B28ABA-0FB2-49B4-85CE-C818FB48EAE8@desy.de>

Dear all,

I am trying to create a FITS file with a bintable extension containing unsigned integers with pyfits but could not get it to work (example below). Does maybe someone know what I am doing wrong here?

Best wishes,
Martin

import numpy as np
import pyfits as pf

phdu = pf.PrimaryHDU(uint=True)

a_uint = np.ones(1, dtype=np.uint32)
                     
events = pf.new_table(
    pf.ColDefs([
        pf.Column(name='EVENT_ID', array=a_uint, format='J', bscale=1, bzero=2**31)
        pf.Column(name='OBS_ID', array=a_uint, format='J', bscale=1, bzero=2**31)
#       ... a few more columns
        ])
    )

hdulist = pf.HDUList([phdu, events])

hdulist.writeto('tmp.fits')

hdulist = None

f = pf.open('tmp.fits')

In [2]: f[1].data
Out[2]: 
FITS_rec([(1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0)], 
      dtype=[('EVENT_ID', '>i4'), ('OBS_ID', '>i4'), ('TIME', '>f4'), ('RA', '>f4'), ('DEC', '>f4'), ('DETX', '>f4'), ('DETY', '>f4'), ('ENERGY', '>f4'), ('HIL_MSW', '>f4'), ('HIL_MSL', '>f4')])

In [3]: f[1].data.field('EVENT_ID')[0].dtype
Out[3]: dtype('float64')


-----------------------------------

Martin Raue

Group Leader - LEXI Young Investigator Group
"Cosmic radiation fields and search for dark matter in the early universe"
http://wwwiexp.desy.de/groups/astroparticle/crf/

University of Hamburg
Institute for Experimental Physics
Luruper Chaussee 149
D-22761 Hamburg
Germany

Phone  : +49-40-8998-2993
Fax    : +49-40-8998-2170
E-Mail : martin.raue at desy.de



From embray at stsci.edu  Mon Apr 30 11:18:25 2012
From: embray at stsci.edu (Erik Bray)
Date: Mon, 30 Apr 2012 11:18:25 -0400
Subject: [AstroPy] Writing unsigned int with pyfits
In-Reply-To: <03B28ABA-0FB2-49B4-85CE-C818FB48EAE8@desy.de>
References: <03B28ABA-0FB2-49B4-85CE-C818FB48EAE8@desy.de>
Message-ID: <4F9EAD41.2050204@stsci.edu>

On 04/30/2012 09:45 AM, Martin Raue wrote:
> Dear all,
>
> I am trying to create a FITS file with a bintable extension containing unsigned integers with pyfits but could not get it to work (example below). Does maybe someone know what I am doing wrong here?
>
> Best wishes,
> Martin
>
> import numpy as np
> import pyfits as pf
>
> phdu = pf.PrimaryHDU(uint=True)
>
> a_uint = np.ones(1, dtype=np.uint32)
>
> events = pf.new_table(
>      pf.ColDefs([
>          pf.Column(name='EVENT_ID', array=a_uint, format='J', bscale=1, bzero=2**31)
>          pf.Column(name='OBS_ID', array=a_uint, format='J', bscale=1, bzero=2**31)
> #       ... a few more columns
>          ])
>      )
>
> hdulist = pf.HDUList([phdu, events])
>
> hdulist.writeto('tmp.fits')
>
> hdulist = None
>
> f = pf.open('tmp.fits')
>
> In [2]: f[1].data
> Out[2]:
> FITS_rec([(1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0)],
>        dtype=[('EVENT_ID', '>i4'), ('OBS_ID', '>i4'), ('TIME', '>f4'), ('RA', '>f4'), ('DEC', '>f4'), ('DETX', '>f4'), ('DETY', '>f4'), ('ENERGY', '>f4'), ('HIL_MSW', '>f4'), ('HIL_MSL', '>f4')])
>
> In [3]: f[1].data.field('EVENT_ID')[0].dtype
> Out[3]: dtype('float64')

I believe that you are doing this correctly.  However, when a column has 
non-trivial BSCALE and/or BZERO values, the values in that column are 
converted to floats on the fly.  That's because the TSCALn and TZEROn 
values themselves are actually treated as floats by the FITS standard.

So while the column format shows up as ints, the actual values given to 
the user are floats with the bscale+bzero applied.  If you save changes 
to the file the values will be converted back to ints.

I think the problem here is that PyFITS supports unsigned ints in image 
data by using the `uint=True` argument when opening a FITS file.  Using 
that argument causes PyFITS to recognize that you want pseudo-unsigned 
ints to be returned as actual unsigned ints, and not floats.  However, 
it seems like it only works for images, and not for table columns. 
That, perhaps, should be fixed.

Erik